The ClassTech program was initiated in 2003 primarily in response to faculty demands for a greater number of classrooms equipped with easy-to-use, well maintained teaching technologies, such as systems capable of displaying a networked computer, videotapes/DVDs and objects using a document camera. However, the overarching goal of ClassTech as it relates to LITRE is to improve student learning outcomes at NC State by expanding and enhancing the availability of learning environments that support a broad diversity of teaching and learning styles.
To evaluate ClassTech in terms of impact on teaching and learning for this LITRE report, the ClassTech assessment team used the following methods:
- ClassTech spring 2005 faculty survey (of those teaching in ClassTech rooms)
The response rate was 47% (n=116) compared to 28% (n=35) in fall 2004 - In-depth interviews of nine faculty
Conducted primarily by Dr. Karen St. Clair of FCTL - In room, quick surveys of students and faculty in ten classes
Conducted in spring 2005 - Interview of ClassTech technical support staff
- Log of equipment usage from a subset of ClassTech rooms
Gathered automatically via some in-room control systems - UPA surveys of sophomores and graduating seniors and the annual ResNet user survey
The 2004 LITRE Quality Enhancement Plan proposed a series of general questions to evaluate the impact of classroom technology use on teaching practices and learning outcomes. In the ongoing effort to improve the outcomes-based assessment process, the ClassTech assessment team revised these questions to guide the 2005 assessment efforts. The revised questions are used in the following summary report. Datasets of the ClassTech surveys with identifying information removed are available upon request for further evaluation. The full 2004-2005 ClassTech LITRE Assessment Report is also available.
Assessment Question 1: Evaluate the impact of using technology on faculty workload, pedagogy, attitudes, and amount of material delivered.
In general, faculty perceptions were favorable related to the impact of classroom technologies on their workload. While sometimes taking more time initially, once faculty identified or created material to be learned using electronic technology, it could be more easily accessed by the students and re-used in subsequent semesters. As expected, having technology installed and available in the classrooms, particularly the projector and in-room computer--the most often used device--obviated the hassles of transporting and setup/tear down time.
The research data did not show any evidence of significant impacts on teaching methods, or changes in program, course outcomes, or assessment methods as a result of having access to classroom technology.
Most frequently, faculty used PowerPoint to display notes or illustrations. Web browsers were also used extensively to provide or illustrate class material through static and interactive sites. In some cases faculty used the technology to aid in learner-centered activities such as looking up information on the fly or carrying out simulations based on students' hypotheses or questions. Faculty also reported examples of using the technology to better demonstrate what was expected of students in their use of technology outside of class.
Attitudes toward using the technology were generally favorable. Respondents indicated having a networked computer was "essential" (51%) or "important" (25%). The document camera was the other piece of equipment that more than half said was either essential (27%) or important (28%). The importance a respondent gave to particular equipment was strongly correlated to the frequency that faculty members indicated they used the equipment.
Almost half (48%) of the faculty survey respondents said the pace of the class was faster when using instructional technology. Half of the respondents said the technology allowed for a wider variety of topics to be covered. Most faculty (61%) said they were able to cover course material in more depth than when they did not have classroom technology available.
Assessment Question 2: Assess faculty and student perceptions of the degree to which technology, used inside and outside the classroom, has impacted learning.
Most of the faculty surveyed did not directly address the issue of learning outcomes. When asked as part of a series of open-ended questions how their use of technology impacted students' learning, most focused instead on how they used technology to facilitate learning. It can only be presumed that these faculty felt using these methods helped their students learn more effectively than non-electronic means. Of those who discussed how students learned, the most frequent response was that students were better able to follow class discussion. Others responses included an improved ability to visualize and understand course concepts and improved participation in class.
When compared with teaching without technology, a majority of faculty said their students were much more (16%) or somewhat more (43%) involved in the learning process when taught using instructional technology. Of the faculty who responded to the an open-ended follow up question related to this, 43% said that the students were more involved in class discussions; 18% said that the students could work along with the instructor on problems and data; 15% said that students showed they were more involved by the types of questions they asked during class; 13% said the students could present their own work during class with use of technology. Of all the faculty respondents, 6% said they felt the technology made the student more passive in class or that the students attend class less.
Student data from the ClassTech surveys, indicated that a strong majority (81%) felt their instructor's use of technology positively affected their learning. Students reported that key ways the technology assisted their learning were through its ability to help them visualize concepts being taught and through the use of real-world examples. The data also indicated that students felt their learning was enhanced by the use of technology when compared to courses that did not use the same technology. Results from UPA surveys of sophomores and graduating seniors also indicate a greater percentage who say they learn better with various in-class instructional technology methods than those who report they do not learn as well with technology use. However, somewhat in contradiction to these reports, fewer than half the respondents in the same UPA surveys felt that the use of any of the specific instructional technology methods listed helped them learn better. In the worst case, for the use of computerized exams/quizzes in a class or lab, more graduating senior respondents indicated they did not learn as well (19%) than those who said they learned better (18%).
Assessment Question 3: Assess satisfaction of students, faculty and technical staff with the use of this technology in academic settings.
The research data indicated that students generally prefer the use of electronic instructional technology. Of the students surveyed in 10 classes, most preferred classes with moderate (71%) or extensive (21%) use of instructional technology. Most agreed (61%) or strongly agreed (13%) that the instructional technology accommodated their needs and learning styles.
The faculty surveyed or interviewed for this research were teaching in technology-equipped classrooms, so the results do not represent all faculty. In general, these faculty made significant use of the technology with 73% reporting using the in-room computer sometimes (24%) or most all (49%) of the class sessions. Actual usage data from a subset of the ClassTech rooms confirms the user surveys. Most felt the technology was important for their teaching, with the in-room computer being rated most important (77% reported it was essential or important for their class). Faculty also indicated that the ClassTech equipment was easy to use. The control system was rated easiest to use, with 98% of those using it saying it was easy or very easy to use, followed by the laptop connection (96%), in-room computer (95%), document camera (88%), overhead transparency projector (88%), VCR/DVD (85%), and microphone (75%). A number of comments were provided for how to improve the functionality of the technology, some of which have been implemented over the summer.
The utilization of ClassTech equipment continues to rise as more instructors experience teaching with in-room technology and incorporate it into their class sessions. Until more rooms have technology installed, a key to making sure faculty who want to use the technology have access to it is to make sure they have priority to be scheduled in technology-equipped rooms. It is clear from the usage data available that some faculty teaching in the ClassTech rooms never use any of the equipment. While this may always be the case, it should be minimized as much as possible as long as faculty demand for technology-equipped classrooms exceeds supply. Registration & Records continues to work with the scheduling officers to improve this process.
Assessment Question 4: Identify technical issues involved in using technology in the classroom, for students, faculty, and technical staff.
The ClassTech staff's goal is for the technology in classrooms to be easy-to-use and always available when needed. The staff have been quite successful, according to the most recent faculty surveys, with providing easy-to-use equipment. The level of availability is something that still needs improvement. Preventive maintenance is carried out through weekly room checks and scheduled maintenance. The number of faculty who found something not working has gone down since last fall, but 21% indicated they found something was not working three or more times during the course of a semester. Of the faculty who requested help from ClassTech, a strong majority were either satisfied (36%) or very satisfied (51%) with the help they received.
Some technical issues can be overcome through better training of faculty on equipment usage. Significant efforts go into providing in-room and online materials and in-person training. More than 52% of the faculty respondents indicated they had some contact with ClassTech staff to receive instruction either before the semester started at an Open House (11%), in a one-to-one training session (12%) or during "meet-and-greet" sessions during the first several days of classes (29%). As ClassTech adds additional classrooms, cost-effective solutions will need to be found that further refine how training appointments are made and carried out while continuing to provide a high level of support.
Another technical concern is the greater-than-expected failure rates with some of the equipment that has been deployed. After initial problems with several document cameras, a superior model was tested and purchased for new installations. While the controls are easier to use and the image quality is much better than the original model, this model has also had its share of mechanical and electronic failures. Issues with a number of the computers have also been surfacing this past spring and summer. While being repaired, the root causes are still under investigation, though it appears overheating is playing a role.
The number of classrooms supported by ClassTech doubled from spring 2005 to fall 2005. ClassTech has added staff to keep up with expected call volume. As the program matures, ClassTech will continue to improve the equipment, training and support procedures to provide faculty highly reliable technology that meets their evolving instructional needs.
Back to Spring 2005 Survery Results
