STUDENT-OUTCOME ASSESSMENT

NCSU's coordinated, universitywide institutional effectiveness efforts began in the mid-1980s, when a task force developed a plan to bring NCSU into compliance with the new SACS criteria. Implementation of this plan began in 1989, when five departments prepared pilot evaluation plans. On the basis of their experience, formal universitywide educational-outcome assessment effort began in 1992. However, assessment of programs at NCSU, began long before the institutional-effectiveness campaign began. To cite just some of the assessments of the past decade:

• Regular ten-year graduate- and undergraduate-program reviews, mandated by NCSU and often involving external reviewers. A number of these reviews incorporated alumni surveys.

• Regular, federally mandated reviews of agriculture programs.

  Periodic, formal reviews of all programs by the UNC General Administration.

• Extensive and exhaustive assessment of the undergraduate curriculum. Three recent assessment efforts, all developed in greater detail in the document Brief History of General Education at NCSU (see self-study library) are the Fairchild Commission (1984-85), a study of the role of the social sciences and humanities; the Bland Commission (1985-87), a study of undergraduate education; the Council on Undergraduate Education study of undergraduate education (1988-92). Each of these studies built on its predecessors. Their combined efforts have led to new general education requirements, which will begin with the class entering in fall 1994.

• Student evaluation of courses. In the 1992 self-study surveys of students, over 90 percent of both undergraduate and graduate students indicated that course syllabi specify objectives. The departments receive student assessments of whether these objectives were achieved through course evaluations. As will be discussed in the education programs chapters, virtually all NCSU departments have long required that students complete a formal course evaluation form at the end of each semester.

• Tracking of such outcomes as initial graduate-placement rates by some programs (e.g., the College of Engineering).

• Review of NCSU programs by twenty-two regional or national accreditation agencies, many of which are moving toward an outcome focus. Among the organizations that review NCSU programs are the Accreditation Board for Engineering and Technology, the American Assembly of Collegiate Schools of Business, and the National Architecture Accrediting Board (see self-study library for a full list of the twenty-two accreditation agencies).

• Various evaluation processes that have been undertaken for each NCSU undergraduate and graduate program (see self-study library for an inventory of departmental-evaluation activities).

Figure 3.3
Departmental planning and program-review processes

Some of the efforts cited above do not meet the SACS requirement of continuous evaluation. Most also did not emphasize outcomes, although many did at least consider some outcomes. Nonetheless the list indicates that educational assessment has been a very important, extensive, and ongoing effort at NCSU over the past decade.

NCSU's current institutional-effectiveness effort can be divided into two categories, each of which is the responsibility of a different administrative unit. Student outcomes are the responsibility of the Division of Undergraduate Studies (for the general education program) and the academic departments (for major courses). Research and extension outcomes are the responsibility of the Office of the Vice-Chancellor for Research, Outreach, and Extension. Most of the University's institutional-effectiveness evaluation efforts to date have been directed at specifying and measuring student outcomes.

General Education Outcomes

Nevertheless, NCSU has taken two important initial steps toward assessing general education outcomes. First, an Ad Hoc Committee on General Education, composed of nine faculty members and a representative from University Planning and Analysis, met throughout spring 1993 to discuss ways of planning and assessing student outcomes within general education. Their deliberations led to a June 1993 report outlining basic principles and future steps. Second, throughout the 1993-94 school year, subcommittees of the Council on Undergraduate Education and interested departments are developing evaluation plans and assessment tools.

Departmental Outcomes

In fall 1989 pilot assessment programs were begun in five departments: Philosophy and Religion, Mechanical and Aerospace Engineering, Forestry, Textile and Apparel Management, and Accounting. The pilot efforts were not totally successful, but the experience suggested two lessons that the Office of University Planning and Analysis attempted to incorporate in the universitywide campaign. First, because developing and tracking objectives are time consuming, the departments need the assistance of consultants. Second, departments should be encouraged to build upon any outcome-oriented assessments that are already in place. Some of the pilot departments also suggested that outcome assessment should be more tightly linked to traditional planning and budgeting. As noted, however, NCSU has elected not to link them tightly.

Building upon the pilot results, a formal, universitywide campaign to require such outcome-assessment measures began in early 1992. The University made a major commitment of personnel and time. UPA designated one staff member as a full-time University assessment coordinator, and each department named a faculty coordinator to oversee his/her department's efforts. (Departments with both undergraduate and graduate programs often named two faculty coordinators.) In addition, five faculty were given release time and training to act as departmental consultants.

The campaign was kicked off by a September 1992 visit by Richard Light, professor of education at Harvard University, who headed a nationally recognized evaluation effort at that university. Light made a well-received presentation to the coordinators and administrators in the morning, then met with smaller groups throughout the day.

Throughout the 1992-93 school year, departmental coordinators met regularly in college-based working groups. Each college's associate dean for academic affairs and the faculty consultant also worked with these groups as they developed assessment procedures and shared ideas. The coordinators then used the skills they developed in these groups to guide their departments in creating student-outcome assessment plans. By fall 1993 all departments had submitted assessment plans to University Planning and Analysis.

Although the process varied from department to department, most of the plans were developed in a participative manner, with all departmental faculty involved (or at least invited to be involved). Faculty participation reflects a long-standing NCSU pattern. As the survey results indicate, a large majority of NCSU faculty believe that there is adequate faculty input on curriculum matters.

Each department's educational-outcome assessment plan has four components: the departmental mission statement; a list of the general purposes of the program; a list of objectives specifying student outcomes (skills, knowledge, and/or values), along with performance measures for each outcome; a list of evaluation strategies for determining whether graduates have achieved those outcomes. These evaluation strategies include use of alumni surveys, placement records, exam results, exit surveys, and others. This evaluation section also contained a statement of how the results will be used.

Table 3.1 provides an inventory of the evaluation approaches used for in the undergraduate programs of every NCSU college and school (see self-study library for complete inventories for every undergraduate and graduate program).

As might be expected, the quality of the plans varied by department, with imprecise or process-oriented objectives a common problem. However, both the faculty consultants and UPA screened the plans and when necessary requested revisions. A less easily solved problem was that even many of the well-expressed objectives depended upon data that were not yet available. For example, many departments proposed to survey new graduates a few months after graduation or to track employment. University Planning and Analysis will be conducting these surveys for the University community, but UPA does not have the staff capacity to conduct its first survey of recent graduates until mid or late 1994. (It hired a survey specialist in summer 1993.)

Despite these problems, many departments proposed acceptable plans, and most plans were further strengthened through feedback from the consultants and University Planning and Analysis. As a result, the objectives for most plans are usually clear, and proposed measurements reasonable. Some objectives already have data available. Of course, this process will continue to unfold. Faculty must continue to develop new means of achieving goals and must continue to adjust them as they confront varying student backgrounds and abilities.

University Planning and Analysis, the faculty consultants, and the academic departments have made remarkable progress toward achieving SACS standards for educational-outcome assessment. Most departments have formulated good plans with measurable outcome-oriented objectives. Accordingly, a strong step has been made toward meeting the SACS criterion requiring result-oriented measures of student attainment. Because the process is so new, it is impossible to state definitively that it will be continuous. However, the NCSU Provost's Office has repeatedly expressed a very strong commitment to continuing and improving the process.

Effectiveness of Student-Outcome Assessment

• When the pass rate on the Engineering in Training exam decreased several years ago, it was determined that students had the most difficulty with the statics and dynamics problems. This information was used to strengthen the statics and dynamics courses.

• The Department of Foreign Languages used the results from exit interviews and student evaluations to analyze problem areas. It then made some courses more speaking-interactive, revised the advanced conversation course, and created a lower-level conversation course.

• The Department of Counselor Education uses assessment data gathered from student performance on certification and departmental tests, evaluation by counseling supervisors, professional observers' evaluations of students' counseling skills, employer ratings, and alumni evaluation of their educational experience. Result-driven decisions over the past three years include the addition of two courses at the master's level and a revision in the focus of a counseling course.

These examples of decisions resulting from outcome data are very important. It is too early to be certain, however, whether such plan-based decisions are common, or are isolated occurrences.

The success of the student-outcome assessment depends on whether the student-outcome information is used consistently. Mainstream planning theory suggests that if plans are to affect decisions, there are two prerequisites: timely, accurate information on how well important goals are being met; and incentives for decision makers to use that information to make changes.

NCSU departments have made substantial progress in acquiring information about student outcomes. Incentives to use that information are more problematic, because it is often uncomfortable to change departmental and college routinescourse requirements, degree requirements, teaching assignments, admission requirements, etc. If a department falls short on its objectives, it will certainly be tempted to just ignore, lower, or change the objectives. It needs incentives to make programmatic changes instead.

Of course, there would be such incentives if student-outcome information were routinely considered when the administration was making decisions about changes in a department's budget share or personnel lines. However, University Planning and Analysis has deliberately kept the institutional-effectiveness effort decentralized and separate from the University planning and budgeting that drives these incentives. According to UPA's institutional-effectiveness plan: "Planning goals are stated differently and serve different purposes than academic-evaluation goals. Planning goals provide guidance for decisions about personnel, resources, and program-development priorities. Academic-evaluation goals specify what students should learn and guide decisions about curriculum content. Consequently, planning and evaluation of program effectiveness require different sets of activities."

The second sentence of the quote seems problematic because results from departmental student-outcome plans would also seem relevant to decisions about departmental personnel needs, resources (e.g., does this department need more equipment to better train its students and raise its placement rate?), and program development (e.g., should this department be allowed to add a new master's program if exit interviews and other assessment procedures show students are unhappy with the way the department is administering a current one?).

Table 3.1
Number of undergraduate departments using various evaluation methods, by college

                                     Capstone              Behavioral        Exit                         Teaching     Prof.   Advisory   Review    Review   Review   Review  
College (# departments)	               course  Portfolio   assessment   interview   Survey   In class	evaluation     exam	  board	   dept.     curr.   prof.	ext.   Other


Agriculture and Life Sciences (11)	 6				               7	3	    10	        1	   1	    11	      11	6	 7	 8	

Design (5)	                         1	   3	       5	     5	       4	1	     5	        1	   1	     5	       5	1	 1		

Education and Psychology (5)	         2		       3	     3	       1	1	     5	        4	   2	     5	       5	4	 1	 1	


Engineering (9)	                         8	   3	       2	     2	       5	             9	        7	   5	     9	       9	8	 1	 9	

Forest Resources (3)	                 1		       1	     3	       2	1	     3	        2	   2	     3	       3	3	 1	 3	

Humanities and Social Sciences (8)	 3	   3	                     5	       6	1	     8	        2	   2	     8	       8	2	 1	 2	


Management (3)		                           1		             1	       3	             3	        2	   1	     3	       3	2		 2	

Physical and Mathematical Sciences (5)		   1	       1	     5	                1	     5		           1	     5	       5	1	 2	 4	

Textiles (2)	                         1	   1		             2	       2	1	     2	        1	   2	     2	       2	2		 2	

• INTRODUCTION
• PLANNING AND EVALUATION (previous)
• PLANNING AND ASSESSING RESEARCH (next)
• PLANNING AND ASSESSING OUTREACH AND EXTENSION
• INSTITUTIONAL RESEARCH
• CONCLUSION
• REFERENCE LIST