Industry-University Cooperative Research Centers (IUCRC),
Program Evaluation Project
|
1.1 OVERVIEW
The Industry University Cooperative Research Center (IUCRC) Program and model grew out of an experimental program (The Experimental R&D Incentives Program) which operated from 1972 to about 1979. NSF began systematically building IUCRCs based on the industrial consortia approach developed and validated through this program beginning in 1980-81. A more detailed description of the history and evolution of the IUCRC program and its evaluation can be found in the sources listed in the publications list (see Section 7) and provided as supplementary reading (Section 9).
The ongoing evaluation of the individual centers and the IUCRC Program began in 1982-83. The evaluation effort has several key features: coordinating mechanisms for the national data collection effort; an on-site evaluator at each center; standardized data collection protocol and instruments; and longitudinal data collection. Although the specifics of the evaluation have changed over time, these features remain the bedrocks of the evaluation effort. Support for the evaluation effort is provided by the IUCRC Program, primarily in the form of a line item funding the on-site evaluator in each center grant. NSF also provides funding for the other coordinating mechanisms and various supplemental studies.
There are three coordinating mechanisms for the evaluation effort. First, NSF IUCRC staff provide overall guidance for the evaluation effort. Second, all evaluators comprise a coordinating committee for the evaluation effort. This group meets twice each year (typically January and June) to share briefings on the findings of different components of the evaluation effort, exchange information, orient new evaluators, and vote on issues related to the evaluation effort. Finally, a team from North Carolina State University (NCSU) has been contracted to coordinate and support the IUCRC evaluation assessment activities. The NCSU team will also maintain and update this Evaluator handbook and analyze structural data collected by NSF from Directors.
The remainder of this handbook focuses on the role and responsibilities of the on-site evaluator.
1.2 REQUIRED AND OPTIONAL ACTIVITIES
The evaluator's role may be described in terms of required and optional activities. This section will provide an overview of both; detailed instructions for administering required instruments and examples of optional instruments are provided in subsequent sections of this handbook. A master copy of each instrument (and disk copy where appropriate) is provided in the relevant section of the handbook. Evaluators should make a copy of each instrument and maintain the master copy for subsequent years.
Required Activities
At a bare minimum, the evaluator is expected to:
1) Attend semi-annual evaluators' meetings (typically in January and June);
2) Attend semi-annual Industrial Advisory Board meetings;
3) Prepare an "Evaluator's Report" when a new center is born and provide a narrative summary of significant Center developments for submission to NCSU and your Center Director;
4) Administer "process/outcome" questionnaires to faculty, and Industrial Advisory Board Members annually;
5) Prepare an annual report based on the process/outcome questionnaire data for your Center and submit to your Center Director;
6) Forward process outcome questionnaire data to the evaluation team at NCSU;
7) Conduct and summarize exit interviews with representatives of any departing companies;
8) Provide information and feedback to NSF; and
9) Provide information and feedback to your Center Director.
Directors are required to include a copy of the Evaluator' Report (i.e., item 3 above) with the renewal application which is due each spring. A summary of required evaluation outputs is provided as Table 1. A calendar of annual evaluation activities is provided as Table 2.
Optional Activities
Evaluators may collect data in addition to the data required by the evaluation effort. For instance, in the past, questionnaires were used to collect data from Center administrators, industry technical monitors, and students. In addition, evaluators are free to add questions to the required instruments in order to provide data that are of local interest. A list of some of the instruments which have been used in the past or by individual evaluators can be found in Section 6.
Many evaluators find that they are called upon to perform other functions in their Centers. These functions are negotiated individually and will depend on the needs of the center, its director and the evaluator's interest to become more involved in the center's operations. Examples of these activities include assisting in planning semi-annual Industrial Advisory Board meetings, developing marketing plans, preparing or contributing to the newsletter, and providing general management consulting.
1.3 EVALUATOR AS ADVISOR/CONSULTANT
It should be noted that the last required activity, "provide information and feedback to your Center Director", provides an opportunity for the evaluator to become an advisor or consultant to their director. In fact, over time some directors have used their evaluator extensively in this role and value the contribution they make as advisors. However, other directors have chosen to not use their evaluators in this way. While we encourage evaluators to become advisors/consultants to their directors, the evolution of this role depends upon the needs of their center director, the evaluator’s willingness to engage in these activities and the relationship, which develops between the director and the evaluator.
1.4 NCSU EVALUATION TEAM
The NCSU Evaluation Team is responsible for coordination of both the process/outcome questionnaire assessment and the Evaluator's Report. If you have any questions about these elements of the evaluation effort you should contact:
Denis O. Gray, Ph.D.
North Carolina State University
Department of Psychology
Box 7801
Raleigh, NC 27695-7801
Tel: (919) 515-1721
Fax: (919) 515-1716
E-mail: denis_gray@ncsu.edu
For general information about coding problems, deadlines, replacement questionnaires, and data entry please contact:
IUCRC Project Manager
North Carolina State University
Department of Psychology
Box 7801
Raleigh, NC 27695-7801
Tel: (919) 515-3237
Fax: (919) 515-1716
E-mail: iucrc@ncsu.edu
SUMMARY OF REQUIRED EVALUATION OUTPUTS
Instrument/ Procedure |
When Collected |
Date Sources |
Reports |
Other |
| Evaluator's Report | Ongoing including attendance at meetings, etc. | All Center participants; Center documents; etc. | Written at end of planning grant, updated annually* | Full Report to Director for inclusion in IUCRC renewal application; copy to NCSU |
| Process/Outcome Questionnaire | Annually; August-November. Commence data collection after one year of operation (exclude planning grant) | IAB Members; Faculty | Report for Center Director | Submit data to NCSU |
| Exit Interviews | When company leaves the Center | IAB Member, technical monitors | Incorporate into "Evaluator Report"; provide informal feedback to Center Director |
* Your Center Director is required to submit this report to NSF with the Center renewal application.
CALENDAR OF EVALUATOR ACTIVITIES AND DEADLINES
DATE |
TASK (Responsibility) |
| Ongoing | Collect data for completeing the Evaluator's Report (all evaluators) Conduct exit interviews for members that have left the center |
| Aug-Nov. | Collect Process/Outcome Questionnaire Data (all evaluators)
|
| November | Submit Supplemental Research Proposals to Research Committee
|
| December | Submit Process/Outcome Questionnaire Data (all evaluators)
|
January (date announced annually) |
Evaluator Meeting (all evaluators)
Meeting of Center Directors |
| March | Submit Evaluator's Report (all evaluators)
Attend your center's IAB meeting |
| April | Submit Supplemental Research Proposals
|
| June (date announced annually) | Evaluator Meeting (all evaluators)
|