Real world problem solving skills have been identified as an important part of success beyond the world of academia (Sternberg, 1997). In addition, many are asking whether undergraduate education is really teaching the skills necessary to address real world problems (Hersh & Merrow, 2005). As awareness of the importance of teaching real-world problem solving in higher education increases, problem-solving skills will become a high priority in General Education outcome assessment.
The Problem-Solving Analysis Protocol (P-SAP) is a written problem-solving protocol and rubric for assessing problem solving skills that can be easily integrated into the normal activities of a class. The tool, created by Pamela Steinke, NC State University, and Peggy Fitch, Central College, is based on the work of researchers in service-learning on cognitive outcomes (Eyler & Giles, 1999) and the reflective judgment framework of intellectual development (King & Kitchener, 1994). The problem-solving protocol presents a real-world issue to the student that is directly relevant to the application of material the student is learning in the course, and by simply changing the issue the protocol can be used in a wide variety of classes. Students answer a series of questions about the causes, consequences and solutions for a problem that arises from the issue. The problem-solving protocol can be used in class as a graded assignment or exam question or as a class exercise to start discussion.
The P-SAP allows two different types of coding for assessment purposes. First, whether the protocol is used as a graded assignment or not, faculty in the discipline can score a sample of protocols for students’ comprehension and application of content knowledge. Many departmental program assessment plans include objectives about students’ ability to apply knowledge but faculty members have difficulty identifying how to assess application. Second, an interdisciplinary team of faculty can also code the same protocol for more general problem-solving skills as related to other intellectual skills such as critical thinking, knowledge transfer and perspective taking. This second application can be scored using the P-SAP rubric.
The P-SAP rubric provides scoring criteria for two dimensions (locus/source and complexity) for each of the four questions in the protocol (questions about problem, cause, solution and analysis of solution). The P-SAP has been tested at several institutions in Iowa, Michigan and Illinois and the rubric has demonstrated high inter-rater reliability (.75-.94 range) and good construct validity with intellectual development and cognitive learning measures.
References
1. Eyler, J. & Giles, D.E., Jr. (1999). Where’s the learning in service-learning? San Francisco: Jossey-Bass.
2. Hersh, R.H. & Merrow, J. (2005). Declining by degrees. New York: Palgrave Macmillan.
3. King, P.M. & Kitchener, K.S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass.
4. Steinke, P. & Fitch, P. (2003). Using written protocols to measure service-learning outcomes. In S. H. Billig & J. Eyler (Eds.), Advances in service-learning research, Vol. 3: Research exploring context, participation, and impacts, 171-194. Greenwich, CT: Information Age Publishing.
5. Sternberg, R. J. (1997). Successful Intelligence: How practical and creative intelligence determine success in life. New York: Plume.P-SAP Links
- Rubric for Coding P-SAP
- P-SAP Questions and Issues
- Using Problem-Solving Protocols for Assessment. Pamela Steinke, NC State, and Peggy Fitch, Central College, 2006 IUPUI Assessment Institute. References.
- Assessing Service Learning. Pamela Steinke, NC State, and Peggy Fitch, Central College, (June 2007). Research and Practice in Assessment, Volume 1, Issue 2.