How long should I run a program before I begin to assess it?

• I would highly recommend assessing your program from its inception. Collecting base-line data can be extremely valuable in understanding how your program is contributing to student development and learning. I see no value in delaying assessment for any amount of time.

How do I create time to do what I want to do? 

• As you know, no one can create time, we can only allocate it. Often, time must be re-allocated. I, of course, can't tell you how to do that. In managing the time, it is important to understand why one would engage in assessment. The reasons to engage in assessment include an understanding of how assessment is used. Assessment can
  • reinforce or emphasize the mission of your unit,
  • modify, shape, and improve programs and/or performance (formative),
  • critique a program's quality or value compared to the program's previously defined principles (summative),
  • inform planning,
  • inform decision making,
  • evaluate programs not personnel,
  • assist in the request for additional funds from the university and external community, and
  • assist in meeting accreditation requirements, models of best practices, and national benchmarks.
• If you create, through your actions, an environment where assessment is valued, where decisions based on assessment results hold weight, where appropriate rewards and recognition are provided, where staff has the creativity and support to assess their programs in meaningful and manageable ways, your staff will soon realize how valuable assessment is. Then, their motivation will come from within.
  

How do I include students in my assessment work? 

• Students can be included in every aspect of your assessment plan. They can assist in the articulation of your program outcomes, assessment methods, and assessment timeline. Students can respond to outcomes you provide, reporting the level of meaning, value, or agreement. They can assist with the interpretation and presentation of data and they can assist with recommendations and decisions that need to be made. Students can help increase response rates and provide meaningful prioritization of outcomes.

How do I assess certain number oriented outcomes in such a way that it can lead to making decisions for continuous improvement? 

• You can do so by searching for information that contributes to the making of the numbers. For example, if you want to know why you did NOT make your intended outcome of increasing your Native American enrollment by 3% in the fall of 2001, you can assess those who would know. That means asking high school counselors, parents, students, and enrolled Native American students questions about your institution's perceived support of Native American students. You may want to survey those Native American students who did not enroll and ask them why they did not. Likewise, you will want to interview those students who did enroll and ask theme why they did. Find out if Native Americans are persisting and why they are or why they aren't. You may want to examine your prospect database and interview your admissions counselors about their contact experiences, etc.
• Finding answers to these questions can help you obtain information that will lead you to making decisions for continuous improvement.
  

Can I see a model or example of outstanding outcomes assessment? 

• Yes, check the examples on-line at http://www2.acs.ncsu.edu/UPA/assmt/resource.htm and at http://www.ncsu.edu/undergrad_affairs/assessment/index.htm
• Also, contact the Division of Student Affairs at University of Montana, Syracuse University, and Slippery Rock University.
  

It is still hard for me to formulate in words what we want the students to come away with. Where can I find help? 

• That is understandable. Many people have difficulty articulating the result of what they do. Often times, their vision of the result is complex; thus, putting it into measurable outcomes becomes very challenging. In other cases, they haven't really thought about what they have wanted the result of what they do everyday to be.
• If complexity is the problem, my suggestion is to take the time to discuss your vision of what the result should be with your colleagues. Ask them to write down their understanding and/or ask you questions to help you clarify. In doing so, you will see your vision become clearer and, ultimately, measurable.
• If initial thinking is the problem, then it may be beneficial for you to visit with your supervisor. He or she may be able to direct you to resources in your field where you can learn of some of the many intended outcomes that would result from your daily hard work.
  

What is the real and clear difference between goals and outcomes? 

• Basically, goals are broad general statements of what you want the program or students to be able to do or know. Outcomes are the measurable, observable end result statements of what those goals look like. For example, compare the goal, "Student Conduct Board Members will value critical thinking" to the outcome, "Student Conduct Board members will demonstrate critical thinking skills by articulating a deeper level (as measured by a rubric based on Facione's and Facione's research) of discussion and reflection when considering student misconduct cases". The goal is a general, broad, non-measurable statement of what you want student conduct board members to do. The outcome is expressing the goal in a specific measurable manner. The outcome is answering the question, "how do I know when students will be exhibiting critical thinking?"
  

Regarding narrowing down the goals - how do I really determine what is more important than the other? 

• There are several questions you can ask yourself in order to help prioritize which outcomes you should be assessing first. Your answers to the following examples may help you begin to prioritize your outcomes.
  • What are our problem areas that we are pretty sure need some improvement, yet are unsure of how to make them better?"
  • Which outcomes represent our "hot buttons"?
  • What is my boss most concerned about our improving?
  • What is my staff most concerned about our improving?
  • What are our students most concerned about our improving?
  • What am I most concerned about our improving?
    

How do I examine specific program components for effectiveness? 

• If you were asking about the most appropriate assessment methods for a particular program component, then I would need to understand what you are assessing in order to answer this question. If you are asking in general, then I would recommend reviewing all of the various methods that are available and asking the following questions when choosing a particular method or methods.
  1. What is my budget?
  2. What is my timeline?
  3. What are my analysis capabilities?
  4. Who needs to see this data?
  5. How easily can I fit this method into my annual responsibilities?
  6. Who needs to make decisions with this data?
  7. Will this kind of evidence help me make the decisions I need to make? How will I document the evidence and the decisions made from that evidence?
• Seeing what others are doing is certainly helpful when choosing appropriate assessment methods, tools, and criteria. You can get a lot of information from http://www2.acs.ncsu.edu/UPA/assmt/resource.htm. Remember, most importantly however, is your ability to answer the question, "will the evidence that I expect to collect from this method, tool, and/or criteria lead me to making decisions for continuous improvement in regard to the outcome I am intending to assess"? While many attempts may seem like "trial and error", the more astute you are at articulating the intended result of your program and how you know if you have reached it, the more likely you will choose the appropriate assessment method and thus, the less you will feel you are engaging in a "trial and error" process.

How do I assess the impact of services our unit provides that are not directly delivered to students by our unit, but are delivered in partnership with other units? 

• There is more than one option to consider here. You can either enter into an assessment plan in partnership, where all the units assess the impact of the partnered service. Or you can design, in partnership, an assessment plan in which your area or one of the other areas alone implements; yet you all use the data to make decisions for continuous improvement. The other option is that you approach assessment of this combined service by stating those outcomes, which assess the areas for which you have responsibility to make decisions.

How do you know if you are doing it right? Everyone has different views.

• Everyone does have different views about how "good" assessment should be conducted. And it is because of that, that you will want to keep the following in mind. When done well, assessment will:
  • be meaningful
  • be manageable
  • be flexible
  • be truth-seeking/objective/ethical
  • inform decisions for continuous improvement or provide evidence of proof
  • move you toward instituting a culture of assessment (e.g. evidence-based decision-making).
• You won't "break" assessment and I have yet to hear of someone who broke their program because they engaged in assessment, so don't worry - just dive-in and get started.

I would like to better understand how to get feedback from students on students.

• Students are excellent in assisting with the evaluation of their peers. They can help with creating rubrics and applying them to their peers' artifacts of works.
• Student self-evaluation is also a powerful means of assessment. I know no other better place to model your work after then Alverno College. See http://www.alverno.edu/academics/ddp.html for more information.

Can I get more information on how to use students as a better resource to design and implement assessment tools?

• Since so much of this depends on how you are assessing, it is difficult to come up with a quick answer. We will have more detailed information on this in our upcoming book on assessment tools for student development and learning.

What self-assessment instruments would you recommend for leadership?

• This is a difficult question to answer without knowing how you are defining leadership and what aspects of leadership you are attempting to assess. In all that you do with assessment, it is most important to be able to identify what it is that you are assessing - - what does [insert leadership skill or quality] look like? How will you know that this particular student or this group of students has achieved it? Once you have answered that question, you can look at some self-assessment instruments to see which one best fits that which you are trying to measure.

If assessment doesn't have to be formal, and it can be a matter of noting things that pop up in conversation…what comes first? If my goal is X and students keep talking about Y shouldn't I regroup and focus on Y?

• That may very well be. For example, if your outcome for your residence halls is to have a safe separate living environment for women, but the women keep telling you that they don't want to have a separate living environment from the men; then you may very well want to change your program outcome to reflect their interest. Thus, the new outcome is a safe living coed environment.
• The point of articulating outcomes first is so that you have some direction for your program in regards to planning and budgeting. If you discover in implementing your program, that your program is meeting unintended outcomes - - invited or uninvited - - you have an opportunity to revisit your primary outcomes. Thus, you improve your stated outcomes, and thus improve your program.