لخّصلي

key takeaways from the reading materials.If you ask students, "What question remains uppermost in your mind as we end this learning experience?" and students' responses are all over the map, you may be able to conclude that they have no major sticking point. But if a number of students raise questions about a point related to a particular learning goal, you have a clue that the curriculum or teaching methods may need to be modified to help students better achieve that learning goal. ? Integrating assessment pieces into an overall picture

• integrate evidence through a humanistic, qualitative approach. 2. If students performed poorly, why? (p 322)

There're possible reasons for disappointing outcomes, including:

• curriculum design,
• teaching methods, learning
• goals,
• support systems,
• and co-curricular. ? Sometimes the design and wording of the assessment itself is the culprit (??????). ? Sometimes a prompt (??????) is unclear; sometimes a multiple-choice question assesses a relatively trivial concept that many students missed;

? And sometimes a rubric is misinterpreted by the faculty using it to assess student work. 3. Additional information may help you understand your evidence (322- 325)

Additional information on students' learning experiences and factors affecting them can help answer questions such as the following:

o Why did students learn X but not Y? o Why did some students master Z but others didn't? o Which courses best prepare students to succeed in the program capstone? o What are the experiences of students most likely to graduate? o What instructional strategies most help students learn to think critically? o Students in which academic programs, if any, have trouble developing quantitative analysis skills? o Which high-impact practices most help students develop information literacy skills? A. Look at learning goals. B. Look at curricular materials. C. Look at the use of research-informed teaching methods and high-impact practices. D. Look at information on student progress, persistence, and success. E. Look at what students bring to the learning experience. F. Ask your students! G. If you can, use learning analytics to connect the pieces. 4. Answering another key question: Stewardship (p 325)

Institutional Effectiveness and Institutional Assessment:

Institutional effectiveness refers to the effectiveness of a college in achieving its mission, goals, and the following responsibilities:

o Meeting stakeholder needs, especially student needs. o Serving the public good. o Stewardship. o Accountability. Institutional assessment gauges institutional effectiveness. Because student learning is the heart of most college missions, the assessment of student learning is a major component of institutional assessment. Information that might be used in concert with evidence of student learning to analyze stewardship includes the following:

o Opportunities, expenditures, and participation rates for faculty and staff professional development on teaching and learning. o Number and dollar value of grants awarded to improve student learning. o Funding for academic programs, support services, and co-curricular experiences. oInstructional facilities, technologies, and other resources. o Class sizes, student/faculty ratios, and/or ratios of students to full-time faculty. ? Documenting and analyzing the effectiveness of your assessment strategies. (p326 - 327)

Whether you need to spend time documenting and analyzing the effectiveness of your assessment tools and processes depends on their setting and how the resulting evidence will be used. Assessments used to inform large-scale, expensive, or life-changing decisions or whose resulting evidence is likely to be challenged need compelling evidence of their effectiveness. Examples include assessments used to inform decisions on who graduates, whether elaborate or expensive changes should be implemented, or whether a program should be terminated. It may not be worthwhile to invest time and resources in documenting and analyzing the effectiveness of assessments used to inform relatively minor decisions, such as changes in teaching strategies or in the curricular design of a single course or co-curricular activity. One way to document the effectiveness of assessment practices is simply to keep records showing that they meet the traits offers other suggestions. Strategies to document and analyze the generalizability, dependability, and consistency of student learning evidence are discussed next. ? Analyzing the generalizability of your evidence (p 327-330)

If you decide that you need to analyze how well your evidence of student learning can be generalized into valid conclusions, consider the following questions:

? Do other assessments corroborate your evidence? Comparing evidence from various assessments against one another is called corroboration or triangulation. ? Does evidence fall in appropriate patterns? - Students at the end of a program should generally do better on an assessment. - Some evidence should predict current or future performance; scores on a pre-calculus tests. - And sometimes students should perform differently by major. ? How well does your sample reflect all your students? If you are assessing sample of student work (Chapter 12), you may want to compare the traits of students in your sample with the traits of the group your sample was drawn from. If 55 percent of all your college's students are women, for example, roughly 55 percent of the students being assessed should be women. ? Analyzing the dependability and consistency of your evidence

? Do faculty and staff assess student work consistently? ? How precise is your estimate from your sample? ? How well does each multiple-choice test item discriminate between high and low scorers? Item discrimination in multiple-choice tests refers to the item's capacity to discriminate between students who have generally learned what the test covers and those who haven't. It makes the following assumptions:

o The purpose of the test is to discriminate between students who have learned the test material well and those who haven't. o Test items that don't help make this discrimination are a waste of testing time. o Students who do well on the overall test should be more likely to answer any one test item correctly than students who do poorly on the overall test. o If students who do well on the overall test are less likely to answer a test item correctly than students who do poorly, the item is not working correctly. Item Analysis

Item analysis is examining the difficulty and discrimination of each multiple-choice test item. It can also include examining the difficulty and discrimination of each option within each item. o Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley & Sons. Part 5 - chapter 25 (page 333- 353) Sharing Evidence of Student Learning ? Student learning evidence is often not shared in ways that promote its use. Evidence can be put to good use only after careful consideration and discussion, and that can happen only if summaries and analyses of evidence are communicated usefully, clearly, and accurately. How you share summaries and analyses of student learning evidence should therefore be planned as carefully as any other part of the assessment process. ? If you're not sure what decisions a particular audience is facing, try to find out before you share evidence with them. ? Don't forget that one of the purposes of reports on assessment plans and processes is to help everyone involved in assessment learn what constitutes good practice. Different people need different information to inform different decisions. Because different audiences want and need different evidence in different formats and in different levels of detail, no one report, database, or website will meet everyone's needs. If you have surveyed graduating seniors, for example, perhaps only co-curricular staff need to see seniors' opinions of co-curricular experiences. Share a story with a clear point Share student learning evidence in ways that tell important, coherent, interesting stories with clear points that plainly connect to the decisions your audiences are making. So, before you share student learning evidence, use your summary and analysis to figure out the big messages you want to convey to each audience. In other words, plan to share not data but information. Data and Information Data are a set of numbers (the singular is datum). Information makes clear the story that the numbers are telling. Focus on sharing what's most important with your most important audiences

Make your efforts worthwhile by identifying your most important audiences and understanding what they most want and need to know. Then share only evidence that each audience can act upon. Don't bother telling your audiences that science majors graduate with stronger math skills than literature majors, for example. But do tell faculty that students at your college spend less time writing than students at peer colleges. If you have some evidence of student learning that no one seems interested in, ask yourself why you collected that evidence in the first place. View sharing evidence as a teaching/learning process

When you share student learning evidence, you are in essence educating your audiences about something important: How well students are learning what you all view as important. Sharing student learning evidence effectively thus means following good teaching practices (List 26.1 in Chapter 26), including the following:

o Understand your audiences and their needs. o Help your audiences see a synthesizing big picture: The big point of your assessment story. o Present your evidence in a variety of ways, as discussed in the next section. o Be balanced and fair, as discussed later in this chapter. o Actively involve your audiences, as discussed at the end of this chapter. Create great visuals and reports

You can share student learning evidence in ways that people ignore, that leave them confused, or that put them to sleep. Or you can share evidence in ways that immediately engage them in discussing and acting on your evidence. Here are suggestions for sharing student learning evidence in ways that will get it discussed and used. Choose the right format for sharing your evidence

Today few people have the time to wade through a lengthy written report, and technologies have made possible a host of alternatives. List 25.1 summarizes some of the many ways to share summaries and analyses of student learning evidence. Formats for Sharing Summaries and Analyses of Student Learning Evidence ? Stacked bar graphs. ? Line graphs ? Pie graphs ? Tables ? Bulleted lists ? Infographics ? Dashboards ? Balanced scorecards ? Other visuals ? Oral presentations ? Written reports

Which format is best? It depends on what your audiences need to see and how they prefer to receive information. Today people increasingly expect and appreciate visual information, but a few may prefer a traditional written report. The best approach may be a combination of modes, such as brief oral remarks accompanied by slides, interactive discussion, and a written executive summary of key evidence. Data Visualization Data visualization is presenting data visually to help people understand its meaning and significance. Keep it short Provide a quick overview of the summaries, analyses, and implications that will be of greatest interest to them - no more than one or two pages, if you're communicating in writing. Pull out just the outcomes that clearly relate to key learning goals of greatest interest to your audience

Keep things simple and clear

? Limit the amount of information ? Use bulleted lists ? Make every visual self-explanatory, ? Avoid jargon. ? Use numbers sparingly, ? Round figures. ? Have a friend review your draft. ? Don't assume a software-generated visual is readable. Make your key points pop (p341)

The best way to make points pop is through well-organized, eye-catching visuals. ? Use engaging, meaningful, self-explanatory titles, headings, and subheadings. ? Use visuals and formatting to draw attention to key points. ? Use traffic-light color coding: ? Cascade from major points to details. ? Sort your evidence in an order that helps convey your point. ? Make it easy for readers to see differences and trends. ? Don't rely on the standard framework for a traditional research report. Make evidence easy to find and access (p 346)

? Decide where and how to make your summaries and analyses available . . . ? Consider embedding summaries and analyses of student learning evidence with other relevant information. Venues for Announcing and Sharing Summaries and Analyses of Student Learning Evidence

o Face-to-face or online presentations at standing or special meetings of relevant groups o Websites o Emails with embedded or attached summaries or hyperlinks to summaries o Social media with hyperlinks to summaries o Newsletters o Alumni magazines o Department memos o Press releases to the student newspaper and/or regional media o Advertisements in the student newspaper o Brochures o Handouts o Posters or banners o Snail-mailed summaries

? Plan multiple communication strategies. ? Should you share your evidence publicly? Be open, honest, balanced, and fair (347-348)

A number of professional organizations engaged in the assessment of human performance have developed statements of ethical standards for assessment. They suggest the following strategies for sharing evidence fairly and Ethically:

o Respect students' privacy and dignity while concurrently giving faculty, administrators, and other decision-makers sufficient evidence to make meaningful decisions. o Give appropriate attribution to the work and ideas of others. o Document dates and sources. Share disappointing outcomes honestly and with context ? Use student learning evidence ethically and appropriately ? Actively involve those who may feel threatened. ? Start with the positives. ? Focus on steps being taken to improve disappointing outcomes. ? Be gentle and sensitive. ? Document the quality of your assessment strategy. ? Acknowledge possible flaws in your assessment strategy. Share your assessment story with accreditors (p350-251) While every accreditor's requirements and expectations are unique, the following suggestions should help you prepare a successful accreditation report. o Understand what your accreditor ???? ?????? is looking for and why. o Focus on four basic questions o View an accreditation report as a cousin of reports to your board o Follow the suggestions in this chapter. Move from sharing to action If your evidence is going to be used, you'll need not only to share it but to make sure, your audiences are actively engaged in discussing it. A face-to-face session is one of the best ways to do this. o Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley & Sons. Part 5 - chapter 26 (page 355- 365) Informing Decisions with Student Learning Evidence Using Evidence of Student Learning to Inform Important Decisions Some valuable ideas you'll find in this chapter: o Hands-on practice is a far more effective way for students to learn than listening to lectures. o Consider giving funding priority to resource requests supported by assessment evidence. o Using a single assessment score as a gatekeeper graduation or progression requirement is an unethical use of evidence of student learning. ? using evidence of student learning to inform those decisions and concludes with guidelines for using evidence ethically and responsibly. Using student learning evidence to recognize and celebrate successes (p 356) Because a fundamental purpose of assessment is betterment- making changes to help students learn and succeed even more effectively - it's easy to focus on problems and not see - let alone celebrate - successful outcomes that need no changes. Recognize and celebrate rather than reward. Recognize and honor assessment efforts. Recognize and honor assessment efforts. Use student learning evidence to give students the best possible education. Disappointing evidence of student learning can lead to decisions to try to improve learning by rethinking curricula, teaching methods, learning goals, support systems, and/or co-curricular. Use student learning evidence to ensure stewardship Evidence of student learning can be used to inform stewardship decisions through resource allocations, program reviews, and, in limited circumstances, performance evaluations. Informing resource allocation decisions (p 360) Two strategies can really help student learning evidence lead to better resource allocation decisions: Give funding priority to resource requests supported by disappointing evidence of student learning. A request for new lab equipment that is based on unsatisfactory student lab skills might receive priority, for example, over a request without such evidence. This can be a powerful strategy to promote a culture of assessment because faculty and staff will quickly learn that student learning evidence literally pays off. Give funding priority to pervasive rather than isolated problems. This can foster collaboration rather than competition over scarce funds. If, for example, there's pervasive evidence that students' information literacy skills are inadequate, funding priority might be given to strategies that will help students, faculty, and staff with this across several programs. Informing program reviews. A program review is a comprehensive evaluation of an academic program that is designed both to foster betterment and to demonstrate stewardship and accountability. Program reviews typically include a self-study conducted by the program's faculty and staff, a visit by one or more external reviewers, and recommendations for changes based on the conclusions of the self-study and the reviewer. Informing performance evaluations Student learning evidence might be used to help inform faculty and staff performance evaluation decisions only under the following circumstances: o Student learning evidence is part of a teaching portfolio documenting teaching effectiveness from a variety of perspectives. o Faculty and staff are evaluated not on student learning evidence per se, but on how they are using that evidence to improve teaching and learning. o Individual faculty and staff are evaluated only on student learning in their own classes and learning experiences, not in broader settings such as an entire program or general education requirement. Changes to assessments don't count Today the most frequent "uses" of student learning evidence are refining the assessment tool or process.Focusing on "intended uses by intended users" (Beld, 2010, p. 6) also leads to a variety of applications of results within departments including redesigns of gateway courses, different approaches to providing feedback on student writing, or even different sequencing of courses. NILOA case study institutions have demonstrated that involving faculty formally, and in detail, in exercises designed to craft a set of specific teaching-related questions that faculty want answered can keep the focus of assessment on utilization. Front-end planning for utilization should also include expectations exercises: What do participants expect the data to reveal, or what might be the action or consequences of this or that result? Leveraging External Processes

Although accreditation has a strong role both in the motivation for assessment and the use of assessment results, external influence on institutions' assessment use is also felt from institutional membership initiatives, national organization or foundation-supported efforts, and state mandates or other accountability initiatives. Even so, results from NILOA's provost survey revealed the most common use of assessment data is related to accreditation--trumping all other uses including informing strategic planning, improving instructional performance, and evaluating units or programs. institutions can further the effective use of assessment results on their campuses by taking advantage of required external initiatives--like accreditation--and reorienting the mandated work around what is meaningful to the campus. The examples also show that the prestige and credibility gained through association with foundations and grants can help advance assessment work, as can the visibility from connections to national projects. Most important, however, these institutional cases exemplify the critical shift from thinking about accreditation as something done solely to satisfy accreditors to using the opportunity accreditation provides to stimulate genuine inquiry toward needed improvement in student learning and institutional performance. Linking Assessment to Internal Processes

Assessment activity and use becomes a regular part of work in the academy when assessment is woven into established structures and processes (see Chapter 4 ). Campus leaders described how some of the practices associated with getting more faculty invested in assessment and making assessment activities more widespread included making it part of standard institutional policies and procedures. Creating organizational structures and positions for assessment within the institution has also made a difference. A director of institutional research at a community college reported that assessment really took hold on campus when an administrative position charged with assessment responsibility--associate dean for student learning--was created (Kinzie, 2010). Another approach to structural adaptation for assessment was employed at the University of Missouri, where the director of the office of assessment has moved efforts forward by working with programs he perceives want support in assessment (Kinzie, 2010). The PRISM system is a consistent, transparent platform for documenting and displaying results and improvement plans. Notably, the featured rotating examples on the PRISM website's homepage in "Spotlights on Program Improvements" illustrate CSU's emphasis on using results to improve.A heartening feature of the case study institutions is that they all saw room for improvement in their assessment efforts (Baker et al., 2012).Support for assessment from faculty and staff has increased with the growing emphasis on authentic and information-rich assessments more aligned with teaching and learning processes including portfolios, scoring rubrics, and surveys of student engagement that provide immediately useful feedback about instructional practice.The rising demand from colleges, universities, and academic departments for help in learning the rudiments of assessment and for opportunities to exchange ideas about assessment practice and resources for continuous improvement prompted the establishment of several formal efforts to advance assessment practice.In the practice of assessment, however, the dominance of accountability language and activities including testing, evaluation, measurement, and comparability can supplant language more frequently associated with student learning such as effort, growth, gain, mastery, and improvement.- The work in these early phases of assessment greatly emphasized a practical and scholarly orientation, made methodological contributions to the emerging scholarship of assessment, and also helped institutions understand student learning.By all accounts, PRISM has intensified and normalized at CSU the use of assessment information in continuous improvement across all levels--in courses, programs, departments, and the university.While many advances in assessment use were in response to external requirements like accreditation, expansion also came and can still come from greater investments in the creation of structures and mechanisms to support and sustain assessment activities.Assessment's true aim is using results, harnessing evidence to inform educational improvements.o Suskie, L. (2018).stewardship, 2.????????2.3.

key takeaways from the reading materials.

• Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley & Sons.
Part 2 – chapter 6 (page 85- 92) Using Evidence of Student Learning
How Will Your Evidence of Student Learning be Used? (page 85)
Assessment has three basic purposes:

1. stewardship,


2. accountability,


3. and, most important, giving students a great education.
   collecting evidence of student learning but the evidence is never really used in any substantive way. Why? Invariably,

• faculty and staff plunged into assessment without thinking first about why they were conducting the assessment (other than to get through the next accreditation review),

• who needed the resulting evidence, and what decisions the evidence would inform.

• The principle of form following function applies here. Once you understand why you are assessing and the decisions that the resulting evidence of student learning is likely to inform, you can design assessment processes to yield evidence that will be truly useful and used rather than sitting on that shelf. (page 86)

Why are you assessing?
Assessment has three fundamental purposes:

1. 
   

   Ensuring and improving educational quality, (chapter 6)

   
   


2. 
   

   stewardship, using evidence of student learning to ensure and improve
   educational quality and for stewardship. (chapter 26)

   
   


3. 
   

   and accountability. using evidence of student learning for accountability (chapter 25)

   
   

o Making sure students get the best possible education. This is – or should be – the fundamental concern of faculty, staff, and college leaders.

o Stewardship. College leaders, board members, and government policy makersm are especially interested in using college resources prudently, effectively, and judiciously.

o Accountability. Board members and external audiences such as accreditors and government policy makers want to be assured of the effectiveness of your college, program, or other learning experience – in other words, that students are getting
o the best possible education with appropriate stewardship of resources. (page 334). Chapter 25

Assessment’s fundamental purpose: Giving students the best
Possible education

Under this traditional or teaching-centered paradigm,

 many faculty viewed their responsibility as providing instruction rather than fostering student learning.

 faculty taught primarily by presenting lectures and assigning readings and homework.

 They believed that, if students didn’t learn the material and earned poor grades, the fault was with the students, not themselves. Such faculty felt no sense of responsibility to reach out to students proactively and help them learn.

Under learning-centered paradigm, (p 86-87)
 Learning-centered (Weimer, 2013) classrooms are those in which students are actively engaged in their learning, and where faculty and students share responsibility for learning. A learning centered college actively fosters these practices.

 learning-centered faculty and staff assume that, if a significant number of their students do not achieve their learning goals, their curricula and teaching methods share at least some of the responsibility.

 under the teaching-centered paradigm

was to give students feedback and grades.

 Under the learning-centered paradigm,

another purpose of assessment is to give feedback to faculty and staff on what is and isn’t working and decide what changes are warranted, if any, to help students learn and succeed even more effectively.

Does assessment really improve learning?

 In a word, no. “Assessment all by itself is an insufficient condition for powerful learning and improvement”.

 It is not the assessment itself, but how faculty, staff, and college leaders use it that can lead to improvement in student learning.

 Why this paucity (ندرة) of systematic evidence that assessment leads to better learning?

1. 
   

   One reason is that disappointing outcomes may be due to a variety of factors, including
    poorly articulated learning goals,
    a curriculum not designed to help students achieve those goals,
    ineffective teaching methods,
    and inadequate or inappropriate support services, among others.
    The specific causes of and solutions for disappointing outcomes are rarely obvious. So it may take some time and trial and error before faculty and staff can identify the best changes to make, implement those changes, and see the impact of those changes.

   
   


2. 
   

   Another reason for the paucity of systematic evidence that assessment leads to better learning is that assessment is action research not experimental research. Action research is context-specific, informal, and designed only to inform individual practice. It does not have the precision, rigor, or generalizability of experimental research. These defining characteristics make it difficult to substantiate the impact of assessment on learning improvement.

   
   

Assessment’s second purpose: Stewardship (p, 88)

Stewardship is the prudent, effective, and judicious care and use of resources entrusted by others.
 critical stewardship responsibility is using good-quality evidence of student learning to help inform resource deployment decisions, including how everyone spends their time.

 Using assessment for stewardship is especially important for programs that:

 may be perceived by some stakeholders as having relatively low impact. Such programs may:
 include co-curricular experiences,
 student support services,
 programs that serve relatively small numbers of students,
 and programs whose goals appear disconnected from college priorities as stated in its mission, strategic goals, and institutional learning goals.
 assessment can be critical to determining whether the value of these kinds of
programs is worth the college’s investment in them.

Assessment’s third purpose: Accountability (page 89)
Accountability is assuring your stakeholders of the effectiveness of your college, program, service, or initiative.

 Accountability includes sharing both successes and steps being taken to make appropriate, evidence-informed improvements.

• What decisions will your evidence inform? Who will make them? (page 89)

Meaningful assessment thus requires a clear understanding of the following:

• Who wants or needs to see this evidence of student learning?

• Why do they want or need to see this evidence?

• What decisions are they making that this evidence should inform?

• How will this evidence help them make a better decision – one in which they
will have more confidence?
While in theory almost anyone might find almost any evidence of student
learning of interest, there are just a few groups most likely to use your evidence to
inform significant decisions.

1. Faculty and staff.


2. College leaders and board members.


3. Government policy makers.


4. Employers.


5. Prospective students and their families.


6. Alumni, foundations, and other donors.


7. Assessment committees and coordinators.

• Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley & Sons.
Part 5 – chapter 24 (page 319- 331) Analyzing Evidence of Student Learning
The most important question you’ll want to answer as you analyze evidence is: (p 320)

1. 
   

   How well your students have learned what you want them to learn.

   
   


2. 
   

   To understand why they have or haven’t achieved your learning goals,

   
   


3. 
   

   You may need to fold additional information into your analysis.

   
   


4. 
   

   You may also want to analyze evidence to answer questions about stewardship.

   
   


5. 
   

   And you may want to evaluate the quality of your assessment strategies to see if they’re yielding.

   
   


6. 
   

   Answering the most important question: How well did students learn what you wanted them to learn? (p 320-322)

   
   

 Analyzing evidence from objective tests, rubrics, and rating scales.

• There are two basic ways to analyze these kinds of evidence.

1. 
   

   One is simply to compare them against one another to identify relative strengths and weaknesses.

   
   


2. 
   

   The other (and better) approach is to compare evidence for each rubric trait, test or rating scale item, or sub-score against targets you’ve established and justify for them.

   
   

 Keep in mind, however, that a statistically significant difference may not be large enough to have any real practical, meaningful significance.

 Analyzing evidence from qualitative assessments.

 If you ask students, “What was the most important thing you learned in this learning experience?” compare their answers against the learning goals of the experience.

 If students’ answers match the learning goals, you’ve probably delivered the curriculum with appropriate balance.
But if a number of students mention ideas that you consider relatively trivial (تافه) and relatively few mention your biggest themes, you have a clue that the learning experience may not have effectively helped students achieve key learning goals.

 If you ask students, “What question remains uppermost in your mind as we end this learning experience?” and students’ responses are all over the map, you may be able to conclude that they have no major sticking point. But if a number of students raise questions about a point related to a particular learning goal, you have a clue that the curriculum or teaching methods may need to be modified to help students better achieve that learning goal.

 Integrating assessment pieces into an overall picture

• integrate evidence through a humanistic, qualitative approach.

2. If students performed poorly, why? (p 322)

There’re possible reasons for disappointing outcomes, including:

• curriculum design,


• teaching methods, learning


• goals,


• support systems,


• and co-curricular.
   Sometimes the design and wording of the assessment itself is the culprit (الجاني).

 Sometimes a prompt (استدعى) is unclear; sometimes a multiple-choice question assesses a relatively trivial concept that many students missed;

 And sometimes a rubric is misinterpreted by the faculty using it to assess
student work.
3. Additional information may help you understand your evidence (322- 325)

Additional information on students’ learning experiences and factors affecting them can help answer questions such as the following:

• Why did students learn X but not Y?

• Why did some students master Z but others didn’t?

• Which courses best prepare students to succeed in the program capstone?

• What are the experiences of students most likely to graduate?

• What instructional strategies most help students learn to think critically?

• Students in which academic programs, if any, have trouble developing
quantitative analysis skills?

• Which high-impact practices most help students develop information literacy skills?

A. Look at learning goals.
B. Look at curricular materials.
C. Look at the use of research-informed teaching methods and high-impact practices.
D. Look at information on student progress, persistence, and success.
E. Look at what students bring to the learning experience.
F. Ask your students!
G. If you can, use learning analytics to connect the pieces.

4. Answering another key question: Stewardship (p 325)

Institutional Effectiveness and Institutional Assessment:

Institutional effectiveness refers to the effectiveness of a college in achieving its mission, goals, and the following responsibilities:

• Meeting stakeholder needs, especially student needs.

• Serving the public good.

• Stewardship.

• Accountability.

Institutional assessment gauges institutional effectiveness.

Because student learning is the heart of most college missions, the assessment of student learning is a major component of institutional assessment.
Information that might be used in concert with evidence of student learning to
analyze stewardship includes the following:

• Opportunities, expenditures, and participation rates for faculty and staff
professional development on teaching and learning.

• Number and dollar value of grants awarded to improve student learning.

• Funding for academic programs, support services, and co-curricular experiences.

•Instructional facilities, technologies, and other resources.

• Class sizes, student/faculty ratios, and/or ratios of students to full-time faculty.

 Documenting and analyzing the effectiveness of your
assessment strategies. (p326 – 327)

Whether you need to spend time documenting and analyzing the effectiveness
of your assessment tools and processes depends on their setting and how the resulting evidence will be used.

Assessments used to inform large-scale, expensive, or life-changing decisions or
whose resulting evidence is likely to be challenged need compelling evidence of their effectiveness.

Examples include assessments used to inform decisions on who graduates, whether elaborate or expensive changes should be implemented, or whether a program should be terminated. It may not be worthwhile to invest time and resources in documenting and analyzing the effectiveness of assessments used to inform relatively minor decisions, such as changes in teaching strategies or in the curricular design of a single course or co-curricular activity.

One way to document the effectiveness of assessment practices is simply to
keep records showing that they meet the traits offers other suggestions. Strategies to document and analyze the generalizability, dependability, and consistency of student learning evidence are discussed next.

 Analyzing the generalizability of your evidence (p 327-330)

If you decide that you need to analyze how well your evidence of student learning can be generalized into valid conclusions, consider the following questions:

 Do other assessments corroborate your evidence?
Comparing evidence from various assessments against one another is called corroboration or triangulation.

 Does evidence fall in appropriate patterns?

• Students at the end of a program should generally do better on an assessment.


• Some evidence should predict current or future performance; scores on
  a pre-calculus tests.


• And sometimes students should perform differently by major.

 How well does your sample reflect all your students?

If you are assessing sample of student work (Chapter 12), you may want to compare the traits of students in your sample with the traits of the group your sample was drawn from. If 55 percent of all your college’s students are women, for example, roughly 55 percent of the students being assessed should be women.

 Analyzing the dependability and consistency of your evidence

 Do faculty and staff assess student work consistently?

 How precise is your estimate from your sample?

 How well does each multiple-choice test item discriminate between high and low scorers?
Item discrimination in multiple-choice tests refers to the item’s capacity to discriminate between students who have generally learned what the test covers and those who haven’t. It makes the following assumptions:

o The purpose of the test is to discriminate between students who have learned the test material well and those who haven’t.

o Test items that don’t help make this discrimination are a waste of testing time.

o Students who do well on the overall test should be more likely to answer any one test item correctly than students who do poorly on the overall test.

o If students who do well on the overall test are less likely to answer a test item correctly than students who do poorly, the item is not working correctly.

Item Analysis

Item analysis is examining the difficulty and discrimination of each multiple-choice test item. It can also include examining the difficulty and discrimination of each option within each item.

• Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley & Sons.
Part 5 – chapter 25 (page 333- 353) Sharing Evidence of Student Learning
 Student learning evidence is often not shared in ways that promote its use. Evidence can be put to good use only after careful consideration and discussion, and that can happen only if summaries and analyses of evidence are communicated usefully, clearly, and accurately. How you share summaries and analyses of student learning evidence should therefore be planned as carefully as any other part of the assessment process.

 If you’re not sure what decisions a particular audience is facing, try to find out before you share evidence with them.

 Don’t forget that one of the purposes of reports on assessment plans and processes is to help everyone involved in assessment learn what constitutes good practice.

Different people need different information to inform different decisions.
Because different audiences want and need different evidence in different
formats and in different levels of detail, no one report, database, or website will meet everyone’s needs. If you have surveyed graduating seniors, for example, perhaps only co-curricular staff need to see seniors’ opinions of co-curricular experiences.

Share a story with a clear point
Share student learning evidence in ways that tell important, coherent, interesting
stories with clear points that plainly connect to the decisions your audiences are
making.
So, before you share student learning evidence, use your summary and analysis to figure out the big messages you want to convey to each audience. In other words, plan to share not data but information.

Data and Information
Data are a set of numbers (the singular is datum).
Information makes clear the story that the numbers are telling.
Focus on sharing what’s most important with your most important audiences

Make your efforts worthwhile by identifying your most important audiences and understanding what they most want and need to know. Then share only evidence that each audience can act upon. Don’t bother telling your audiences that science majors graduate with stronger math skills than literature majors, for example. But do tell faculty that students at your college spend less time writing than students at peer colleges. If you have some evidence of student learning that no one seems interested in, ask yourself why you collected that evidence in the first place.

View sharing evidence as a teaching/learning process

When you share student learning evidence, you are in essence educating your
audiences about something important: How well students are learning what you all
view as important.
Sharing student learning evidence effectively thus means following good teaching practices (List 26.1 in Chapter 26), including the following:

• Understand your audiences and their needs.
• Help your audiences see a synthesizing big picture: The big point of your
assessment story.
• Present your evidence in a variety of ways, as discussed in the next section.
• Be balanced and fair, as discussed later in this chapter.
• Actively involve your audiences, as discussed at the end of this chapter.

Create great visuals and reports

You can share student learning evidence in ways that people ignore, that leave
them confused, or that put them to sleep. Or you can share evidence in ways
that immediately engage them in discussing and acting on your evidence. Here
are suggestions for sharing student learning evidence in ways that will get it
discussed and used.

Choose the right format for sharing your evidence

Today few people have the time to wade through a lengthy written report, and
technologies have made possible a host of alternatives. List 25.1 summarizes some
of the many ways to share summaries and analyses of student learning evidence.

Formats for Sharing Summaries and Analyses of Student Learning Evidence
 Stacked bar graphs.
 Line graphs
 Pie graphs
 Tables
 Bulleted lists
 Infographics
 Dashboards
 Balanced scorecards
 Other visuals
 Oral presentations
 Written reports

Which format is best? It depends on what your audiences need to see and how
they prefer to receive information. Today people increasingly expect and appreciate visual information, but a few may prefer a traditional written report. The best approach may be a combination of modes, such as
brief oral remarks accompanied by slides, interactive discussion, and a written executive summary of key evidence.

Data Visualization
Data visualization is presenting data visually to help people understand its meaning and significance.

Keep it short
Provide a quick overview of the summaries, analyses, and implications that
will be of greatest interest to them – no more than one or two pages, if you’re
communicating in writing. Pull out just the outcomes that clearly relate to key
learning goals of greatest interest to your audience

Keep things simple and clear

 Limit the amount of information
 Use bulleted lists
 Make every visual self-explanatory,
 Avoid jargon.
 Use numbers sparingly,
 Round figures.
 Have a friend review your draft.
 Don’t assume a software-generated visual is readable.
Make your key points pop (p341)

The best way to make points pop is through well-organized, eye-catching visuals.

 Use engaging, meaningful, self-explanatory titles, headings, and subheadings.
 Use visuals and formatting to draw attention to key points.
 Use traffic-light color coding:
 Cascade from major points to details.
 Sort your evidence in an order that helps convey your point.
 Make it easy for readers to see differences and trends.
 Don’t rely on the standard framework for a traditional research report.

Make evidence easy to find and access (p 346)

 Decide where and how to make your summaries and analyses available . . .

 Consider embedding summaries and analyses of student learning evidence with other relevant information.

Venues for Announcing and Sharing Summaries and Analyses of Student Learning Evidence

• Face-to-face or online presentations at standing or special meetings of relevant groups
• Websites
• Emails with embedded or attached summaries or hyperlinks to summaries
• Social media with hyperlinks to summaries
• Newsletters
• Alumni magazines
• Department memos
• Press releases to the student newspaper and/or regional media
• Advertisements in the student newspaper
• Brochures
• Handouts
• Posters or banners
• Snail-mailed summaries

 Plan multiple communication strategies.

 Should you share your evidence publicly?

Be open, honest, balanced, and fair (347-348)

A number of professional organizations engaged in the assessment of human performance have developed statements of ethical standards for assessment.

They suggest the following strategies for sharing evidence fairly and
Ethically:

o Respect students’ privacy and dignity while concurrently giving faculty, administrators, and other decision-makers sufficient evidence to make meaningful decisions.

o Give appropriate attribution to the work and ideas of others.

o Document dates and sources.
Share disappointing outcomes honestly and with context
 Use student learning evidence ethically and appropriately
 Actively involve those who may feel threatened.
 Start with the positives.
 Focus on steps being taken to improve disappointing outcomes.
 Be gentle and sensitive.
 Document the quality of your assessment strategy.
 Acknowledge possible flaws in your assessment strategy.

Share your assessment story with accreditors (p350-251)
While every accreditor’s requirements and expectations are unique, the following suggestions should help you prepare a successful accreditation report.
• Understand what your accreditor جهات متعددة is looking for and why.
• Focus on four basic questions
• View an accreditation report as a cousin of reports to your board
• Follow the suggestions in this chapter.
Move from sharing to action
If your evidence is going to be used, you’ll need not only to share it but to make
sure, your audiences are actively engaged in discussing it.
A face-to-face session is one of the best ways to do this.

• Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley & Sons.
Part 5 – chapter 26 (page 355- 365) Informing Decisions with Student Learning Evidence
Using Evidence of Student Learning to Inform Important Decisions
Some valuable ideas you’ll find in this chapter:
o Hands-on practice is a far more effective way for students to learn than listening to lectures.
o Consider giving funding priority to resource requests supported by assessment evidence.
o Using a single assessment score as a gatekeeper graduation or progression requirement is an unethical use of evidence of student learning.

 using evidence of student learning to inform those decisions and concludes with guidelines for using evidence ethically and responsibly.
Using student learning evidence to recognize and celebrate successes (p 356)
Because a fundamental purpose of assessment is betterment– making changes
to help students learn and succeed even more effectively – it’s easy to focus on
problems and not see – let alone celebrate – successful outcomes that need no
changes.
Recognize and celebrate rather than reward.
Recognize and honor assessment efforts.
Recognize and honor assessment efforts.
Use student learning evidence to give students the best possible education.
Disappointing evidence of student learning can lead to decisions to try to improve learning by rethinking curricula, teaching methods, learning goals, support systems, and/or co-curricular.

Use student learning evidence to ensure stewardship
Evidence of student learning can be used to inform stewardship decisions through resource allocations, program reviews, and, in limited circumstances, performance evaluations.
Informing resource allocation decisions (p 360)
Two strategies can really help student learning evidence lead to better resource
allocation decisions:
Give funding priority to resource requests supported by disappointing evidence of student learning.
A request for new lab equipment that is based on unsatisfactory student lab skills might receive priority, for example, over a request without such evidence. This can be a powerful strategy to promote a culture of assessment because faculty and staff will quickly learn that student learning
evidence literally pays off.
Give funding priority to pervasive rather than isolated problems.
This can foster collaboration rather than competition over scarce funds. If, for
example, there’s pervasive evidence that students’ information literacy skills are
inadequate, funding priority might be given to strategies that will help students,
faculty, and staff with this across several programs.

Informing program reviews.
A program review is a comprehensive evaluation of an academic program that is designed both to foster betterment and to demonstrate stewardship and accountability. Program reviews typically include a self-study conducted by the program’s faculty and staff, a visit by one or more external reviewers, and recommendations for changes based on the conclusions of the self-study and the reviewer.
Informing performance evaluations
Student learning evidence might be used to help inform faculty and staff performance evaluation decisions only under the following circumstances:
• Student learning evidence is part of a teaching portfolio documenting teaching effectiveness from a variety of perspectives.
• Faculty and staff are evaluated not on student learning evidence per se, but on
how they are using that evidence to improve teaching and learning.
• Individual faculty and staff are evaluated only on student learning in their
own classes and learning experiences, not in broader settings such as an entire
program or general education requirement.
Changes to assessments don’t count
Today the most frequent “uses” of student learning evidence are refining the
assessment tool or process. Rubrics are tweaked for clarity; survey administration
procedures are revised to achieve a better response rate; multiple-choice test
questions are rewritten. This makes sense the first time a new, untested assessment
is used. People in higher education are very good at research, and good research
often includes refining the methodology after a pilot study. If your analysis of
student learning evidence identifies anything that isn’t working appropriately, of course you should revise your assessments before using them again.

Use student learning evidence for broad impact
Using evidence of student learning to identify and
address broader issues is not only prudent but a requirement of some accreditors
and of the Excellence in Assessment designation of the National Institute for
Learning Outcomes Assessment.
Meaningful advancements in quality and effectiveness require people to knit together assessment pieces in order to identify broad, pervasive issues (Chapter 24) and to support plans to address those issues with resources.

Use student learning evidence fairly, ethically, and responsibly
• Make assessments planned and purposeful.
• Actively involve those with a stake in decisions.
• Focus assessments on important learning goals.
• Communicate evidence of student learning widely and transparently.
• Don’t brush disappointing evidence under the carpet.
• Don’t let student learning evidence dictate decisions.
• Don’t base a major decision only on the outcome of a single assessment.
• Don’t hold people accountable for things they can’t do.
• Don’t use student learning evidence punitively.
• Don’t penalize students for our mistakes.
• Discourage others from making inappropriate interpretations.
• Keep faculty, students, and staff informed.

Kuh, George D., Stanley O. Ikenberry, Natasha A. Jankowski, Timothy Reese Cain, Peter T. Ewell, Pat Hutchings, and Jillian Kinzie.
Using evidence of student learning to improve higher education. John Wiley & Sons, 2015. Page 51-91.
This chapter describes what productive use of assessment results looks
like by drawing on a range of examples from diverse campus settings.
Toward this end, the chapter is organized around the following questions:

1. What is the difference between doing assessment and using assessment results?


2. What approaches have been successfully employed on campuses
   that have achieved greater use of assessment results?


3. What principles might guide the fi eld in furthering the use of
   assessment results to improve student learning?
   Based on our attempt to answer these questions, we distill principles
   for guiding effective assessment use that point to ways to enhance student
   learning. To put our analysis in context, we briefly discuss the history of
   assessment practice and its lessons for productive use.

A Brief History of Assessment Practice

• 
  

  Assessment practice has a longer history, beginning in the 1930s as a practical and scholarly inquiry project to measure and understand cognitive gains.

  
  


• 
  

  full 40 years later, work shifted to study the net effects of college and student persistence, and at the same time, some campuses, including Alverno College, placed greater emphasis on mastery learning and on assessing and certifying individual student accomplishments.

  
  


• 
  

  The work in these early phases of assessment greatly emphasized a practical and scholarly orientation, made methodological contributions to the emerging scholarship of assessment, and also helped institutions understand student learning.

  
  


• 
  

  in the 1980s, the purpose of assessment in higher education expanded in response to pressures for accountability.

  
  


• 
  

  assessment practice focused narrowly on the adoption of standardized tests to address concerns about educational quality expressed in high‐profi le public documents.

  
  


• 
  

  Barbara Wright (1997) asserted that assessment could serve both purposes and, most important, that assessment is a process that converts data into information that is used to improve student learning. In the practice of assessment, however, the dominance of accountability language and activities including testing, evaluation, measurement, and comparability can supplant language more frequently associated with student learning such as effort, growth, gain, mastery, and improvement.

  
  


• 
  

  According to Peter Ewell (2009), since the early days of the assessment movement, assessment’s two purposes have not rested comfortably together. Alongside the growth in assessment activity and the emphasis on accountability and using evidence to improve emerged a natural curiosity about how assessment is accomplished—in particular, a desire for concrete examples about good practice. The rising demand from colleges, universities, and academic departments for help in learning the rudiments of assessment and for opportunities to exchange ideas about assessment practice and resources for continuous improvement prompted the establishment of several formal efforts to advance assessment practice.

  
  


• 
  

  As this brief history of assessment practice suggests, the growth of institutional assessment practice corresponded with a shift of emphasis in assessment’s purpose: a desire on the part of some to move away from compliance‐driven data collection and toward the development of systematic ways to use evidence to improve the quality of undergraduate education. Engaging in assessment has become a condition of doing business for colleges and universities because of accreditation requirements and the need to show results (Ewell, 2009). Support for assessment from faculty and staff has increased with the growing emphasis on authentic and information‐rich assessments more aligned with teaching and learning processes including portfolios, scoring rubrics, and surveys of student engagement that provide immediately useful feedback about instructional practice.

  
  

Doing Assessment Versus Using Results
 assessment’s purpose is to answer questions, shape better policies, make better decisions— all designed to improve student success and strengthen institutional performance. Doing assessment, simply performing assessment activities, is not the same as using assessment results. Considerable assessment activity can occur at a college or university—administering standardized tests to all students, documenting pass rates on licensure exams, writing reports about the results, for example. Until the institution uses the assembled evidence to answer questions about educational quality—about what students know and can do—and then uses the answers to guide change leading to improvement, it is just doing assessment. Assessment’s true aim is using results, harnessing evidence to inform educational improvements. Linda Suskie (2004) put it plainly: “If an assessment doesn’t help improve teaching and learning activities, why bother with it?”

The Assessment cycle (56-58)

The familiar visual depiction of the assessment process as an iterative
sequence, represented as a six‐stage cycle in Figure 3.2, illustrates the difference
between assessment activity and use.
The first two stages of the assessment cycle—planning and identifying
outcomes and gathering data—are usually associated with assessment
activity; they are followed by the stages associated with use: analyzing
and sharing results, identifying and implementing changes, and assessing
the impact of changes; finally, in the model’s most important feature, the
cycle begins again. The first two stages of the cycle are generally considered
easy, while analyzing and sharing results are harder, demanding careful
interpretation and connections to units on campus—often requiring
diplomacy, particularly when data do not comport with dominant beliefs.
The final stages—acting on evidence, that is, implementing changes and
assessing their impact—are the hardest in the cycle. All of the cycle’s use
stages require significant oversight and effort to complete.

• 
  

  Gathering data is far less risky and complicated than acting on the evidence in the data.

  
  


• 
  

  The final stage of any assessment and improvement plan involves taking action on results and then going back to assess whether or not this action was effective.

  
  


• 
  

  Institutions of higher education find it difficult to convert data into action an even more challenging to measure the impact of improvement initiatives.

  
  


• 
  

  Furthermore, while most assessment activity is associated with some sort of campus improvement or change, this is usually related to improving processes, such as better teaching methods, enhanced advising approaches, better faculty development programs, or more resources for a program, and usually not with direct evidence of improvements to student learning (Banta et al., 2009).

  
  


• 
  

  Equal attention must be afforded to the difficult task of assessing impact on student learning.

  
  


• 
  

  Another stumbling block to using assessment results is getting caught up in creating the “perfect” research design.

  
  


• 
  

  Assessment must work with the best design possible— given what is practical, feasible, and most likely to produce reasonable results.

  
  


• 
  

  The most feasible design must also answer a specific question of interest to campus leaders about educational quality or student learning.

  
  


• 
  

  In assessment, as Barbara Walvoord (2010) observed, “You are not
  trying to achieve the perfect research design; you are trying to gather enough data to provide a reasonable basis for action” (p. 5). Notably, enough data is not about quantity—although one way to strengthen the usability of assessment results is to line up multiple points of evidence.

  
  

Rather, data are enough when they provide evidence that is reasonably sufficient—perhaps good enough—to answer a specific question.

Assessment must, first and foremost, provide useful information to inform improvements and address questions of interest around student learning.
Assessment is not about the technical adequacy of the design. Effective assessment requires persistent emphasis on utility and, as noted in Chapters 1 and 2, abiding concern for consequential validity—that is, for the impacts and consequences of the activity, the evidence it generates, and fitness of the evidence for the intended use. This means shifting the emphasis from collecting data to taking action and carrying out the commitment to measure the impact of actions taken.

Good Practice: Examples of Effective Use

Distinct Levels of Use

In this section, we discuss two distinct levels
of use:
institution and program/course. For the most part,
course‐level uses are closely aligned with program‐level uses. Additional distinctions in course‐level uses are highlighted in Chapters 2 , 5 , and 9 of this volume.

Institutional examples illustrating effective use of student learning
evidence for institution‐level improvement include strengthening connections
between assessment and institutional goals by setting institutional
priorities and strategic planning, informing institutional decision making,
incorporating results into accreditation efforts, framing assessment
at the institution level, revising institutional outcomes, improving student
engagement and success, creating a culture of teaching and learning,
enhancing faculty collaboration across campus, and reflecting on assessment
processes and institutional practices.

Program‐level uses in all NILOA institutional examples focus on specifi
c problems or questions regarding student learning, emphasizing what
is of most interest to faculty members. As noted earlier, focusing on the
interests and questions of faculty members helps achieve faculty engagement
with and ownership of assessment efforts, as well as leading more
directly to assessment use to improve academic programs and courses.

Focusing on faculty interest in problems of teaching and learning and emphasizing the connection to centers for teaching excellence increase the likelihood that assessment results will lead to improvements in student learning.

Distinguishing the differences in assessment use between these two levels
and the specific activities and approaches aligned with effective use is
important as we strive to understand and improve campus assessment
practice. Clarifying the intended level of use for assessment at the outset
might help connect assessment activity more directly to use.

Begin with Use in Mind (61-63)

Institutions that effectively use assessment results focus sharply from the beginning of any assessment initiative on how results will be used. The emphasis on utilization is strong in the related field of educational evaluation and is formally expressed in Michael Patton’s (2008) utilization‐focused evaluation (U‐FE) model.

The backward‐design approach, taken from classroom instruction, begins with the identification of the intended learning outcomes. Once these are determined, instruction and practice that support them as well as evidence that demonstrates them are identified with the purpose of facilitating students’ movement toward the outcomes. The combination of these two models—the utilization focused/backward‐design model of assessment— begins with the question, “Who are the likely users of the evidence we want to gather, and how will they use it?” This approach is thought to increase the likelihood that faculty and staff will be able to act on the evidence of student learning they collect. Focusing on “intended uses by intended users” (Beld, 2010, p. 6) also leads to a variety of applications of results within departments including redesigns of gateway courses, different approaches to providing feedback on student writing, or even different sequencing of courses.

NILOA case study institutions have demonstrated that involving faculty formally, and in detail, in exercises designed to craft a set of specific teaching‐related questions that faculty want answered can keep the focus of assessment on utilization. Front‐end planning for utilization should also include expectations exercises: What do participants expect the data to reveal, or what might be the action or consequences of this or that result?
Leveraging External Processes

Although accreditation has a strong role both in the motivation for assessment
and the use of assessment results, external influence on institutions’
assessment use is also felt from institutional membership initiatives,
national organization or foundation‐supported efforts, and state mandates
or other accountability initiatives. Even so, results from NILOA’s
provost survey revealed the most common use of assessment data is
related to accreditation—trumping all other uses including informing
strategic planning, improving instructional performance, and evaluating
units or programs.

institutions can further the effective use of assessment results on their campuses by taking advantage of required external initiatives—like accreditation—and reorienting the mandated work around what is meaningful to the campus. The examples also show that the prestige and credibility gained through association with foundations and grants can help advance assessment work, as can the visibility from connections to national projects. Most important, however, these institutional cases exemplify the critical shift from thinking about accreditation as something done solely to satisfy accreditors to using the opportunity accreditation provides to stimulate genuine inquiry toward needed improvement in student learning and
institutional performance.

Linking Assessment to Internal Processes

Assessment activity and use becomes a regular part of work in the academy when assessment is woven into established structures and processes (see Chapter 4 ). Campus leaders described how some of the practices associated with getting more faculty invested in assessment and making assessment activities more widespread
included making it part of standard institutional policies and procedures.

Creating organizational structures and positions for assessment within
the institution has also made a difference. A director of institutional
research at a community college reported that assessment really took
hold on campus when an administrative position charged with assessment
responsibility—associate dean for student learning—was created
(Kinzie, 2010).

Another approach to structural adaptation for assessment was
employed at the University of Missouri, where the director of the office
of assessment has moved efforts forward by working with programs he
perceives want support in assessment (Kinzie, 2010).

The PRISM system is a consistent, transparent platform for documenting and displaying results and improvement plans. Notably, the featured rotating examples on the PRISM website’s homepage in “Spotlights on Program Improvements”
illustrate CSU’s emphasis on using results to improve. By all accounts,
PRISM has intensified and normalized at CSU the use of assessment
information in continuous improvement across all levels—in courses,
programs, departments, and the university.

The online system has fostered a shared understanding of assessment and learning by making it possible for all faculty and staff members to view the plans of other units and to explore best practices in assessment across divisions. Simply having
access to past plans and other unit reports has accelerated collaboration
on the creation of rubrics and interview protocols and on the use of data in planning and improvement initiatives. Access to institution‐wide results and information has prompted greater use of data in planning and reporting.

Another important link between meaningful assessment practice and
internal structures at institutions has been evident in the work fostered
in centers for teaching and learning, a development we return to in
Chapter 9 . To bring faculty work in assessment into the spotlight, and
to strengthen the connection between assessment and real improvements
in teaching and learning, many institutions have found welcome connections
between assessment and the scholarship of teaching and learning,
or the work of centers for teaching and learning.

Another important link between meaningful assessment practice and
internal structures at institutions has been evident in the work fostered
in centers for teaching and learning, a development we return to in
Chapter 9 . To bring faculty work in assessment into the spotlight, and
to strengthen the connection between assessment and real improvements
in teaching and learning, many institutions have found welcome connections
between assessment and the scholarship of teaching and learning,
or the work of centers for teaching and learning. A structure. While many advances in assessment use were in response to external requirements like accreditation, expansion also came and can still come from greater investments in the creation of structures and mechanisms to support and sustain assessment activities.

The Quest to Close the Assessment Loop

Completing the assessment cycle—that is, measuring the impact of the action taken to improve student learning and, the ultimate stage, gaining evidence of improved student learning—is assessment’s nirvana, perhaps.
As Trudy Banta and Charles Blaich (2011) report, few institutions have achieved that goal: realizing actual change in student learning after using assessment results to bring about change. However distant that goal may seem, assessment practice seems to demonstrate that a necessary condition of effective assessment use is the quest to close the assessment loop.

A heartening feature of the case study institutions is that they all saw room for improvement in their assessment efforts (Baker et al., 2012). In particular, they acknowledged that assessment is an ongoing process in which closing the assessment loop begins the assessment process anew. Commenting on the continual nature of the process of assessing student learning, leaders at case study sites indicated they did not see their work with assessment as complete but, rather, as continuing to evolve. No one was completely satisfied with their current use of assessment results and all wanted to learn more about their students in order to improve student learning. Recognition of the evolving cycle of assessment and interest in improving student learning at these institutions helped stimulate continuous effort for improvement.

Taking time to reflect on assessment results, documenting what changes
were made, and most important, examining whether the implemented
changes have been successful, are all vital steps. Case study institutions
were committed to the level of reflection required for this and valued the
time spent and the chance to make shared meaning of the results.