Accrediting agencies, legislators, parents, funders and employers of college students and graduates are demanding that colleges provide more and better evidence that they are fulfilling their educational missions. Higher education costs continue to escalate and people want to know that their investments are producing results. RMU and other accredited colleges and universities are constantly gathering data in order to measure how well they are serving students and improving their operations. The umbrella term for these activities is "outcomes assessment".
What is Outcomes Assessment?
Outcomes assessment describes the measurement activities that colleges conduct to determine what and how well their students are learning and how well student learning is being supported. Outcomes assessments differ from course grades in their scope. They are broader measures than individual grades, often covering learning across multiple courses or an entire academic program. Outcomes are also sought from activities outside the classroom as well as from alumni and external parties such as employers of RMU's graduates. The comprehensiveness of a college's outcomes assessment program, the findings from those assessments and its ability to prove that it uses those results to improve its services determine whether a particular school has an effective continuous improvement program. An effective outcomes assessment program provides actionable data and documented evidence that the college is fulfilling its mission.
Robert Morris University is presently accredited by the Middle States Commission on Higher Education. The Commission's outcomes assessment requirements are contained in Standards 7 and 14 of its publication, Characteristics of Excellence in Higher Education: Eligibility Requirements and Standards for Accreditation.
Outcomes Assessments at RMU
RMU's outcomes assessment program has several major components:
- The University's 2007-2012 five year strategic plan contains numerous quantifiable measures of success. The degree of attainment of these measures will be directly indicative of RMU's effectiveness at carrying out its educational mission.
- Pursuit of program accreditations for specific academic schools or majors has been an important objective at RMU over the past decade. This highly successful effort has resulted in the attainment of seven discipline-specific accreditations (Actuarial Science, Business, Computer Science, Education, Engineering, Nuclear Medicine and Nursing). Two-thirds of all RMU undergraduates are now enrolled in accredited academic programs.
- The most direct evidence of student learning available can be seen in academic programs where students are required to pass comprehensive licensing examinations in order to work in their chosen career. Currently two RMU majors have such a requirement prior to graduation (Education and Nursing) and several other majors (Accounting, Actuarial Science and Engineering) require licensing examinations soon afterwards.
- The University's contract with its faculty union provides incentives for faculty members to engage in course-level assessments of learning outcomes.
- A wide variety of other assessment measures are employed at RMU. These measure areas such as student satisfaction with RMU's facilities and services, student engagement, employer satisfaction with RMU graduates, job placement rates, and adherence to published industry standards.
- RMU has provided training for its faculty and staff utilizing several nationally-known experts in outcomes assessment.
The outcomes assessment effort at RMU is administered in a decentralized way with each academic school and student support area given responsibility for implementation of assessments that are pertinent to each unit. These efforts are coordinated and directed by senior officers of the University.
Assessment data is helpful only if it is evaluated and then used to make necessary improvements. Several examples are provided of how assessment is used as part of a continuous improvement process at RMU. The University also has a well-established, formal process to document its assessments and the improvements made on the basis of assessment data.
Assessments and Measures - Strategic Plan
Specialty Accreditations
RMU has attained discipline-specific accreditations for many of its academic programs to accompany its University-wide accreditation from the Middle States Commission on Higher Education. These program-specific accreditations include very specific and stringent requirements that RMU measure student learning in the accredited discipline. The possession of the accreditation itself is validation that the accredited academic program has passed in-depth, third-party scrutiny of its faculty, curriculum and outcomes.
Specialty Accreditations and Their Assessments
Licensing Examinations
The applied nature of many of RMU's majors lends itself well to assessment. The graduates of several RMU undergraduate majors (Education and Nursing) literally cannot work in their chosen fields unless they pass national comprehensive examinations given at RMU during their senior years. These examinations are written by recognized experts in the field and a passing grade provides evidence that the test-taker has a body of knowledge sufficient to perform in their chosen career.
Graduates in other majors such as Accounting, Actuarial Science, and Engineering must pass licensing examinations but they do so after they have left RMU so tracking their pass rates is not possible. In these cases RMU must rely upon feedback from employers and alumni in order to determine how successful the University is in preparing graduates for the examinations for those careers.
Assessment Results - Professional Licensing Examinations
NCLEX-RN (Nursing)
PRAXIS (Education)
Other Outcomes
RMU and other colleges also attempt to measure other types of outcomes, including student success in the general education portion of their studies (communications skills, social science and humanities courses taken by all undergraduates regardless of major), student satisfaction with RMU's facilities and services, students' degree of participation and engagement with in- and out-of-class learning and service activities, and alumni/employer satisfaction with the quality of education received at RMU.
General indicators of student success are also measured, including the one-year retention rate and six-year graduation rate of each entering freshman class, graduates' job placement rates, and other indicators.
Assessment Results - Other Outcomes
Alumni surveys (AICUP, others)
Compliance with Established Standards (Student Affairs, Others)
Employer Surveys
Financial and Other Audits
Job Placement Rates
National Survey of Student Engagement
Noel-Levitz Student Satisfaction Inventory
One-Year and Six-Year Retention Rates
Outcomes of RMU's General Education Courses
US News Ranking
Examples of how RMU uses assessment results to improve teaching and learning
The effort expended on outcomes assessment does no good if the results are not evaluated and used to decide if improvements need to be made to some aspect of RMU's operations. Every individual assessment administered at RMU is undertaken with that mandate and the responsible manager is charged with insuring that it is both carried out and documented. While the scope of RMU's assessment effort precludes a complete list of these documented improvements on this website, two specific examples will show how a fully implemented assessment loop works.
Nursing - NCLEX-RN Results Improvement Program
Education - PRAXIS Results Improvement Program
Where does RMU need to improve in outcomes assessment?
No college has an outcomes assessment effort that is either perfect or complete. There are always ways in which the comprehensiveness of an assessment effort, use of assessment results for improvement, or the administration and governance of the assessment effort can be improved. RMU is working hard to try to improve the three following aspects of its outcomes assessment effort:
- General Oversight - Outcomes assessment at RMU is a decentralized effort that is administered through each academic school and administrative department. The University is currently reviewing alternatives for governance of its outcomes assessment process.
- Assessment of General Education - General Education courses are often referred to as "Core" courses in the academic world. These are the courses in English, History, Humanities, etc. that every college student takes to assure that they have a well-rounded education.
RMU, like many other colleges, has had difficulty assessing student achievement in its general education curriculum for both practical and organizational reasons. RMU's Core curriculum does have stated learning outcome goals. However, the 13 Core courses are spread across six different academic departments. Since every RMU undergraduate must take these courses, there are hundreds of sections taught and dozens of faculty members teaching them each semester. RMU's Core curriculum does not exist as a separate administrative unit with dedicated management; rather, it is a conceptual entity run as a cooperative joint venture. Changes to the Core Curriculum are thus difficult to implement.
Another problem confronting RMU and other colleges is that desired outcomes of a general education program are often difficult to both articulate and measure. What does it mean to be "well-rounded" or "globally aware" or other similar-cited desirable outcomes for general education? And even if you know what these mean, how do you measure them, particularly when students take only one class in a specific discipline as part of their Core education?
RMU continues to work on addressing this issue and it is far from unique in this regard.
- Non-Accredited Disciplines - Pursuit of program accreditations and the satisfaction of licensing requirements have been the main means for establishing an effective outcomes assessment system at RMU. However, not every major or academic program is covered by such accreditations or requirements. Non-accredited programs in general have not been as proactive in installing assessment procedures as those programs who are subject to specialty accreditation requirements. RMU continues to address this issue as an important part of its strategic plan.
Conclusion
Outcomes assessment provides RMU with evidence of how well it is carrying out all the activities that support the University's Mission. RMU has made great progress in using the findings from these measurement activities for institutional improvement. The University continues to aggressively seek other opportunities to measure its outcomes as well as to improve the processes that it employs to evaluate what it has learned from its assessment program.