As a facet of its Strategic Plan and prior to the Spellings Commission’s call for greater accountability and transparency of academic measures in higher education, PCOM has embarked on an effort to create and implement College- and program-wide assessment reports through the use of a variety of assessments and outcomes indicators.
DASHBOARD INDICATORS: A BASE LINE TO INFORM INSTITUTIONAL ACCOUNTABILITY
Akin to the dashboard gauges of an automobile that reflect the car’s performance and operational ranges at a glance, dashboard indicators, visual models of progress measured against appropriate benchmarks, help to monitor the collective operations of the College. Such assessment tools provide a multimodal approach to analyzing data and report on a combination of outcomes-based information such as enrollment and retention, student perceptions, student learning, degree completion, board exam scores, pass rates on licensure xaminations, internship acceptance rates, employer surveys and post-graduation achievements. These indicators stress the inherent interrelationships between evaluation and program implementation, determining whether specific academic program initiatives are having the impact intended, whether desired outcomes are being met or whether there are any unexpected developments. Data collected can also form the basis for a summative evaluation conducted at a future date.
“Updated annually, these assessments are excellent qualitative and quantitative indicators of our DO and graduate programs’ effectiveness,” reports Robert G. Cuzzolino, EdD, vice president for graduate programs and planning. “They help to identify strengths and weaknesses in student achievement, and they illustrate how well our students are acquiring competencies in relation to their academic programs. As such, they provide quality assurance for our institution.”
Robert A. DiTomasso, PhD, chair, department of psychology, and chair of the Student Learning Outcomes Committee, was among the first to use dashboard indicators at the College for the psychology graduate programs in 1999 during preparation for American Psychological Association accreditation. “Dashboard indicators determine the impact of our teaching and curriculum strategies at various stages. Our measures are obtained at the time of student program entry, during program learning, and at program completion and beyond. Each of our programs has specific goals and objectives; by measuring ongoing progress, faculty and staff can eliminate the risk of waiting until students have experienced an entire program to assess likely outcomes.” For example, some of the data Dr. DiTomasso has collected has been used to hone the clinical psychology doctoral program. “Among our findings was a recommendation to move the research sequence into the first year of study to enable students to begin their theses projects earlier, thus reducing the time and expense it takes to complete the program.”
VALUE OF ANNUAL ASSESSMENTS
PCOM’s enhanced assessments support appraisal workflow management and accreditation compliance. Outcomes indicators allow senior administrators and program directors to view appraisals and see where specific academic areas are in the review process. Such assessments allow the College to determine top competencies and trends across the institution to provide quality assurance.
COMLEX COMPARISON STUDY
An internal study of COMLEX I scores has resulted in the development of a unique criteria model for identifying the characteristics that differentiate the academic performance of osteopathic medical students who pass the exam versus those who fail on the first attempt. The six-step model outlines a profile of risk factors for COMLEX difficulty. In application, it is an evidence-based approach for the admissions process, assisting in more subjective screenings of candidates. Likewise, it can inform the development of an early identification system and additional support for students.