Should medical school admissions be a lottery? Part 2
In a counterargument to my previous blog post discussing Mazer BL. Accepting randomness in medical school admissions: The case for a lottery. Medical Teacher. 2020 Oct 17;1-3; this publication compared students admitted to Maastricht University Medical School via an outcomes-based selection procedure to those students rejected through an outcomes-based procedure and then admitted through a national, grade-point-average based lottery. These two groups were compared on their performance on 7 CanMEDS roles on their clinical rotation assessments.
As you know, medical school admissions committees use several tools to try to determine an applicant’s cognitive and non-cognitive traits to determine who will make the best doctor and match their school’s mission statement. Tools include GPA, MCAT, situational judgement tests such as CASPer and MMI, and in person interviews. Typically cognitive measures have shown to have predictive value for success in the preclinical years and on future testing such as USMLE. Whereas, personal assessments have been predictive of performance in the clinical years of medical school.
This study focused on the predictive value of the Maastricht University Medical School (MUMS) in the Netherlands admissions process for student performance in the 7 CanMEDS roles during clinical rotations including reflective medicine, surgical medicine, mother and child, neurosciences, and family/social medicine.
Maastricht had two distinct admission pathways for the years 2011, 2012 and 2013. One admissions pathway was an outcomes-based selection procedure that screes for the following competencies: knowledge, transfer of knowledge, textual comprehension and verbal/inductive reasoning, communication and strength of arguments, collaboration, organization, social and medical consciousness, ethical awareness, empathy and reflection. They reviewed applicant portfolios consisting of GPA/academic activities, extracurricular activities, open ended questions to assess fit with problem based learning, and open ended questions regarding fit with MUMS. There is a second round focused on competencies that requires a situational justice test and a written aptitude test involving questions managing relevant situations and testing planning skills and fluid intelligence.
The other admissions pathway was a pure lottery based system for students above a certain academic GPA who were initially rejected through the outcomes based admissions procedure.
They found that “students who were admitted via the local outcomes-based selection procedure (SP students) outperformed their initially rejected, lottery-entry counterparts (SN students) on an increasing number of CanMEDS roles and on national progress tests as they advanced through the clinical years of medical school. In other words, the local outcomes-based selection procedure had not only predictive value in the clinical phase of medical school, but also this predictive value increased over time.”
Another point is that all students performed relatively well, independent of which admission group they were in and both improved over time with clinical training - pointing out that most applicants who apply are capable of completing medical studies. The SP students ultimately performed better in Communicating, Collaborating, Organizing, Advocating and Professional behavior.
This paper states its conclusions are an argument for the labor intensive selection procedures over a simple lottery procedure if the goal is to select for doctors who will ultimately excel at the predetermined CanMEDS competencies they are aiming for. Shortcomings of this paper include that it is a single institution using an admissions selection process specific to their institution.
What do you think? Lottery since everyone selected ultimately did meet their clinical competencies OR intensive selection process to start with students who will ultimately perform better on those competencies?
