MCCQE Part II standard-setting exercise

The Medical Council of Canada Qualifying Examination (MCCQE) Part II held in late October 2018 was based on a new examination Blueprint. Following the first session of the new exam, a full and detailed standard-setting exercise was conducted in December 2018.

The Medical Council of Canada Qualifying Examination (MCCQE) Part II held in late October 2018 was based on a new examination Blueprint.

The new Blueprint was created following extensive practice analysis, from the results of four areas of research, input from subject matter experts, and consultation with external stakeholders. The new Blueprint test specifications incorporate two categories: Physician Activities and Dimensions of Care.

Following the first session of the new exam, a full and detailed standard-setting exercise was conducted over three days in December 2018. This was completed by a panel of 20 physicians from across the country. The purpose of the exercise was to establish the standard for minimal competence for entering independent practice, hence determining the pass score for the October 2018 session and future sessions.

Brief interviews held with multiple panel members towards the conclusion of the standard-setting exercise revealed extensive support for the new Blueprint. They indicated strong support for this exam in terms of establishing a standard for minimal competence and for what this means for candidates, regulators and the public who rely on the MCC’s assessments. Further, they strongly commended the rigour of the standard-setting process itself.

Those interviewed overwhelmingly identified that the new Blueprint is representative of what physicians encounter in everyday practice and assesses the foundational skills expected of a candidate in order to practice independently. The following are some of the representative comments provided by panel members who were interviewed on December 12:

Regarding the new Blueprint:

“I think it’s pretty accurate, in terms of practice, what we encounter in real life. And, well, it’s fair for the residents who are taking the exam.”

“…it is, in my opinion, more tailored to what you will come across as a physician on a day-to-day basis.”

“…more focused on assessing the key requirements graduates need to have… The content reflects more what’s going on now…”

“…it raises the bar on public safety…the public can be reassured…”

“…it really forces the candidate to be more well-rounded, have a more comprehensive approach and really be ready for anything.”

Regarding the standard-setting process:

“I find it very reassuring, as a practicing physician, knowing that the quality, the standard is really very high in terms of establishing a very valid examination process.”

“…there’s a tremendous amount of rigour.”

“…ensures that the standard that is being set is reproducible, it’s believable. It’s the right process…”

“…it’s a very strict process. It’s very fair for the students and it’s very scientific.”

The MCC’s Central Examination Committee (CEC) convened in January to review and approve the pass score for the new exam and to verify the results for the October 2018 cohort. The CEC were provided detailed information that outlined the processes and procedures for the standard-setting exercise, and the impacts to candidates based on the recommended pass score from the standard-setting panel. The pass score was endorsed by the members of the CEC.

The information above helps explain the extensive process that was undertaken to establish the new Blueprint and to set the standard for minimal competence. As a result of the standard-setting exercise, the October 2018 MCCQE Part II results were released on January 14. The new scale now ranges from 50 to 250 with a mean of 150 and a standard deviation of 20. On this new scale, the pass score that was recommended and approved is 138. This pass score will remain in effect until the next standard-setting exercise.