Session
A Comparative Analysis of Curriculum Reform Impact Using Cognitive Diagnostic Assessment
Background: Integration of basic science education with clinical application during the pre-clerkship years enhances knowledge mastery and long-term retention in clinical settings (1–5). In 2017, our institution implemented a redesigned, competency-based curriculum known as Rocket Medicine, which emphasizes seamless integration of foundational and clinical sciences. The transition provided a unique opportunity to evaluate student content mastery by comparing performance under both curricula. The proficiency in core clinical skills—such as ‘Diagnosis’, ‘Management’, and ‘Prevention’, as well as in ‘Foundation’—were analyzed before and after the implementation of Rocket Medicine.
Methods: We have shown that a Cognitive Diagnostic Assessment (CDA) method, known as the Deterministic Input, Noisy “And” Gate (DINA) model (6), can be used to analyze students’ content mastery based on their performance on NBME subject exams (7). In this study, percentage content mastery for the clinical and foundational domains was calculated across the core clerkships from both the Legacy and Rocket Medicine curricula (AY 2017–2025). This was done by developing a question-by-content Q-matrix, constructed using the corresponding item analysis reports provided by the NBME, as previously described (7). One-way ANOVA and 2-sample t-tests were used to determine significance (P<0.05).
Results:
We observed significant improvements in content mastery in all four tested skills by students in the Rocket Medicine curriculum compared to those in the Legacy curriculum. Specifically, Rocket Medicine students demonstrated markedly higher proficiency in clinical skills like Diagnosis, Management and Prevention across the core clerkships (Table 1, column 3). The Diagnosis skill mastery was significantly higher in the Internal Medicine, Ob/Gyn, Surgery, Psychiatry and Neurology clerkships following the implementation of Rocket Medicine. Rocket Medicine students from the Pediatrics, Ob/Gyn, and Psychiatry clerkships exhibited significantly high Management skill mastery compared to the Legacy students. Similarly, the Ob/Gyn, Surgery and Neurology clerkship students from the Rocket Medicine showed significantly higher mastery in Prevention compared to their Legacy counterparts. The “Foundation” skill mastery was higher in the Rocket Medicine students mostly, but was statistically significant only in the Ob/Gyn clerkship. These gains were consistent across successive cohorts, including the most recent one (column 4, Table 1), suggesting a sustained impact of the curriculum.
Discussion:
One of the major challenges in medical education is effectively balancing knowledge acquisition with competency development. Our findings indicate that integrating clinical and foundational sciences during the pre-clerkship phase significantly enhances students’ retention of foundational knowledge and mastery of clinical competencies such as diagnosis and management. The use of CDA offered a robust, quantitative method to evaluate and compare student mastery across clerkships, serving as a novel tool for assessing curricular modifications. This approach will allow curricular leaders to pinpoint specific content areas of strength and those requiring improvement, thereby supporting data-driven, continuous quality improvement in the medical curriculum.
Limitations:
While the present study offers valuable insights into broad trends in content mastery and retention, it does not account for variations at the level of individual organ systems or disciplines. Further in-depth analysis is necessary to zero in on specific curricular areas requiring focused revision.
Table:
Clerkship
Skills Legacy vs Rocket Medicine ; 2023-2024
(% content mastery), * p< 0.05
Internal Medicine
Diagnosis- 81 vs 94%* ; 92%
Management-81 vs 88%; 92%
Prevention-69 vs 75%;74.4%
Foundation-84 vs 87%;85.6%
Family Medicine
Diagnosis- 84 vs 84%; 75%
Management- 83 vs 85%; 68%
Prevention- 78 vs 83%; 73%
Foundation- 78 vs 74%; 78%
Pediatrics
Diagnosis-89 vs 94%; 95.3%
Management-83 vs 92%*; 93.6%
Prevention-74 vs 81%;92.5%
Foundation-84 vs 90.4%;
Ob/Gyn
Diagnosis-81 vs 93%*; 95.4%
Management-79 vs 92%*; 94.7%
Prevention-79 vs 91%*; 90.4%
Foundation-80 vs 92%*; 92.9%
Surgery
Diagnosis- 82 vs 96%*; 94.7%
Management-82 vs 88%; 94.3%
Prevention-71 vs 88%*; 74.6%
Foundation-78 vs 83%; 80.7%
Psychiatry
Diagnosis-87 vs 94%*; 95.2%
Management-87 vs 94%*; 95.2%
Prevention-66 vs 63%; 95.2%
Foundation-85 vs 84%; 72%
Neurology
Diagnosis-88 vs 94%*; 92.5%
Management-87 vs 91%; 80.9%
Prevention-68 vs 81%*; 81%
Foundation-88 vs 88%; 75.9%
Table legend: Percent content mastery in foundational and clinical skills across various clerkships. Column 2 lists the assessed skill (knowledge) domains; Column 3 compares content mastery percentages between the Legacy and Rocket Medicine curricula; Column 4 presents data from the most recent academic year.

Bindu Menon
Associate Professor, Department of Medical Education, The University of Toledo College of Medicine and Life Sciences
Toledo, Ohio, United States
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top