To evaluate the utility and correlation of an open-ended evaluation (NeuroCaRE) with clinical core competency grading and NBME scores.

Students are usually graded using the clinical core competencies and a factual knowledge evaluation (NBME). However, evaluations can be limited due to lack of direct observation or limited patient’s experiences.  NBME as a multiple choice question test has the bias of “cueing” and may not test actual practical knowledge. A different testing method is needed to evaluate reasoning (“knows how” of Miller’s pyramid).

The 90-minute Neurology Case Reasoning Exam (NeuroCaRE) includes 4 clinical vignettes with 3 questions each (12 questions total). The questions are “essay-type” (students type their answers and reasoning) and assess knowledge of key features of history/exam, differential diagnosis generation, radiographic or video evidence interpretation and formulation of an assessment and plan. Each question has 2 to 3 score points depending on difficult level. Grading is done using a rubric and the clarity of the explanation written by the student. Questions are different for every clerkship rotation to prevent memorization.

129 third year medical students took the Neurocare exam in 2018. Scores showed low to moderate correlation with the NBME score (Pearson coefficient of 0.5), and clinical core competency score (Pearson coefficient of 0.1). Scores were lower in questions regarding neuromuscular disorders and altered mental status and better in ischemic stroke and headache cases.

NeuroCaRE had low/moderate correlation with the NBME clinical core competencies grading which indicates the constructs being evaluated are different and complement student's assessment. NBME is a factual knowledge tests and the clinical core competencies include teamwork, oral presentations and communication skills with patients and other health providers which are not evaluated using this test. Clinical reasoning for neuromuscular diseases and altered mental status presented was more challenging for students.