TY - JOUR

T1 - Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy

AU - Jørgensen, Morten

AU - Savran, Mona Meral

AU - Christakopoulos, Christos

AU - Bek, Toke

AU - Grauslund, Jakob

AU - Toft, Peter Bjerre

AU - Ziemssen, Focke

AU - Konge, Lars

AU - Sørensen, Torben Lykke

AU - Subhi, Yousif

PY - 2019

Y1 - 2019

N2 - PURPOSE: Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy.METHODS: The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass-fail score and the consequences of its use).RESULTS: The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass-fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification.CONCLUSION: We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.

AB - PURPOSE: Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy.METHODS: The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass-fail score and the consequences of its use).RESULTS: The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass-fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification.CONCLUSION: We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.

U2 - 10.1111/aos.14065

DO - 10.1111/aos.14065

M3 - Journal article

C2 - 30816642

VL - 97

SP - 700

EP - 706

JO - Acta Ophthalmologica

JF - Acta Ophthalmologica

SN - 1755-375X

IS - 7

ER -