잠시만 기다려 주세요. 로딩중입니다.

Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions

Bourque Jimmy, Skinner Haley, Dupre Jonathan, Bacchus Maria, Ainslie Martha, Ma Irene W. Y., Cole Gary,
소속 상세정보
 ( Bourque Jimmy ) - Royal College of Physicians and Surgeons of Canada Exam Quality and Analytics Unit
 ( Skinner Haley ) - Royal College of Physicians and Surgeons of Canada Exam Quality and Analytics Unit
 ( Dupre Jonathan ) - Royal College of Physicians and Surgeons of Canada Exam Quality and Analytics Unit
 ( Bacchus Maria ) - University of Calgary Department of Medicine
 ( Ainslie Martha ) - University of Manitoba Department of Medicine
 ( Ma Irene W. Y. ) - University of Calgary Department of Medicine
 ( Cole Gary ) - Royal College of Physicians and Surgeons of Canada Exam Quality and Analytics Unit

Abstract


Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters’ knowledge of correct answers’ impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Ebel scores.

Methods: Data were drawn from a Royal College of Physicians and Surgeons of Canada certification exam. Ebel’s method was applied to 203 MCQs by 49 raters. Facility indices came from 194 candidates. We computed Fleiss’ kappa and the Pearson correlation between Ebel scores and item facility indices. We investigated differences in the Ebel score (correct answers provided or not) and differences between internists and other specialists with t-tests.

Results: Kappa was below 0.15 for facility and relevance. The correlation between Ebel scores and facility indices was low when correct answers were provided and negligible when they were not. The Ebel score was the same, whether the correct answers were provided or not. Inter-rater agreement and Ebel scores was not differentbetween internists and other specialists.

Conclusion: Inter-rater agreement and correlations between item Ebel scores and facility indices wee consistently low; furthermore, raters’ knowledge of correct answer and rater specialty had no effect on Ebel scores in the present setting.

키워드

Canada; Certification; MedicineSpecialization; Standard-setting

원문 및 링크아웃 정보

 

등재저널 정보