Journal of Educational Evaluation for Health Professions

Research article

Comparison of standard-setting methods for the Korean Radiological Technologist Licensing Examination: Angoff, Ebel, bookmark, and Hofstee: Janghee Park, Duck-Sun Ahn, Mi Kyoung Yim, Jaehyoung Lee; J Educ Eval Health Prof. 2018;15:32. Published online December 26, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.32

18,911 View
247 Download
11 Web of Science
8 Crossref

Purpose
This study aimed to compare the possible standard-setting methods for the Korean Radiological Technologist Licensing Examination, which has a fixed cut score, and to suggest the most appropriate method.
Methods
Six radiological technology professors set standards for 250 items on the Korean Radiological Technologist Licensing Examination administered in December 2016 using the Angoff, Ebel, bookmark, and Hofstee methods.
Results
With a maximum percentile score of 100, the cut score for the examination was 71.27 using the Angoff method, 62.2 using the Ebel method, 64.49 using the bookmark method, and 62 using the Hofstee method. Based on the Hofstee method, an acceptable cut score for the examination would be between 52.83 and 70, but the cut score was 71.27 using the Angoff method.
Conclusion
The above results suggest that the best standard-setting method to determine the cut score would be a panel discussion with the modified Angoff or Ebel method, with verification of the rated results by the Hofstee method. Since no standard-setting method has yet been adopted for the Korean Radiological Technologist Licensing Examination, this study will be able to provide practical guidance for introducing a standard-setting process.

Citations

Citations to this article as recorded by

Setting standards for a diagnostic test of aviation English for student pilots
Maria Treadaway, John Read
Language Testing.2024;[Epub] CrossRef
The challenges inherent with anchor-based approaches to the interpretation of important change in clinical outcome assessments
Kathleen W. Wyrwich, Geoffrey R. Norman
Quality of Life Research.2023; 32(5): 1239. CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef
Comparison of the validity of bookmark and Angoff standard setting methods in medical performance tests
Majid Yousefi Afrashteh
BMC Medical Education.2021;[Epub] CrossRef
Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
Journal of Educational Evaluation for Health Professions.2021; 18: 25. CrossRef
Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination
Mi Kyoung Yim, Sujin Shin
Journal of Educational Evaluation for Health Professions.2020; 17: 14. CrossRef
Performance of the Ebel standard-setting method for the spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisting of multiple-choice questions
Jimmy Bourque, Haley Skinner, Jonathan Dupré, Maria Bacchus, Martha Ainslie, Irene W. Y. Ma, Gary Cole
Journal of Educational Evaluation for Health Professions.2020; 17: 12. CrossRef
Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination
Janghee Park, Mi Kyoung Yim, Na Jin Kim, Duck Sun Ahn, Young-Min Kim
Journal of Educational Evaluation for Health Professions.2020; 17: 28. CrossRef

Review Articles

Reconsidering the Cut Score of Korean National Medical Licensing Examination: Duck Sun Ahn, Sowon Ahn; J Educ Eval Health Prof. 2007;4:1. Published online April 28, 2007; DOI: https://doi.org/10.3352/jeehp.2007.4.1

42,019 View
177 Download
4 Crossref

Abstract PDF

After briefly reviewing theories of standard setting we analyzed the problems of the current cut scores. Then, we reported the results of need assessment on the standard setting among medical educators and psychometricians. Analyses of the standard setting methods of developed countries were reported as well. Based on these findings, we suggested the Bookmark and the modified Angoff methods as alternative methods for setting standard. Possible problems and challenges were discussed when these methods were applied to the National Medical Licensing Examination.

Citations

Citations to this article as recorded by

Predicting medical graduates’ clinical performance using national competency examination results in Indonesia
Prattama Santoso Utomo, Amandha Boy Timor Randita, Rilani Riskiyana, Felicia Kurniawan, Irwin Aras, Cholis Abrori, Gandes Retno Rahayu
BMC Medical Education.2022;[Epub] CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef
Applying the Bookmark method to medical education: Standard setting for an aseptic technique station
Monica L. Lypson, Steven M. Downing, Larry D. Gruppen, Rachel Yudkowsky
Medical Teacher.2013; 35(7): 581. CrossRef
Standard Setting in Student Assessment: Is a Defensible Method Yet to Come?
A Barman
Annals of the Academy of Medicine, Singapore.2008; 37(11): 957. CrossRef

A Pychometric Approach to Setting a Passing Score on Korean National Medical Licensing Examination: Guemin Lee; J Educ Eval Health Prof. 2004;1(1):5-14. Published online January 31, 2004; DOI: https://doi.org/10.3352/jeehp.2004.1.1.5

29,901 View
146 Download
5 Crossref

Abstract PDF

National Health Personnel Licensing Examination Board (hereafter NHPLEB) has used 60% correct responses of overall tests and 40% correct responses of each subject area test as a criterion to give physician licenses to satisfactory candidates. The 60%-40% criterion seems reasonable to laypersons without pychometric or measurement knowledge, but it may causes several severe problems on pychometrician's perspective. This paper pointed out several problematic cases that can be encountered by using the 60%-40% criterion, and provided several pychometric alternatives that could overcome these problems. A fairly new approach, named Bookmark standard setting method, was introduced and explained in detail as an example. This paper concluded with five considerations when the NHPLEB decides to adopt a pychometric standard setting approach to set a cutscore for a licensure test like medical licensing examination.

Citations

Citations to this article as recorded by

Analysis on Validity and Academic Competency of Mock Test for Korean Medicine National Licensing Examination Using Item Response Theory
Han Chae, Eunbyul Cho, SeonKyoung Kim, DaHye Choi, Seul Lee
Keimyung Medical Journal.2023; 42(1): 7. CrossRef
Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination
Mi Kyoung Yim, Sujin Shin
Journal of Educational Evaluation for Health Professions.2020; 17: 14. CrossRef
Comparison of results between modified-Angoff and bookmark methods for estimating cut score of the Korean medical licensing examination
Mikyoung Yim
Korean Journal of Medical Education.2018; 30(4): 347. CrossRef
Presidential address: launching the Korea Health Personnel Licensing Examination Institute, a government-supported special foundation from December 23, 2015
Chang Hwi Kim
Journal of Educational Evaluation for Health Professions.2016; 13: 20. CrossRef
Reconsidering the Cut Score of Korean National Medical Licensing Examination
Duck Sun Ahn, Sowon Ahn
Journal of Educational Evaluation for Health Professions.2007; 4: 1. CrossRef

First
Prev
Page of 1
Next
Last

Search