Journal of Educational Evaluation for Health Professions

Research article

Proposal for setting a passing score for the Korean Nursing Licensing Examination: Janghee Park, Mi Kyoung Yim, Sujin Shin, Rhayun Song, Jun-Ah Song, Inyoung Lee, Heejeong Kim, Minjae Lee; J Educ Eval Health Prof. 2025;22:25. Published online September 8, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.25

1,433 View
215 Download

Abstract PDF Supplementary Material: Purpose
The Korean Nursing Licensing Examination (KNLE) is planning to transition to a computer-based test (CBT). This study aims to propose a reasonable and efficient method for setting passing scores.
Methods
A standard-setting (passing score setting) analysis was conducted using an expert panel over the past 3 years of the national nursing examination. The standard-setting method was modified from Angoff, and the validity of the passing score was verified through the Hofstee method. The standard-setting workshop was conducted in 2 stages: first, a pilot workshop for 2 subjects, followed by a second workshop where 6 additional subjects were selected based on the pilot results. For items with an actual correct answer rate of 90% or higher, the estimated correct answer rate for minimum competency was calculated using the observed correct answer rate. A survey and discussion with the expert panel were also conducted regarding the standard-setting procedures and results.
Results
The passing score for the national nursing examination was calculated using the new method, and the score was slightly higher than the existing score. The nursing subject had similar results; however, the legal subjects varied.
Conclusion
The modified Angoff and Hofstee methods were successfully applied to the KNLE. Using the actual correct answer rate as an indicator to derive expected minimum competency was shown to be effective. This approach could streamline future standard-setting processes, particularly when converting to CBT.

Technical report

Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study: Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee; J Educ Eval Health Prof. 2022;19:19. Published online August 22, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.19

5,493 View
251 Download
3 Web of Science
3 Crossref

Purpose
This study aimed to develop the examination objectives based on nursing competency of the Korean Nursing Licensing Examination.
Methods
This is a validity study to develop the examination objectives based on nursing competency. Data were collected in December 2021. We reviewed the literature related to changing nurse roles and on the learning objectives for the Korea Medical Licensing Examination and other health personnel licensing examinations. Thereafter, we created a draft of the nursing problems list for examination objectives based on the literature review, and the content validity was evaluated by experts. A final draft of the examination objectives is presented and discussed.
Results
A total of 4 domains, 12 classes, and 85 nursing problems for the Korean Nursing Liscensing Examination were developed. They included the essentials of objectives, related factors, evaluation goals, related activity statements, related clients, related settings, and specific outcomes.
Conclusion
This study developed a draft of the examination objectives based on clinical competency that were related to the clinical situations of nurses and comprised appropriate test items for the licensing examination. Above results may be able to provide fundamental data for item development that reflects future nursing practices.

Citations

Citations to this article as recorded by

A validity study of COMLEX-USA Level 3 with the new test design
Xia Mao, John R. Boulet, Jeanne M. Sandella, Michael F. Oliverio, Larissa Smith
Journal of Osteopathic Medicine.2024; 124(6): 257. CrossRef
A Survey on Perceptions of the Direction of Korean Medicine Education and National Licensing Examination
Han-Byul Cho, Won-Suk Sung, Jiseong Hong, Yeonseok Kang, Eun-Jung Kim
Healthcare.2023; 11(12): 1685. CrossRef
Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study
Kyunghee Kim, So Young Kang, Younhee Kang, Youngran Kweon, Hyunjung Kim, Youngshin Song, Juyeon Cho, Mi-Young Choi, Hyun Su Lee
Journal of Educational Evaluation for Health Professions.2023; 20: 18. CrossRef

Research article

Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination: Mi Kyoung Yim, Sujin Shin; J Educ Eval Health Prof. 2020;17:14. Published online April 22, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.14

11,291 View
223 Download
15 Web of Science
12 Crossref

Abstract PDF Supplementary Material

Purpose
This study explored the possibility of using the Angoff method, in which panel experts determine the cut score of an exam, for the Korean Nursing Licensing Examination (KNLE). Two mock exams for the KNLE were analyzed. The Angoff standard setting procedure was conducted and the results were analyzed. We also aimed to examine the procedural validity of applying the Angoff method in this context.
Methods
For both mock exams, we set a pass-fail cut score using the Angoff method. The standard setting panel consisted of 16 nursing professors. After the Angoff procedure, the procedural validity of establishing the standard was evaluated by investigating the responses of the standard setters.
Results
The descriptions of the minimally competent person for the KNLE were presented at the levels of general and subject performance. The cut scores of first and second mock exams were 74.4 and 76.8, respectively. These were higher than the traditional cut score (60% of the total score of the KNLE). The panel survey showed very positive responses, with scores higher than 4 out of 5 points on a Likert scale.
Conclusion
The scores calculated for both mock tests were similar, and were much higher than the existing cut scores. In the second simulation, the standard deviation of the Angoff rating was lower than in the first simulation. According to the survey results, procedural validity was acceptable, as shown by a high level of confidence. The results show that determining cut scores by an expert panel is an applicable method.

Citations

Citations to this article as recorded by

The Nursing Student Licensure Examination: A Scoping Review
Flavia Pantaleo, Alessandro Stievano, Chiara Mastroianni, Giorgia Petrucci, Natascia Mazzitelli, Michela Piredda, Maria Grazia De Marinis, Anna Marchetti
Nursing Reports.2025; 15(8): 299. CrossRef
Optimal cutoff score for the Malay version of parenting and child tooth brushing assessment (M-PACTA)
Yu Jie Chin, Jennifer Geraldine Doss, Sabri Musa, Danaee Mahmoud, Shani Ann Mani
BMC Oral Health.2025;[Epub] CrossRef
Angoff methods in standard setting in health professional education: a systematic review and meta-analysis
Kannan Sridharan, Gowri Sivaramakrishnan
BMC Medical Education.2025;[Epub] CrossRef
Comparing the passing rates of nursing students participating in the objective structured clinical examination using fixed score, Angoff, yes/no Angoff, and 3-level Angoff methods†
Fatemeh Maghsoodi, Mohammadreza Yazdankhahfard, Shahnaz Pouladi, Kamran Mirzaei, Amin Beigzadeh
Frontiers of Nursing.2025; 12(4): 511. CrossRef
Experts’ prediction of item difficulty of multiple-choice questions in the Ethiopian Undergraduate Medicine Licensure Examination
Shewatatek Gedamu Wonde, Tefera Tadesse, Belay Moges, Stefan K. Schauber
BMC Medical Education.2024;[Epub] CrossRef
Standard setting methods in objective structured clinical examination (OSCE): A comparative study of five methods
Reshma Ansari, Norhafizah Ab Manan, Nur Ain Mahat, Norfaizatul Shalida Omar, Atikah Abdul Latiff, Sara Idris, Azli Shahril Othman
Journal of Medical Education Development.2024; 17(56): 87. CrossRef
Comparing Estimated and Real Item Difficulty Using Multi-Facet Rasch Analysis
Ayfer SAYIN, Sebahat GÖREN
Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi.2023; 14(4): 440. CrossRef
Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study
Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee
Journal of Educational Evaluation for Health Professions.2022; 19: 19. CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef
Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
Journal of Educational Evaluation for Health Professions.2021; 18: 25. CrossRef

Research Article

Validation of a clinical critical thinking skills test in nursing: Sujin Shin, Dukyoo Jung, Sungeun Kim; J Educ Eval Health Prof. 2015;12:1. Published online January 27, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.1

37,448 View
308 Download
6 Web of Science
11 Crossref

Abstract PDF

Purpose
The purpose of this study was to develop a revised version of the clinical critical thinking skills test (CCTS) and to subsequently validate its performance. Methods: This study is a secondary analysis of the CCTS. Data were obtained from a convenience sample of 284 college students in June 2011. Thirty items were analyzed using item response theory and test reliability was assessed. Test-retest reliability was measured using the results of 20 nursing college and graduate school students in July 2013. The content validity of the revised items was analyzed by calculating the degree of agreement between instrument developer intention in item development and the judgments of six experts. To analyze response process validity, qualitative data related to the response processes of nine nursing college students obtained through cognitive interviews were analyzed. Results: Out of initial 30 items, 11 items were excluded after the analysis of difficulty and discrimination parameter. When the 19 items of the revised version of the CCTS were analyzed, levels of item difficulty were found to be relatively low and levels of discrimination were found to be appropriate or high. The degree of agreement between item developer intention and expert judgments equaled or exceeded 50%. Conclusion: From above results, evidence of the response process validity was demonstrated, indicating that subjects respondeds as intended by the test developer. The revised 19-item CCTS was found to have sufficient reliability and validity and will therefore represents a more convenient measurement of critical thinking ability.

Citations

Citations to this article as recorded by

Clinical Judgment Model‐Based Critical Reflection Program for Newly Graduated Nurses: A Nonrandomized Controlled Trial
Ae Ran Kim, Jeong Hee Hong, Kyeongsug Kim, Yuna Kim, Jung Min Lee, Heejin Lee, Ji Hyun Yoon, Mi Soon Kim
Research in Nursing & Health.2025; 48(2): 234. CrossRef
Utilizing cognitive interview in the item refinement of the Blended Teaching Assessment Tool (BTAT) for Health Professions Education
Maria Teresita B. Dalusong, Glenda Sanggalang Ogerio, Valentin C. Dones, Maria Elizabeth M. Grageda
Philippine Journal of Health Research and Development.2025; 29(2): 54. CrossRef
Meta-analysis in Physical Therapy Education Research
Mitch Wolden, Brent Hill, Sara Farquhar Voorhees
Journal of Physical Therapy Education.2019; 33(1): 78. CrossRef
Effects of a work-based critical reflection program for novice nurses
Yeon Hee Kim, Ja Min, Soon Hee Kim, Sujin Shin
BMC Medical Education.2018;[Epub] CrossRef
Measurement of critical thinking, clinical reasoning, and clinical judgment in culturally diverse nursing students – A literature review
Christine L. Sommers
Nurse Education in Practice.2018; 30: 91. CrossRef
Individual and School Factors Affecting Critical Thinking Ability among Nursing Students
Sujin Shin, Inhee Park, Eunhee Hwang, Dukyoo Jung, Kon Hee Kim
Korean Medical Education Review.2018; 20(1): 44. CrossRef
The Health Professions Education Pathway: Preparing Students, Residents, and Fellows to Become Future Educators
H. Carrie Chen, Maria A. Wamsley, Amin Azzam, Katherine Julian, David M. Irby, Patricia S. O'Sullivan
Teaching and Learning in Medicine.2017; 29(2): 216. CrossRef
Cultivating Critical Thinking Using Virtual Interactive Case Studies
Susan M. Burke
Journal of Pediatric Nursing.2017; 33: 94. CrossRef
Encouraging Critical Clinical Thinking (CCT) Skills in First-Year Veterinary Students
Duncan C. Ferguson, Leslie Klis McNeil, David J. Schaeffe, Eric M. Mills
Journal of Veterinary Medical Education.2017; 44(3): 531. CrossRef
Developing a Foundation for Interprofessional Education Within Nursing and Medical Curricula
Trisha Leann Horsley, Trent Reed, Keith Muccino, Donna Quinones, Viva Jo Siddall, Janet McCarthy
Nurse Educator.2016; 41(5): 234. CrossRef
Supervision in psychiatry
Joanna MacDonald, Pete M. Ellis
Current Opinion in Psychiatry.2012; 25(4): 322. CrossRef

First
Prev
Page of 1
Next
Last

Search