Journal of Educational Evaluation for Health Professions

Research articles

Validity evaluation of a mock examination including multimedia-assisted items for the Korean Nursing Licensing Examination: a mixed-methods study with a counterbalanced design: Sujin Shin, Inyoung Lee, Heejeong Kim, Rhayun Song, Jun-Ah Song, Hyun Sook Yi, Janghee Park, Mi Kyoung Yim, Minjae Lee, Subin Yu; J Educ Eval Health Prof. 2026;23:21. Published online July 21, 2026; DOI: https://doi.org/10.3352/jeehp.2026.23.21 [Epub ahead of print]

630 View
106 Download

Abstract PDF: Purpose
This study aimed to evaluate the validity of multimedia-assisted items (MAIs) in a mock examination by examining their psychometric properties and nursing students’ perceptions.
Methods
This methodological study used a within-group counterbalanced design in which fourth-year nursing students in South Korea completed an online mock examination consisting of paired text-based items and MAIs. Psychometric properties were examined using classical test theory and item response theory, and learner perceptions and response processes were explored through post-examination surveys and focus group interviews.
Results
Among 515 participants, MAIs demonstrated overall difficulty and discrimination comparable to those of text-based items in both classical test theory and item response theory analyses, with similar test characteristic curves across counterbalanced groups, supporting measurement equivalence across item formats. Post-test surveys indicated that participants perceived MAIs as realistic reflections of clinical practice and as more helpful for problem solving and ability assessment than text-based items. Qualitative findings from focus group interviews (n=15) further supported the educational value of MAIs for assessing clinical judgment while emphasizing the need for appropriate multimedia use. An order effect was also observed: presenting MAIs first led to higher item response theory discrimination parameters in subsequent text-based items (P=0.020).
Conclusion
MAIs demonstrated psychometric properties comparable to those of text-based items and were positively perceived as valid tools for assessing clinical judgment and practical competence. These findings support the feasibility of incorporating multimedia-based assessment into the Korean Nursing Licensing Examination as part of a rigorous computer-based testing framework.

Proposal for setting a passing score for the Korean Nursing Licensing Examination: Janghee Park, Mi Kyoung Yim, Sujin Shin, Rhayun Song, Jun-Ah Song, Inyoung Lee, Heejeong Kim, Minjae Lee; J Educ Eval Health Prof. 2025;22:25. Published online September 8, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.25

2,830 View
246 Download

Abstract PDF Supplementary Material: Purpose
The Korean Nursing Licensing Examination (KNLE) is planning to transition to a computer-based test (CBT). This study aims to propose a reasonable and efficient method for setting passing scores.
Methods
A standard-setting (passing score setting) analysis was conducted using an expert panel over the past 3 years of the national nursing examination. The standard-setting method was modified from Angoff, and the validity of the passing score was verified through the Hofstee method. The standard-setting workshop was conducted in 2 stages: first, a pilot workshop for 2 subjects, followed by a second workshop where 6 additional subjects were selected based on the pilot results. For items with an actual correct answer rate of 90% or higher, the estimated correct answer rate for minimum competency was calculated using the observed correct answer rate. A survey and discussion with the expert panel were also conducted regarding the standard-setting procedures and results.
Results
The passing score for the national nursing examination was calculated using the new method, and the score was slightly higher than the existing score. The nursing subject had similar results; however, the legal subjects varied.
Conclusion
The modified Angoff and Hofstee methods were successfully applied to the KNLE. Using the actual correct answer rate as an indicator to derive expected minimum competency was shown to be effective. This approach could streamline future standard-setting processes, particularly when converting to CBT.

Technical report

Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study: Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee; J Educ Eval Health Prof. 2022;19:19. Published online August 22, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.19

6,596 View
261 Download
3 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to develop the examination objectives based on nursing competency of the Korean Nursing Licensing Examination.
Methods
This is a validity study to develop the examination objectives based on nursing competency. Data were collected in December 2021. We reviewed the literature related to changing nurse roles and on the learning objectives for the Korea Medical Licensing Examination and other health personnel licensing examinations. Thereafter, we created a draft of the nursing problems list for examination objectives based on the literature review, and the content validity was evaluated by experts. A final draft of the examination objectives is presented and discussed.
Results
A total of 4 domains, 12 classes, and 85 nursing problems for the Korean Nursing Liscensing Examination were developed. They included the essentials of objectives, related factors, evaluation goals, related activity statements, related clients, related settings, and specific outcomes.
Conclusion
This study developed a draft of the examination objectives based on clinical competency that were related to the clinical situations of nurses and comprised appropriate test items for the licensing examination. Above results may be able to provide fundamental data for item development that reflects future nursing practices.

Citations

Citations to this article as recorded by

A validity study of COMLEX-USA Level 3 with the new test design
Xia Mao, John R. Boulet, Jeanne M. Sandella, Michael F. Oliverio, Larissa Smith
Journal of Osteopathic Medicine.2024; 124(6): 257. CrossRef
A Survey on Perceptions of the Direction of Korean Medicine Education and National Licensing Examination
Han-Byul Cho, Won-Suk Sung, Jiseong Hong, Yeonseok Kang, Eun-Jung Kim
Healthcare.2023; 11(12): 1685. CrossRef
Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study
Kyunghee Kim, So Young Kang, Younhee Kang, Youngran Kweon, Hyunjung Kim, Youngshin Song, Juyeon Cho, Mi-Young Choi, Hyun Su Lee
Journal of Educational Evaluation for Health Professions.2023; 20: 18. CrossRef

First
Prev
Page of 1
Next
Last

Search