KHPLEI
Articles
- Page Path
- HOME > J Educ Eval Health Prof > Volume 22; 2025 > Article
-
Research article
Proposal for setting a passing score for the Korean Nursing Licensing Examination -
Janghee Park1
, Mi Kyoung Yim2*, Sujin Shin3, Rhayun Song4, Jun-Ah Song5, Inyoung Lee6, Heejeong Kim7, Minjae Lee8 -
DOI: https://doi.org/10.3352/jeehp.2025.22.25
Published online: September 8, 2025
1Department of Medical Education, College of Medicine, Kangwon National University, Chuncheon, Korea
2Korea Health Personnel Licensing Examination Institute, Seoul, Korea
3College of Nursing, Ewha Womans University, Seoul, Korea
4College of Nursing, Chungnam National University, Daejeon, Korea
5College of Nursing, Korea University, Seoul, Korea
6Department of Nursing, Dongnam Health University, Suwon, Korea
7Division of Health Science, Department of Nursing Science, Baekseok University, Cheonan, Korea
8College of Nursing, Ewha Womans University, Seoul, Korea
- *Corresponding email: mkyim@kuksiwon.or.kr
Editor: A Ra Cho
, The Catholic University of Korea, Korea
© 2025 Korea Health Personnel Licensing Examination Institute
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
- 1,088 Views
- 187 Download
Abstract
-
Purpose
- The Korean Nursing Licensing Examination (KNLE) is planning to transition to a computer-based test (CBT). This study aims to propose a reasonable and efficient method for setting passing scores.
-
Methods
- A standard-setting (passing score setting) analysis was conducted using an expert panel over the past 3 years of the national nursing examination. The standard-setting method was modified from Angoff, and the validity of the passing score was verified through the Hofstee method. The standard-setting workshop was conducted in 2 stages: first, a pilot workshop for 2 subjects, followed by a second workshop where 6 additional subjects were selected based on the pilot results. For items with an actual correct answer rate of 90% or higher, the estimated correct answer rate for minimum competency was calculated using the observed correct answer rate. A survey and discussion with the expert panel were also conducted regarding the standard-setting procedures and results.
-
Results
- The passing score for the national nursing examination was calculated using the new method, and the score was slightly higher than the existing score. The nursing subject had similar results; however, the legal subjects varied.
-
Conclusion
- The modified Angoff and Hofstee methods were successfully applied to the KNLE. Using the actual correct answer rate as an indicator to derive expected minimum competency was shown to be effective. This approach could streamline future standard-setting processes, particularly when converting to CBT.
- Background/rationale
- Computer-based testing (CBT), which integrates information technology, has been introduced to licensure exams and offers advantages over paper-based tests, such as diverse assessment methods, shorter testing times, and more reliable evaluations. Additionally, CBT enables longitudinal data collection to track learning trends, allows for tailored items based on difficulty and purpose, and provides examinees with rapid results through automated scoring [1]. The national nursing examination is designed to verify the minimum job competency required for new nurses to perform their duties. Currently, the Korean Nursing Licensing Examination (KNLE) consists of a knowledge-based written test that primarily evaluates basic competencies for clinical practice. However, it remains difficult to develop job-oriented items in this format [2,3]. Although various methods for assessing practical competency exist, CBT is regarded as the most practical alternative for the national nursing examination, as it enables the inclusion of diverse, practice-oriented items that enhance both validity and reliability. Nonetheless, challenges remain due to limited infrastructure, the need for appropriate large-scale test design, and the planned integration of 8 subjects into 2 domains—nursing and health and medical law—by 2028. In this context, establishing concrete measures for setting passing scores in line with these structural changes is essential.
- Among standard-setting methods, the modified Angoff approach is the most frequently employed for setting passing scores in licensure examinations, including the US Medical Licensing Examination (USMLE) and the KNLE. Other widely used methods include the Ebel, Bookmark, and Hofstee methods. Ahn et al. [4] simulated these approaches for Korean physicians, radiologists, and pharmacists. Based on their findings, they proposed that the most appropriate approach involves calculating the passing score using either the modified Angoff or Ebel method and then verifying that the derived score falls within an acceptable range of passing scores and failure rates using the Hofstee method [4].
- In licensing examinations, the Angoff method remains the most widely applied criterion-setting approach. In this method, evaluators perform repeated individual assessments and discussions, supported by feedback materials, to review item content and determine the final passing score [5]. In high-stakes licensing contexts, passing scores are not determined through annual workshops but through periodic evaluations of all items. For example, the National Council Licensure Examination for Registered Nurses (NCLEX-RN) sets scores every 3 years, the USMLE every 3 to 4 years, and the Canadian Medical Licensing Examination every 3 to 5 years [6-8]. Passing scores are not established annually because test content and examinee ability vary each year, and repeated evaluations demand substantial time and effort while contributing to rater fatigue and decreased reliability [9]. Therefore, developing an economical and efficient method to set reasonable passing scores through expert evaluation is necessary.
- Objectives
- This study aimed to propose a reasonable and efficient method for setting passing scores for the national nursing examination under the introduction of CBT and changes in test subjects.
Introduction
- Ethics statement
- All panelists voluntarily participated in the standard-setting workshop with informed consent. Anonymity was maintained, data use was restricted to research purposes, and all identifiable information was destroyed at the study’s conclusion.
- Study design
- This study is a methodological study that employed the modified Angoff and Hofstee methods, conducting expert panel evaluations to establish the passing score for the KNLE. The Angoff method estimates the percentage of correct responses for each item by a minimally competent person, sums these estimates to determine the passing score, and is widely applied because of its simplicity and ease of explanation [10]. In the Hofstee method, panelists respond to 4 questions regarding the assumed knowledge level (k) and fail rate (f) of test-takers. These questions specify the minimum and maximum acceptable passing scores as well as the allowable fail-rate range [11].
- Data
- The passing score for the KNLE was determined using exam items and data from the 61st (2021), 62nd (2022), and 63rd (2023) examinations, each consisting of 295 items across 8 subjects (Table 1). After excluding a small number of withdrawals and violations, data from 22,927, 24,172, and 24,011 candidates were analyzed, yielding passing rates of 95.8%, 96.6%, and 97.3%, respectively. Exam data were provided by the KHPLEI. Standard-setting was conducted in 2 stages: a pilot workshop using adult and fundamental nursing (initially 300 items, reduced to 295 after excluding 5), followed by the main workshop.
- Participants
- The standard-setting workshop panel included 17 female faculty members. The first workshop comprised 4 experts in adult nursing and 2 in fundamental nursing. The second workshop added specialists in maternal, pediatric, psychiatric, community health, and nursing management, as well as one expert in health and medical law. Most panelists were in their 50s, all had experience as KNLE examiners or reviewers, 4 had prior experience in setting passing scores, and all had at least 10 years of teaching or examination-related experience. To ensure validity, specialties were represented in multiples when possible; although only one expert specialized in health and medical law, the content was reviewed collectively during the rating process.
- Procedure
- Fig. 1 illustrates the study process. Following orientation and training on defining the minimally competent person, the modified Angoff and subsequently the Hofstee assessments were conducted. The detailed workshop schedule is provided in Appendix 1.
- The rater composition method divided raters by subject and allowed cross-sharing across related subjects. This approach, which followed the basic procedure of the modified Angoff method, was based on the study by Yim [12] and Yim and Shin [13] rather than requiring all items to be evaluated by all panelists.
- After orientation and training, the panel defined the minimally competent person and conducted the Angoff method. First, 30 items were evaluated with feedback and discussion, after which the remaining items were reviewed. This iterative clarification of the minimally competent person and estimation of response probability was critical for determining passing scores and ensuring procedural validity [4,12-14].
- The Hofstee method was implemented after the third round of the modified Angoff method. The workshop was concluded after checking the results and conducting a post-survey.
- In the modified Angoff method, panelists estimated the expected correct response rate for a minimally competent person, resolving wide discrepancies through discussion or review of response data. Because evaluating large numbers of items can induce fatigue and error, item rating may be divided or limited according to characteristics such as item difficulty. In this study, items with ≥90% correct rates—considered to have little impact on the passing score—used the actual correct rate as an estimate of minimally competent performance [4,9,14,15].
- The average correct answer rate and number of items with ≥90% correct rates for each subject in the 3 exams are presented in Table 2. In the 61st–63rd exams, adult and fundamental nursing included 13–26 items with ≥90% correct rates (14.3%–80% across subjects). The second workshop reviewed 6 subjects with 35 items each, including 20 for health and medical law.
- Following both the pilot and main workshops, a panelist perception survey was administered. This survey evaluated participants’ understanding of the preliminary training, the appropriateness of the procedures, and the perceived validity of the results. Responses were rated on a 5-point scale using the questionnaire developed by Yim and Shin [13].
Methods
- Simulation: estimation of the expected correct answer rate for a minimally competent person
- A simulation estimated the expected correct response rate of a minimally competent person for KNLE items with ≥90% correct rates. Values were calculated as 90%, 80%, 70%, or 60% of the actual rates; these estimates were then used to derive passing scores and compared with panel ratings (Fig. 2).
- Across the 61st to 63rd KNLE, passing scores varied depending on the applied proportion of correct answer rates. The values most closely aligned with panel-based ratings were obtained when applying 80% of the actual correct answer rate. Therefore, 80% was adopted as the expected correct answer rate for minimally competent examinees, rather than relying solely on panel ratings for items with a difficulty level of 90% or higher (Fig. 2).
- The passing score using the Hofstee method for validation
- Using the Hofstee method, the passing scores for the 61st, 62nd, and 63rd KNLE were determined as 65, 67, and 69, respectively, based on the panel’s averaged evaluations and the cumulative distribution of examinees (Fig. 3).
- The passing score using modified Angoff method
- Table 3 presents the passing scores for the 61st–63rd KNLE calculated using the modified Angoff method.
- All 8 subjects were rated by panelists. The passing scores for the 61st, 62nd, and 63rd KNLE were 65.3 (90.7%), 66.5 (92.5%), and 67.5 (94.1%), all lower than the actual passing rates of 94.8%, 96.6%, and 97.3%, respectively.
- The passing score was analyzed based on the integrated subjects, which include all items of the KNLE subjects except for the health and medical laws subject, because the KNLE subjects will be integrated in the future. For nursing, passing scores (65.5–67.8) and rates (90.5%–94.2%) were almost identical to the totals, while for health and medical laws, passing scores (61.5–62.9) and rates (82.1%–88.6%) were markedly lower, by 3.8–5.3 percentage points. A summary of the modified Angoff passing scores by the participants is presented in Table 4.
- Panel survey
- The frequency of responses to the 13 questions regarding the 2 workshop procedures and scores out of the 5 are shown in Table 5. All survey items scored ≥4.3, with the highest ratings for understanding pre-education and strong scores for procedure, results, and clarity in defining the minimally competent person, indicating effective consensus-building.
- The results of the panel’s responses to the considerations in setting the passing scores are presented in Table 6. Most panelists preferred either the modified Angoff or Hofstee methods. Views differed on whether all or only some items should be reviewed. Among those favoring partial review, opinions varied regarding extraction ratios, although 88.2% emphasized the importance of considering item difficulty and discrimination.
Results
- The KNLE’s fixed passing score faces limitations amid reforms such as subject integration, job-oriented items, and the transition to CBT/computer adaptive test, leading this study to examine more suitable standard-setting methods.
- First, using both the modified Angoff and Hofstee methods for setting cut scores was found to be efficient and effective in determining passing standards. By applying methods proposed in previous studies, passing scores were calculated using the widely applied and easily understood modified Angoff method, and the validity of these scores was confirmed by verifying through the Hofstee method that the scores derived from the modified Angoff method fell within acceptable ranges of passing scores and failure rates [14]. The passing scores were 65.3, 66.5, and 67.5 using the modified Angoff method, which may overestimate scores when test difficulty is low or panelists assume a higher minimum competency. However, because the KNLE after COVID-19 (coronavirus disease 2019) was considered less difficult, with higher pass rates, the Hofstee method was also applied, yielding scores of 65, 67, and 69, which closely aligned with the Angoff results.
- Second, through simulation, the panel confirmed that for items with an average correct response rate of 90% or higher, the expected correct response rate for a minimally competent person was assessed as similar to 80% of the actual correct response rate. Utilizing this finding, when panelists are tasked with estimating the expected correct response rate of minimally competent person in standard-setting, excluding items with high actual correct response rates (90% or above) from direct panel evaluation and instead using the expected correct response rates calculated from 80% of the actual correct response rates can reduce the number of items panelists must evaluate, thereby decreasing their fatigue. In a survey conducted after participation in the workshop, panelists indicated a preference for a passing score setting method that combines the modified Angoff and Hofstee methods, acknowledging that it is unnecessary to rate all items and that rating a subset of items is feasible. When setting a passing score, it is necessary to consider a method for efficiently calculating it by evaluating only some items [9].
- Panelists further favored item selection methods based on difficulty and discrimination. In medical licensing examinations, excluding items with ≥90% correct response rates while estimating minimally competent performance at 60% of the actual correct rate has produced passing scores close to 60 [15]. Similarly, this study confirmed that when items had an average correct response rate of 90% or higher, estimating minimally competent performance at 80% of the actual correct rate aligned well with panel ratings.
- Third, the analysis of all items rated by the panel in various combinations revealed differences in the passing scores. Currently, the KNLE requires at least 60% overall and no less than 40% in any subject, but analysis of 2020–2024 data showed higher failure rates and lower passing scores in health and medical laws (61.5–62.9) than in nursing (65.5–67.8), indicating the need to set separate passing scores for the 2 subjects in preparation for their integration.
- Fourth, the methods implemented to enhance the validity of the standard-setting process were found to be effective. Internal validity was strengthened by including multiple experienced panelists per specialty, which facilitated consensus on minimal competence and accurate interpretation of item performance. Practice rating and discussion of sample items also reinforced this process. External validity was supported by the convergence of passing scores derived from panel ratings and actual response data with Hofstee criteria, as well as by consistency across 3 years of exam data.
- Based on the study results, several recommendations are proposed. Because evaluating all items annually is impractical, passing scores could be set every 3 years, following the NCLEX-RN model, and maintained until the next cycle. Additionally, panel selection should consider both the National Council of State Boards of Nursing’s criteria and the inclusion of experts with experience in national examinations or clinical nurse education.
- When the passing score was set by dividing it into nursing and health and medical laws using the same method, the passing score for health and medical laws was lower than that for integrated nursing. Over the past 5 years, more examinees failed health and medical laws than other subjects. With the KNLE restructuring into nursing and health and medical laws, applying a single passing score would be inappropriate; thus, subject-specific standards are needed to ensure fairness and validity.
- A potential limitation of this study is the subjectivity introduced by panel composition, which may influence passing score determinations. To mitigate this, future studies should diversify panel membership and consider consultation with international experts. Furthermore, because panelists in high-stakes examinations may not rate all items, it is necessary to apply similar standard-setting approaches across national licensing examinations in other health professions.
- In conclusion, in the modified Angoff method, items with ≥90% correct response rates were set at 80% of the actual correct rate, while all other items were directly estimated by panelists. This approach reduced workload while focusing judgment on critical items. It was preferred by most panelists, and the Hofstee method provided confirmatory evidence for validating the derived passing score.
Discussion
Authors’ contributions
Conceptualization: JP, MKY, SS. Data curation: JP, MKY, SS, ML. Formal analysis: JP, MKY, SS, ML. Funding acquisition: SS. Methodology: JP, MKY, SS, RS, JAS, HK, IL, ML. Project administration: JP, MKY, SS, RS, JAS, HK, IL, ML. Writing–original draft: JP, MKY. Writing–review & editing: JP, MKY, SS, ML.
Conflict of interest
Mi Kyoung Yim is a staff member of the Korea Health Personnel Licensing Examination Institute; however, this study is not an official opinion of the Institute. No potential conflict of interest relevant to this article was reported.
Funding
This work was supported by the Korea Health Personnel Licensing Examination Institute of Korea in 2024 (No., RE12-2504-05).
Acknowledgments
None.
Supplementary materials
Fig. 2.Comparison of the passing score using the existing passing score, the score evaluated by the panel, and the actual correct answer rate. Fixed: the current passing score of the Korean Nursing Licensing Examination; Total: the passing score estimated by the panelists who rated all items during the workshop; 90%, 80%, 70%, and 60%: application rate of expected correct answer rate for minimally competent person using actual correct answer rate. For example, if the actual correct response rate is 95%, 80% of 95% is applied as the expected correct response rate for minimally competent person.
Fig. 3.Hofstee method passing score setting graph (A–C). KNLE, Korean Nursing Licensing Examination.
Table 1.Korean Nursing Licensing Examination: subject-by-subject items and passing criteria
Table 2.Average correct answer rate by subject and number of items with a correct answer rate of 90% or higher
Table 3.Results of setting passing scores using the modified Angoff method
Hofstee passing score: 65 (61st), 67 (62nd), and 69 (63rd KNLE). All subjects: The 8 subjects of the Korean Nursing Licensing Examination (adult nursing, fundamental nursing, maternal nursing, pediatric nursing, psychiatric nursing, community health nursing, nursing management, and health and medical laws). Nursing: A single subject comprising all KNLE items except for those in health and medical laws.
SD, standard deviation; KNLE, Korean Nursing Licensing Examination.
a)Calculation of passing rate based on total score of all subjects of actual test takers (out of 295 points). b)Calculation of passing rate based on the total score of 7 nursing subjects of actual applicants (out of 275 points). c)Calculation of passing rate based on actual applicants’ scores on health and medical laws (out of 20 points).
Table 4.The summary of the modified Angoff passing scores
Table 5.Panel response results on the procedure and results for setting the passing score (N=17)
Table 6.Panel response results on considerations for setting the passing score (N=17)
- 1. Chun KH, Jin HK, Yoon JH, Kim MG, Choi KH, Kim E, Kim H, Kim JK, Kim G, Kim K, Lee JY, Chung EK, Lee YS, Rhie SJ. Novel innovative computer-based test (Inno-CBT) item types for national licensing examinations for health care professionals. BMC Med Educ 2023;23:560. https://doi.org/10.1186/s12909-023-04444-5 ArticlePubMedPMC
- 2. Tak YR, Kang YH, Song RY, Song J, Han M. A study on the improvement of the national examination and license management system for nurses according to changes in the health care environment. Korea Health Personnel Licensing Examination Institute; 2019.
- 3. Shin S, Song R, Song J, Kim H, Hong E, Lee M. Basic study on the implementation plan of the CBT for the national nursing examination. Korea Health Personnel Licensing Examination Institute; 2024.
- 4. Ahn DS, Park JH, Yim MK, Lee JH. A study about standard setting for national licensing exams for health professions. Korea Health Personnel Licensing Examination Institute; 2018.
- 5. Cizek GJ. Setting performance standards: concepts, methods, and perspectives. Lawrence Erlbaum Associate; 2001.
- 6. MCC Qualifying Examination. Score interpretation [Internet]. Medical Council of Canada; 2024 [cited 2024 Apr 28]. Available from: https://mcc.ca/examinations-assessments/exam-inquiries/score-interpretation/
- 7. National Board of Medical Examiners. Examination Results and Scoring [Internet]. United States Medical Licensing Examination; 2024 [cited 2024 Apr 28]. Available from: https://www.usmle.org/scores-transcripts/examination-results-and-scoring
- 8. National Council of State Boards of Nursing. Passing standard [Internet]. National Council of State Boards of Nursing; 2023 [cited 2024 Mar 5]. Available from: https://www.nclex.com/passing-standard.page
- 9. Ferdous AA, Plake BS. Item selection strategy for reducing the number of items rated in an Angoff standard setting study. Educ Psychol Meas 2007;67:193-206. https://doi.org/10.1177/0013164406288160 Article
- 10. Angoff WH. Scales, norms, and equivalent scores. Educational Testing Service; 1984.
- 11. Hofstee WK. The case for compromise in educational selection and grading. In: Anderson SB, Helmick JS, editors. On educational testing. Jossey-Bass; 1983. p. 109-127.
- 12. Yim M. Comparison of results between modified-Angoff and bookmark methods for estimating cut score of the Korean medical licensing examination. Korean J Med Educ 2018;30:347-357. https://doi.org/10.3946/kjme.2018.110 ArticlePubMedPMC
- 13. Yim MK, Shin S. Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination. J Educ Eval Health Prof 2020;17:14. https://doi.org/10.3352/jeehp.2020.17.14 ArticlePubMedPMC
- 14. Park J, Ahn DS, Yim MK, Lee J. Comparison of standard-setting methods for the Korea Radiological Technologist Licensing Examination: Angoff, Ebel, Bookmark, and Hofstee. J Educ Eval Health Prof 2018;15:32. https://doi.org/10.3352/jeehp.2018.15.32 ArticlePubMedPMC
- 15. Park J, Ahn D, Lim M, Kim Y, Kim N. Seeking an efficient method to set a passing score for the Korean medical licensing examination. Korea Health Personnel Licensing Examination Institute; 2020.
References
Appendix
Appendix 1.
Figure & Data
References
Citations
Citations to this article as recorded by 
PubReader
Cite
XML DownloadProposal for setting a passing score for the Korean Nursing Licensing Examination
Fig. 1. Procedure for setting the passing score.
Fig. 2. Comparison of the passing score using the existing passing score, the score evaluated by the panel, and the actual correct answer rate. Fixed: the current passing score of the Korean Nursing Licensing Examination; Total: the passing score estimated by the panelists who rated all items during the workshop; 90%, 80%, 70%, and 60%: application rate of expected correct answer rate for minimally competent person using actual correct answer rate. For example, if the actual correct response rate is 95%, 80% of 95% is applied as the expected correct response rate for minimally competent person.
Fig. 3. Hofstee method passing score setting graph (A–C). KNLE, Korean Nursing Licensing Examination.
Graphical abstract
Fig. 1.
Fig. 2.
Fig. 3.
Graphical abstract
Proposal for setting a passing score for the Korean Nursing Licensing Examination
| Subject | No. of questions |
|---|---|
| Adult nursing | 70 |
| Basic nursing | 30 |
| Maternal nursing | 35 |
| Pediatric nursing | 35 |
| Psychiatric nursing | 35 |
| Regional nursing | 35 |
| Management nursing | 35 |
| Health and medical laws | 20 |
| Total | 295 |
| Round | No. of candidates | Category | Subject (no. of items) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Adult nursing (70) | Fundamentals of nursing (30) | Maternal nursing (35) | Pediatric nursing (35) | Psychiatric nursing (35) | Community health nursing (35) | Nursing management (35) | Health and medical laws (20) | |||
| 61st (2021) | 22,927 | Correct answer rate (%) | 74.89 | 77.61 | 78.05 | 85.55 | 85.58 | 74.06 | 68.87 | 77.90 |
| More than 90% | 20 (28.6) | 13 (43.3) | 7 (20) | 20 (57.1) | 18 (51.4) | 13 (37.1) | 5 (14.3) | 4 (20) | ||
| 62nd (2022) | 24,172 | Correct answer rate (%) | 74.02 | 87.86 | 82.30 | 80.84 | 85.55 | 68.12 | 83.13 | 73.44 |
| More than 90% | 18 (25.7) | 19 (63.3) | 16 (45.7) | 16 (45.7) | 20 (57.1) | 13 (22.9) | 15 (42.9) | 6 (30) | ||
| 63rd (2023) | 24,011 | Correct answer rate (%) | 75.15 | 91.66 | 75.01 | 80.87 | 91.82 | 85.38 | 77.85 | 78.97 |
| More than 90% | 26 (37.1) | 22 (73.3) | 13 (37.1) | 13 (37.1) | 28 (80) | 13 (40) | 13 (37.1) | 6 (30) | ||
| Subjects | Round | Mean±SD | Final passing score (points) | Passing rate by passing score (%) | Actual pass rate (%) | |||
|---|---|---|---|---|---|---|---|---|
| Round 1 | Round 2 | Round 3 | 100 points out of 100 | Raw score | ||||
| 1. All subjects (8 subjects) | 61st | 67.4±4.4 | 66.3±3.9 | 65.3±3.6 | 65.3 | 193 | 90.7a) | 94.8 |
| 62nd | 67.4±5.6 | 67.2±5.0 | 66.5±4.6 | 66.5 | 197 | 92.5a) | 96.6 | |
| 63rd | 68.5±4.9 | 68.4±4.6 | 67.5±3.7 | 67.5 | 200 | 94.1a) | 97.3 | |
| 2. Nursing | 61st | 67.5±4.3 | 66.5±3.8 | 65.3±3.5 | 65.5 | 181 | 90.5b) | 94.8 |
| 62nd | 67.7±5.4 | 67.5±4.8 | 66.8±4.4 | 66.8 | 184 | 92.9b) | 96.6 | |
| 63rd | 68.8±4.8 | 68.7±4.4 | 67.8±3.5 | 67.8 | 187 | 94.2b) | 97.3 | |
| 3. Health and medical laws | 61st | 66.7±7.5 | 62.9±4.3 | 61.7±4.2 | 61.7 | 13 | 86.2c) | 94.8 |
| 62nd | 62.4±7.3 | 62.1±6.9 | 61.5±6.4 | 61.5 | 13 | 82.1c) | 96.6 | |
| 63rd | 63.7±5.0 | 63.7±4.5 | 62.9±5.2 | 62.9 | 13 | 88.6c) | 97.3 | |
| Round | Subject | No. of items | Passing score (100 points) |
|---|---|---|---|
| 61st | By subject | ||
| Adult | 70 | 66.4 | |
| Fundamental | 30 | 73.3 | |
| Maternal | 35 | 58.9 | |
| Pediatric | 35 | 67.4 | |
| Psychiatric | 35 | 66.2 | |
| Community | 35 | 65.0 | |
| Management | 35 | 61.1 | |
| Law | 20 | 61.7 | |
| Total score (all subjects) | 295 | 65.3 | |
| Nursing and law | |||
| Nursing | 275 | 65.5 | |
| Law | 20 | 61.7 | |
| 62nd | By subject | ||
| Adult | 70 | 68.9 | |
| Fundamental | 30 | 75.4 | |
| Maternal | 35 | 63.4 | |
| Pediatric | 35 | 63.9 | |
| Psychiatric | 35 | 67.6 | |
| Community | 35 | 60.4 | |
| Management | 35 | 65.7 | |
| Law | 20 | 61.5 | |
| Total subjects | 295 | 66.5 | |
| Nursing and law | |||
| Nursing | 275 | 66.8 | |
| Law | 20 | 61.5 | |
| 63rd | By subject | ||
| Adult | 70 | 69.0 | |
| Fundamental | 30 | 76.8 | |
| Maternal | 35 | 60.5 | |
| Pediatric | 35 | 62.8 | |
| Psychiatric | 35 | 73.3 | |
| Community | 35 | 66.8 | |
| Management | 35 | 65.1 | |
| Law | 20 | 62.9 | |
| Total subjects | 295 | 67.5 | |
| Nursing and law | |||
| Nursing | 275 | 67.8 | |
| Law | 20 | 62.9 |
| Question | Response frequency | Mean±SD | ||||
|---|---|---|---|---|---|---|
| Not at all | No | Commonly | Yes | Very much so | ||
| 1. The pre-training helped me understand the purpose of setting the passing score. | 0 | 0 | 0 | 4 | 13 | 4.8±0.4 |
| 2. Pre-training made clear the tasks to be done. | 0 | 0 | 0 | 3 | 14 | 4.8±0.4 |
| 3. The definition of ‘minimally competent person’ was clear. | 0 | 0 | 0 | 6 | 11 | 4.6±0.5 |
| 4. It was easy to assume the response probability of the ‘minimally competent person’. | 0 | 1 | 2 | 4 | 10 | 4.4±0.9 |
| 5. The achievement level technology was easy. | 0 | 0 | 0 | 10 | 6 | 4.4±0.5 |
| 6. I was able to fill out the score sheet following the instructions. | 0 | 0 | 0 | 5 | 12 | 4.7±0.5 |
| 7. The discussion after the individual assessment helped me. | 0 | 0 | 0 | 2 | 15 | 4.9±0.3 |
| 8. Group discussions helped with the evaluation. | 0 | 0 | 0 | 2 | 15 | 4.9±0.3 |
| 9. The information provided for discussion was helpful to me. | 0 | 0 | 0 | 5 | 12 | 4.7±0.5 |
| 10. The time provided for discussion was adequate. | 0 | 0 | 1 | 5 | 11 | 4.6±0.6 |
| 11. In my group, everyone had ample opportunity to express their opinions. | 0 | 0 | 0 | 4 | 13 | 4.8±0.4 |
| 12. The split scores I have assessed are appropriate. | 0 | 1 | 1 | 7 | 8 | 4.3±0.8 |
| 13. The final split score determined is appropriate. | 0 | 0 | 0 | 9 | 8 | 4.5±0.5 |
| Question | Category | Frequency (%) |
|---|---|---|
| How to set a desirable passing score | Modified Angoff+Hofstee | 10 (58.8) |
| Modified Angoff | 6 (35.3) | |
| Need for a standard-setting method that reviews all items | Yes | 4 (36.4) |
| No | 7 (63.6) | |
| Need for a standard-setting method that reviews only some items | Yes | 9 (52.9) |
| No | 8 (47.1) | |
| Optimal extraction ratio when reviewing some items among all items | 0.2 | 1 (5.9) |
| 0.3 | 2 (11.8) | |
| 0.35 | 1 (5.9) | |
| 0.5 | 4 (23.5) | |
| 0.6 | 2 (11.8) | |
| 0.7 | 4 (23.5) | |
| 0.8 | 2 (11.8) | |
| Item characteristics to consider when reviewing some items (multiple responses allowed) | Random sampling | 4 (23.5) |
| Item difficulty | 15 (88.2) | |
| Item discrimination | 15 (88.2) |
| Time | Program | Remarks |
|---|---|---|
| 10:00–11:00 | Orientation (research introduction, theoretical background and methods of standard-setting) & pre-survey | |
| 11:00–11:30 | Discussion on minimally competent person | |
| 11:30–12:20 | Modified Angoff round 1-1: individual rating, feedback, and discussion | 10% of items |
| 12:20–13:20 | Lunch & break | |
| 13:20–14:20 | Modified Angoff round 1-2: individual rating, feedback, and discussion | 90% of items |
| 14:20–15:10 | Modified Angoff round 2: individual rating, feedback, and discussion | |
| 15:10–16:00 | Modified Angoff round 3: individual rating, feedback, and discussion; final passing score confirmation | |
| 16:00–16:30 | Hofstee rating | |
| 16:30–17:00 | Post-survey, results discussion & closing |
Table 1. Korean Nursing Licensing Examination: subject-by-subject items and passing criteria
Passing criteria: 60% of the total points, Score more than 40% in each subject.
Table 2. Average correct answer rate by subject and number of items with a correct answer rate of 90% or higher
Values are presented as % or number (%).
Table 3. Results of setting passing scores using the modified Angoff method
Hofstee passing score: 65 (61st), 67 (62nd), and 69 (63rd KNLE). All subjects: The 8 subjects of the Korean Nursing Licensing Examination (adult nursing, fundamental nursing, maternal nursing, pediatric nursing, psychiatric nursing, community health nursing, nursing management, and health and medical laws). Nursing: A single subject comprising all KNLE items except for those in health and medical laws. SD, standard deviation; KNLE, Korean Nursing Licensing Examination. a)Calculation of passing rate based on total score of all subjects of actual test takers (out of 295 points). b)Calculation of passing rate based on the total score of 7 nursing subjects of actual applicants (out of 275 points). c)Calculation of passing rate based on actual applicants’ scores on health and medical laws (out of 20 points).
Table 4. The summary of the modified Angoff passing scores
Table 5. Panel response results on the procedure and results for setting the passing score (N=17)
SD, standard deviation.
Table 6. Panel response results on considerations for setting the passing score (N=17)
