Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study

Purpose This study aims to suggest the number of test items in each of 8 nursing activity categories of the Korean Nursing Licensing Examination, which comprises 134 activity statements including 275 items. The examination will be able to evaluate the minimum ability that nursing graduates must have to perform their duties. Methods Two opinion surveys involving the members of 7 academic societies were conducted from March 19 to May 14, 2021. The survey results were reviewed by members of 4 expert associations from May 21 to June 4, 2021. The results for revised numbers of items in each category were compared with those reported by Tak and his colleagues and the National Council License Examination for Registered Nurses of the United States. Results Based on 2 opinion surveys and previous studies, the suggestions for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination in this study are as follows: 50 items for management of care and improvement of professionalism, 33 items for safety and infection control, 40 items for management of potential risk, 28 items for basic care, 47 items for physiological integrity and maintenance, 33 items for pharmacological and parenteral therapies, 24 items for psychosocial integrity and maintenance, and 20 items for health promotion and maintenance. Twenty other items related to health and medical laws were not included due to their mandatory status. Conclusion These suggestions for the number of test items for each activity category will be helpful in developing new items for the Korean Nursing Licensing Examination.


Introduction Background/rationale
The Korean Nursing Licensing Examination (KNLE) is a representative criterion-referenced assessment, based on the latest guidelines for nursing practice. It evaluates the minimum knowledge and skills that nursing graduates must possess to efficiently perform their duties [1]. However, the subjects that this licensing examination should cover are under constant debate. It has been suggested that the learning objectives of the 8 major nursing subjects overlap and that the KNLE may not be adequately equipped to appropriately evaluate the job competency of new nursing graduates [2,3]. Hence, continuing efforts are being made to verify the job-based knowledge of nursing graduates and to resolve the issue of overlap among the objectives of nursing subjects [4][5][6][7]. As part of this effort, Kim et al. [1] reviewed the feasibility of linking nurse jobs with the licensing examination. Song et al. [3] proposed subjects for an integrated nursing examination by extracting 1,303 statements from 481 knowledge statements. Consequently, 134 activity statements based on graduate nurse job activities in the field of nursing practice and 481 knowledge statements have been developed for the KNLE. These knowledge statements can be grouped into the following 8 activity categories: management of care and improvement of professionalism, safety and infection control, management of potential risk, basic care, physiological integrity and maintenance, pharmacological and parenteral therapies, psychosocial integrity and maintenance, and health promotion and maintenance. However, despite ongoing efforts to design a more appropriate licensing examination, the process of sharing information, collecting opinions, and securing agreement among academic societies and related organizations, such as those involved in nursing education, and nursing practice experts is still insufficient.

Objectives
This study aims to suggest the number of test items in each of 8 nursing activity categories of the KNLE.
The specific goals of the study were as follows: first, to elicit opinions on the distribution of the number of 5 test items from 7 major academic societies affiliated with the Korean Society of Nursing Science; and second, to obtain perspectives from 4 professional associations on those opinions from 7 academic societies.
It is hoped that the results will help resolve redundancy among the objectives of the 8 nursing subjects by proposing job-based item development for the KNLE.

Ethics statement
Institutional Review Board approval was not required for this process, because the survey questionnaire items did not contain information about the individual characteristics of the participants. Instead, it had information about the policies of societies and associations. No sensitive or individually identifiable information was obtained from the participants.

Study design
This was a survey-based descriptive study and content analysis, described according to the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) statement and 6 steps for conducting thematic analysis [8].

Setting
A job-based item model for the KNLE to resolve redundancy among the objectives of the 8 nursing subjects was provided by the authors (Supplement 1). Two opinion surveys were conducted from members of 7 academic societies to assess the suitability of this model. The first opinion survey was conducted from March 19 to 31, 2021. The second opinion survey was conducted from April 30 to May 14, 2021, after sharing the results of the first survey. Approximately 12-27 members from all academic societies and associations participated in the opinion-collection process. Next, 4 experts from 4 nursing associations reviewed the opinion-survey results from May 21 to June 4, 2021. The review results were summarized by 6 content analysis. Official e-mails were sent to each academic society and association, and the executives of each academic society and association gave their opinions. After summarizing the survey and analysis results, these results were compared with those reported by Tak et al. [9] and the National Council License Examination for Registered Nurses (NCLEX-RN) of the United States [10]. The overall workflow of the study is depicted in Fig. 1. Although a public hearing was held, the content was not reflected in this article.
For more information on the public hearing, refer to Supplement 2.

Participants
Seven academic societies participated in this survey, including the following: the Korean Academic

Variables
The primary outcome was the number of items for each category.

Data sources/measurement
The results of the opinion survey of the KNLE among the 7 academic societies and 4 associations were used as data sources. The survey questionnaire for 7 academic societies consisted of the items and the responses by the participants (Supplement 3). A more precise source report for the article is available in

Bias
There was no bias in selecting participants because this was a descriptive study in which no comparison was conducted.

Study size
We did not perform any sample size estimation because this was a descriptive study without any further analysis.

Statistical methods
The collected quantitative data were organized through content analysis by focusing on key concepts and analyzed with descriptive statistics such as frequencies and percentages. Table 1 presents the participating societies and associations.

Number of items suggested for the 8 categories by 7 academic societies
The KNLE items comprising 8 activity categories based on the results of the second opinion survey of the 7 member societies are listed in Table 2 (Dataset 1). The exam items included the following: management of care and improvement of professionalism, 41 items; health promotion and maintenance, 33 items; basic care, 37 items; physiological integrity and maintenance, 63 items; psychosocial integrity and maintenance, 41 items; safety and infection control, 16 items; pharmacological and parenteral therapies, 11 items; and management of potential risk, 31 items. In total, 273 items were confirmed. Two items on nursing history were exempted. Another 20 test items related to medical laws were also exempted. 8

Opinions of experts from 4 professional associations
The 4 professional associations emphasized that the safety and infection control activity category and pharmacological and parenteral therapies activity category play major roles in clinical and community nursing practice in real-life situations. Therefore, in the guideline on item-making, the proportion of items in the pharmacological and parenteral therapy activity category increased from 10% (in the guideline on item-making presented by Tak et al. [9] in 2019) to 12%, and that of the safety and infection control activity category also increased from 10% to 12%. The results of the opinion survey of the 4 expert associations on the KNLE standards for each activity category presented by the 7 academic societies mentioned above are summarized in Table 3.

Comparison with research on other examination items
We compared the results obtained with those of the NCLEX-RN evaluation conducted in 2019 [9] and those reported by Tak et al. [9] in a study that was also conducted in 2019 (Table 4). Among the items presented by Tak et al. [9], the percentage of items related to the management of care and improvement of professionalism, basic care, psychosocial integrity and maintenance, and health promotion and maintenance activity categories were the same as that determined by the opinion surveys conducted for this study. In the examination subjects developed by Tak et al. [9], the percentage of items in the management of potential risk activity category was 16% and that of items in the physiological integrity and maintenance activity category was 20%, which were higher than those in the NCLEX-RN Test Plan (2019) [10]. Referring to the opinion-survey results, we noted that the maximum percentage of the management of risk potential activity category in the distribution of the NCLEX-RN Test Plan (2019) was 15% (9%-15%) [10]. The percentage of the physiological integrity and maintenance activity category in the NCLEX-RN Test Plan (2019) was adjusted to 17%, corresponding to the maximum range of the distribution (11%-17%) [10]. In the opinion survey of the academies, the percentages of items in the safety and infection control and pharmacological and parenteral therapies activity categories were 4%-6%p lower than those in the item model suggested by 9 Tak et al. [9] and 6%-10%p lower than those in the NCLEX-RN Test Plan (2019) [10].

Interpretation
This study proposes a guideline for item-making based on the item model of Tak et al. [9]  In the expert opinion survey of the associations, some differences were noted in the understanding of the following 4 activity categories: management of potential risk, basic care, physiological integrity and maintenance, and pharmacological and parenteral therapies. To gain a clear understanding of the 8 activity categories, the NCLEX-RN Test Plan should be referred to [10] and the definition and goals for each activity category should be presented more clearly. An opinion was expressed that the 8 activity categories should be arranged systematically in a sequence. Furthermore, a specialist society needs to be established to collect opinions and reach a consensus regarding clinical and community nursing practice. Additionally, the 481 knowledge statements have different levels of specificity for each group. Some of the knowledge statements are not linked to any major subject, while others are linked to 2 or more major subjects. Therefore, the process of collecting opinions and reaching a consensus regarding knowledge statements should be modified and take precedence over other procedures. Because the competency of new nursing graduates is directly related to the quality of nursing and patient safety [11,12], it is necessary to prepare a guideline on item-making that can redirect the aim of the examination to a knowledge-verification model to evaluate examinees' competency.

Limitations
Given the basic research nature of this study, which investigated the opinions of experts on the jobbased integrated national examination model and a guideline on item-making for new nursing graduates, there is a possibility that it lacks objective justification.

Generalizability
The problems and directions for improvement proposed in this study relate to the KNLE. The study results are based on the cultural and situational background of the Republic of Korea. Hence, the results of this study have limited applicability to nursing licensing examinations of other countries or licensing examinations for other job groups.

Suggestions for further studies
Future studies should increase the level of objective legitimacy achieved in this study. For this, a task force team composed of nursing education and practice experts should be formed, and a consensus should be reached through sufficient discussion and careful collection of experts' opinions.

Conclusion
The study results suggest that the distribution of items in the following 4 activity categories should be maintained: management of care and improvement of professionalism, basic care, physiological integrity and maintenance, and health promotion and maintenance. However, the number of items of the management of potential risk and the psychosocial integrity and maintenance area should be reduced. The number of items in pharmacological and parenteral therapies, and safety and infection control should be increased. KK, YK, YS.

Conflict of interest
No potential conflict of interest relevant to this article was reported.

Funding
This work was supported by the Korea Health Personnel Licensing Examination Institute (Fund ref ID: 10.13039/501100003647) research fund (RE02-2111-05).

Data availability
Data files are available from Harvard Dataverse: ~~ Dataset 1. Raw response data from 7 academic societies on the 2 rounds of the opinion survey.

Supplementary materials
Supplementary Examination are still based on the major fields of nursing, including nursing management, fundamental nursing, women's health nursing, adult nursing, children's health nursing, psychiatric and mental health nursing, community health nursing, and medical law. Therefore, competency, activity, or job-based integrated subjects should be developed based on consensus among interested groups.