1Department of Medical Education, Soonchunhyang University College of Medicine, Asan, Korea
2Research institute for healthcare policy, Seoul, Korea
3Korea Health Personnel Licensing Examination Institute, Seoul, Korea
© 2018, Korea Health Personnel Licensing Examination Institute
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Authors’ contributions
Conceptualization: JP, MKY, DSA, JL. Data curation and analysis: JP, MKY, JL. Funding acquisition & project administration: DSA. Writing–original draft: JP. Writing–review & editing: JP, MKY, DSA.
Conflict of interest
No potential conflict of interest relevant to this article was reported.
Funding
This work was supported by the National Health Personnel Licensing Examination Board of Korea (Fundref ID: 10.13039/501100003647) research fund (2017).
Subject | No. of items | Allocation | Period |
---|---|---|---|
Radiation theory | 90 | 90 | 1 |
Medical regulations | 20 | 20 | |
Radiation application | 90 | 90 | 2 |
Practical skills | 50 | 50 | 3 |
Total | 250 | 250 |
Cut-off score | Total score of 100 | |
---|---|---|
Angoff | 178.17 | 71.27 |
Ebel | 155.5 | 62.2 |
Bookmark | 161.23 | 64.49 |
Hofstee | 155 | 62 |
Total | 250 | 100 |
Pass or fail | Angoff | Ebel | Bookmark | Hofstee |
---|---|---|---|---|
Fail | 970 (37.0) | 532 (20.3) | 593 (22.6) | 532 (20.3) |
Pass | 1,652 (63.0) | 2,090 (79.7) | 2,029 (77.4) | 2,090 (79.7) |
Angoff | Ebel | Bookmark | |
---|---|---|---|
Ebel | 0.658 | ||
Bookmark | 0.706 | 0.932 | |
Hofstee | 0.647 | 0.983 | 0.917 |
Subject | No. of items | Allocation | Period |
---|---|---|---|
Radiation theory | 90 | 90 | 1 |
Medical regulations | 20 | 20 | |
Radiation application | 90 | 90 | 2 |
Practical skills | 50 | 50 | 3 |
Total | 250 | 250 |
Time | Content |
---|---|
9:00–09:30 | Introduction of the standard-setting methods |
9:30–10:30 | Angoff & Ebel: rating of individual item of total items, agreement by group |
10:30–12:00 | Angoff & Ebel: total items, agreement of all panelists |
12:00–13:00 | Lunch |
13:00–15:00 | Angoff & Ebel: total items, agreement of all panelists |
15:00–15:40 | Bookmark |
15:40–16:10 | Hofstee |
16:10–16:30 | Survey of panelists |
16:30–17:00 | Discussion |
Cut-off score | Total score of 100 | |
---|---|---|
Angoff | 178.17 | 71.27 |
Ebel | 155.5 | 62.2 |
Bookmark | 161.23 | 64.49 |
Hofstee | 155 | 62 |
Total | 250 | 100 |
Pass or fail | Angoff | Ebel | Bookmark | Hofstee |
---|---|---|---|---|
Fail | 970 (37.0) | 532 (20.3) | 593 (22.6) | 532 (20.3) |
Pass | 1,652 (63.0) | 2,090 (79.7) | 2,029 (77.4) | 2,090 (79.7) |
Angoff | Ebel | Bookmark | |
---|---|---|---|
Ebel | 0.658 | ||
Bookmark | 0.706 | 0.932 | |
Hofstee | 0.647 | 0.983 | 0.917 |
Relevance | Level of difficulty | No. of questions (A) | Expected rate of correct answers (B) | A×B |
---|---|---|---|---|
Essential | Easy | 4 | 0.8 | 3.2 |
Medium | 125 | 0.7 | 87.5 | |
Hard | 1 | 0.5 | 0.5 | |
Subtotal | 130 | 91.2 | ||
Important | Easy | 0 | 0.7 | 0 |
Medium | 44 | 0.6 | 26.4 | |
Hard | 1 | 0.4 | 0.4 | |
Subtotal | 45 | 26.8 | ||
Additional | Easy | 2 | 0.6 | 1.2 |
Medium | 72 | 0.5 | 36 | |
Hard | 1 | 0.3 | 0.3 | |
Subtotal | 75 | 37.5 | ||
Total | 155.5/250 | |||
62.2/100 |
Subject panel | Radiation theory |
Medical regulation |
Radiation application |
Practical skills |
Total score | Pass (%) | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
OIB | Ability | Score | OIB | Ability | Score | OIB | Ability | Score | OIB | Ability | Score | |||
P1 | 47 | -0.92 | 55 | 11 | -1.46 | 13 | 44 | -0.99 | 59 | 30 | -0.52 | 35 | 161.35 | 76.60 |
P2 | 46 | -1.10 | 51 | 14 | -1.05 | 15 | 42 | -1.02 | 58 | 30 | -0.52 | 35 | 159.33 | 77.70 |
P3 | 50 | -0.86 | 56 | 12 | -1.21 | 14 | 42 | -1.02 | 58 | 25 | -0.70 | 33 | 160.87 | 76.60 |
P4 | 50 | -0.86 | 56 | 13 | -1.07 | 15 | 47 | -0.82 | 63 | 27 | -0.58 | 35 | 167.58 | 71.70 |
P5 | 47 | -0.92 | 55 | 10 | -1.50 | 12 | 45 | -0.95 | 60 | 26 | -0.65 | 34 | 160.54 | 76.60 |
P6 | 46 | -1.10 | 51 | 12 | -1.21 | 14 | 45 | -0.95 | 60 | 22 | -0.74 | 33 | 157.68 | 78.30 |
Cut-off score | 160.71 | 76.60 |
Relevance | Range | Mean ± standard deviation | Final |
---|---|---|---|
Fail rate (%) | Maximum | 30.00 ± 3.16 | 20.00 |
Minimum | 11.17 ± 2.04 | ||
Cut score | Maximum | 70.00 ± 7.07 | 62.00 |
Minimum | 52.83 ± 6.01 |
Values are presented as number (%).
OIB, ordered item booklet.