Skip Navigation
Skip to contents

JEEHP : Journal of Educational Evaluation for Health Professions

OPEN ACCESS
SEARCH
Search

Articles

Page Path
HOME > J Educ Eval Health Prof > Volume 12; 2015 > Article
Technical Report
Reforms of the Korean Medical Licensing Examination regarding item development and performance evaluation
Mi Kyoung Yim*orcid

DOI: https://doi.org/10.3352/jeehp.2015.12.6
Published online: March 17, 2015

National Health Personnel Licensing Examination Board, Seoul, Korea

*Corresponding email: mkyim@kuksiwon.or.kr

: 

• Received: October 14, 2014   • Accepted: March 13, 2015

©2015, National Health Personnel Licensing Examination Board of the Republic of Korea.

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

  • 36,195 Views
  • 189 Download
  • 5 Web of Science
  • 6 Crossref
  • Purpose:
    The Korean Medical Licensing Examination (KMLE) has undergone a variety of innovative reforms implemented by the National Health Personnel Licensing Examination Board (NHPLEB) in order to make it a competency-based test. The purpose of this article is to describe the ways in which the KMLE has been reformed and the effect of those innovations on medical education in Korea.
  • Methods:
    Changes in the KMLE were traced from 1994 to 2014 by reviewing the adoption of new policies by the NHPLEB and the relevant literature.
  • Results:
    The most important reforms that turned the examination into a competency-based test were the following: First, the subjects tested on the exam were revised; second, R-type items were introduced; third, the proportion of items involving problem-solving skills was increased; and fourth, a clinical skills test was introduced in addition to the written test. The literature shows that the above reforms have resulted in more rigorous licensure standards and have improved the educational environment of medical schools in Korea.
  • Conclusion:
    The reforms of the KMLE have led to improvements in how the competency of examinees is evaluated, as well as improvements in the educational system in medical schools in Korea.
The purpose of the Korean Medical Licensing Examination (KMLE) is to determine if examinees have sufficient knowledge and skills to be a primary care physician, as well as a proper clinical demeanor; therefore, its items must draw on these three domains. Before 1994, the KMLE was a written test with an abundance of items evaluating factual knowledge. The National Health Personnel Licensing Examination Board (NHPLEB) reformed the written test and incorporated a clinical skills test, in order to make the KMLE a competency-based test. The KMLE currently consists of a written test and a clinical skills test. Medical school graduates must pass both components of the test to receive medical licenses in Korea. This paper aims to summarize these reforms of the KMLE and to assess their effect on the qualification rate of examinees and the environment of medical education in Korea.
Reforms to the KMLE were reviewed, along with the corresponding results. A particular focus was placed on assessing what was changed, what each change achieved, and how the educational environment of medical schools in Korea was affected.
Several innovations transformed the KMLE into a competency-based test.
First, the subjects tested on the exam were reformed through job analysis. Before 1994, the subjects were based on the curricula of medical schools. There were 15 subjects: internal medicine, surgery, obstetrics and gynecology, pediatrics, neurology, psychiatry, dermatology, urology, ophthalmology, otorhinolaryngology, radiology, laboratory medicine, anesthesiology, preventive medicine, and medical law. In 1995, the examination subjects were reduced to the following core medical areas: internal medicine, surgery, pediatrics, obstetrics and gynecology, psychiatry, preventive medicine, and medical law [1]. In 2002, a second reform reduced the number of subjects to three: general medicine, specialized medicine, and medical law.
Second, extended matching (R-type) items were developed and introduced to the KMLE. The total number of test items was increased to 550, but subsequently was gradually decreased; however, the proportion of problem-solving items and R-type items continuously increased [2]. The number of multiple true-false (K-type) items decreased over time, and K-type questions were ultimately eliminated from the exam in 2010, while the proportion of R-type items increased to 13% in 2013 from 5% in 2002 (Fig. 1).
Third, problem-solving items were continuously developed and introduced. The number of recall items decreased from 26% in 2002 to 12% in 2013, and the number of interpretation items decreased from 48% in 2002 to 26% in 2013. In contrast, the number of problem-solving items increased from 25% in 2002 to 62% in 2013 (Fig. 2). These reforms of the written test involved an increase in items dealing with clinical cases [3].
Fourth, a clinical skills test was introduced in 2009 alongside the written test [4]. The clinical skills test consists of six objective structured clinical examination stations where examinees perform the indicated tasks for five minutes in each station, and another six clinical performance examination stations, where examinees encounter standardized patients for ten minutes in each station. The clinical skills test evaluates the clinical competence and demeanor of each examinee. The KMLE was the first licensing examination in Asia to incorporate a clinical skills test. The evaluation sheet of the clinical skills test contains five categories: history taking, physical examination, physician and patient interaction, attitude toward patients, and basic technical skills.
As a result of introducing the clinical skills test into the KMLE, the clinical performance abilities of medical graduates improved [5,6]. A survey indicated that adopting the clinical skills test in the KMLE improved the quality of medical school education (Table 1) [5]. A high score on the written test does not always guarantee a satisfactory score on the clinical skills test, as an analysis of examination results has shown that some examinees who received high scores on the written test failed the clinical skills test [7]. Some examinees who failed the clinical skills test in 2013 also failed the written test, with a mean score of 213 (range, 123 to 239; SD=23.5). However, some examinees who failed the clinical skills test passed the written test, with a mean score of 274 (range, 240 to 330; SD=20.4). The cutoff score for passing the written test in 2013 was 240.
Consolidating the examination subjects made it possible to evaluate the overall medical competency of examinees, since patients’ complaints and problems are not classified by specialization during presentation to a primary care physician. The inclusion of a greater proportion of problem-solving items reflects the practical situations that medical graduates face in the field. The introduction of more items based on case studies helps measure the ability of examinees to cope with real-world situations. The discriminating index of R-type items is higher than that of other types; therefore, the proportion of R-type items will increase to 15% [3]. Introducing the clinical skills test to the KMLE affected the educational environment of medical schools in Korea. Most schools have been equipped with centers to develop clinical skills, and have introduced objective structured clinical examinations and clinical performance examinations to test the clinical competence of their students. The KMLE allows each examinee’s abilities to be examined in a variety of domains.
The NHPLEB has improved the KMLE in a step-by-step manner, with cooperation from the Korean medical community. Innovative reforms have resulted in more rigorous licensure standards. The following challenges and reforms are currently being considered: first, the written test will be replaced by a computer-based test beginning in 2020 [8]; second, a scientific standard-setting method must be considered with regard to educational measurement; third, items that deal with medical ethics may be included as essential items in the test; fourth, a hybrid item combining objective structured clinical examinations and clinical performance examinations may be created; and fifth, the subject of general medicine will be removed in 2016, after which general and specialized medicine will be consolidated into a single topic dealing with medicine.
In conclusion, the reforms of the KMLE have succeeded in improving the evaluation of examinees’ competency and the educational system in medical schools in Korea. Although reforms are still in progress, it is to be hoped that the KMLE may become a model for other countries.

No potential conflict of interest relevant to this article was reported.

Audio recording of the abstract.
jeehp-12-06-abstract-recording.avi
Fig. 1.
Chronological change of the percentage of items of a given type in the Korean Medical Licensing Examination. A-type, one best answer type; K-type, multiple true-false type; R-type, extended matching type.
jeehp-12-06f1.gif
Fig. 2.
Chronological change of the percentage of items testing a given level of knowledge in the Korean Medical Licensing Examination.
jeehp-12-06f2.gif
Table 1.
Changes in educational facilities and curricular content in 41 medical schools after the 2009 introduction of the clinical skills test into the KMLE
Facility & curricular content Before 2009 After 2009
Clinical skills training center 27 (65.9) 40 (97.6)
Medical interviewing skills 30 (73.2) 40 (97.6)
Physical examination course 37 (90.2) 41 (100.0)
Introduction to clinical medicine 35 (85.4) 41 (100.0)
Standardized patient program for proving feedback 19 (46.3) 35 (85.4)
Inpatient practice clerkships 38 (92.7) 41 (100.0)
Outpatient practice clerkships 32 (78.0) 38 (92.7)
Writing medical records 33 (80.5) 38 (92.7)
Medical ethics 36 (87.8) 41 (100.0)
Clinical performance examinations 31 (75.6) 41 (100.0)
Objective structured clinical examination to test simple procedural skills 32 (78.0) 41 (100.0)
Special preparatory course for the clinical skills test of the KMLE 16 (39.0) 40 (97.6)

Values are presented as number (%).

KMLE, Korean Medical Licensing Examination.

Reproduced from Park HK. J Korean Med Assoc. 2012;55:116-123, with permission of Korean Medical Association [5].

  • 1. National Health Personnel Licensing Examination Board. Twenty years history of National Health Personnel Licensing Examination. Seoul: National Health Personnel Licensing Examination Board; 2012.
  • 2. National Health Personnel Licensing Examination Board. Statistics annual report. Seoul: National Health Personnel Licensing Examination Board; 2013.
  • 3. Yim MK, Huh S. Usability of extended-matching type items in the Korean Medical Licensing Examinations (2002, 2003). Korean J Med Educ 2004;16:219-226. http://dx.doi.org/10.3946/kjme.2004.16.2.219 ArticlePDF
  • 4. Kim KS. Introduction and administration of the clinical skill test of the medical licensing examination, Republic of Korea (2009). J Educ Eval Health Prof 2010;7:4. http://dx.doi.org/10.3352/jeehp.2010.7.4 ArticlePubMedPMCPDF
  • 5. Park HK. The impact of introducing the Korean Medical Licensing Examination clinical skills assessment on medical education. J Korean Med Assoc 2012;55:116-123. http://dx.doi.org/10.5124/jkma.2012.55.2.116 Article
  • 6. Kim SM, Park I, Chang HS, Park EC. Improvement in clinical performance of interns and residents through clinical skills assessment of the Korean Medical Licensing Examination. Korean J Med Educ 2012;24:329-338. http://dx.doi.org/10.3946/kjme.2012.24.4.329 ArticlePubMedPMCPDF
  • 7. Hwang JY, Jeong HS. Relationship between the content of the medical knowledge written examination and clinical skill score in medical students. Korean J Med Educ 2011;23:305-314. http://dx.doi.org/10.3946/kjme.2011.23.4.305 ArticlePubMedPMCPDF
  • 8. Chung MH. President’s address: research and development on computer-based testing and extension of clinical skill examination to a variety of fields. J Educ Eval Health Prof 2013;10:1. http://dx.doi.org/10.3352/jeehp.2013.10.1 ArticlePubMedPMCPDF

Figure & Data

References

    Citations

    Citations to this article as recorded by  
    • A Survey on Perceptions of the Direction of Korean Medicine Education and National Licensing Examination
      Han-Byul Cho, Won-Suk Sung, Jiseong Hong, Yeonseok Kang, Eun-Jung Kim
      Healthcare.2023; 11(12): 1685.     CrossRef
    • Impact of anesthetist licensing examination on quality of education in Ethiopia: a qualitative study of faculty and student perceptions
      Yohannes Molla Asemu, Tegbar Yigzaw, Firew Ayalew Desta, Tewodros Abebaw Melese, Leulayehu Akalu Gemeda, Fedde Scheele, Thomas van den Akker
      BMC Medical Education.2023;[Epub]     CrossRef
    • Attitudes to proposed assessment of pharmacy skills in Korean pharmacist licensure examination
      Joo Hee Kim, Ju-Yeun Lee, Young Sook Lee, Chul-Soon Yong, Nayoung Han, Hye Sun Gwak, Jungmi Oh, Byung Koo Lee, Sukhyang Lee
      Journal of Educational Evaluation for Health Professions.2017; 14: 6.     CrossRef
    • Is there an agreement among the items of the Korean physical therapist licensing examination, learning objectives of class subjects, and physical therapists’ job descriptions?
      Min-Hyeok Kang, Oh-Yun Kwon, Yong-Wook Kim, Ji-Won Kim, Tae-Ho Kim, Tae-Young Oh, Jong-Hyuk Weon, Tae-Sik Lee, Jae-Seop Oh
      Journal of Educational Evaluation for Health Professions.2016; 13: 3.     CrossRef
    • The past, present, and future of traditional medicine education in Korea
      Sang Yun Han, Hee Young Kim, Jung Hwa Lim, Jinhong Cheon, Young Kyu Kwon, Hyungwoo Kim, Gi Young Yang, Han Chae
      Integrative Medicine Research.2016; 5(2): 73.     CrossRef
    • Medical students’ satisfaction with the Applied Basic Clinical Seminar with Scenarios for Students, a novel simulation-based learning method in Greece
      Panteleimon Pantelidis, Nikolaos Staikoglou, Georgios Paparoidamis, Christos Drosos, Stefanos Karamaroudis, Athina Samara, Christodoulos Keskinis, Michail Sideris, George Giannakoulas, Georgios Tsoulfas, Asterios Karagiannis
      Journal of Educational Evaluation for Health Professions.2016; 13: 13.     CrossRef

    Figure
    We recommend
    Related articles

    JEEHP : Journal of Educational Evaluation for Health Professions