Post-hoc simulation study to adopt a computerized adaptive testing (CAT) for a Korean Medical License Examination

Seo, Dong Gi; Choi, Jeongwook

doi:2018.15.0.14

J Educ Eval Health Prof > Epub ahead of print

Research article

J Educ Eval Health Prof. 2018; 15: 14.
Published online May 17, 2018.
DOI: https://doi.org/10.3352/jeehp.2018.15.14
[Epub ahead of print]

Post-hoc simulation study to adopt a computerized adaptive testing (CAT) for a Korean Medical License Examination

Dong Gi Seo

, Jeongwook Choi

Department of Psychology, College of Social Science, Hallym University, Chuncheon, Korea

Correspondence Dong Gi Seo ,Email: wmotive@hallym.ac.kr

Editor: Sun Huh, Hallym University, Korea

Submitted: April 17, 2018 Accepted after revision: May 17, 2018

Abstract

Purpose:
Computerized adaptive testing (CAT) has been adopted in license examinations due to a test efficiency and accuracy. Many research about CAT have been published to prove the efficiency and accuracy of measurement. This simulation study investigated scoring method and item selection methods to implement CAT in Korean medical license examination (KMLE).
Method:
This study used post-hoc (real data) simulation design. The item bank used in this study was designed with all items in a 2017 KMLE. All CAT algorithms for this study were implemented by a 'catR' package in R program.
Results:
In terms of accuracy, Rasch and 2parametric logistic (PL) model performed better than 3PL model. Modal a Posteriori (MAP) or Expected a Posterior (EAP) provided more accurate estimates than MLE and WLE. Furthermore Maximum posterior weighted information (MPWI) or Minimum expected posterior variance (MEPV) performed better than other item selection methods. In terms of efficiency, Rasch model was recommended to reduce test length.
Conclusion:
Simulation study should be performed under varied test conditions before adopting a live CAT. Based on a simulation study, specific scoring and item selection methods should be predetermined before implementing a live CAT.

Keywords: Algorithms; Computers; Korea; Logistic models; Research design