To evaluate the usefulness of computerized adaptive testing (CAT) in medical school, the General Examination for senior medical students was administered as a paper and pencil test (P&P) and using CAT. The General Examination is a graduate examination, which is also a preliminary examination for the Korean Medical Licensing Examination (KMLE). The correlations between the results of the CAT and P&P and KMLE were analyzed. The correlation between the CAT and P&P was 0.8013 (p=0.000); that between the CAT and P&P was 0.7861 (p=0.000); and that between the CAT and KMLE was 0.6436 (p=0.000). Six out of 12 students with an ability estimate below 0.52 failed the KMLE. The results showed that CAT could replace P&P in medical school. The ability of CAT to predict whether students would pass the KMLE was 0.5 when the criterion of the theta value was set at -0.52 that was chosen arbitrarily for the prediction of pass or failure.
Citations
Citations to this article as recorded by
Analysis on Validity and Academic Competency of Mock Test for Korean Medicine National Licensing Examination Using Item Response Theory Han Chae, Eunbyul Cho, SeonKyoung Kim, DaHye Choi, Seul Lee Keimyung Medical Journal.2023; 42(1): 7. CrossRef
Application of Computerized Adaptive Testing in Medical Education Sun Huh Korean Journal of Medical Education.2009; 21(2): 97. CrossRef
Estimation of an Examinee's Ability in the Web-Based Computerized Adaptive Testing Program IRT-CAT Yoon-Hwan Lee, Jung-Ho Park, In-Yong Park Journal of Educational Evaluation for Health Professions.2006; 3: 4. CrossRef
An examinee's ability can be evaluated precisely using computerized adaptive testing (CAT), which is shorter than written tests and more efficient in terms of the duration of the examination. We used CAT for the second General Examination of 98 senior students in medical college on November 27, 2004. We prepared 1,050 pre-calibrated test items according to item response theory, which had been used for the General Examination administered to senior students in 2003. The computer was programmed to pose questions until the standard error of the ability estimate was smaller than 0.01. To determine the students' attitude toward and evaluation of CAT, we conducted surveys before and after the examination, via the Web. The mean of the students' ability estimates was 0.3513 and its standard deviation was 0.9097 (range -2.4680 to +2.5310). There was no significant difference in the ability estimates according to the responses of students to items concerning their experience with CAT, their ability to use a computer, or their anxiety before and after the examination (p>0.05). Many students were unhappy that they could not recheck their responses (49%), and some stated that there were too few examination items (24%). Of the students, 79 % had no complaints concerning using a computer and 63% wanted to expand the use of CAT. These results indicate that CAT can be implemented in medical schools without causing difficulties for users.
Citations
Citations to this article as recorded by
Computer‐Based Testing and Construction of an Item Bank Database for Medical Education in Korea Sun Huh Korean Medical Education Review.2014; 16(1): 11. CrossRef
Can computerized tests be introduced to the Korean Medical Licensing Examination? Sun Huh Journal of the Korean Medical Association.2012; 55(2): 124. CrossRef
Application of Computerized Adaptive Testing in Medical Education Sun Huh Korean Journal of Medical Education.2009; 21(2): 97. CrossRef
The goal of accreditation is to ensure that the education provided by an institution of higher education meets an acceptable level of quality. This study developed standards and criteria for accreditation of baccalaureate nursing education programs, by comparing accreditation in South Korea and in the United States, and validating standards and criteria. A main comparative analysis was made between Nursing League for Nursing Accrediting Commission (NLNAC) standards, Commission on Collegiate Nursing Education (CCNE) standards for accreditation of baccalaureate nursing education programs, and Korean Accreditation Board of Nursing standards for accreditation of nursing education programs. The research team developed and validated standards and criteria for South Korean baccalaureate nursing education programs. Using the results of the analysis, revisions are proposed to standards for accreditation of baccalaureate nursing education programs, and categorized into 24 criteria and six domains: mission and governance, curriculum and teaching-learning practices, students, faculty, resources, effectiveness. Further studies are required to refine the standards and criteria and make them sophisticated enough to be applied globally.
Citations
Citations to this article as recorded by
Effect of Symptom Severity on the Quality of Life in Women with Urinary Incontinence: A Comparative Study Figen Kazankaya, Sezer Er Güneri The Anatolian Journal of General Medical Research.2024; : 40. CrossRef
Development and Psychometric Testing of a Self-Rated Scale Based on National Nursing Core Competency Standards Al D. Biag, Vanessa L. Belen Journal of Nursing Measurement.2022; 30(1): 75. CrossRef
The Positive Effects of Accreditation on Graduate Medical Education Programs in Singapore Kathleen D. Holt, Rebecca S. Miller, Lauren M. Byrne, Susan H. Day Journal of Graduate Medical Education.2019; 11(4s): 213. CrossRef
Analysis of Survey of Interdisciplinary Unification of Biomedical Laboratory Science Hong Sung Kim, Ji-Hyuk Kang, Man-Gil Yang, Chang-Eun Park, Kyung-A Shin, Pil Seung Kwon The Korean Journal of Clinical Laboratory Science.2018; 50(3): 289. CrossRef
Development and Evaluation of Korean Nurses’ Core Competency Scale (KNCCS) Won-Hee Lee, Sanghee Kim, Jihyoun An Open Journal of Nursing.2017; 07(05): 599. CrossRef
Direction of the Community Health Nursing Practice Education Focusing on the Nursing Education Accreditation Criteria Kyung Ja June, Young Ran Chin, Hee Girl Kim, Chun Mi Kim, Yeon Yi Song, Souk Young Kim, Hanju Lee, Ihn Sook Jeong, Ki Soon Seo, Kyung Won Choi Journal of Korean Academy of Community Health Nursing.2016; 27(2): 183. CrossRef
A study on Korean nursing students' educational outcomes Kasil Oh, Yang Heui Ahn, Hyang-Yeon Lee, Sook-Ja Lee, In-Ja Kim, Kyung-Sook Choi, Myung-Sook Ko Journal of Educational Evaluation for Health Professions.2011; 8: 3. CrossRef
Faculty Perceptions of the Improvement and Effort for the Achievement of Quality in Nursing Education through Accreditation in Baccalaureate Nursing Programs Kon Hee Kim Korean Medical Education Review.2011; 13(2): 59. CrossRef
Confirmatory factor analysis based on a measurement model of a structural equation model was used to test the construct validity of 13 subjects in the Korean Dental Licensing Examination (KDLE). The results of 1,086 examinees who wrote the KDLE in 2004 were analyzed. The thirteen subjects were classified into 62 major categories and 122 intermediate categories. There were 364 items. A hierarchical model was constructed, including major and intermediate categories. The impact of the variables was determined by the standardized regression coefficient that related latent and measured variables in the measurement model. The KDLE showed a high goodness-of-fit with a root mean square error of approximation of 0.030 and a non-normed fit index of 0.998. When the latent variables for the major and intermediate categories were analyzed, the standardized regression coefficients of all of the subjects, with the exception of Health and Medical Legislation, were significant. From the result, we concluded that the 13 subjects showed constructive validity. In addition, the study model and data were very compatible. The subject Health and Medical Legislation had a low explanatory impact with respect to testing the ability of dentists to perform their jobs. This study suggests that similar psychometric studies are needed before integrating or deleting subjects on the KDLE, and to improve item development.
This study aimed to develop standard items for the Korean Dental Hygienists' Licensing Examination; these items were also earmarked for use in developing the curriculum for dental hygienists, and in writing a job description, based on a job analysis using the Developing A Curriculum Method (DACUM). It also aimed to understand the significance and frequency of task elements that dental hygienists perform. Data were collected by means of a mail survey, in the form of self-entry, from a sample of dental hygienists registered with the Korean Dental Hygienists' Association. In all, 260 responses were analyzed. The tasks of dental hygienists were divided into four categories, 93 tasks, and 494 task elements. There were 281 elements (61%) that scored higher than 3.5 in significance, in the 4-scale items, and 480 elements (98%) that scored higher than 3.0. There were 30 elements (6%) that scored higher than 3.5 in frequency, and 140 elements (29%) that scored higher than 3.0 in frequency. Overall, 130 out of 494 elements (27%) scored higher than 3.0 for both significance and frequency. Therefore, those 130 elements should be included as items in the Korean Dental Hygienists' Licensing Examination. The results can also be used for curriculum development and as the basis of a job description for dental hygienists.
Citations
Citations to this article as recorded by
Performance of ChatGPT on the Korean National Examination for Dental Hygienists Soo-Myoung Bae, Hye-Rim Jeon, Gyoung-Nam Kim, Seon-Hui Kwak, Hyo-Jin Lee Journal of Dental Hygiene Science.2024; 24(1): 62. CrossRef
The Dental Hygienists’ Perception of the National Practical Examination Da-Kyung Ko, Sung-Suk Bae Journal of Dental Hygiene Science.2016; 16(6): 488. CrossRef
This study prepared a basic framework for the development and improvement of Korean Dental Technicians' Licensing Examination, based on actual test questions. A peer review was conducted to ensure relevance to current practices in dental technology. For the statistical analysis, 1000 dental laboratory technicians were selected; specialists in dental laboratory technology (laboratory owners, educators, etc.) were involved in creating valid and reliable questions. Results indicated that examination subjects should be divided into three categories: basic dental laboratory theory, dental laboratory specialties, and a practical examination. To ensure relevance to current practice, there should be less emphasis on basic dental laboratory theory, including health-related laws, and more emphasis on dental laboratory specialties. Introduction to dental anatomy should be separated from oral anatomy and tooth morphology; and fixed prosthodontics should be separated from crown and bridge technology and dental ceramics technology. Removable orthodontic appliance technology should be renamed 'orthodontic laboratory technology'. There should be less questions related to health related law, oral anatomy, dental hygiene, dental materials science and inlay, while the distribution ratio of questions related to tooth morphology should be maintained. There should be a decrease in the distribution ratio of questions related to crown and bridge technology, dental ceramics technology, complete dentures and removable partial dentures technology, and orthodontic laboratory technology. In the practical examination, the current multiple choice test should be replaced with tooth carving using wax or plaster. In dental laboratory specialties, subjects related to contemporary dental laboratory technology should be included in the test items.
Citations
Citations to this article as recorded by
A study of the opening status of the compulsory courses for the dental technologist license Soon-Suk Kwon Journal of Korean Acedemy of Dental Technology.2021; 43(3): 106. CrossRef
A study on the reorganization of the curriculum for produce of field-oriented dental technicians Kim Jae-Hong, Kim Ki-Baek, Kim Won-Soo Journal of Korean Acedemy of Dental Technology.2019; 41(4): 359. CrossRef
The Study on the improvement of dental technician practical examination II Eun-Jeong Bae, Ki-Baek Kim, Woong-Chul Kim, Ji-Hwan Kim, Chin-Ho Yu, Gyu-Sun Lee Journal of Korean Acedemy of Dental Technology.2014; 36(1): 39. CrossRef
The Study on the improvement of dental technician practical examination I Eun-Jeong Bae, Woong-Chul Kim, In-Sung Chung, Sang-Yong Nam, Ji-Hwan Kim Journal of Korean Acedemy of Dental Technology.2012; 34(4): 413. CrossRef
This study tested the validity of subjects in the Korean Nurses' Licensing Examination (KNLE). To determine the validity of test items in the KNLE, the items testing each subject in the examination and all of the test items were compared. The homogeneity and proper degree of conceptual diversity of the items in the examination were tested by comparing the frameworks of the test items in the National Council Licensure Examination for Registered Nurses (NCLEX-RN) and the KNLE. The validity of the KNLE was determined by examining the correlations between the scores that the same individuals obtained on the KNLE in 2002 and their college marks. This study showed that basic core items appeared repeatedly in the KNLE, and items in the examination were selected according to the importance of each subject examined. Therefore, a new test system based on united subjects should be considered to solve these problems. Further studies are needed to develop a framework for classifying the test items that should be included as united subjects through considering the job description and goals of study of nurses.
Citations
Citations to this article as recorded by
Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination In Sook Park, Yeon Ok Suh, Hae Sook Park, So Young Kang, Kwang Sung Kim, Gyung Hee Kim, Yeon-Hee Choi, Hyun-Ju Kim Journal of Educational Evaluation for Health Professions.2017; 14: 20. CrossRef
Relevance of the test content of the Korean Nursing Licensing Examination to nursing job In Sook Park, Yeon Ok Suh, Hae Sook Park, Soo Yeon Ahn, So Young Kang, Kwang Sung Kim Journal of Educational Evaluation for Health Professions.2016; 13: 23. CrossRef
The Relevance between Pathophysiological Subject and Examination Workbook Items for National Nurse Licensure Examination in South Korea and the United States Myung Sook Park, Hee Jung Choi, Youn Jung Kim, Hee Kyung Chang, Sun Ju Chang, Haeyoung Lee Journal of Korean Biological Nursing Science.2016; 18(4): 264. CrossRef
A Six-Year Study of Relationship between Academic Performance in Dental Hygiene School and Performance on the Korean Dental Hygiene Licensing Examination at Yonsei University So-Jung Mun, Hie-Jin Noh, Hyun-Sun Jeon, Ji-Eun Heo, Won-Gyun Chung Journal of dental hygiene science.2014; 14(3): 332. CrossRef
A Comparative Study of the Nurse Licensure Exam Systems between the United States and South Korea Chanyeong Kwak, Soon-Nyoung Yun, Shin-Jeong Kim Korean Journal of Adult Nursing.2013; 25(6): 622. CrossRef
To test the applicability of item response theory (IRT) to the Korean Nurses' Licensing Examination (KNLE), item analysis was performed after testing the unidimensionality and goodness-of-fit. The results were compared with those based on classical test theory. The results of the 330-item KNLE administered to 12,024 examinees in January 2004 were analyzed. Unidimensionality was tested using DETECT and the goodness-of-fit was tested using WINSTEPS for the Rasch model and Bilog-MG for the two-parameter logistic model. Item analysis and ability estimation were done using WINSTEPS. Using DETECT, Dmax ranged from 0.1 to 0.23 for each subject. The mean square value of the infit and outfit values of all items using WINSTEPS ranged from 0.1 to 1.5, except for one item in pediatric nursing, which scored 1.53. Of the 330 items, 218 (42.7%) were misfit using the two-parameter logistic model of Bilog-MG. The correlation coefficients between the difficulty parameter using the Rasch model and the difficulty index from classical test theory ranged from 0.9039 to 0.9699. The correlation between the ability parameter using the Rasch model and the total score from classical test theory ranged from 0.9776 to 0.9984. Therefore, the results of the KNLE fit unidimensionality and goodness-of-fit for the Rasch model. The KNLE should be a good sample for analysis according to the IRT Rasch model, so further research using IRT is possible.
Citations
Citations to this article as recorded by
Item difficulty index, discrimination index, and reliability of the 26 health professions licensing examinations in 2022, Korea: a psychometric study Yoon Hee Kim, Bo Hyun Kim, Joonki Kim, Bokyoung Jung, Sangyoung Bae Journal of Educational Evaluation for Health Professions.2023; 20: 31. CrossRef
Study on the Academic Competency Assessment of Herbology Test using Rasch Model Han Chae, Soo Jin Lee, Chang-ho Han, Young Il Cho, Hyungwoo Kim Journal of Korean Medicine.2022; 43(2): 27. CrossRef
Can computerized tests be introduced to the Korean Medical Licensing Examination? Sun Huh Journal of the Korean Medical Association.2012; 55(2): 124. CrossRef
Over the last two decades, there have been a number of significant changes in the evaluation system in medical education in Korea. One major improvement in this respect has been the listing of learning objectives at medical schools and the construction of a content outline for the Korean Medical Licensing Examination that can be used as a basis of evaluation. Item analysis has become a routine method for obtaining information that often provides valuable feedback concerning test items after the completion of a written test. The use of item response theory in analyzing test items has been spreading in medical schools as a way to evaluate performance tests and computerized adaptive testing. A series of recent studies have documented an upward trend in the adoption of the objective structured clinical examination (OSCE) and clinical practice examination (CPX) for measuring skill and attitude domains, in addition to tests of the knowledge domain. There has been an obvious increase in regional consortiums involving neighboring medical schools that share the planning and administration of the OSCE and CPX; this includes recruiting and training standardized patients. Such consortiums share common activities, such as case development and program evaluation. A short history and the pivotal roles of four organizations that have brought about significant changes in the examination system are discussed briefly.
Citations
Citations to this article as recorded by
Presidential address: Adoption of a clinical skills examination for dental licensing, implementation of computer-based testing for the medical licensing examination, and the 30th anniversary of the Korea Health Personnel Licensing Examination Institute Yoon-Seong Lee Journal of Educational Evaluation for Health Professions.2022; 19: 1. CrossRef
Effectiveness of Medical Education Assessment Consortium Clinical Knowledge Mock Examination (2011‐2016) Sang Yeoup Lee, Yeli Lee, Mi Kyung Kim Korean Medical Education Review.2018; 20(1): 20. CrossRef
Long for wonderful leadership in a new era of the Korean Association of Medical Colleges Young Hwan Lee Korean Journal of Medical Education.2014; 26(3): 163. CrossRef
Major Reforms and Issues of the Medical Licensing Examination Systems in Korea Sang-Ho Baik Korean Medical Education Review.2013; 15(3): 125. CrossRef
A Study on the Feasibility of a National Practical Examination in the Radiologic Technologist Soon-Yong Son, Tae-Hyung Kim, Jung-Whan Min, Dong-Kyoon Han, Sung-Min Ahn Journal of the Korea Academia-Industrial cooperation Society.2011; 12(5): 2149. CrossRef
The Relationship between Senior Year Examinations at a Medical School and the Korean Medical Licensing Examination Ki Hoon Jung, Ho Keun Jung, Kwan Lee Korean Journal of Medical Education.2009; 21(1): 17. CrossRef
What Qualities Do Medical School Applicants Need to Have? - Secondary Publication Yera Hur, Sun Kim Yonsei Medical Journal.2009; 50(3): 427. CrossRef