Journal of Educational Evaluation for Health Professions

Research article

Medical students’ thought process while solving problems in 3 different types of clinical assessments in Korea: clinical performance examination, multimedia case-based assessment, and modified essay question: Sejin Kim, Ikseon Choi, Bo Young Yoon, Min Jeong Kwon, Seok-jin Choi, Sang Hyun Kim, Jong-Tae Lee, Byoung Doo Rhee; J Educ Eval Health Prof. 2019;16:10. Published online May 9, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.10

17,434 View
291 Download
1 Web of Science
4 Crossref

Purpose
This study aimed to explore students’ cognitive patterns while solving clinical problems in 3 different types of assessments—clinical performance examination (CPX), multimedia case-based assessment (CBA), and modified essay question (MEQ)—and thereby to understand how different types of assessments stimulate different patterns of thinking.
Methods
A total of 6 test-performance cases from 2 fourth-year medical students were used in this cross-case study. Data were collected through one-on-one interviews using a stimulated recall protocol where students were shown videos of themselves taking each assessment and asked to elaborate on what they were thinking. The unit of analysis was the smallest phrases or sentences in the participants’ narratives that represented meaningful cognitive occurrences. The narrative data were reorganized chronologically and then analyzed according to the hypothetico-deductive reasoning framework for clinical reasoning.
Results
Both participants demonstrated similar proportional frequencies of clinical reasoning patterns on the same clinical assessments. The results also revealed that the three different assessment types may stimulate different patterns of clinical reasoning. For example, the CPX strongly promoted the participants’ reasoning related to inquiry strategy, while the MEQ strongly promoted hypothesis generation. Similarly, data analysis and synthesis by the participants were more strongly stimulated by the CBA than by the other assessment types.
Conclusion
This study found that different assessment designs stimulated different patterns of thinking during problem-solving. This finding can contribute to the search for ways to improve current clinical assessments. Importantly, the research method used in this study can be utilized as an alternative way to examine the validity of clinical assessments.

Citations

Citations to this article as recorded by

Evaluation of modified essay questions (MEQs) as an assessment tool in third-year medical students’ modular summative assessment
Ayman Elsamanoudy, Mohamed Shehata, Amer Almarabheh, Zienab Alrefaie
BMC Medical Education.2024;[Epub] CrossRef
Future directions of online learning environment design at medical schools: a transition towards a post-pandemic context
Sejin Kim
Kosin Medical Journal.2023; 38(1): 12. CrossRef
Clinical Reasoning Training based on the analysis of clinical case using a virtual environment
Sandra Elena Lisperguer Soto, María Soledad Calvo, Gabriela Paz Urrejola Contreras, Miguel Ángel Pérez Lizama
Educación Médica.2021; 22(3): 139. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Research Articles

Learning Style Scales: a valid and reliable questionnaire: Abdolghani Abdollahimohammad, Rogayah Ja’afar; J Educ Eval Health Prof. 2014;11:22. Published online August 18, 2014; DOI: https://doi.org/10.3352/jeehp.2014.11.22

31,144 View
353 Download
7 Web of Science
7 Crossref

Abstract PDF

Purpose
Learning-style instruments assist students in developing their own learning strategies and outcomes, in eliminating learning barriers, and in acknowledging peer diversity. Only a few psychometrically validated learning-style instruments are available. This study aimed to develop a valid and reliable learning-style instrument for nursing students. Methods: A cross-sectional survey study was conducted in two nursing schools in two countries. A purposive sample of 156 undergraduate nursing students participated in the study. Face and content validity was obtained from an expert panel. The LSS construct was established using principal axis factoring (PAF) with oblimin rotation, a scree plot test, and parallel analysis (PA). The reliability of LSS was tested using Cronbach’s α, corrected item-total correlation, and test-retest. Results: Factor analysis revealed five components, confirmed by PA and a relatively clear curve on the scree plot. Component strength and interpretability were also confirmed. The factors were labeled as perceptive, solitary, analytic, competitive, and imaginative learning styles. Cronbach’s α was > 0.70 for all subscales in both study populations. The corrected item-total correlations were > 0.30 for the items in each component. Conclusion: The LSS is a valid and reliable inventory for evaluating learning style preferences in nursing students in various multicultural environments.

Citations

Citations to this article as recorded by

The Fallacy of Learning Styles in Medical Education: An Evidence-Based Critique
Saurabh RamBihariLal Shrivastava, Prateek Saurabh Shrivastava
Archives of Medicine and Health Sciences.2024; 12(2): 278. CrossRef
Self‐directed learning readiness and learning styles among Omani nursing students: Implications for online learning during the COVID‐19 pandemic
Cherry Ann C. Ballad, Leodoro Jabien Labrague, Arcalyd Rose R. Cayaban, Oscar M. Turingan, Siham Mahmoud Al Balushi
Nursing Forum.2022; 57(1): 94. CrossRef
Development of a novel, theoretically motivated scale to assess cognitive learning styles related to the autism spectrum
Seyed Mohammad Mahdi Moshirian Farahi, Craig Leth-Steensen
BMC Psychiatry.2022;[Epub] CrossRef
An Examination of Pre-Service Teachers’ Interpersonal Dispositions in the Readiness Assurance Stage of Team-Based Learning
Shenghua Zha, Pamela Moore, Brandon Browning, Jennifer Fetner, Yolany Lagos Ortiz
Action in Teacher Education.2021; 43(1): 4. CrossRef
Instruments to evaluate undergraduate healthcare student learning styles globally: A scoping review
Daniel Gonçalves Campos, Juliany Lino Gomes Silva, Melissa Jarvill, Roberta Cunha M. Rodrigues, Ana Railka de Souza Oliveira Kumakura, Daniel Gonçalves Campos
Nurse Education Today.2021; 107: 105141. CrossRef
Associations of learning style with cultural values and demographics in nursing students in Iran and Malaysia
Abdolghani Abdollahimohammad, Rogayah Ja’afar
Journal of Educational Evaluation for Health Professions.2015; 12: 42. CrossRef
Learning style preferences of nursing students at two universities in Iran and Malaysia
Abdolghani Abdollahimohammad, Rogayah Ja’afar
Journal of Educational Evaluation for Health Professions.2014; 11: 30. CrossRef

Assessing the reliability and validity of the Revised Two Factor Study Process Questionnaire (R-SPQ2F) in Ghanaian medical students: Victor Mogre, Anthony Amalba; J Educ Eval Health Prof. 2014;11:19. Published online August 15, 2014; DOI: https://doi.org/10.3352/jeehp.2014.11.19

27,581 View
215 Download
9 Web of Science
11 Crossref

Abstract PDF

Purpose
We investigated the validity and reliability of the Revised Two Factor Study Process Questionnaire (R-SPQ2F) in preclinical students in Ghana. Methods: The R-SPQ2F was administered to 189 preclinical students of the University for Development Studies, School of Medicine and Health Sciences. Both descriptive and inferential statistics with Cronbach’s alpha test and factor analysis were done. Results: The mean age of the students was 22.69 ± 0.18 years, 60.8% (n = 115) were males and 42.3% (n = 80) were in their second year of medical training. The students had higher mean deep approach scores (31.23 ± 7.19) than that of surface approach scores (22.62 ± 6.48). Findings of the R-SPQ2F gave credence to a solution of two-factors indicating deep and surface approaches accounting for 49.80% and 33.57%, respectively, of the variance. The scales of deep approach (Cronbach’s alpha, 0.80) and surface approach (Cronbach’s alpha, 0.76) and their subscales demonstrated an internal consistency that was good. The factorial validity was comparable to other studies. Conclusion: Our study confirms the construct validity and internal consistency of the R-SPQ2F for measuring approaches to learning in Ghanaian preclinical students. Deep approach was the most dominant learning approach among the students. The questionnaire can be used to measure students’ approaches to learning in Ghana and in other African countries.

Citations

Citations to this article as recorded by

A comparison of two learning approach inventories and their utility in predicting examination performance and study habits
Andrew R. Thompson
Advances in Physiology Education.2024; 48(2): 164. CrossRef
Learning approaches in students of the areas of Health Sciences belonging to a regional university in Chile
Mónica Cardona-V, Aurora Henríquez-Melgarejo, Trinidad Cifuentes Muñoz, Carolina Luengo, Ángel Roco-Videla
Salud, Ciencia y Tecnología.2024;[Epub] CrossRef
Relationship between learning approach, Bloom’s taxonomy, and student performance in an undergraduate Human Anatomy course
Andrew R. Thompson, Logan P. O. Lake
Advances in Health Sciences Education.2023; 28(4): 1115. CrossRef
Evaluation of Construct Validity and Reliability of the Arabic and English Versions of Biggs Study Process Scale Among Saudi University Students
Nadeem Shafique Butt, Muhammad Abid Bashir, Sami Hamdan Alzahrani, Zohair Jamil Gazzaz, Ahmad Azam Malik
Sage Open.2023;[Epub] CrossRef
Development and Preliminary Validation of the Physical Education-Study Process Questionnaire : Insights for Physical Education University Students
Amayra Tannoubi, Noomen Guelmami, Tore Bonsaksen, Nasr Chalghaf, Fairouz Azaiez, Nicola Luigi Bragazzi
Frontiers in Public Health.2022;[Epub] CrossRef
The Association Between Learning Styles, Time Management Skills and Pharmacology Academic Performance Among First Year Medical Students in Universiti Putra Malaysia (UPM) During the COVID-19 Pandemic
Azmah Sa'at, Suryati Mohd. Thani, Safuraa Salihan, Nur Izah Ab. Razak, Siti Saleha Masrudin
Malaysian Journal of Medicine and Health Sciences.2022; 18(s14): 94. CrossRef
Outcomes of 2 Multimodal Human Anatomy Courses Among Doctor of Physical Therapy Students (Entry-Level): A Quasi-experimental Study
Sara F. Maher, Deborah J. Doherty
Journal of Physical Therapy Education.2021; 35(1): 38. CrossRef
Study Approaches of Life Science Students Using the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F)
Miguel Leiva-Brondo, Jaime Cebolla-Cornejo, Rosa Peiró, Nuria Andrés-Colás, Cristina Esteras, María Ferriol, Hugo Merle, María José Díez, Ana Pérez-de-Castro
Education Sciences.2020; 10(7): 173. CrossRef
Assessing ‘approaches to learning’ in Botswana, Ghana and Kenya
Caine Rolleston, Rebecca Schendel, Ana M Grijalva Espinosa
Research in Comparative and International Education.2019; 14(1): 118. CrossRef
Psychometric properties of the revised two-factor study process questionnaire r-spq-2f - spanish version
Clara Vergara-Hernández, Miguel Simancas-Pallares, Zoila Carbonell-Muñoz
Duazary.2019; 16(2): 205. CrossRef
Enfoques de Aprendizaje según el R-SPQ-2F: Análisis de sus propiedades psicométricas en estudiantes universitarios de Buenos Aires
Agustín Freiberg Hoffmann, María Mercedes Fernández Liporace
Revista Colombiana de Psicología.2016;[Epub] CrossRef

Revised Subjects of the Current Korean Oriental Pharmacists' Licensing Examination: Jong-Pil Lim, Seon-Pyo Hong, Young-Mi Lee, Hoon Jeon; J Educ Eval Health Prof. 2007;4:4. Published online December 20, 2007; DOI: https://doi.org/10.3352/jeehp.2007.4.4

29,241 View
156 Download

Abstract PDF: This study is designed to draw out new integrated subjects of the Korean Oriental Pharmacists??Licensing Examination (KOPLE). In 2004, for the revision of subjects, we have analyzed the curriculums of the Oriental Pharmacy department, the oriental pharmacist?占퐏 (OP?占퐏) job description book, and the elementary items of KOPLE. We also examined the system of the Chinese Herb Pharmacists??Examination and other health personnel licensing examinations and studied the data of items and compared them with KOPLE. We heard the public opinion on the present KOPLE. We developed a subfield of 18 subjects, a middle category of 188 items, and a small category of 1,026 items. We proposed a new KOPLE that consists of three subjects: basic oriental pharmacy, applied oriental pharmacy, and laws and regulations.

Original Articles

Construct Validity of the Korean Dental Licensing Examination using Confirmatory Factor Analysis: Mi Kyoung Yim, Yoon Hee Kim; J Educ Eval Health Prof. 2005;2(1):75-86. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.75

35,131 View
164 Download

Abstract PDF: Confirmatory factor analysis based on a measurement model of a structural equation model was used to test the construct validity of 13 subjects in the Korean Dental Licensing Examination (KDLE). The results of 1,086 examinees who wrote the KDLE in 2004 were analyzed. The thirteen subjects were classified into 62 major categories and 122 intermediate categories. There were 364 items. A hierarchical model was constructed, including major and intermediate categories. The impact of the variables was determined by the standardized regression coefficient that related latent and measured variables in the measurement model. The KDLE showed a high goodness-of-fit with a root mean square error of approximation of 0.030 and a non-normed fit index of 0.998. When the latent variables for the major and intermediate categories were analyzed, the standardized regression coefficients of all of the subjects, with the exception of Health and Medical Legislation, were significant. From the result, we concluded that the 13 subjects showed constructive validity. In addition, the study model and data were very compatible. The subject Health and Medical Legislation had a low explanatory impact with respect to testing the ability of dentists to perform their jobs. This study suggests that similar psychometric studies are needed before integrating or deleting subjects on the KDLE, and to improve item development.

The Validity of Subjects in Korean Dental Technicians' Licensing Examination: Woong-chul Kim, Won-chul Lee, Young-seok Sohn, Sae-yoon Oh, Bu-sob Kim, Chin-ho Yu, Ji-hwan Kim; J Educ Eval Health Prof. 2005;2(1):43-57. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.43

27,277 View
158 Download
4 Crossref

Abstract PDF

This study prepared a basic framework for the development and improvement of Korean Dental Technicians' Licensing Examination, based on actual test questions. A peer review was conducted to ensure relevance to current practices in dental technology. For the statistical analysis, 1000 dental laboratory technicians were selected; specialists in dental laboratory technology (laboratory owners, educators, etc.) were involved in creating valid and reliable questions. Results indicated that examination subjects should be divided into three categories: basic dental laboratory theory, dental laboratory specialties, and a practical examination. To ensure relevance to current practice, there should be less emphasis on basic dental laboratory theory, including health-related laws, and more emphasis on dental laboratory specialties. Introduction to dental anatomy should be separated from oral anatomy and tooth morphology; and fixed prosthodontics should be separated from crown and bridge technology and dental ceramics technology. Removable orthodontic appliance technology should be renamed 'orthodontic laboratory technology'. There should be less questions related to health related law, oral anatomy, dental hygiene, dental materials science and inlay, while the distribution ratio of questions related to tooth morphology should be maintained. There should be a decrease in the distribution ratio of questions related to crown and bridge technology, dental ceramics technology, complete dentures and removable partial dentures technology, and orthodontic laboratory technology. In the practical examination, the current multiple choice test should be replaced with tooth carving using wax or plaster. In dental laboratory specialties, subjects related to contemporary dental laboratory technology should be included in the test items.

Citations

Citations to this article as recorded by

A study of the opening status of the compulsory courses for the dental technologist license
Soon-Suk Kwon
Journal of Korean Acedemy of Dental Technology.2021; 43(3): 106. CrossRef
A study on the reorganization of the curriculum for produce of field-oriented dental technicians
Kim Jae-Hong, Kim Ki-Baek, Kim Won-Soo
Journal of Korean Acedemy of Dental Technology.2019; 41(4): 359. CrossRef
The Study on the improvement of dental technician practical examination II
Eun-Jeong Bae, Ki-Baek Kim, Woong-Chul Kim, Ji-Hwan Kim, Chin-Ho Yu, Gyu-Sun Lee
Journal of Korean Acedemy of Dental Technology.2014; 36(1): 39. CrossRef
The Study on the improvement of dental technician practical examination I
Eun-Jeong Bae, Woong-Chul Kim, In-Sung Chung, Sang-Yong Nam, Ji-Hwan Kim
Journal of Korean Acedemy of Dental Technology.2012; 34(4): 413. CrossRef

A study of the validity of the Korean Nurses' Licensing Examination: Hyang Yeon Lee, Cho Ja Kim, Sook Ja Lee, Ho Ran Park, In Sook Lee, Hoo Ja Kim, Young Mi Park; J Educ Eval Health Prof. 2005;2(1):31-42. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.31

33,085 View
192 Download
5 Crossref

Abstract PDF

This study tested the validity of subjects in the Korean Nurses' Licensing Examination (KNLE). To determine the validity of test items in the KNLE, the items testing each subject in the examination and all of the test items were compared. The homogeneity and proper degree of conceptual diversity of the items in the examination were tested by comparing the frameworks of the test items in the National Council Licensure Examination for Registered Nurses (NCLEX-RN) and the KNLE. The validity of the KNLE was determined by examining the correlations between the scores that the same individuals obtained on the KNLE in 2002 and their college marks. This study showed that basic core items appeared repeatedly in the KNLE, and items in the examination were selected according to the importance of each subject examined. Therefore, a new test system based on united subjects should be considered to solve these problems. Further studies are needed to develop a framework for classifying the test items that should be included as united subjects through considering the job description and goals of study of nurses.

Citations

Citations to this article as recorded by

Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
In Sook Park, Yeon Ok Suh, Hae Sook Park, So Young Kang, Kwang Sung Kim, Gyung Hee Kim, Yeon-Hee Choi, Hyun-Ju Kim
Journal of Educational Evaluation for Health Professions.2017; 14: 20. CrossRef
Relevance of the test content of the Korean Nursing Licensing Examination to nursing job
In Sook Park, Yeon Ok Suh, Hae Sook Park, Soo Yeon Ahn, So Young Kang, Kwang Sung Kim
Journal of Educational Evaluation for Health Professions.2016; 13: 23. CrossRef
The Relevance between Pathophysiological Subject and Examination Workbook Items for National Nurse Licensure Examination in South Korea and the United States
Myung Sook Park, Hee Jung Choi, Youn Jung Kim, Hee Kyung Chang, Sun Ju Chang, Haeyoung Lee
Journal of Korean Biological Nursing Science.2016; 18(4): 264. CrossRef
A Six-Year Study of Relationship between Academic Performance in Dental Hygiene School and Performance on the Korean Dental Hygiene Licensing Examination at Yonsei University
So-Jung Mun, Hie-Jin Noh, Hyun-Sun Jeon, Ji-Eun Heo, Won-Gyun Chung
Journal of dental hygiene science.2014; 14(3): 332. CrossRef
A Comparative Study of the Nurse Licensure Exam Systems between the United States and South Korea
Chanyeong Kwak, Soon-Nyoung Yun, Shin-Jeong Kim
Korean Journal of Adult Nursing.2013; 25(6): 622. CrossRef

Review Article

A Pychometric Approach to Setting a Passing Score on Korean National Medical Licensing Examination: Guemin Lee; J Educ Eval Health Prof. 2004;1(1):5-14. Published online January 31, 2004; DOI: https://doi.org/10.3352/jeehp.2004.1.1.5

30,883 View
150 Download
5 Crossref

Abstract PDF

National Health Personnel Licensing Examination Board (hereafter NHPLEB) has used 60% correct responses of overall tests and 40% correct responses of each subject area test as a criterion to give physician licenses to satisfactory candidates. The 60%-40% criterion seems reasonable to laypersons without pychometric or measurement knowledge, but it may causes several severe problems on pychometrician's perspective. This paper pointed out several problematic cases that can be encountered by using the 60%-40% criterion, and provided several pychometric alternatives that could overcome these problems. A fairly new approach, named Bookmark standard setting method, was introduced and explained in detail as an example. This paper concluded with five considerations when the NHPLEB decides to adopt a pychometric standard setting approach to set a cutscore for a licensure test like medical licensing examination.

Citations

Citations to this article as recorded by

Analysis on Validity and Academic Competency of Mock Test for Korean Medicine National Licensing Examination Using Item Response Theory
Han Chae, Eunbyul Cho, SeonKyoung Kim, DaHye Choi, Seul Lee
Keimyung Medical Journal.2023; 42(1): 7. CrossRef
Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination
Mi Kyoung Yim, Sujin Shin
Journal of Educational Evaluation for Health Professions.2020; 17: 14. CrossRef
Comparison of results between modified-Angoff and bookmark methods for estimating cut score of the Korean medical licensing examination
Mikyoung Yim
Korean Journal of Medical Education.2018; 30(4): 347. CrossRef
Presidential address: launching the Korea Health Personnel Licensing Examination Institute, a government-supported special foundation from December 23, 2015
Chang Hwi Kim
Journal of Educational Evaluation for Health Professions.2016; 13: 20. CrossRef
Reconsidering the Cut Score of Korean National Medical Licensing Examination
Duck Sun Ahn, Sowon Ahn
Journal of Educational Evaluation for Health Professions.2007; 4: 1. CrossRef

First
Prev
Page of 1
Next
Last

Search