Journal of Educational Evaluation for Health Professions

Reviews

Application of artificial intelligence chatbots, including ChatGPT, in education, scholarly work, programming, and content generation and its prospects: a narrative review: Tae Won Kim; J Educ Eval Health Prof. 2023;20:38. Published online December 27, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.38

12,423 View
1,171 Download
18 Web of Science
19 Crossref

This study aims to explore ChatGPT’s (GPT-3.5 version) functionalities, including reinforcement learning, diverse applications, and limitations. ChatGPT is an artificial intelligence (AI) chatbot powered by OpenAI’s Generative Pre-trained Transformer (GPT) model. The chatbot’s applications span education, programming, content generation, and more, demonstrating its versatility. ChatGPT can improve education by creating assignments and offering personalized feedback, as shown by its notable performance in medical exams and the United States Medical Licensing Exam. However, concerns include plagiarism, reliability, and educational disparities. It aids in various research tasks, from design to writing, and has shown proficiency in summarizing and suggesting titles. Its use in scientific writing and language translation is promising, but professional oversight is needed for accuracy and originality. It assists in programming tasks like writing code, debugging, and guiding installation and updates. It offers diverse applications, from cheering up individuals to generating creative content like essays, news articles, and business plans. Unlike search engines, ChatGPT provides interactive, generative responses and understands context, making it more akin to human conversation, in contrast to conventional search engines’ keyword-based, non-interactive nature. ChatGPT has limitations, such as potential bias, dependence on outdated data, and revenue generation challenges. Nonetheless, ChatGPT is considered to be a transformative AI tool poised to redefine the future of generative technology. In conclusion, advancements in AI, such as ChatGPT, are altering how knowledge is acquired and applied, marking a shift from search engines to creativity engines. This transformation highlights the increasing importance of AI literacy and the ability to effectively utilize AI in various domains of life.

Citations

Citations to this article as recorded by

The Development and Validation of an Artificial Intelligence Chatbot Dependence Scale
Xing Zhang, Mingyue Yin, Mingyang Zhang, Zhaoqian Li, Hansen Li
Cyberpsychology, Behavior, and Social Networking.2025; 28(2): 126. CrossRef
Readability, quality and accuracy of generative artificial intelligence chatbots for commonly asked questions about labor epidurals: a comparison of ChatGPT and Bard
D. Lee, M. Brown, J. Hammond, M. Zakowski
International Journal of Obstetric Anesthesia.2025; 61: 104317. CrossRef
ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial
Christian Burisch, Abhav Bellary, Frank Breuckmann, Jan Ehlers, Serge C Thal, Timur Sellmann, Daniel Gödde
JMIR Research Protocols.2025; 14: e63887. CrossRef
The effect of incorporating large language models into the teaching on critical thinking disposition: An “AI + Constructivism Learning Theory” attempt
Peng Wang, Kexin Yin, Mingzhu Zhang, Yuanxin Zheng, Tong Zhang, Yanjun Kang, Xun Feng
Education and Information Technologies.2025;[Epub] CrossRef
The Impact of Adaptive Learning Technologies, Personalized Feedback, and Interactive AI Tools on Student Engagement: The Moderating Role of Digital Literacy
Husam Yaseen, Abdelaziz Saleh Mohammad, Najwa Ashal, Hesham Abusaimeh, Ahmad Ali, Abdel-Aziz Ahmad Sharabati
Sustainability.2025; 17(3): 1133. CrossRef
Artificial Intelligence in Nursing: New Opportunities and Challenges
Estel·la Ramírez‐Baraldes, Daniel García‐Gutiérrez, Cristina García‐Salido
European Journal of Education.2025;[Epub] CrossRef
Can ChatGPT be used as a scientific source of information on tooth extraction?
Shiori Yamamoto, Masakazu Hamada, Kyoko Nishiyama, Ayako Motoki, Yusei Fujita, Narikazu Uzawa
Journal of Oral and Maxillofacial Surgery, Medicine, and Pathology.2025;[Epub] CrossRef
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Artificial Intelligence: Fundamentals and Breakthrough Applications in Epilepsy
Wesley Kerr, Sandra Acosta, Patrick Kwan, Gregory Worrell, Mohamad A. Mikati
Epilepsy Currents.2024;[Epub] CrossRef
A Developed Graphical User Interface-Based on Different Generative Pre-trained Transformers Models
Ekrem Küçük, İpek Balıkçı Çiçek, Zeynep Küçükakçalı, Cihan Yetiş, Cemil Çolak
ODÜ Tıp Dergisi.2024; 11(1): 18. CrossRef
Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases
Mohamad-Hani Temsah, Abdullah N. Alhuzaimi, Mohammed Almansour, Fadi Aljamaan, Khalid Alhasan, Munirah A. Batarfi, Ibraheem Altamimi, Amani Alharbi, Adel Abdulaziz Alsuhaibani, Leena Alwakeel, Abdulrahman Abdulkhaliq Alzahrani, Khaled B. Alsulaim, Amr Jam
Journal of Medical Systems.2024;[Epub] CrossRef
Authentic assessment in medical education: exploring AI integration and student-as-partners collaboration
Syeda Sadia Fatima, Nabeel Ashfaque Sheikh, Athar Osama
Postgraduate Medical Journal.2024; 100(1190): 959. CrossRef
Comparative performance analysis of large language models: ChatGPT-3.5, ChatGPT-4 and Google Gemini in glucocorticoid-induced osteoporosis
Linjian Tong, Chaoyang Zhang, Rui Liu, Jia Yang, Zhiming Sun
Journal of Orthopaedic Surgery and Research.2024;[Epub] CrossRef
Can AI-Generated Clinical Vignettes in Japanese Be Used Medically and Linguistically?
Yasutaka Yanagita, Daiki Yokokawa, Shun Uchida, Yu Li, Takanori Uehara, Masatomi Ikusaka
Journal of General Internal Medicine.2024; 39(16): 3282. CrossRef
ChatGPT vs. sleep disorder specialist responses to common sleep queries: Ratings by experts and laypeople
Jiyoung Kim, Seo-Young Lee, Jee Hyun Kim, Dong-Hyeon Shin, Eun Hye Oh, Jin A Kim, Jae Wook Cho
Sleep Health.2024; 10(6): 665. CrossRef
Technology integration into Chinese as a foreign language learning in higher education: An integrated bibliometric analysis and systematic review (2000–2024)
Binze Xu
Language Teaching Research.2024;[Epub] CrossRef
The Transformative Power of Generative Artificial Intelligence for Achieving the Sustainable Development Goal of Quality Education
Prema Nedungadi, Kai-Yu Tang, Raghu Raman
Sustainability.2024; 16(22): 9779. CrossRef
Is AI the new course creator
Sheri Conklin, Tom Dorgan, Daisyane Barreto
Discover Education.2024;[Epub] CrossRef
Emergency Medicine Assistants in the Field of Toxicology, Comparison of ChatGPT-3.5 and GEMINI Artificial Intelligence Systems
Hatice Aslı Bedel, Cihan Bedel, Fatih Selvi, Ökkeş Zortuk, Yusuf Karanci
Acta medica Lituanica.2024; 31(2): 294. CrossRef

How to review and assess a systematic review and meta-analysis article: a methodological study (secondary publication): Seung-Kwon Myung; J Educ Eval Health Prof. 2023;20:24. Published online August 27, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.24

13,951 View
906 Download
18 Web of Science
18 Crossref

Abstract PDF Supplementary Material

Systematic reviews and meta-analyses have become central in many research fields, particularly medicine. They offer the highest level of evidence in evidence-based medicine and support the development and revision of clinical practice guidelines, which offer recommendations for clinicians caring for patients with specific diseases and conditions. This review summarizes the concepts of systematic reviews and meta-analyses and provides guidance on reviewing and assessing such papers. A systematic review refers to a review of a research question that uses explicit and systematic methods to identify, select, and critically appraise relevant research. In contrast, a meta-analysis is a quantitative statistical analysis that combines individual results on the same research question to estimate the common or mean effect. Conducting a meta-analysis involves defining a research topic, selecting a study design, searching literature in electronic databases, selecting relevant studies, and conducting the analysis. One can assess the findings of a meta-analysis by interpreting a forest plot and a funnel plot and by examining heterogeneity. When reviewing systematic reviews and meta-analyses, several essential points must be considered, including the originality and significance of the work, the comprehensiveness of the database search, the selection of studies based on inclusion and exclusion criteria, subgroup analyses by various factors, and the interpretation of the results based on the levels of evidence. This review will provide readers with helpful guidance to help them read, understand, and evaluate these articles.

Citations

Citations to this article as recorded by

Testing the distinction between sadism and psychopathy: A metanalysis
Bruno Bonfá-Araujo, Gisele Magarotto Machado, Ariela Raissa Lima-Costa, Fernanda Otoni, Mahnoor Nadeem, Peter K. Jonason
Personality and Individual Differences.2025; 235: 112973. CrossRef
Impact of peripheral immune cells in experimental neonatal hypoxia-ischemia: A systematic review and meta-analysis
Ricardo Ribeiro Nunes, Luz Elena Durán-Carabali, Nícolas Heller Ribeiro, Dienifer Hermann Sirena, Isadora D’Ávila Tassinari, Carlos Alexandre Netto, Ana Helena Paz, Luciano Stürmer de Fraga
International Immunopharmacology.2025; 145: 113682. CrossRef
Systematic review and meta-analysis of the prevalence of frailty and pre-frailty amongst older hospital inpatients in low- and middle-income countries
Sean Lawlor Davidson, Jim Lee, Luke Emmence, Emily Bickerstaff, George Rayers, Elizabeth Davidson, Jenny Richardson, Heather Anderson, Richard Walker, Catherine Dotchin
Age and Ageing.2025;[Epub] CrossRef
Effect of Motivational Interviewing and Exercise on Chronic Low Back Pain: A Systematic Review and Meta‐Analysis
Olayinka Akinrolie, Uchechukwu B. Abioke, Francis O. Kolawole, Nicole Askin, Ebuka M. Anieto, Serena A. Itua, Oluwatoyin G. Akin, Blessing Eromosele, Opeyemi A. Idowu, Henrietta O. Fawole
Musculoskeletal Care.2025;[Epub] CrossRef
Smoking and Risk of Fatty Liver Disease: A Meta-Analysis of Cohort Studies
Moonhyung Lee, Seung-Kwon Myung, Sang Hee Lee, Yoosoo Chang
Gastroenterology Insights.2025; 16(1): 1. CrossRef
The Influence of Study Quality, Age, and Geographic Factors on PCOS Prevalence—A Systematic Review and Meta-analysis
Mina Amiri, Sana Hatoum, Richard P Buyalos, Ali Sheidaei, Ricardo Azziz
The Journal of Clinical Endocrinology & Metabolism.2025;[Epub] CrossRef
Relationship Between Bullshit, Cognitive Skills, and Belief Systems: A Meta‐Analytic Review
Geraldy Sepúlveda‐Páez, Marcelo Leiva‐Bianchi, Rodrigo Ferrer‐Urbina, Javier Escudero‐Pastén, Fabiola Salas
Applied Cognitive Psychology.2025;[Epub] CrossRef
Adjuvant chemotherapy in localized, resectable extremity and truncal soft tissue sarcoma and survival outcomes – A systematic review and meta‐analysis of randomized controlled trials
Megan H. Goh, Marcos R. Gonzalez, Hillary M. Heiling, Emanuele Mazzola, Joseph J. Connolly, Edwin Choy, Gregory M. Cote, Dimitrios Spentzos, Santiago A. Lozano‐Calderon
Cancer.2025;[Epub] CrossRef
Relationship Between Sleep Disturbances and In Vitro Fertilization Outcomes in Infertile Women: A Systematic Review and Meta‐Analysis
Farangis Habibi, Roya Nikbakht, Shayesteh Jahanfar, Mohammad Ahmadi, Maryam Eslami, Marzieh Azizi, Zohreh Shahhosseini
Brain and Behavior.2025;[Epub] CrossRef
Intimate partner violence among women of reproductive age during the COVID-19 pandemic in Ethiopia: a systematic review and meta-analysis
Alemayehu Sayih Belay, Gizachew Yilak, Aychew Kassie, Legese Fekede Abza, Ambaw Abebaw Emrie, Mulat Ayele, Befkad Derese Tilahun, Eyob Shitie Lake
BMJ Public Health.2025; 3(1): e001161. CrossRef
Meta-analysis of Reliability and Validity of the Bergen Social Media Addiction Scale (BSMAS)
Rossella Bottaro, Mark D. Griffiths, Palmira Faraci
International Journal of Mental Health and Addiction.2025;[Epub] CrossRef
Efficacy of PCSK9 inhibitors on pediatric patients with heterozygous familial hypercholesterolemia: a systematic review and meta-analysis of randomized controlled trials
Alesia Prillya Mauna, Joshua Eldad Frederich Lasanudin
Cardiology Plus.2025;[Epub] CrossRef
The Role of BIM in Managing Risks in Sustainability of Bridge Projects: A Systematic Review with Meta-Analysis
Dema Munef Ahmad, László Gáspár, Zsolt Bencze, Rana Ahmad Maya
Sustainability.2024; 16(3): 1242. CrossRef
The association between long noncoding RNA ABHD11-AS1 and malignancy prognosis: a meta-analysis
Guangyao Lin, Tao Ye, Jing Wang
BMC Cancer.2024;[Epub] CrossRef
The impact of indoor carbon dioxide exposure on human brain activity: A systematic review and meta-analysis based on studies utilizing electroencephalogram signals
Nan Zhang, Chao Liu, Caixia Hou, Wenhao Wang, Qianhui Yuan, Weijun Gao
Building and Environment.2024; 259: 111687. CrossRef
Efficacy of mechanical debridement with adjunct antimicrobial photodynamic therapy against peri-implant subgingival oral yeasts colonization: A systematic review and meta-analysis
Dena Ali, Jenna Alsalman
Photodiagnosis and Photodynamic Therapy.2024; 50: 104399. CrossRef
The effectiveness and usability of online, group-based interventions for people with severe obesity: a systematic review and meta-analysis
Madison Milne-Ives, Lorna Burns, Dawn Swancutt, Raff Calitri, Ananya Ananthakrishnan, Helene Davis, Jonathan Pinkney, Mark Tarrant, Edward Meinert
International Journal of Obesity.2024;[Epub] CrossRef
Non-invasive brain stimulation enhances motor and cognitive performances during dual tasks in patients with Parkinson’s disease: a systematic review and meta-analysis
Hajun Lee, Beom Jin Choi, Nyeonju Kang
Journal of NeuroEngineering and Rehabilitation.2024;[Epub] CrossRef

Research articles

Empirical effect of the Dr Lee Jong-wook Fellowship Program to empower sustainable change for the health workforce in Tanzania: a mixed-methods study: Masoud Dauda, Swabaha Aidarus Yusuph, Harouni Yasini, Issa Mmbaga, Perpetua Mwambinngu, Hansol Park, Gyeongbae Seo, Kyoung Kyun Oh; J Educ Eval Health Prof. 2025;22:6. Published online January 20, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.6 [Epub ahead of print]

902 View
142 Download

Abstract PDF: Purpose
This study evaluated the Dr Lee Jong-wook Fellowship Program’s impact on Tanzania’s health workforce, focusing on relevance, effectiveness, efficiency, impact, and sustainability in addressing healthcare gaps.
Methods
A mixed-methods research design was employed. Data were collected from 97 out of 140 alumni through an online survey, 35 in-depth interviews, and one focus group discussion. The study was conducted from November to December 2023 and included alumni from 2009 to 2022. Measurement instruments included structured questionnaires for quantitative data and semi-structured guides for qualitative data. Quantitative analysis involved descriptive and inferential statistics (Spearman’s rank correlation, non-parametric tests) using Python ver. 3.11.0 and Stata ver. 14.0. Thematic analysis was employed to analyze qualitative data using NVivo ver. 12.0.
Results
Findings indicated high relevance (mean=91.6, standard deviation [SD]=8.6), effectiveness (mean=86.1, SD=11.2), efficiency (mean=82.7, SD=10.2), and impact (mean=87.7, SD=9.9), with improved skills, confidence, and institutional service quality. However, sustainability had a lower score (mean=58.0, SD=11.1), reflecting challenges in follow-up support and resource allocation. Effectiveness strongly correlated with impact (ρ=0.746, P<0.001). The qualitative findings revealed that participants valued tailored training but highlighted barriers, such as language challenges and insufficient practical components. Alumni-led initiatives contributed to knowledge sharing, but limited resources constrained sustainability.
Conclusion
The Fellowship Program enhanced Tanzania’s health workforce capacity, but it requires localized curricula and strengthened alumni networks for sustainability. These findings provide actionable insights for improving similar programs globally, confirming the hypothesis that tailored training positively influences workforce and institutional outcomes.

Reliability and construct validation of the Blended Learning Usability Evaluation–Questionnaire with interprofessional clinicians in Canada: a methodological study: Anish Kumar Arora, Jeff Myers, Tavis Apramian, Kulamakan Kulasegaram, Daryl Bainbridge, Hsien Seow; J Educ Eval Health Prof. 2025;22:5. Published online January 16, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.5 [Epub ahead of print]

524 View
125 Download

Abstract PDF: Purpose
To generate Cronbach’s alpha and further mixed methods construct validity evidence for the Blended Learning Usability Evaluation–Questionnaire (BLUE-Q).
Methods
Forty interprofessional clinicians completed the BLUE-Q after finishing a 3-month long blended learning professional development program in Ontario, Canada. Reliability was assessed with Cronbach’s α for each of the 3 sections of the BLUE-Q and for all quantitative items together. Construct validity was evaluated through the Grand-Guillaume-Perrenoud et al. framework, which consists of 3 elements: congruence, convergence, and credibility. To compare quantitative and qualitative results, descriptive statistics, including means and standard deviations for each Likert scale item of the BLUE-Q were calculated.
Results
Cronbach’s α was 0.95 for the pedagogical usability section, 0.85 for the synchronous modality section, 0.93 for the asynchronous modality section, and 0.96 for all quantitative items together. Mean ratings (with standard deviations) were 4.77 (0.506) for pedagogy, 4.64 (0.654) for synchronous learning, and 4.75 (0.536) for asynchronous learning. Of the 239 qualitative comments received, 178 were identified as substantive, of which 88% were considered congruent and 79% were considered convergent with the high means. Among all congruent responses, 69% were considered confirming statements and 31% were considered clarifying statements, suggesting appropriate credibility. Analysis of the clarifying statements assisted in identifying 5 categories of suggestions for program improvement.
Conclusion
The BLUE-Q demonstrates high reliability and appropriate construct validity in the context of a blended learning program with interprofessional clinicians, making it a valuable tool for comprehensive program evaluation, quality improvement, and evaluative research in health professions education.

Educational/Faculty development material

The role of large language models in the peer-review process: opportunities and challenges for medical journal reviewers and editors: Jisoo Lee, Jieun Lee, Jeong-Ju Yoo; J Educ Eval Health Prof. 2025;22:4. Published online January 16, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.4 [Epub ahead of print]

812 View
118 Download

Abstract PDF: The peer review process ensures the integrity of scientific research. This is particularly important in the medical field, where research findings directly impact patient care. However, the rapid growth of publications has strained reviewers, causing delays and potential declines in quality. Generative artificial intelligence, especially large language models (LLMs) such as ChatGPT, may assist researchers with efficient, high-quality reviews. This review explores the integration of LLMs into peer review, highlighting their strengths in linguistic tasks and challenges in assessing scientific validity, particularly in clinical medicine. Key points for integration include initial screening, reviewer matching, feedback support, and language review. However, implementing LLMs for these purposes will necessitate addressing biases, privacy concerns, and data confidentiality. We recommend using LLMs as complementary tools under clear guidelines to support, not replace, human expertise in maintaining rigorous peer review standards.

Research articles

Empathy and tolerance of ambiguity in medical students and doctors participating in art-based observational training at the Rijksmuseum in Amsterdam, the Netherlands: a before-and-after study: Stella Anna Bult, Thomas van Gulik; J Educ Eval Health Prof. 2025;22:3. Published online January 14, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.3

697 View
112 Download

Abstract PDF Supplementary Material: Purpose
This research presents an experimental study using validated questionnaires to quantitatively assess the outcomes of art-based observational training in medical students, residents, and specialists. The study tested the hypothesis that art-based observational training would lead to measurable effects on judgement skills (tolerance of ambiguity) and empathy in medical students and doctors.
Methods
An experimental cohort study with pre- and post-intervention assessments was conducted using validated questionnaires and qualitative evaluation forms to examine the outcomes of art-based observational training in medical students and doctors. Between December 2023 and June 2024, 15 art courses were conducted in the Rijksmuseum in Amsterdam. Participants were assessed on empathy using the Jefferson Scale of Empathy (JSE) and tolerance of ambiguity using the Tolerance of Ambiguity in Medical Students and Doctors (TAMSAD) scale.
Results
In total, 91 participants were included; 29 participants completed the JSE and 62 completed the TAMSAD scales. The results showed statistically significant post-test increases for mean JSE and TAMSAD scores (3.71 points for the JSE, ranging from 20 to 140, and 1.86 points for the TAMSAD, ranging from 0 to 100). The qualitative findings were predominantly positive.
Conclusion
The results suggest that incorporating art-based observational training in medical education improves empathy and tolerance of ambiguity. This study highlights the importance of art-based observational training in medical education in the professional development of medical students and doctors.

Pharmacy students’ perspective on remote flipped classrooms in Malaysia: a qualitative study: Wei Jin Wong, Shaun Wen Huey Lee, Ronald Fook Seng Lee; J Educ Eval Health Prof. 2025;22:2. Published online January 14, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.2 [Epub ahead of print]

536 View
104 Download

Abstract PDF: Purpose
This study aimed to explore pharmacy students’ perceptions of remote flipped classrooms in Malaysia, focusing on their learning experiences and identifying areas for potential improvement to inform future educational strategies.
Methods
A qualitative approach was employed, utilizing inductive thematic analysis. Twenty Bachelor of Pharmacy students (18 women, 2 men; age range, 19–24 years) from Monash University participated in 8 focus group discussions over 2 rounds during the coronavirus disease 2019 pandemic. Participants were recruited via convenience sampling. The focus group discussions, led by experienced academics, were conducted in English via Zoom, recorded, and transcribed for analysis using NVivo. Themes were identified through emergent coding and iterative discussions to ensure thematic saturation.
Results
Five major themes emerged: flexibility, communication, technological challenges, skill-based learning challenges, and time-based effects. Students appreciated the flexibility of accessing and reviewing pre-class materials at their convenience. Increased engagement through anonymous question submission was noted, yet communication difficulties and lack of non-verbal cues in remote workshops were significant drawbacks. Technological issues, such as internet connectivity problems, hindered learning, especially during assessments. Skill-based learning faced challenges in remote settings, including lab activities and clinical examinations. Additionally, prolonged remote learning led to feelings of isolation, fatigue, and a desire to return to in-person interactions.
Conclusion
Remote flipped classrooms offer flexibility and engagement benefits but present notable challenges related to communication, technology, and skill-based learning. To improve remote education, institutions should integrate robust technological support, enhance communication strategies, and incorporate virtual simulations for practical skills. Balancing asynchronous and synchronous methods while addressing academic success and socioemotional wellness is essential for effective remote learning environments.

Validation of the 21st Century Skills Assessment Scale for public health students in Thailand: a methodological study: Suphawadee Panthumas, Kaung Zaw, Wirin Kittipichai; J Educ Eval Health Prof. 2024;21:37. Published online December 10, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.37

931 View
167 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to develop and validate the 21st Century Skills Assessment Scale (21CSAS) for Thai public health (PH) undergraduate students using the Partnership for 21st Century Skills framework.
Methods
A cross-sectional survey was conducted among 727 first- to fourth-year PH undergraduate students from 4 autonomous universities in Thailand. Data were collected using self-administered questionnaires between January and March 2023. Exploratory factor analysis (EFA) was used to explore the underlying dimensions of 21CSAS, while confirmatory factor analysis (CFA) was conducted to test the hypothesized factor structure using Mplus software (Muthén & Muthén). Reliability and item discrimination were assessed using Cronbach’s α and the corrected item-total correlation, respectively.
Results
EFA performed on a dataset of 300 students revealed a 20-item scale with a 6-factor structure: (1) creativity and innovation; (2) critical thinking and problem-solving; (3) information, media, and technology; (4) communication and collaboration; (5) initiative and self-direction; and (6) social and cross-cultural skills. The rotated eigenvalues ranged from 2.12 to 1.73. CFA performed on another dataset of 427 students confirmed a good model fit (χ2/degrees of freedom=2.67, comparative fit index=0.93, Tucker-Lewis index=0.91, root mean square error of approximation=0.06, standardized root mean square residual=0.06), explaining 34%–71% of variance in the items. Item loadings ranged from 0.58 to 0.84. The 21CSAS had a Cronbach’s α of 0.92.
Conclusion
The 21CSAS proved be a valid and reliable tool for assessing 21st century skills among Thai PH undergraduate students. These findings provide insights for educational system to inform policy, practice, and research regarding 21st-century skills among undergraduate students.

Simulation-based teaching versus traditional small group teaching for first-year medical students among high and low scorers in respiratory physiology, India: a randomized controlled trial: Nalini Yelahanka Channegowda, Dinker Ramanand Pai, Shivasakthy Manivasakan; J Educ Eval Health Prof. 2025;22:8. Published online February 21, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.8 [Epub ahead of print]

278 View
86 Download

Abstract PDF: Purpose
Although it is widely utilized in clinical subjects for skill training, using simulation-based education (SBE) for teaching basic science concepts to phase I medical students or pre-clinical students is limited. Simulation-based education/teaching is preferred in cardiovascular and respiratory physiology when compared to other systems because it is easy to recreate both the normal physiological component and alterations in the simulated environment, thus a promoting deep understanding of the core concepts.
Methods
A block randomized study was conducted among 107 phase 1 (first-year) medical undergraduate students at a Deemed to be University in India. Group A received SBE and Group B traditional small group teaching. The effectiveness of the teaching intervention was assessed using pre- and post-tests. Student feedback was obtained through a self administered structured questionnaire via an anonymous online survey and by in-depth interview.
Results
The intervention group showed a statistically significant improvement in post-test scores compared to the control group. A sub-analysis revealed that high scorers performed better than low scorers in both groups, but the knowledge gain among low scorers was more significant in the intervention group.
Conclusion
This teaching strategy offers a valuable supplement to traditional methods, fostering a deeper comprehension of clinical concepts from the outset of medical training.

Review

Immersive simulation in nursing and midwifery education: a systematic review: Lahoucine Ben Yahya, Aziz Naciri, Mohamed Radid, Ghizlane Chemsi; J Educ Eval Health Prof. 2024;21:19. Published online August 8, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.19

4,144 View
418 Download
1 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Purpose
Immersive simulation is an innovative training approach in health education that enhances student learning. This study examined its impact on engagement, motivation, and academic performance in nursing and midwifery students.
Methods
A comprehensive systematic search was meticulously conducted in 4 reputable databases—Scopus, PubMed, Web of Science, and Science Direct—following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The research protocol was pre-registered in the PROSPERO registry, ensuring transparency and rigor. The quality of the included studies was assessed using the Medical Education Research Study Quality Instrument.
Results
Out of 90 identified studies, 11 were included in the present review, involving 1,090 participants. Four out of 5 studies observed high post-test engagement scores in the intervention groups. Additionally, 5 out of 6 studies that evaluated motivation found higher post-test motivational scores in the intervention groups than in control groups using traditional approaches. Furthermore, among the 8 out of 11 studies that evaluated academic performance during immersive simulation training, 5 reported significant differences (P<0.001) in favor of the students in the intervention groups.
Conclusion
Immersive simulation, as demonstrated by this study, has a significant potential to enhance student engagement, motivation, and academic performance, surpassing traditional teaching methods. This potential underscores the urgent need for future research in various contexts to better integrate this innovative educational approach into nursing and midwifery education curricula, inspiring hope for improved teaching methods.

Citations

Citations to this article as recorded by

The power of moulage: Teaching with wound moulage simulations in nursing education
Romaine Meichtry, Rabea Krings, Alina Klein, Monika Droz, Claudia Schlegel
Teaching and Learning in Nursing.2025;[Epub] CrossRef
NursingXR: Advancing Nursing Education Through Virtual Reality-Based Training
Mohammad F. Obeid, Ahmed Ewais, Mohammad R. Asia
Applied Sciences.2025; 15(6): 2949. CrossRef
Application of Virtual Reality, Artificial Intelligence, and Other Innovative Technologies in Healthcare Education (Nursing and Midwifery Specialties): Challenges and Strategies
Galya Georgieva-Tsaneva, Ivanichka Serbezova, Silvia Beloeva
Education Sciences.2024; 15(1): 11. CrossRef

Research article

Inter-rater reliability and content validity of the measurement tool for portfolio assessments used in the Introduction to Clinical Medicine course at Ewha Womans University College of Medicine: a methodological study: Dong-Mi Yoo, Jae Jin Han; J Educ Eval Health Prof. 2024;21:39. Published online December 10, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.39

637 View
151 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to examine the reliability and validity of a measurement tool for portfolio assessments in medical education. Specifically, it investigated scoring consistency among raters and assessment criteria appropriateness according to an expert panel.
Methods
A cross-sectional observational study was conducted from September to December 2018 for the Introduction to Clinical Medicine course at the Ewha Womans University College of Medicine. Data were collected for 5 randomly selected portfolios scored by a gold-standard rater and 6 trained raters. An expert panel assessed the validity of 12 assessment items using the content validity index (CVI). Statistical analysis included Pearson correlation coefficients for rater alignment, the intraclass correlation coefficient (ICC) for inter-rater reliability, and the CVI for item-level validity.
Results
Rater 1 had the highest Pearson correlation (0.8916) with the gold-standard rater, while Rater 5 had the lowest (0.4203). The ICC for all raters was 0.3821, improving to 0.4415 after excluding Raters 1 and 5, indicating a 15.6% reliability increase. All assessment items met the CVI threshold of ≥0.75, with some achieving a perfect score (CVI=1.0). However, items like “sources” and “level and degree of performance” showed lower validity (CVI=0.72).
Conclusion
The present measurement tool for portfolio assessments demonstrated moderate reliability and strong validity, supporting its use as a credible tool. For a more reliable portfolio assessment, more faculty training is needed.

Review

Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review: Xiaojun Xu, Yixiao Chen, Jing Miao; J Educ Eval Health Prof. 2024;21:6. Published online March 15, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.6

7,128 View
604 Download
14 Web of Science
18 Crossref

Abstract PDF Supplementary Material

Background
ChatGPT is a large language model (LLM) based on artificial intelligence (AI) capable of responding in multiple languages and generating nuanced and highly complex responses. While ChatGPT holds promising applications in medical education, its limitations and potential risks cannot be ignored.
Methods
A scoping review was conducted for English articles discussing ChatGPT in the context of medical education published after 2022. A literature search was performed using PubMed/MEDLINE, Embase, and Web of Science databases, and information was extracted from the relevant studies that were ultimately included.
Results
ChatGPT exhibits various potential applications in medical education, such as providing personalized learning plans and materials, creating clinical practice simulation scenarios, and assisting in writing articles. However, challenges associated with academic integrity, data accuracy, and potential harm to learning were also highlighted in the literature. The paper emphasizes certain recommendations for using ChatGPT, including the establishment of guidelines. Based on the review, 3 key research areas were proposed: cultivating the ability of medical students to use ChatGPT correctly, integrating ChatGPT into teaching activities and processes, and proposing standards for the use of AI by medical students.
Conclusion
ChatGPT has the potential to transform medical education, but careful consideration is required for its full integration. To harness the full potential of ChatGPT in medical education, attention should not only be given to the capabilities of AI but also to its impact on students and teachers.

Citations

Citations to this article as recorded by

AI-assisted patient education: Challenges and solutions in pediatric kidney transplantation
MZ Ihsan, Dony Apriatama, Pithriani, Riza Amalia
Patient Education and Counseling.2025; 131: 108575. CrossRef
Exploring predictors of AI chatbot usage intensity among students: Within- and between-person relationships based on the technology acceptance model
Anne-Kathrin Kleine, Insa Schaffernak, Eva Lermer
Computers in Human Behavior: Artificial Humans.2025; 3: 100113. CrossRef
AI-powered standardised patients: evaluating ChatGPT-4o’s impact on clinical case management in intern physicians
Selcen Öncü, Fulya Torun, Hilal Hatice Ülkü
BMC Medical Education.2025;[Epub] CrossRef
UsmleGPT: An AI application for developing MCQs via multi-agent system
Zhehan Jiang, Shicong Feng
Software Impacts.2025; 23: 100742. CrossRef
ChatGPT’s Performance on Portuguese Medical Examination Questions: Comparative Analysis of ChatGPT-3.5 Turbo and ChatGPT-4o Mini
Filipe Prazeres
JMIR Medical Education.2025; 11: e65108. CrossRef
Chatbots in neurology and neuroscience: Interactions with students, patients and neurologists
Stefano Sandrone
Brain Disorders.2024; 15: 100145. CrossRef
ChatGPT in education: unveiling frontiers and future directions through systematic literature review and bibliometric analysis
Buddhini Amarathunga
Asian Education and Development Studies.2024; 13(5): 412. CrossRef
Evaluating the performance of ChatGPT-3.5 and ChatGPT-4 on the Taiwan plastic surgery board examination
Ching-Hua Hsieh, Hsiao-Yun Hsieh, Hui-Ping Lin
Heliyon.2024; 10(14): e34851. CrossRef
Preparing for Artificial General Intelligence (AGI) in Health Professions Education: AMEE Guide No. 172
Ken Masters, Anne Herrmann-Werner, Teresa Festl-Wietek, David Taylor
Medical Teacher.2024; 46(10): 1258. CrossRef
A Comparative Analysis of ChatGPT and Medical Faculty Graduates in Medical Specialization Exams: Uncovering the Potential of Artificial Intelligence in Medical Education
Gülcan Gencer, Kerem Gencer
Cureus.2024;[Epub] CrossRef
Research ethics and issues regarding the use of ChatGPT-like artificial intelligence platforms by authors and reviewers: a narrative review
Sang-Jun Kim
Science Editing.2024; 11(2): 96. CrossRef
Innovation Off the Bat: Bridging the ChatGPT Gap in Digital Competence among English as a Foreign Language Teachers
Gulsara Urazbayeva, Raisa Kussainova, Aikumis Aibergen, Assel Kaliyeva, Gulnur Kantayeva
Education Sciences.2024; 14(9): 946. CrossRef
Exploring the perceptions of Chinese pre-service teachers on the integration of generative AI in English language teaching: Benefits, challenges, and educational implications
Ji Young Chung, Seung-Hoon Jeong
Online Journal of Communication and Media Technologies.2024; 14(4): e202457. CrossRef
Unveiling the bright side and dark side of AI-based ChatGPT : a bibliographic and thematic approach
Chandan Kumar Tiwari, Mohd. Abass Bhat, Abel Dula Wedajo, Shagufta Tariq Khan
Journal of Decision Systems.2024; : 1. CrossRef
Artificial Intelligence in Medical Education and Mentoring in Rehabilitation Medicine
Julie K. Silver, Mustafa Reha Dodurgali, Nara Gavini
American Journal of Physical Medicine & Rehabilitation.2024; 103(11): 1039. CrossRef
The Potential of Artificial Intelligence Tools for Reducing Uncertainty in Medicine and Directions for Medical Education
Sauliha Rabia Alli, Soaad Qahhār Hossain, Sunit Das, Ross Upshur
JMIR Medical Education.2024; 10: e51446. CrossRef
A Systematic Literature Review of Empirical Research on Applying Generative Artificial Intelligence in Education
Xin Zhang, Peng Zhang, Yuan Shen, Min Liu, Qiong Wang, Dragan Gašević, Yizhou Fan
Frontiers of Digital Education.2024; 1(3): 223. CrossRef
Artificial intelligence in medical problem-based learning: opportunities and challenges
Yaoxing Chen, Hong Qi, Yu Qiu, Juan Li, Liang Zhu, Xiaoling Gao, Hao Wang, Gan Jiang
Global Medical Education.2024;[Epub] CrossRef

Research articles

Comparison of virtual and in-person simulations for sepsis and trauma resuscitation training in Singapore: a randomized controlled trial: Matthew Jian Wen Low, Gene Wai Han Chan, Zisheng Li, Yiwen Koh, Chi Loong Jen, Zi Yao Lee, Lenard Tai Win Cheng; J Educ Eval Health Prof. 2024;21:33. Published online November 18, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.33

843 View
157 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to compare cognitive, non-cognitive, and overall learning outcomes for sepsis and trauma resuscitation skills in novices with virtual patient simulation (VPS) versus in-person simulation (IPS).
Methods
A randomized controlled trial was conducted on junior doctors in 1 emergency department from January to December 2022, comparing 70 minutes of VPS (n=19) versus IPS (n=21) in sepsis and trauma resuscitation. Using the nominal group technique, we created skills assessment checklists and determined Bloom’s taxonomy domains for each checklist item. Two blinded raters observed participants leading 1 sepsis and 1 trauma resuscitation simulation. Satisfaction was measured using the Student Satisfaction with Learning Scale (SSLS). The SSLS and checklist scores were analyzed using the Wilcoxon rank sum test and 2-tailed t-test respectively.
Results
For sepsis, there was no significant difference between VPS and IPS in overall scores (2.0; 95% confidence interval [CI], -1.4 to 5.4; Cohen’s d=0.38), as well as in items that were cognitive (1.1; 95% CI, -1.5 to 3.7) and not only cognitive (0.9; 95% CI, -0.4 to 2.2). Likewise, for trauma, there was no significant difference in overall scores (-0.9; 95% CI, -4.1 to 2.3; Cohen’s d=0.19), as well as in items that were cognitive (-0.3; 95% CI, -2.8 to 2.1) and not only cognitive (-0.6; 95% CI, -2.4 to 1.3). The median SSLS scores were lower with VPS than with IPS (-3.0; 95% CI, -1.0 to -5.0).
Conclusion
For novices, there were no major differences in overall and non-cognitive learning outcomes for sepsis and trauma resuscitation between VPS and IPS. Learners were more satisfied with IPS than with VPS (clinicaltrials.gov identifier: NCT05201950).

Mentorship and self-efficacy are associated with lower burnout in physical therapists in the United States: a cross-sectional survey study: Matthew Pugliese, Jean-Michel Brismée, Brad Allen, Sean Riley, Justin Tammany, Paul Mintken; J Educ Eval Health Prof. 2023;20:27. Published online September 27, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.27

6,431 View
452 Download
4 Web of Science
5 Crossref

Abstract PDF Supplementary Material

Purpose
This study investigated the prevalence of burnout in physical therapists in the United States and the relationships between burnout and education, mentorship, and self-efficacy.
Methods
This was a cross-sectional survey study. An electronic survey was distributed to practicing physical therapists across the United States over a 6-week period from December 2020 to January 2021. The survey was completed by 2,813 physical therapists from all states. The majority were female (68.72%), White or Caucasian (80.13%), and employed full-time (77.14%). Respondents completed questions on demographics, education, mentorship, self-efficacy, and burnout. The Burnout Clinical Subtypes Questionnaire 12 (BCSQ-12) and self-reports were used to quantify burnout, and the General Self-Efficacy Scale (GSES) was used to measure self-efficacy. Descriptive and inferential analyses were performed.
Results
Respondents from home health (median BCSQ-12=42.00) and skilled nursing facility settings (median BCSQ-12=42.00) displayed the highest burnout scores. Burnout was significantly lower among those who provided formal mentorship (median BCSQ-12=39.00, P=0.0001) compared to no mentorship (median BCSQ-12=41.00). Respondents who received formal mentorship (median BCSQ-12=38.00, P=0.0028) displayed significantly lower burnout than those who received no mentorship (median BCSQ-12=41.00). A moderate negative correlation (rho=-0.49) was observed between the GSES and burnout scores. A strong positive correlation was found between self-reported burnout status and burnout scores (rrb=0.61).
Conclusion
Burnout is prevalent in the physical therapy profession, as almost half of respondents (49.34%) reported burnout. Providing or receiving mentorship and higher self-efficacy were associated with lower burnout. Organizations should consider measuring burnout levels, investing in mentorship programs, and implementing strategies to improve self-efficacy.

Citations

Citations to this article as recorded by

Wellness and Stress Management Practices Among Healthcare Professionals and Health Professional Students
Asli C. Yalim, Katherine Daly, Monica Bailey, Denise Kay, Xiang Zhu, Mohammed Patel, Laurie C. Neely, Desiree A. Díaz, Denyi M. Canario Asencio, Karla Rosario, Melissa Cowan, Magdalena Pasarica
American Journal of Health Promotion.2025; 39(2): 204. CrossRef
Final results of the National Oncology Mentorship Program 2023 and its impact on burnout and professional fulfilment
Udit Nindra, Gowri Shivasabesan, Rhiannon Mellor, Weng Ng, Wei Chua, Deme Karikios, Bethan Richards, Jia Liu
Internal Medicine Journal.2025; 55(2): 233. CrossRef
Incidence of Shared Clinical Instruction in Physical Therapy Clinical Education in the United States
Nicki Silberman, Lori Hochman, Jaya Rachwani
Journal of Physical Therapy Education.2025;[Epub] CrossRef
Interprofessional education to support alcohol use screening and future team-based management of stress-related disorders in vulnerable populations
Taylor Fitzpatrick-Schmidt, Scott Edwards
Frontiers in Education.2024;[Epub] CrossRef
Prevalence of Stress and Burnout in Physical Therapist Clinical Instructors
Ryan J. Pontiff, Peggy Gleeson, Katy Mitchell, Rupal M. Patel
Journal of Physical Therapy Education.2024;[Epub] CrossRef

Educational/Faculty development material

The 6 degrees of curriculum integration in medical education in the United States: Julie Youm, Jennifer Christner, Kevin Hittle, Paul Ko, Cinda Stone, Angela D. Blood, Samara Ginzburg; J Educ Eval Health Prof. 2024;21:15. Published online June 13, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.15

3,778 View
442 Download

Abstract PDF Supplementary Material: Despite explicit expectations and accreditation requirements for integrated curriculum, there needs to be more clarity around an accepted common definition, best practices for implementation, and criteria for successful curriculum integration. To address the lack of consensus surrounding integration, we reviewed the literature and herein propose a definition for curriculum integration for the medical education audience. We further believe that medical education is ready to move beyond “horizontal” (1-dimensional) and “vertical” (2-dimensional) integration and propose a model of “6 degrees of curriculum integration” to expand the 2-dimensional concept for future designs of medical education programs and best prepare learners to meet the needs of patients. These 6 degrees include: interdisciplinary, timing and sequencing, instruction and assessment, incorporation of basic and clinical sciences, knowledge and skills-based competency progression, and graduated responsibilities in patient care. We encourage medical educators to look beyond 2-dimensional integration to this holistic and interconnected representation of curriculum integration.

First
Prev
Page of 7
Next
Last

Most download articles