Journal of Educational Evaluation for Health Professions

Volume 19; 2022

Editorial

Presidential address: reflection on work from April 2019 to 2022 and appreciation to the staff and volunteers: Yoon-Seong Lee; J Educ Eval Health Prof. 2022;19:39. Published online December 30, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.39

769 View
78 Download
1 Citations

PDF

Citations

Citations to this article as recorded by

Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
Sun Huh
Journal of Educational Evaluation for Health Professions.2023; 20: 5. CrossRef

Reviews

Factors associated with medical students’ scores on the National Licensing Exam in Peru: a systematic review: Javier Alejandro Flores-Cohaila; J Educ Eval Health Prof. 2022;19:38. Published online December 29, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.38

2,099 View
221 Download
1 Citations

Abstract PDF Supplementary Material

Purpose
This study aimed to identify factors that have been studied for their associations with National Licensing Examination (ENAM) scores in Peru.
Methods
A search was conducted of literature databases and registers, including EMBASE, SciELO, Web of Science, MEDLINE, Peru’s National Register of Research Work, and Google Scholar. The following key terms were used: “ENAM” and “associated factors.” Studies in English and Spanish were included. The quality of the included studies was evaluated using the Medical Education Research Study Quality Instrument (MERSQI).
Results
In total, 38,500 participants were enrolled in 12 studies. Most (11/12) studies were cross-sectional, except for one case-control study. Three studies were published in peer-reviewed journals. The mean MERSQI was 10.33. A better performance on the ENAM was associated with a higher-grade point average (GPA) (n=8), internship setting in EsSalud (n=4), and regular academic status (n=3). Other factors showed associations in various studies, such as medical school, internship setting, age, gender, socioeconomic status, simulations test, study resources, preparation time, learning styles, study techniques, test-anxiety, and self-regulated learning strategies.
Conclusion
The ENAM is a multifactorial phenomenon; our model gives students a locus of control on what they can do to improve their score (i.e., implement self-regulated learning strategies) and faculty, health policymakers, and managers a framework to improve the ENAM score (i.e., design remediation programs to improve GPA and integrate anxiety-management courses into the curriculum).

Citations

Citations to this article as recorded by

Performance of ChatGPT on the Peruvian National Licensing Medical Examination: Cross-Sectional Study
Javier A Flores-Cohaila, Abigaíl García-Vicente, Sonia F Vizcarra-Jiménez, Janith P De la Cruz-Galán, Jesús D Gutiérrez-Arratia, Blanca Geraldine Quiroga Torres, Alvaro Taype-Rondan
JMIR Medical Education.2023; 9: e48039. CrossRef

Medical students’ satisfaction level with e-learning during the COVID-19 pandemic and its related factors: a systematic review: Mahbubeh Tabatabaeichehr, Samane Babaei, Mahdieh Dartomi, Peiman Alesheikh, Amir Tabatabaee, Hamed Mortazavi, Zohreh Khoshgoftar; J Educ Eval Health Prof. 2022;19:37. Published online December 20, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.37

1,630 View
170 Download
2 Citations

Abstract PDF Supplementary Material

Purpose
This review investigated medical students’ satisfaction level with e-learning during the coronavirus disease 2019 (COVID-19) pandemic and its related factors.
Methods
A comprehensive systematic search was performed of international literature databases, including Scopus, PubMed, Web of Science, and Persian databases such as Iranmedex and Scientific Information Database using keywords extracted from Medical Subject Headings such as “Distance learning,” “Distance education,” “Online learning,” “Online education,” and “COVID-19” from the earliest date to July 10, 2022. The quality of the studies included in this review was evaluated using the appraisal tool for cross-sectional studies (AXIS tool).
Results
A total of 15,473 medical science students were enrolled in 24 studies. The level of satisfaction with e-learning during the COVID-19 pandemic among medical science students was 51.8%. Factors such as age, gender, clinical year, experience with e-learning before COVID-19, level of study, adaptation content of course materials, interactivity, understanding of the content, active participation of the instructor in the discussion, multimedia use in teaching sessions, adequate time dedicated to the e-learning, stress perception, and convenience had significant relationships with the satisfaction of medical students with e-learning during the COVID-19 pandemic.
Conclusion
Therefore, due to the inevitability of online education and e-learning, it is suggested that educational managers and policymakers choose the best online education method for medical students by examining various studies in this field to increase their satisfaction with e-learning.

Citations

Citations to this article as recorded by

Physician Assistant Students’ Perception of Online Didactic Education: A Cross-Sectional Study
Daniel L Anderson, Jeffrey L Alexander
Cureus.2023;[Epub] CrossRef
Mediating Role of PERMA Wellbeing in the Relationship between Insomnia and Psychological Distress among Nursing College Students
Qian Sun, Xiangyu Zhao, Yiming Gao, Di Zhao, Meiling Qi
Behavioral Sciences.2023; 13(9): 764. CrossRef

Research article

Suggestion of more suitable study designs and the corresponding reporting guidelines in articles published in the Journal of Educational Evaluation for Health Professions from 2021 to September 2022: a descriptive study: Soo Young Kim; J Educ Eval Health Prof. 2022;19:36. Published online December 26, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.36

915 View
91 Download
3 Citations

Abstract PDF Supplementary Material

Purpose
This study aimed to suggest a more suitable study design and the corresponding reporting guidelines in the papers published in the Journal of Educational Evaluation for Health Professionals from January 2021 to September 2022.
Methods
Among 59 papers published in the Journal of Educational Evaluation for Health Professionals from January 2021 to September 2022, research articles, review articles, and brief reports were selected. The followings were analyzed: first, the percentage of articles describing the study design in the title, abstracts, or methods; second, the portion of articles describing reporting guidelines; third, the types of study design and corresponding reporting guidelines; and fourth, the suggestion of a more suitable study design based on the study design algorithm for medical literature on interventions, systematic reviews & other review types, and epidemiological studies overview.
Results
Out of 45 articles, 44 described study designs (97.8%). Out of 44, 19 articles were suggested to be described with more suitable study designs, which mainly occurred in before-and-after studies, diagnostic research, and non-randomized trials. Of the 18 reporting guidelines mentioned, 8 (44.4%) were considered perfect. STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) was used for descriptive studies, before-and-after studies, and randomized controlled trials; however, its use should be reconsidered.
Conclusion
Some declarations of study design and reporting guidelines were suggested to be described with more suitable ones. Education and training on study design and reporting guidelines for researchers are needed, and reporting guideline policies for descriptive studies should also be implemented.

Citations

Citations to this article as recorded by

Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
Sun Huh
Journal of Educational Evaluation for Health Professions.2023; 20: 5. CrossRef
A comprehensive perspective on the interaction between gut microbiota and COVID-19 vaccines
Ming Hong, Tin Lan, Qiuxia Li, Binfei Li, Yong Yuan, Feng Xu, Weijia Wang
Gut Microbes.2023;[Epub] CrossRef
Why do editors of local nursing society journals strive to have their journals included in MEDLINE? A case study of the Korean Journal of Women Health Nursing
Sun Huh
Korean Journal of Women Health Nursing.2023; 29(3): 147. CrossRef

Educational/Faculty development material

Common models and approaches for the clinical educator to plan effective feedback encounters: Cesar Orsini, Veena Rodrigues, Jorge Tricio, Margarita Rosel; J Educ Eval Health Prof. 2022;19:35. Published online December 19, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.35

2,542 View
471 Download
1 Citations

Abstract PDF Supplementary Material

Giving constructive feedback is crucial for learners to bridge the gap between their current performance and the desired standards of competence. Giving effective feedback is a skill that can be learned, practiced, and improved. Therefore, our aim was to explore models in clinical settings and assess their transferability to different clinical feedback encounters. We identified the 6 most common and accepted feedback models, including the Feedback Sandwich, the Pendleton Rules, the One-Minute Preceptor, the SET-GO model, the R2C2 (Rapport/Reaction/Content/Coach), and the ALOBA (Agenda Led Outcome-based Analysis) model. We present a handy resource describing their structure, strengths and weaknesses, requirements for educators and learners, and suitable feedback encounters for use for each model. These feedback models represent practical frameworks for educators to adopt but also to adapt to their preferred style, combining and modifying them if necessary to suit their needs and context.

Citations

Citations to this article as recorded by

Feedback conversations: First things first?
Katharine A. Robb, Marcy E. Rosenbaum, Lauren Peters, Susan Lenoch, Donna Lancianese, Jane L. Miller
Patient Education and Counseling.2023; 115: 107849. CrossRef

Research articles

Physical therapy students’ perception of their ability of clinical and clinical decision-making skills enhanced after simulation-based learning courses in the United States: a repeated measures design: Fabian Bizama, Mansoor Alameri, Kristy Jean Demers, Derrick Ferguson Campbell; J Educ Eval Health Prof. 2022;19:34. Published online December 19, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.34

1,798 View
160 Download
3 Citations

Abstract PDF Supplementary Material

Purpose
It aimed to investigate physical therapy students’ perception of their ability of clinical and clinical decision-making skills after a simulation-based learning course in the United States.
Methods
Survey questionnaires were administered to voluntary participants, including 44 second and third-year physical therapy students of the University of St. Augustine for Health Sciences during 2021–2022. Thirty-six questionnaire items consisted of 4 demographic items, 1 general evaluation, 21 test items for clinical decision-making skills, and 4 clinical skill items. Descriptive and inferential statistics evaluated differences in students’ perception of their ability in clinical decision-making and clinical skills, pre- and post-simulation, and post-first clinical experience during 2021–2022.
Results
Friedman test revealed a significant increase from pre- to post-simulation in perception of the ability of clinical and clinical decision-making skills total tool score (P<0.001), clinical decision-making 21-item score (P<0.001), and clinical skills score (P<0.001). No significant differences were found between post-simulation and post-first clinical experience. Post-hoc tests indicated a significant difference between pre-simulation and post-simulation (P<0.001) and between pre-simulation and post-first clinical experience (P<0.001). Forty-three students (97.6%) either strongly agreed (59.1%) or agreed (38.5%) that simulation was a valuable learning experience.
Conclusion
The above findings suggest that simulation-based learning helped students begin their first clinical experience with enhanced clinical and clinical decision-making skills.

Citations

Citations to this article as recorded by

Simulación clínica mediada por tecnología: un escenario didáctico a partir de recursos para la formación de los profesionales en rehabilitación
Cyndi Yacira Meneses Castaño, Isabel Jimenez Becerra, Paola Teresa Penagos Gomez
Educación Médica.2023; 24(4): 100810. CrossRef
Self-Efficacy with Telehealth Examination: the Doctor of Physical Therapy Student Perspective
Derrick F. Campbell, Jean-Michel Brismee, Brad Allen, Troy Hooper, Manuel A. Domenech, Kathleen J. Manella
Philippine Journal of Physical Therapy.2023; 2(2): 12. CrossRef
Physiotherapists' training in oncology rehabilitation from entry‐level to advanced education: A qualitative study
Gianluca Bertoni, Valentina Conti, Marco Testa, Ilaria Coppola, Stefania Costi, Simone Battista
Physiotherapy Research International.2023;[Epub] CrossRef

Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study: Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park; J Educ Eval Health Prof. 2022;19:33. Published online December 12, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.33

1,136 View
101 Download
2 Citations

Abstract PDF Supplementary Material

Purpose
This study aims to apply the yes/no Angoff and Hofstee methods to actual Korean Medical Licensing Examination (KMLE) 2022 written examination data to estimate cut scores for the written KMLE.
Methods
Fourteen panelists gathered to derive the cut score of the 86th KMLE written examination data using the yes/no Angoff method. The panel reviewed the items individually before the meeting and shared their respective understanding of the minimum-competency physician. The standard setting process was conducted in 5 rounds over a total of 800 minutes. In addition, 2 rounds of the Hofstee method were conducted before starting the standard setting process and after the second round of yes/no Angoff.
Results
For yes/no Angoff, as each round progressed, the panel’s opinion gradually converged to a cut score of 198 points, and the final passing rate was 95.1%. The Hofstee cut score was 208 points out of a maximum 320 with a passing rate of 92.1% at the first round. It scored 204 points with a passing rate of 93.3% in the second round.
Conclusion
The difference between the cut scores obtained through yes/no Angoff and Hofstee methods did not exceed 2% points, and they were within the range of cut scores from previous studies. In both methods, the difference between the panelists decreased as rounds were repeated. Overall, our findings suggest the acceptability of cut scores and the possibility of independent use of both methods.

Citations

Citations to this article as recorded by

Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
Sun Huh
Journal of Educational Evaluation for Health Professions.2023; 20: 5. CrossRef
Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea
Hyunjoo Pai
Journal of Educational Evaluation for Health Professions.2023; 20: 8. CrossRef

Brief report

Self-directed learning quotient and common learning types of pre-medical students in Korea by the Multi-Dimensional Learning Strategy Test 2nd edition: a descriptive study: Sun Kim, A Ra Cho, Chul Woon Chung; J Educ Eval Health Prof. 2022;19:32. Published online November 28, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.32

1,009 View
115 Download

Abstract PDF Supplementary Material: This study aimed to find the self-directed learning quotient and common learning types of pre-medical students through the confirmation of 4 characteristics of learning strategies, including personality, motivation, emotion, and behavior. The response data were collected from 277 out of 294 target first-year pre-medical students from 2019 to 2021, using the Multi-Dimensional Learning Strategy Test 2nd edition. The most common learning type was a self-directed type (44.0%), stagnant type (33.9%), latent type (14.4%), and conscientiousness type (7.6%). The self-directed learning index was high (29.2%), moderate (24.6%), somewhat high (21.7%), somewhat low (14.4%), and low (10.1%). This study confirmed that many students lacked self-directed learning capabilities for learning strategies. In addition, it was found that the difficulties experienced by each student were different, and the variables resulting in difficulties were also diverse. It may provide insights into how to develop programs that can help students increase their self-directed learning capability.

Research articles

Medical student selection process enhanced by improving selection algorithms and changing the focus of interviews in Australia: a descriptive study: Boaz Shulruf, Gary Mayer Velan, Sean Edward Kennedy; J Educ Eval Health Prof. 2022;19:31. Published online November 28, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.31

1,238 View
103 Download

Abstract PDF Supplementary Material: Purpose
The study investigates the efficacy of new features introduced to the selection process for medical school at the University of New South Wales, Australia: (1) considering the relative ranks rather than scores of the Undergraduate Medicine and Health Sciences Admission Test and Australian Tertiary Admission Rank; (2) structured interview focusing on interpersonal interaction and concerns should the applicants become students; and (3) embracing interviewers’ diverse perspectives.
Methods
Data from 5 cohorts of students were analyzed, comparing outcomes of the second year in the medicine program of 4 cohorts of the old selection process and 1 of the new process. The main analysis comprised multiple linear regression models for predicting academic, clinical, and professional outcomes, by section tools and demographic variables.
Results
Selection interview marks from the new interview (512 applicants, 2 interviewers each) were analyzed for inter-rater reliability, which identified a high level of agreement (kappa=0.639). No such analysis was possible for the old interview since it required interviewers to reach a consensus. Multivariate linear regression models utilizing outcomes for 5 cohorts (N=905) revealed that the new selection process was much more effective in predicting academic and clinical achievement in the program (R2=9.4%–17.8% vs. R2=1.5%–8.4%).
Conclusion
The results suggest that the medical student selection process can be significantly enhanced by employing a non-compensatory selection algorithm; and using a structured interview focusing on interpersonal interaction and concerns should the applicants become students; as well as embracing interviewers’ diverse perspectives.

Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study: Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah; J Educ Eval Health Prof. 2022;19:30. Published online November 7, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.30

1,157 View
115 Download
1 Citations

Abstract PDF Supplementary Material

Purpose
Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.
Methods
Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.
Results
Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.
Conclusion
The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.

Citations

Citations to this article as recorded by

Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2023;[Epub] CrossRef

Enhanced numeracy skills following team-based learning in United States pharmacy students: a longitudinal cohort study: Rob Edwin Carpenter, Leanne Coyne, Dave Silberman, Jody Kyoto Takemoto; J Educ Eval Health Prof. 2022;19:29. Published online October 27, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.29

1,259 View
143 Download

Abstract PDF Supplementary Material: Purpose
The literature suggests that the ability to numerate cannot be fully understood without accounting for the social context in which mathematical activity is represented. Team-based learning (TBL) is an andragogical approach with theoretical links to sociocultural and community-of-practice learning. This study aimed to quantitatively explore the impact of TBL instruction on numeracy development in 2 cohorts of pharmacy students and identify the impact of TBL instruction on numeracy development from a social perspective for healthcare education.
Methods
Two cohorts of students were administered the Health Science Reasoning Test-Numeracy (HSRT-N) before beginning pharmacy school. Two years after using TBL as the primary method of instruction, both comprehensive and domain data from the HSRT-N were analyzed.
Results
In total, 163 pharmacy student scores met the inclusion criteria. The students’ numeracy skills measured by HSRT-N improved after 2 years of TBL instruction.
Conclusion
Numeracy was the most significantly improved HSRT-N domain in pharmacy students following two years of TBL instruction. Although a closer examination of numeracy development in TBL is warranted, initial data suggest that TBL instruction may be an adequate proxy for advancing numeracy in a cohort of pharmacy students. TBL may encourage a social practice of mathematics to improve pharmacy students’ ability to numerate critically.

Factors affecting nursing and health technician students' satisfaction with distance learning during the COVID-19 pandemic in Morocco: a descriptive study: Aziz Naciri, Mohamed Radid, Abderrahmane Achbani, Mohamed Amine Baba, Ahmed Kharbach, Ghizlane Chemsi; J Educ Eval Health Prof. 2022;19:28. Published online October 17, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.28

2,022 View
229 Download
1 Citations

Abstract PDF Supplementary Material

Purpose
Distance learning describes any learning based on the use of new multimedia technologies and the internet to allow students to acquire new knowledge and skills at a distance. This study aimed to determine satisfaction levels with distance learning and associated factors among nursing and health technician students during the coronavirus disease 2019 pandemic in Morocco.
Methods
An descriptive study was conducted between April and June 2022 among nursing and health technician students using a self-administered instrument. The student satisfaction questionnaire consists of 24 questions categorized into 6 subscales: instructor, technology, course setup, interaction, outcomes, and overall satisfaction. It was based on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). Univariate and multivariate logistic regression analyses were conducted to identify factors associated with student satisfaction during distance learning.
Results
A total of 330 students participated in this study, and 176 students (53.3%) were satisfied with the distance learning activities. A mean score higher than 2.8 out of 5 was obtained for all subscales. Multiple regression analysis showed that students’ year of study (adjusted odds ratio [aOR], 2.34; 95% confidence interval [CI], 1.28–4.27) and internet quality (aOR, 0.47; 95% CI, 0.29–0.77) were the significant factors associated with students’ satisfaction during distance learning.
Conclusion
This study highlights the satisfaction level of students and factors that influenced it during distance learning. A thorough understanding of student satisfaction with digital environments will contribute to the successful implementation of distance learning devices in nursing.

Citations

Citations to this article as recorded by

Satisfaction with online education among students, faculty, and parents before and after the COVID-19 outbreak: Evidence from a meta-analysis
Tianyuan Xu, Ling Xue
Frontiers in Psychology.2023;[Epub] CrossRef

Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study: Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf; J Educ Eval Health Prof. 2022;19:27. Published online October 17, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.27

1,339 View
109 Download

Abstract PDF Supplementary Material: Purpose
Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method.
Methods
The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school’s final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station.
Results
On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts.
Conclusion
The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.

Acceptability of the 8-case objective structured clinical examination of medical students in Korea using generalizability theory: a reliability study: Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu; J Educ Eval Health Prof. 2022;19:26. Published online September 8, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.26

1,691 View
203 Download
1 Citations

Abstract PDF Supplementary Material

Purpose
This study investigated whether the reliability was acceptable when the number of cases in the objective structured clinical examination (OSCE) decreased from 12 to 8 using generalizability theory (GT).
Methods
This psychometric study analyzed the OSCE data of 439 fourth-year medical students conducted in the Busan and Gyeongnam areas of South Korea from July 12 to 15, 2021. The generalizability study (G-study) considered 3 facets—students (p), cases (c), and items (i)—and designed the analysis as p×(i:c) due to items being nested in a case. The acceptable generalizability (G) coefficient was set to 0.70. The G-study and decision study (D-study) were performed using G String IV ver. 6.3.8 (Papawork, Hamilton, ON, Canada).
Results
All G coefficients except for July 14 (0.69) were above 0.70. The major sources of variance components (VCs) were items nested in cases (i:c), from 51.34% to 57.70%, and residual error (pi:c), from 39.55% to 43.26%. The proportion of VCs in cases was negligible, ranging from 0% to 2.03%.
Conclusion
The case numbers decreased in the 2021 Busan and Gyeongnam OSCE. However, the reliability was acceptable. In the D-study, reliability was maintained at 0.70 or higher if there were more than 21 items/case in 8 cases and more than 18 items/case in 9 cases. However, according to the G-study, increasing the number of items nested in cases rather than the number of cases could further improve reliability. The consortium needs to maintain a case bank with various items to implement a reliable blueprinting combination for the OSCE.

Citations

Citations to this article as recorded by

Applying the Generalizability Theory to Identify the Sources of Validity Evidence for the Quality of Communication Questionnaire
Flávia Del Castanhel, Fernanda R. Fonseca, Luciana Bonnassis Burg, Leonardo Maia Nogueira, Getúlio Rodrigues de Oliveira Filho, Suely Grosseman
American Journal of Hospice and Palliative Medicine®.2023;[Epub] CrossRef

Medical students’ self-assessed efficacy and satisfaction with training on endotracheal intubation and central venous catheterization with smart glasses in Taiwan: a non-equivalent control-group pre- and post-test study: Yu-Fan Lin, Chien-Ying Wang, Yen-Hsun Huang, Sheng-Min Lin, Ying-Ying Yang; J Educ Eval Health Prof. 2022;19:25. Published online September 2, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.25

1,980 View
214 Download
1 Citations

Abstract PDF Supplementary Material

Purpose
Endotracheal intubation and central venous catheterization are essential procedures in clinical practice. Simulation-based technology such as smart glasses has been used to facilitate medical students’ training on these procedures. We investigated medical students’ self-assessed efficacy and satisfaction regarding the practice and training of these procedures with smart glasses in Taiwan.
Methods
This observational study enrolled 145 medical students in the 5th and 6th years participating in clerkships at Taipei Veterans General Hospital between October 2020 and December 2021. Students were divided into the smart glasses or the control group and received training at a workshop. The primary outcomes included students’ pre- and post-intervention scores for self-assessed efficacy and satisfaction with the training tool, instructor’s teaching, and the workshop.
Results
The pre-intervention scores for self-assessed efficacy of 5th- and 6th-year medical students in endotracheal intubation and central venous catheterization procedures showed no significant difference. The post-intervention score of self-assessed efficacy in the smart glasses group was better than that of the control group. Moreover, 6th-year medical students in the smart glasses group showed higher satisfaction with the training tool, instructor’s teaching, and workshop than those in the control group.
Conclusion
Smart glasses served as a suitable simulation tool for endotracheal intubation and central venous catheterization procedures training in medical students. Medical students practicing with smart glasses showed improved self-assessed efficacy and higher satisfaction with training, especially for procedural steps in a space-limited field. Simulation training on procedural skills with smart glasses in 5th-year medical students may be adjusted to improve their satisfaction.

Citations

Citations to this article as recorded by

The use of smart glasses in nursing education: A scoping review
Charlotte Romare, Lisa Skär
Nurse Education in Practice.2023; 73: 103824. CrossRef

First
Prev
Page of 3
Next
Last

Previous issues