Journal of Educational Evaluation for Health Professions

Research article

Agreement between medical students’ peer assessments and faculty assessments in advanced resuscitation skills examinations in South Korea: Jinwoo Jeong, Song Yi Park, Kyung Hoon Sun; J Educ Eval Health Prof. 2021;18:4. Published online March 25, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.4

5,300 View
289 Download

Abstract PDF Supplementary Material: Purpose
In medical education, peer assessment is considered to be an effective learning strategy. Although several studies have examined agreement between peer and faculty assessments regarding basic life support (BLS), few studies have done so for advanced resuscitation skills (ARS) such as intubation and defibrillation. Therefore, this study aimed to determine the degree of agreement between medical students’ and faculty assessments of ARS examinations.
Methods
This retrospective explorative study was conducted during the emergency medicine (EM) clinical clerkship of fourth-year medical students from April to July 2020. A faculty assessor (FA) and a peer assessor (PA) assessed each examinee’s resuscitation skills (including BLS, intubation, and defibrillation) using a checklist that consisted of 20 binary items (performed or not performed) and 1 global proficiency rating using a 5-point Likert scale. The prior examinee assessed the next examinee after feedback and training as a PA. All 54 students participated in peer assessment. The assessments of 44 FA/PA pairs were analyzed using the intraclass correlation coefficient (ICC) and Gwet’s first-order agreement coefficient.
Results
The PA scores were higher than the FA scores (mean±standard deviation, 20.2±2.5 [FA] vs. 22.3±2.4 [PA]; P<0.001). The agreement was poor to moderate for the overall checklist (ICC, 0.55; 95% confidence interval [CI], 0.31 to 0.73; P<0.01), BLS (ICC, 0.19; 95% CI, -0.11 to 0.46; P<0.10), intubation (ICC, 0.51; 95% CI, 0.26 to 0.70; P<0.01), and defibrillation (ICC, 0.49; 95% CI, 0.23 to 0.68; P<0.01).
Conclusion
Senior medical students showed unreliable agreement in ARS assessments compared to faculty assessments. If a peer assessment is planned in skills education, comprehensive preparation and sufficient assessor training should be provided in advance.

Brief Report

Clinical performance of medical students in Korea in a whole-task emergency station in the objective structured clinical examination with a standardized patient complaining of palpitations: Song Yi Park, Hyun-Hee Kong, Min-Jeong Kim, Yoo Sang Yoon, Sang-Hwa Lee, Sunju Im, Ji-Hyun Seo; J Educ Eval Health Prof. 2020;17:42. Published online December 16, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.42

5,161 View
135 Download
2 Web of Science
2 Crossref

This study assessed the clinical performance of 150 third-year medicalstudents in Busan, Korea in a whole-task emergency objective structured clinical examination station that simulated a patient with palpitations visiting the emergency department. The examination was conducted from November 25 to 27, 2019. Clinical performance was assessed as the number and percentage of students who performed history-taking (HT), a physical examination (PE), an electrocardiography (ECG) study, patient education (Ed), and clinical reasoning (CR), which were items on the checklist. It was found that 18.0% of students checked the patient’s pulse, 51.3% completed an ECG study, and 57.9% explained the results to the patient. A sizable proportion (38.0%) of students did not even attempt an ECG study. In a whole-task emergency station, students showed good performance on HT and CR, but unsatisfactory results for PE, ECG study, and Ed. Clinical skills educational programs for subjected student should focus more on PE, timely diagnostic tests, and sufficient Ed.

Citations

Citations to this article as recorded by

Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef
Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
Journal of Educational Evaluation for Health Professions.2021; 18: 25. CrossRef

Research articles

Use of graded responsibility and common entrustment considerations among United States emergency medicine residency programs: Jason Lai, Benjamin Holden Schnapp, David Simon Tillman, Mary Westergaard, Jamie Hess, Aaron Kraut; J Educ Eval Health Prof. 2020;17:11. Published online April 20, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.11

6,077 View
98 Download
1 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
The Accreditation Council for Graduate Medical Education (ACGME) requires all residency programs to provide increasing autonomy as residents progress through training, known as graded responsibility. However, there is little guidance on how to implement graded responsibility in practice and a paucity of literature on how it is currently implemented in emergency medicine (EM). We sought to determine how EM residency programs apply graded responsibility across a variety of activities and to identify which considerations are important in affording additional responsibilities to trainees.
Methods
We conducted a cross-sectional study of EM residency programs using a 23-question survey that was distributed by email to 162 ACGME-accredited EM program directors. Seven different domains of practice were queried.
Results
We received 91 responses (56.2% response rate) to the survey. Among all domains of practice except for managing critically ill medical patients, the use of graded responsibility exceeded 50% of surveyed programs. When graded responsibility was applied, post-graduate year (PGY) level was ranked an “extremely important” or “very important” consideration between 80.9% and 100.0% of the time.
Conclusion
The majority of EM residency programs are implementing graded responsibility within most domains of practice. When decisions are made surrounding graded responsibility, programs still rely heavily on the time-based model of PGY level to determine advancement.

Citations

Citations to this article as recorded by

Do you see what I see?: exploring trends in organizational culture perceptions across residency programs
Jennifer H. Chen, Paula Costa, Aimee Gardner
Global Surgical Education - Journal of the Association for Surgical Education.2024;[Epub] CrossRef
Guiding Fellows to Independent Practice
Maybelle Kou, Aline Baghdassarian, Kajal Khanna, Nazreen Jamal, Michele Carney, Daniel M. Fein, In Kim, Melissa L. Langhan, Jerri A. Rose, Noel S. Zuckerbraun, Cindy G. Roskind
Pediatric Emergency Care.2022; 38(10): 517. CrossRef

Efficacy of an asynchronous electronic curriculum in emergency medicine education in the United States: Alisa Wray, Kathryn Bennett, Megan Boysen-Osborn, Warren Wiechmann, Shannon Toohey; J Educ Eval Health Prof. 2017;14:29. Published online December 11, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.29

32,161 View
245 Download
11 Web of Science
8 Crossref

Abstract PDF Supplementary Material

Purpose
The aim of this study was to measure the effect of an iPad-based asynchronous curriculum on emergency medicine resident performance on the in-training exam (ITE). We hypothesized that the implementation of an asynchronous curriculum (replacing 1 hour of weekly didactic time) would result in non-inferior ITE scores compared to the historical scores of residents who had participated in the traditional 5-hour weekly didactic curriculum.
Methods
The study was a retrospective, non-inferiority study. conducted at the University of California, Irvine Emergency Medicine Residency Program. We compared ITE scores from 2012 and 2013, when there were 5 weekly hours of didactic content, with scores from 2014 and 2015, when 1 hour of conference was replaced with asynchro-nous content. Examination results were compared using a non-inferiority data analysis with a 10% margin of difference.
Results
Using a non-inferiority test with a 95% confidence interval, there was no difference between the 2 groups (before and after implementation of asynchronous learning), as the confidence interval for the change of the ITE was −3.5 to 2.3 points, whereas the 10% non-inferiority margin was 7.8 points.
Conclusion
Replacing 1 hour of didactic conference with asynchronous learning showed no negative impact on resident ITE scores.

Citations

Citations to this article as recorded by

An Asynchronous Curriculum: Learner Perspectives on Incorporating Asynchronous Learning Into In-Person and Virtual Emergency Residency Didactics
Emily L Jameyfield, Semhar Tesfai, Alejandro A Palma, Adriana S Olson
Cureus.2023;[Epub] CrossRef
A randomized controlled trial to examine the effect of the Pediatric Opioid Analgesia Self‐Instruction System (PedOASIS) tool on pediatric hematology/oncology trainee education
Rebecca E. MacDonell‐Yilmaz, Anarina Murillo, Jennifer G. Welch
Pediatric Blood & Cancer.2023;[Epub] CrossRef
Development and Validation of Pediatric Opioid Analgesia Self-Instruction System (PedOASIS): An Opioid Knowledge Tool for Pediatric Clinicians
Rebecca E. MacDonell-Yilmaz, Angela Anderson, Priya Hirway, Jennifer G. Welch
Journal of Pediatric Hematology/Oncology.2022; 44(1): e204. CrossRef
Utilizing social media for cardiovascular education
Christina Mansour, Nooshin Beygui, Mamas A Mamas, Purvi J Parwani
Heart.2022; 108(15): 1240. CrossRef
Nephrology Trainee Education Needs Assessment: Five Years and a Pandemic Later
Benjamin S. Ko, Kurtis A. Pivert, Rob Rope, Anna M. Burgner, Joshua S. Waitzman, Susan M. Halbach, Suzanne M. Boyle, Lili Chan, Stephen M. Sozio
Kidney Medicine.2022; 4(11): 100548. CrossRef
Planning Engaging, Remote, Synchronous Didactics in the COVID-19 Pandemic Era
Ronald Rivera, Jonathan Smart, Sangeeta Sakaria, Alisa Wray, Warren Wiechmann, Megan Boysen-Osborn, Shannon Toohey
JMIR Medical Education.2021; 7(2): e25213. CrossRef
Pediatric Resident Engagement With an Online Critical Care Curriculum During the Intensive Care Rotation*
Dennis A. Daniel, Sue E. Poynter, Christopher P. Landrigan, Charles A. Czeisler, Jeffrey P. Burns, Traci A. Wolbrink
Pediatric Critical Care Medicine.2020; 21(11): 986. CrossRef
Asynchronous Curriculum “Socially Synchronized”: Learning Via Competition
Jon Smart, Adriana Segura Olson, Andrew Muck
Western Journal of Emergency Medicine.2018; 20(1): 6. CrossRef

Research Articles

Impact of a novel, resource appropriate resuscitation curriculum on Nicaraguan resident physician’s management of cardiac arrest: Breena R. Taira, Aristides Orue, Edward Stapleton, Luis Lovato, Sitaram Vangala, Lucia Solorzano Tinoco, Orlando Morales; J Educ Eval Health Prof. 2016;13:25. Published online June 9, 2016; DOI: https://doi.org/10.3352/jeehp.2016.13.25

27,959 View
208 Download
4 Web of Science
6 Crossref

Abstract PDF

Purpose
Project Strengthening Emergency Medicine, Investing in Learners in Latin America (SEMILLA) created a novel, language and resource appropriate course for the resuscitation of cardiac arrest for Nicaraguan resident physicians. We hypothesized that participation in the Project SEMILLA resuscitation program would significantly improve the physician’s management of simulated code scenarios. Methods: Thirteen Nicaraguan resident physicians were evaluated while managing simulated cardiac arrest scenarios before, immediately, and at 6 months after participating in the Project SEMILLA resuscitation program. This project was completed in 2014 in Leon, Nicaragua. The Cardiac Arrest Simulation Test (CASTest), a validated scoring system, was used to evaluate performance on a standardized simulated cardiac arrest scenario. Mixed effect logistic regression models were constructed to assess outcomes. Results: On the pre-course simulation exam, only 7.7% of subjects passed the test. Immediately post-course, the subjects achieved a 30.8% pass rate and at 6 months after the course, the pass rate was 46.2%. Compared with pre-test scores, the odds of passing the CASTest at 6 months after the course were 21.7 times higher (95% CI 4.2 to 112.8, P<0.001). Statistically significant improvement was also seen on the number of critical items completed (OR=3.75, 95% CI 2.71-5.19), total items completed (OR=4.55, 95% CI 3.4-6.11), and number of “excellent” scores on a Likert scale (OR=2.66, 95% CI 1.85-3.81). Conclusions: Nicaraguan resident physicians demonstrate improved ability to manage simulated cardiac arrest scenarios after participation in the Project SEMILLA resuscitation course and retain these skills.

Citations

Citations to this article as recorded by

Симуляційні технології у навчанні інтернів та безперервній професійній освіті дипломованих лікарів-анестезіологів
Kateryna Bielka, Мariya Smirnova, Hanna Fomina
The Ukrainian Scientific Medical Youth Journal.2023; 139(2): 64. CrossRef
Analysis of Unstructured Text-Based Data Using Machine Learning Techniques: The Case of Pediatric Emergency Department Records in Nicaragua
Giulia Lorenzoni, Silvia Bressan, Corrado Lanera, Danila Azzolina, Liviana Da Dalt, Dario Gregori
Medical Care Research and Review.2021; 78(2): 138. CrossRef
Assessing the impact of resuscitation residents on the treatment of cardiopulmonary resuscitation patients
David M. Lee, David A. Berger, Patrick A. Wloszczynski, Patrick Karabon, Lihua Qu, Michael J. Burla
The American Journal of Emergency Medicine.2021; 41: 46. CrossRef
Resuscitation Resident Impact in the Treatment of Sepsis
Michael J Burla, Nashid Shinthia, Judith A Boura, Lihua QU, David A Berger
Cureus.2020;[Epub] CrossRef
A Review of Anesthesia Simulation in Low-Income Countries
Oleg Turkot, Michael C. Banks, Seung Woo Lee, Adam Dodson, Shirley Duarte, Mwemezi Kaino, Howard Nelson-Williams, Serkan Toy, John Sampson
Current Anesthesiology Reports.2019; 9(1): 1. CrossRef
Community-based in situ simulation: bringing simulation to the masses
Barbara M. Walsh, Marc A. Auerbach, Marcie N. Gawel, Linda L. Brown, Bobbi J. Byrne, Aaron Calhoun
Advances in Simulation.2019;[Epub] CrossRef

Psychometric properties of a novel knowledge assessment tool of mechanical ventilation for emergency medicine residents in the northeastern United States: Jeremy B. Richards, Tania D. Strout, Todd A. Seigel, Susan R. Wilcox; J Educ Eval Health Prof. 2016;13:10. Published online February 16, 2016; DOI: https://doi.org/10.3352/jeehp.2016.13.10

28,019 View
179 Download
4 Web of Science
4 Crossref

Abstract PDF

Purpose
Prior descriptions of the psychometric properties of validated knowledge assessment tools designed to determine Emergency medicine (EM) residents understanding of physiologic and clinical concepts related to mechanical ventilation are lacking. In this setting, we have performed this study to describe the psychometric and performance properties of a novel knowledge assessment tool that measures EM residents’ knowledge of topics in mechanical ventilation.
Methods
Results from a multicenter, prospective, survey study involving 219 EM residents from 8 academic hospitals in northeastern United States were analyzed to quantify reliability, item difficulty, and item discrimination of each of the 9 questions included in the knowledge assessment tool for 3 weeks, beginning in January 2013.
Results
The response rate for residents completing the knowledge assessment tool was 68.6% (214 out of 312 EM residents). Reliability was assessed by both Cronbach’s alpha coefficient (0.6293) and the Spearman-Brown coefficient (0.6437). Item difficulty ranged from 0.39 to 0.96, with a mean item difficulty of 0.75 for all 9 questions. Uncorrected item discrimination values ranged from 0.111 to 0.556. Corrected item-total correlations were determined by removing the question being assessed from analysis, resulting in a range of item discrimination from 0.139 to 0.498.
Conclusion
Reliability, item difficulty and item discrimination were within satisfactory ranges in this study, demonstrating acceptable psychometric properties of this knowledge assessment tool. This assessment indicates that this knowledge assessment tool is sufficiently rigorous for use in future research studies or for assessment of EM residents for evaluative purposes.

Citations

Citations to this article as recorded by

Comparison of three methods for teaching mechanical ventilation in an emergency setting to sixth-year medical students: a randomized trial
Fernando Sabia Tallo, Letícia Sandre Vendrame, André Luciano Baitello
Revista da Associação Médica Brasileira.2020; 66(10): 1409. CrossRef
Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2016
Nicole M. Dubosh, Jaime Jordan, Lalena M. Yarris, Edward Ullman, Joshua Kornegay, Daniel Runde, Amy Miller Juve, Jonathan Fisher, Teresa Chan
AEM Education and Training.2019; 3(1): 58. CrossRef
Mechanical Ventilation Training During Graduate Medical Education: Perspectives and Review of the Literature
Jonathan M. Keller, Dru Claar, Juliana Carvalho Ferreira, David C. Chu, Tanzib Hossain, William Graham Carlos, Jeffrey A. Gold, Stephanie A. Nonas, Nitin Seam
Journal of Graduate Medical Education.2019; 11(4): 389. CrossRef
Development and validation of a questionnaire to assess the knowledge of mechanical ventilation in urgent care among students in their last-year medical course in Brazil
Fernando Sabia Tallo, Simone de Campos Vieira Abib, Andre Luciano Baitello, Renato Delascio Lopes
Clinics.2019; 74: e663. CrossRef

Emergency medicine and internal medicine trainees’ smartphone use in clinical settings in the United States: Sonja E. Raaum, Christian Arbelaez, Carlos Eduardo Vallejo, Andres M. Patino, Jorie M. Colbert-Getz, Caroline K. Milne; J Educ Eval Health Prof. 2015;12:48. Published online October 29, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.48

27,424 View
147 Download
5 Web of Science
6 Crossref

Abstract PDF

Purpose
Smartphone technology offers a multitude of applications (apps) that provide a wide range of functions for healthcare professionals. Medical trainees are early adopters of this technology, but how they use smartphones in clinical care remains unclear. Our objective was to further characterize smartphone use by medical trainees at two United States academic institutions, as well as their prior training in the clinical use of smartphones. Methods: In 2014, we surveyed 347 internal medicine and emergency medicine resident physicians at the University of Utah and Brigham and Women’s Hospital about their smartphone use and prior training experiences. Scores (0%–100%) were calculated to assess the frequency of their use of general features (email, text) and patient-specific apps, and the results were compared according to resident level and program using the Mann-Whitney U-test. Results: A total of 184 residents responded (response rate, 53.0%). The average score for using general features, 14.4/20 (72.2%) was significantly higher than the average score for using patient-specific features and apps, 14.1/44 (33.0%, P<0.001). The average scores for the use of general features, were significantly higher for year 3–4 residents, 15.0/20 (75.1%) than year 1–2 residents, 14.1/20 (70.5%, P=0.035), and for internal medicine residents, 14.9/20 (74.6%) in comparison to emergency medicine residents, 12.9/20 (64.3%, P= 0.001). The average score reflecting the use of patient-specific apps was significantly higher for year 3–4 residents, 16.1/44 (36.5%) than for year 1–2 residents, 13.7/44 (31.1%; P=0.044). Only 21.7% of respondents had received prior training in clinical smartphone use. Conclusion: Residents used smartphones for general features more frequently than for patient-specific features, but patient-specific use increased with training. Few residents have received prior training in the clinical use of smartphones.

Citations

Citations to this article as recorded by

Prevalence and patterns of mobile device usage among physicians in clinical practice: A systematic review
Judith Kraushaar, Sabine Bohnet-Joschko
Health Informatics Journal.2023; 29(2): 146045822311692. CrossRef
SMARTPHONE MEDICAL APPLICATION USE AND ASSOCIATED FACTORS AMONG PHYSICIAN AT REFERRAL HOSPITALS IN AMHARA REGION NORTH ETHIOPIA: A CROSS-SECTIONAL STUDY, 2019. (Preprint)
Gizaw Hailiye, Binyam Cheklu Tilahun, Habtamu Alganeh Guadie, Ashenafi Tazebew Amare
JMIR mHealth and uHealth.2020;[Epub] CrossRef
Online webinar training to analyse complex atrial fibrillation maps: A randomized trial
João Mesquita, Natasha Maniar, Tina Baykaner, Albert J. Rogers, Mark Swerdlow, Mahmood I. Alhusseini, Fatemah Shenasa, Catarina Brizido, Daniel Matos, Pedro Freitas, Ana Rita Santos, Gustavo Rodrigues, Claudia Silva, Miguel Rodrigo, Yan Dong, Paul Clopton
PLOS ONE.2019; 14(7): e0217988. CrossRef
Learning strategies among adult CHD fellows
Jouke P. Bokma, Joshua A. Daily, Adrienne H. Kovacs, Erwin N. Oechslin, Helmut Baumgartner, Paul Khairy, Barbara J.M. Mulder, Gruschen R. Veldtman
Cardiology in the Young.2019; 29(11): 1356. CrossRef
Incorporating sleep medicine content into medical school through neuroscience core curricula
Rachel Marie E. Salas, Roy E. Strowd, Imran Ali, Madhu Soni, Logan Schneider, Joseph Safdieh, Bradley V. Vaughn, Alon Y. Avidan, Jane B. Jeffery, Charlene E. Gamaldo
Neurology.2018; 91(13): 597. CrossRef
E-Scripts and Cell Phones
Susie T. Harris, Paul D. Bell, Elizabeth A. Baker
The Health Care Manager.2017; 36(4): 320. CrossRef

First
Prev
Page of 1
Next
Last

Search