Journal of Educational Evaluation for Health Professions

Research articles

Development and validity evidence for the resident-led large group teaching assessment instrument in the United States: a methodological study: Ariel Shana Frey-Vogel, Kristina Dzara, Kimberly Anne Gifford, Yoon Soo Park, Justin Berk, Allison Heinly, Darcy Wolcott, Daniel Adam Hall, Shannon Elliott Scott-Vernaglia, Katherine Anne Sparger, Erica Ye-pyng Chung; J Educ Eval Health Prof. 2024;21:3. Published online February 23, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.3

961 View
193 Download

Abstract PDF Supplementary Material: Purpose
Despite educational mandates to assess resident teaching competence, limited instruments with validity evidence exist for this purpose. Existing instruments do not allow faculty to assess resident-led teaching in a large group format or whether teaching was interactive. This study gathers validity evidence on the use of the Resident-led Large Group Teaching Assessment Instrument (Relate), an instrument used by faculty to assess resident teaching competency. Relate comprises 23 behaviors divided into 6 elements: learning environment, goals and objectives, content of talk, promotion of understanding and retention, session management, and closure.
Methods
Messick’s unified validity framework was used for this study. Investigators used video recordings of resident-led teaching from 3 pediatric residency programs to develop Relate and a rater guidebook. Faculty were trained on instrument use through frame-of-reference training. Resident teaching at all sites was video-recorded during 2018–2019. Two trained faculty raters assessed each video. Descriptive statistics on performance were obtained. Validity evidence sources include: rater training effect (response process), reliability and variability (internal structure), and impact on Milestones assessment (relations to other variables).
Results
Forty-eight videos, from 16 residents, were analyzed. Rater training improved inter-rater reliability from 0.04 to 0.64. The Φ-coefficient reliability was 0.50. There was a significant correlation between overall Relate performance and the pediatric teaching Milestone (r=0.34, P=0.019).
Conclusion
Relate provides validity evidence with sufficient reliability to measure resident-led large-group teaching competence.

Medical residents and attending physicians’ perceptions of feedback and teaching in the United States: a qualitative study: Madeleine Matthiesen, Michael S. Kelly, Kristina Dzara, Arabella Simpkin Begin; J Educ Eval Health Prof. 2022;19:9. Published online April 26, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.9

8,922 View
367 Download
3 Web of Science
3 Crossref

Purpose
Residents and attendings agree on the importance of feedback to resident education. However, while faculty report providing frequent feedback, residents often do not perceive receiving it, particularly in the context of teaching. Given the nuanced differences between feedback and teaching, we aimed to explore resident and attending perceptions of feedback and teaching in the clinical setting.
Methods
We conducted a qualitative study of internal medicine residents and attendings from December 2018 through March 2019 at the Massachusetts General Hospital to investigate perceptions of feedback in the inpatient clinical setting. Residents and faculty were recruited to participate in focus groups. Data were analyzed using thematic analysis to explore perspectives and barriers to feedback provision and identification.
Results
Five focus groups included 33 total participants in 3 attending (n=20) and 2 resident (n=13) groups. Thematic analysis of focus group transcripts identified 7 themes which organized into 3 thematic categories: (1) disentangling feedback and teaching, (2) delivering high-quality feedback, and (3) experiencing feedback in the group setting. Residents and attendings highlighted important themes in discriminating feedback from teaching. They indicated that while feedback is reactive in response to an action or behavior, teaching is proactive and oriented toward future endeavors.
Conclusion
Confusion between the critical concepts of teaching and feedback may be minimized by allowing them to each have their intended impact, either in response to prior events or aimed toward those yet to take place.

Citations

Citations to this article as recorded by

Resident Assessment of Clinician Educators According to Core ACGME Competencies
Bailey A. Pope, Patricia A. Carney, Mary C. Brooks, Doug R. Rice, Ashly A. Albright, Stephanie A. C. Halvorson
Journal of General Internal Medicine.2024; 39(3): 377. CrossRef
Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education
Laura Wong, Ethan Sacoransky, Wilma Hopman, Omar Islam, Andrew D. Chung, Benjamin Y. M. Kwan
Medical Education Online.2024;[Epub] CrossRef
Through the Looking Glass: Comparing Hospitalists' and Internal Medicine Residents' Perceptions of Feedback
Andrew V Raikhel, Helene Starks, Gabrielle Berger, Jeffrey Redinger
Cureus.2024;[Epub] CrossRef

First
Prev
Page of 1
Next
Last

Search