Journal of Educational Evaluation for Health Professions

Research articles

Content validity test of a safety checklist for simulated participants in simulation-based education in the United Kingdom: a methodological study: Matthew Bradley; J Educ Eval Health Prof. 2022;19:21. Published online August 25, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.21

1,621 View
157 Download

Abstract PDF Supplementary Material: Purpose
Simulation training is an ever-growing means of healthcare education and often involves simulated participants (SPs), commonly known as actors. Simulation-based education (SBE) can sometimes endanger SPs, and as such we have created a safety checklist for them to follow. This study describes how we developed the checklist through a quality improvement project, and then evaluated feedback responses to assess whether SPs felt our checklist was safe.
Methods
The checklist was provided to SPs working in an acute trust simulation service when delivering multidisciplinary SBE over 4 months. Using multiple plan–do–study–act cycles, the checklist was refined by reflecting on SP feedback to ensure that the standards of the safe simulation were met. We collected 21 responses from September to December 2021 after SPs completed an SBE event.
Results
The responses showed that 100% of SPs felt safe during SBE when using our checklist. The average “confidence in safety” rating before using the checklist was 6.8/10, which increased significantly to 9.2/10 after using the checklist (P<0.0005). The checklist was refined throughout the 4 months and implemented in adult and pediatric SBE as a standard operating procedure.
Conclusion
We recommend using our safety checklist as a standard operating procedure to improve the confidence and safety of SPs during safe and effective simulations.

Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea: Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu; J Educ Eval Health Prof. 2021;18:25. Published online September 27, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.25

5,333 View
295 Download
2 Web of Science
3 Crossref

Purpose
Setting standards is critical in health professions. However, appropriate standard setting methods do not always apply to the set cut score in performance assessment. The aim of this study was to compare the cut score when the standard setting is changed from the norm-referenced method to the borderline group method (BGM) and borderline regression method (BRM) in an objective structured clinical examination (OSCE) in medical school.
Methods
This was an explorative study to model the implementation of the BGM and BRM. A total of 107 fourth-year medical students attended the OSCE at 7 stations for encountering standardized patients (SPs) and at 1 station for performing skills on a manikin on July 15th, 2021. Thirty-two physician examiners evaluated the performance by completing a checklist and global rating scales.
Results
The cut score of the norm-referenced method was lower than that of the BGM (P<0.01) and BRM (P<0.02). There was no significant difference in the cut score between the BGM and BRM (P=0.40). The station with the highest standard deviation and the highest proportion of the borderline group showed the largest cut score difference in standard setting methods.
Conclusion
Prefixed cut scores by the norm-referenced method without considering station contents or examinee performance can vary due to station difficulty and content, affecting the appropriateness of standard setting decisions. If there is an adequate consensus on the criteria for the borderline group, standard setting with the BRM could be applied as a practical and defensible method to determine the cut score for OSCE.

Citations

Citations to this article as recorded by

Analyzing the Quality of Objective Structured Clinical Examination in Alborz University of Medical Sciences
Suleiman Ahmadi, Amin Habibi, Mitra Rahimzadeh, Shahla Bahrami
Alborz University Medical Journal.2023; 12(4): 485. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

First
Prev
Page of 1
Next
Last

Search