Skip Navigation
Skip to contents

JEEHP : Journal of Educational Evaluation for Health Professions

OPEN ACCESS
SEARCH
Search

Articles

Page Path
HOME > J Educ Eval Health Prof > Volume 13; 2016 > Article
Research article
Randomized study of effectiveness of computerized ultrasound simulators for an introductory course for residents in Brazil
Jack Philip Silva1*orcid, Trevor Plescia1orcid, Nathan Molina1orcid, Ana Claudia de Oliveira Tonelli2orcid, Mark Langdorf1orcid, John Christian Fox1orcid

DOI: https://doi.org/10.3352/jeehp.2016.13.16
Published online: April 4, 2016

1Department of Medicine, University of California Irvine School of Medicine, Irvine, CA, USA

2Department of Internal Medicine, Hospital de Clinicas de Porto Alegre, Porto Alegre, Brazil

*Corresponding email: jpsilva@uci.edu

: 

• Received: February 27, 2016   • Accepted: April 2, 2016

© 2016, Korea Health Personnel Licensing Examination Institute

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

  • 42,784 Views
  • 193 Download
  • 10 Web of Science
  • 14 Crossref
  • 12 Scopus
  • Purpose:
    This study aimed to assess the impact of ultrasound simulation (SonoSim) on educational outcomes of an introductory point-of-care ultrasound course compared to hands-on training with live models alone.
  • Methods:
    Fifty-three internal medicine residents without ultrasound experience were randomly assigned to control or experimental groups. They participated in an introductory point-of-care ultrasound course covering eight topics in eight sessions from June 23, 2014 until July 18, 2014. Both participated in lecture and hands-on training, but experimental group received an hour of computerized simulator training instead of a second hour of hands-on training. We assessed clinical knowledge and image acquisition with written multiple-choice and practical exams, respectively. Of the 53 enrolled, 40 participants (75.5%) completed the course and all testing.
  • Results:
    For the 30-item written exam, mean score of the experimental group was 23.1±3.4 (n=21) vs. 21.8±4.8 (n=19), (P>0 .05). For the practical exam, mean score for both groups was 8.7 out of 16 (P>0 .05).
  • Conclusion:
    The substitution of eight hours of ultrasound simulation training for live model scanning in a 24 hour training course did not enhance performance on written and image acquisition tests in an introductory ultrasound course for residents. This result suggests that ultrasound simulation technology used as a substitute for live model training on an hour-for-hour basis, did not improve learning outcomes. Further investigation into simulation as a total replacement for live model training will provide a clearer picture of the efficacy of ultrasound simulators in medical education.
The benefits of point of care (POC) ultrasound are well-known; it is inexpensive, noninvasive, portable, and increasingly present in rural and less developed areas [1]. Early exposure to ultrasound in medical education increases users’ aptitude in diagnostic and procedural sonography [2,3]. The use of simulated clinical experiences (e.g., standardized patients and high-tech mannequins) to augment conventional teaching methods is well-established in medical schools, but the utility of simulation in ultrasound education has yet to be fully investigated [4]. This article discusses the impact of SonoSim ver. 2.8.1. (Ultrasound Training Solution, Santa Monica, CA, USA) ultrasound simulators on learning outcomes for medical residents taking an introductory ultrasound course in a hospital, Brazil.
Fifty-three internal medicine residents of the Universidade Federal de Ciências da Saûde de Porto Alegre at Santa Casa de Misericordia Hospital in Porto Alegre, Rio Grande do Sul, Brazil participated in an introductory POC ultrasound course of eight three-hour sessions over four weeks. Residents had no prior experience with ultrasound. Each session covered one topic: knobology, focused assessment with sonography for trauma (FAST) protocol, head and neck, abdominal, cardiac, pulmonary, musculoskeletal, and procedures. We provided an hour-long lecture and two hours of hands-on training for each session. For the hands-on training portion, we randomly assigned subjects to control ‘live model only’ (LM) group (two hours scanning live models) or the ‘live model plus simulator’ (LM+S) experimental group (one hour with live models and one hour of self-instruction using the SonoSim training modules).
Three well-trained ‘blinded for peer review’ medical students gave the lectures and hands-on instruction using one another as live models. The student-instructor ratio was less than or equal to 6 to 1. Four ultrasound devices (SonoSite S Series, Bothell, WA, USA) and a procedure training block (CAE Blue Phantom Regional Anesthesia Ultrasound Training Block, Sarasota, FL, USA) were used for training. The LM group was allowed two hours of practice with a live model, while the LM+S group spent one hour with live models and one hour with the simulator. During each live model session, subjects were divided into small groups in which they practiced insonating key structures highlighted in the prior lecture while learning the basic function of the ultrasound devices.
The simulator used in this study was a SonoSim. It provides integrated hands-on ultrasound training, didactic instruction, and assessment using real patient ultrasound cases. Through its replication of the tactile experience of probe manipulation, the simulator is designed to mimic the experience of scanning an actual patient. Users are allowed to practice image acquisition and interpretation in a risk-free environment and test their knowledge and skills in clinical scenarios and module knowledge assessment questions [5]. During the hour with the simulator, the LM group used the software training modules pertaining to the focus of that session. Their learning was largely independent, relying on the instruction and feedback provided by the software.
Fifty-three participants were enrolled and randomized before instruction began. Of the 53 participants enrolled, 40 (75.5%) met the necessary criteria (including attendance of all eight sessions and completion of both the written and practical exams), leaving the groups with n=21 in LM+S and n=19 in LM. This sample size was shown to be sufficient to reach a power, 0.8 with an expected standard deviation of written exam test scores, 5 (α=0.05). The outcome measures were written exam and practical exam scores. An unpaired t-test was performed on the written and practical exam scores of the LM and LM+S groups. Statistical significance was set at P<0.05. Data was analyzed with STATA ver. 9.0 (Stata Co., College Station, TX, USA).
Written exam
The written exam had 30 image or video-based multiple choice questions (MCQ) assessing the subjects’ clinical knowledge of ultrasound functionality, diagnostic methods, and image interpretation (Supplementary file 1). We did not administer a pre-test because enrollment requirements stipulated that participants have no prior ultrasound experience.
Practical exam
The 16-point practical exam was designed to assess the subjects’ ability to operate the ultrasound device, properly insonate key POC ultrasound structures, and utilize advanced imaging functions (Appendix 1). Subjects were allowed six minutes to scan a live model and save five images (Fig. 1): (1) right upper quadrant organs and landmarks; (2) parasternal long axis view of the heart; (3) parasternal short axis view of the heart; (4) visceral-parietal pleural interface (VPPI) on M-mode (for detection of pneumothorax); and (5) right carotid artery/internal jugular vein of the neck with color Doppler.
For full credit, each window needed to include landmarks or structures specified by the exam instructions, e.g., the left ventricle, left atrium, interventricular septum, mitral valve and aortic outflow tract for the parasternal long axis view. Some images required the use of special device functions, including M-mode for the insonation of the VPPI and color Doppler for the neck vasculature.
The subjects had linear, phased-array, and curvilinear probes during the exam. Proctors, present for timing purposes only, paused the time when the subjects verbally expressed a desire to change transducers. Subjects were not provided feedback about the quality of their images or any instruction for improvement. When they were satisfied with their images, the subjects saved them with a coded label to be used for identification during the grading process.
Images were graded back in the United States by faculty at the ‘blinded for peer review’ department of ultrasound education with student identity and timing of assessment blinded. Scores were calculated as the sum of points earned from each image (16 points total) through successful acquisition of items specified by the grading rubric. Points were granted to images with potential diagnostic utility in a clinical setting. The study was approved by the institutional review board (IRB) as exempt under IRB number HS# 2014-1170.
Of the 53 students who participated in the course, 40 (75.5%) met the attendance requirements for all eight sessions and completed the 30-item MCQ written exam. The mean score of the LM+S group was 23.1±3.4 (n=21) vs. 21.8±4.8 (n=19) in the LM group (Fig. 2). There was no significant difference between the written exam scores of the two groups (P>0.05). Of 53 students who participated in the course, 41 (77.4%) met the attendance requirements for all eight sessions and completed the 16-item practical exam. The mean score for the LM+ S group was 8.7±4.8 (n=22) while the mean score for the LM group was also 8.7±3.9 (n=19) (P>0.05) (Fig. 2). Raw data of the results were available from Supplementary file 2.
Technology is a vital component of effective simulation-based education in ultrasound [6]. Simulators such as the SonoSim Ultrasound Training Solution, Phantom Task Trainers, and SonoTrainer have already been adopted into medical education programs throughout the United States [7]. These technologies have demonstrated promise to teach specific clinical applications of ultrasound, e.g., the FAST protocol, the detection of fetal anomalies, and advanced procedures [8,9]. Yet the efficacy of ultrasound simulation, its educational outcomes, and effects on clinical competence remain unclear. A 2011 systematic review of ultrasound procedural simulation concluded that there was little evidence to suggest that the extensive use of simulation-based ultrasound education would yield improved competence [10].
This study investigated the effectiveness of ultrasound simulator technology in conjunction with traditional teaching methods, with the hope that the simulators could overcome the logistical barriers to providing ultrasound training to students. A similar educational study showed that an ultrasound curriculum based upon Podcast lectures, peer instruction, and a standardized formative evaluation for each learning session led to improved educational outcomes compared to a traditional lecture format with dedicated practice sessions [11]. The study in Brazil aimed to build on that success and investigate simulation as a possible alternative to live model training. By combining these two modalities, it could be possible to provide ultrasound education in institutions lacking local expertise. Podcast lectures are a solution to lack of ultrasound faculty, and simulator training could reduce the cost of ultrasound machines and live models.
The use of ultrasound simulators as a partial replacement for live model training did not significantly affect learning outcomes in this study. This result can be interpreted in two ways: the simulators were an unnecessary adjunct to traditional learning methods, or conversely, the simulators served as an adequate replacement for live model training when used for half of the dedicated learning time. Either hypothesis is consistent with showing no difference in post-simulation scores.
Proficiency of the English language was a requirement to participate in this study, but inevitably, students had variable fluency, which we did not assess. Individuals aware of their limited grasp of English signed up for the course because they did not want to miss the opportunity to learn ultrasound. This could have led to a discrepancy between an individual’s knowledge and their written exam score. Furthermore language barrier may have impaired learning itself. Our students had imperfect attendance; we excluded subjects who did not sign into every session but it is possible that some who did may have left early to handle medical emergencies. Nearly all of the internal medicine residents that were accessible for the study participated, but residents from other specialties that could have satisfied the requirements for participation were not available. The addition of a third group that received only simulation training would have provided a clearer picture of the effectiveness of the simulators.
In conclusion, the substitution of eight hours of ultrasound simulation training for live model scanning in a 24 hour training course did not enhance novice student performance on written and image acquisition tests in an international introductory ultrasound course for residents. This result suggests that ultrasound simulation technology, as used here to substitute for live model training on an hour-for-hour basis, did not improve learning outcomes. Further investigation into simulation as a total replacement for live model training will provide a clearer picture of the efficacy of ultrasound simulators in medical education.

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Audio recording of the abstract.
jeehp-13-16-abstract-recording.avi
Supplementary file 1. Data files of written exam.
jeehp-13-16-supple1.pptx
Supplementary file 2. Raw data of the results.
jeehp-13-16-supple2.xlsx
Fig. 1.
Ultrasound images used as an answer key for the practical exam. (A) Right upper quadrant view. (B) Parasternal long axis view. (C) Parasternal short axis view. (D) Visceral-parietal pleural interface with M-mode. (E) Carotid/internal jugular vein with color Doppler.
jeehp-13-16f1.gif
Fig. 2.
Comparison of mean scores of the live model (LM) and live model plus simulator (LM+S) groups for the (A) written and (B) practical examinations. Vertical bars represent standard deviation.
jeehp-13-16f2.gif
  • 1. Sippel S, Muruganandan K, Levine A, Shah S. Review article: use of ultrasound in the developing world. Int J Emerg Med 2011;4:72. http://dx.doi.org/10.1186/1865-1380-4-72 ArticlePubMedPMC
  • 2. Fernandez-Frackelton M, Peterson M, Lewis RJ, Perez JE, Coates WC. A bedside ultrasound curriculum for medical students: prospective evaluation of skill acquisition. Teach Learn Med 2007;19:14-19. http://dx.doi.org/10.1080/10401330709336618 ArticlePubMed
  • 3. Hoppmann R, Cook T, Hunt P, Fowler S, Paulman L, Wells J, Richeson N, Thomas L, Wilson B, Neuffer F. Ultrasound in medical education: a vertical curriculum at the University of South Carolina School of Medicine. J S C Med Assoc 2006;102:330-334. PubMed
  • 4. Akaike M, Fukutomi M, Nagamune M, Fujimoto A, Tsuji A, Ishida K, Iwata T. Simulation-based medical education in clinical skills laboratory. J Med Invest 2012;59:28-35. http://dx.doi.org/10.2152/jmi.59.28 ArticlePubMed
  • 5. SonoSim. The SonoSim Ultrasound Training Solution [Internet]. Santa Monica (CA): SonoSim Inc.; 2012 [cited 2014 Sep 15]. Available from: http://sonosim.com/our-solution
  • 6. Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hamstra SJ. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA 2011;306:978-988. http://dx.doi.org/10.1001/jama.2011.1234 ArticlePubMed
  • 7. Maddox RW, Schmid RJ. New frontiers in medical education: simulation technology at Campbell University School of Osteopathic Medicine. N C Med J 2014;75:59-61. http://dx.doi.org/10.18043/ncm.75.1.59 ArticlePubMed
  • 8. Chung GK, Gyllenhammer RG, Baker EL, Savitsky E. Effects of simulation-based practice on focused assessment with sonography for trauma (FAST) window identification, acquisition, and diagnosis. Mil Med 2013;178(10 Supple):87-97. http://dx.doi.org/10.7205/MILMED-D-13-00208 ArticlePubMed
  • 9. Maul H, Scharf A, Baier P, Wustemann M, Günter HH, Gebauer G, Sohn C. Ultrasound simulators: experience with the SonoTrainer and comparative review of other training systems. Ultrasound Obstet Gynecol 2004;24:581-585. http://dx.doi.org/0.1002/uog.1119 ArticlePubMed
  • 10. Sidhu HS, Olubaniyi BO, Bhatnagar G, Shuen V, Dubbins P. Role of simulation-based education in ultrasound practice training. J Ultrasound Med 2012;31:785-791. ArticlePubMed
  • 11. Fox JC, Chiem AT, Rooney KP, Maldonaldo G. Web-based lectures, peer instruction and ultrasound-integrated medical education. Med Educ 2012;46:1109-1110. http://dx.doi.org/10.1111/medu.12039 ArticlePubMed
Appendix
The 16-point practical exam grade rubric designed to assess the subjects’ ability to operate the ultrasound device, properly insonate key point of care ultrasound structures, and utilize advanced imaging functions
A. Morison’s pouch (hepatorenal recess)
 1. Morison’s pouch (kidney+liver)
 2. Diaphragm
 3. Mirror image of liver
B. Parasternal long axis view
 1. Parasternal long axis view
 2. Left ventricle
 3. Left atrium
 4. Interventricular septum
 5. Mitral valve
 6. Aortic outflow tract
C. Parasternal short axis view
 1. Parasternal short axis view
 2. Mitral valve
D. The visceral-parietal pleural interface (VPPI)
 1. VPPI
 2. M-mode, showing ‘sky-ocean-beach.’ Sky representing skin and subcutaneous fat, ocean representing muscle fibers, and beach representing lunch parenchyma.
E. Neck vessels in their short axis (cross-section)
 1. Right common carotid artery
 2. Right internal jugular vein
 3. Use color Doppler to show the direction of blood flow
Total score: ________ /16

Figure & Data

References

    Citations

    Citations to this article as recorded by  
    • Ultrasound diagnostics in solving situational problems in the discipline of clinical pathophysiology
      A. B. Makarov, V. N. Tsygan, A. V. Lemeshchenko, M. V. Rezvantsev, T. A. Krivolutskaya, T. A. Bammatov
      Pacific Medical Journal.2024; (4): 97.     CrossRef
    • Point-of-Care Ultrasound in the Intensive Care Unit
      Yonatan Y. Greenstein, Keith Guevarra
      Clinics in Chest Medicine.2022; 43(3): 373.     CrossRef
    • SonoSim ultrasound simulator training for novice residents
      Antoine Frère, Esther Samba, Corinne Lejus-Bourdeau
      European Journal of Anaesthesiology.2021; 38(7): 791.     CrossRef
    • Ultrasound in Medical Education: Can Students Teach Themselves?
      Brandon Cowan, Abigail Brackney, Mallikarjuna Barremkala
      Medical Science Educator.2021; 31(5): 1663.     CrossRef
    • Determining if simulation is effective for training in ultrasound: A narrative review
      Kate Bradley, Ann Quinton, Aamer Aziz
      Sonography.2020; 7(1): 22.     CrossRef
    • Impact of a 4-hour Introductory eFAST Training Intervention Among Ultrasound-Naïve U.S. Military Medics
      Jonathan D Monti, Michael D Perreault
      Military Medicine.2020; 185(5-6): e601.     CrossRef
    • The long-term effect of short point of care ultrasound course on physicians’ daily practice
      Ortal Tuvali, Re'em Sadeh, Sergio Kobal, Shaked Yarza, Yael Golan, Lior Fuchs, Etsuro Ito
      PLOS ONE.2020; 15(11): e0242084.     CrossRef
    • Developing Neuraxial and Regional Pain Procedural Skills Through Innovative 3-Dimensional Printing Technology
      Zachary C. Headman, Marcus C. Matson, Robert P. Schneider, James L. Potter, Debra L. Loguda-Summers, Shalini Bhatia, Tatyana Kondrashova
      Journal of Osteopathic Medicine.2020; 120(4): 273.     CrossRef
    • The Use of Ultrasound Simulators to Strengthen Scanning Skills in Medical Students: A Randomized Controlled Trial
      Cathie‐Kim Le, John Lewis, Peter Steinmetz, Alina Dyachenko, Sharon Oleskevich
      Journal of Ultrasound in Medicine.2019; 38(5): 1249.     CrossRef
    • Impact of an intensive education programme of diagnostic lung and lower limb ultrasound on physiotherapist knowledge: A pilot study
      George Ntoumenopoulos, Selina M. Parry, Aymeric Le Neindre
      Australasian Journal of Ultrasound in Medicine.2018; 21(2): 104.     CrossRef
    • Point-of-Care Ultrasound for Internal Medicine: An International Perspective
      Irene W. Y. Ma, Chiara Cogliati, Frank H. Bosch, Ana Claudia Tonelli de Oliveira, Vincenzo Arienti, Michiel J. Blans, Barry Chan, Anjali Bhagra
      Southern Medical Journal.2018; 111(7): 439.     CrossRef
    • Développement d’un examen clinique objectif structuré (ECOS) pour évaluer les compétences des étudiants en médecine vasculaire
      J. Risse, T. Busato, V. Dufrost, M. Perri, S. Zuily, D. Wahl
      JMV-Journal de Médecine Vasculaire.2017; 42(3): 141.     CrossRef
    • Enhancing Learning Experience Using Ultrasound Simulation in Undergraduate Medical Education: Student Perception
      Tatyana Kondrashova, Crystal Coleman
      Medical Science Educator.2017; 27(3): 489.     CrossRef
    • Evaluation of a pilot programme on diagnostic thoracic ultrasound curriculum for acute care physiotherapists
      George Ntoumenopoulos, Hwee Kuan Ong, Hong Chuen Toh, Rafael Pulido Saclolo, Wen Duu Sewa
      Australasian Journal of Ultrasound in Medicine.2017; 20(4): 147.     CrossRef

    Figure
    We recommend

    JEEHP : Journal of Educational Evaluation for Health Professions