Introduction

Traditional education reflects passive transfer of information from lecturer to students in a large group. This has been shown inferior to active learning [1], which promotes more thorough and lasting understanding [2]. Active learning is a process whereby students engage in activities, such as reading, writing, discussion, or problem solving that promotes analysis, synthesis, and evaluation of class content [3]. Team-based learning (TBL) is one example of active learning, and uses a “flipped classroom” (FC) approach to promote knowledge permanence. In a FC, students receive a first exposure to material prior to class, for example, by reading or watching podcasts at home. In TBL, students then collaborate through group interactions in class to solve clinical problems and reflect on their learning [4]. Medical education has been slow to adopt these practices. While low-fidelity simulation has been used in ACLS for more than 30 years, small-group learning as an adjunct has not been studied [5]. ACLS training, as dictated by the American Heart Association (AHA), is typically done with classroom didactics, and reinforced with simulation. A recent report from the AHA indicates a paradigm shift in their core training course to be implemented with the 2015 guidelines, to involve FC with small- group learning, simulations, and online learning modules [6]. We aim to be the first to assess and report this FC/TBL model for ACLS training in the University of California-Irvine School of Medicine. We hypothesized that FC/TBL would enhance learning as shown by improved performance on multimodal written testing in 4th year (senior) medical students, compared to 3 recent historical control classes taught the same content in lecture based (LB) format.

Methods

The School of Medicine has 104 entering students per class. While the ACLS course is “mandatory,” a variable number enrolled in the course each year due to conflicts with MBA and MPH coursework, and delayed graduation for those pursuing PhD. Ninety five final year students from the School of Medicine participated in the FC/TBL ACLS course in 2015.

The simulation part of the course was taught in 8 rooms, each with a crash cart/defibrillator/transcutaneous pacemaker, airway/intubation mannequin and high-fidelity simulator (Laerdahl Sim-Man, Wappingers Falls, New York), as part of a 6,000 square foot simulation center. The TBL component was taught in a large lecture hall with tiered seating.

We taught late (March) final-year students 2010 AHA ACLS in a FC/TBL model in 2015, and compared written test performance to controls of 3 classes in LB format (2012-14). There were 27.5 scheduled hours in the FC/TBL model (TBL 10.5, podcasts 9, small-group simulation 8 hours), and 20 hours (12 lecture, small-group simulation 8 hours) in LB format. In essence, 12 hours of classroom lectures (2012-14) were replaced by 9 hours of recorded podcasts (2015), with the same material presented by the same instructor, and 10.5 hours of TBL. Comparison of teaching formats is shown in Table 1.

We used ungraded 10-question quizzes at the beginning of each class session to both encourage and gauge student compliance with assigned podcast viewing. Three to 4 multiple choice (MC) questions were drawn from each 20-45 minute podcast.

The TBL group application exercises covered 13 cardiac- and peri-arrest cases; the LB format had none. These TBL exercises were:

1. Acute coronary syndrome (ACS)

2. Ventricular fibrillation (VF)

3. Refractory VF

4. Post-cardiac arrest care

5. Respiratory distress

6. Pulseless electrical activity (PEA) case 1 septic shock

7. PEA case 2 hyperkalemia

8. Asystole

9. Symptomatic bradycardia

10. Paroxysmal supraventricular tachycardia (PSVT, SVT) with good perfusion

11. Ventricular tachycardia (VT), stable

12. VT, unstable

13. Acute ischemic stroke

We used 8 cardiac- and peri-arrest simulation cases with high-fidelity mannequins for both formats, in small groups of 5-9 with 1 instructor per group. These were:

1. ACS/ventricular fibrillation (VF) cardiac arrest/3rd degree atrioventricular block (AVB)/ST elevation myocardial infarction (STEMI) diagnosis

2. atrial fibrillation (AF) with rapid ventricular response (RVR)

3. stable then unstable (VT)

4. PSVT

5. PEA

6. symptomatic bradycardia

7. PEA case 2 hyperkalemia

8. unknown SVT/rate 150/atrial flutter with 2:1 conduction vs. PSVT vs. sinus tachycardia.

9. Torsade de pointe/polymorphic VT

The 3 written evaluation were multiple choice question (MCQ) test (not published due to copyright by the American Heart Association), cardiac rhythm test (Appendix 1), and clinical management test (Appendix 2). The 50 items of MCQ were developed by the AHA, which covered the content of the ACLS Provider Manual [7]. The questions focused on basic and advanced airway management, algorithm application, resuscitative pharmacology, and special situations like drowning and stroke recognition. Passing score set by the AHA for the MCQ test was over 84% correct.

The rhythm knowledge evaluation consisted of 20 examples of brady- and tachyarrhythmias, heart blocks, asystole/agonal rhythm, multifocal atrial tachycardia, and ventricular fibrillation, to which the students were required to match rhythm diagnoses on a one-to-one basis. Passing score defined by the instructor for Appendix 1 test was at least 18/20 correctly matched.

The clinical management “therapeutic modalities” was a fill-in-the-blank test including 7 clinical scenarios: acute coronary syndrome, symptomatic bradycardia, pulseless electrical activity, refractory ventricular fibrillation, stable and then unstable ventricular tachycardia, third-degree heart block, and asystole (Appendix 2). Passing score established by the instructor was >87% correct for Appendix 2 test.

All written evaluation tools were based on content from the ACLS Provider Manual or obtained from the AHA. Two expert ACLS instructors/experienced clinicians (anesthesiologist and emergency physician and 1 regional faculty, evaluated all testing protocols and tools prior to implementation of the course. Although we weighted the 3 components equally in the composite “correct answer” score, the maximum possible written test points were 50 (MCQ), 20 (rhythm test, Appendix 1) and 61 (“therapeutic modalities” test, Appendix 2). Written testing was constant across all 4 classes and was allotted 3 hours.

Statistical methods: The data were analyzed using Stata (version 14.0, StataCorp, College Station, TX). We used the Kruskal-Wallis rank sum test to assess differences between FC/TBL class and the aggregate scores of the 3 control classes taught in LB format. Confidence intervals for differences in proportions were calculated. We set statistical significance at P<0.05. The study was approved by the institutional review board of University of California, Irvine (IRB Number: HS# 2014-1195 ).

Results

Total 354 students took the course (259 in LB format 2012-14, and 95 in FC/TBL format 2015). The average entering MCAT scores for the classes tested (2015 vs 2012-14) were similar: 32.1 vs. 31.8 average, and average college grade point average was 3.68 for both control and experimental groups, indicating no difference in baseline academic achievement.

Two of 3 tests had statistical improvement for the FC/TBL format. For all tests combined, median scores increased from 93.5% (IQR 90.6, 95.4) to 95.1% (92.8, 96.7, P=0.0001). For the 7 case fill-in-the-blank tests, scores improved from 94.1% correct for LB (89.6, 97.2) to 96.6% for FC/TBL (92.4, 99.20 P=0.0001). For the 50 MCQ, scores improved from 88% correct for LB (84, 92) to 90% for FC/TBL (86, 94, P=0.0002). For the 20 rhythm test matching, students did well (median 100% both formats). More students failed 1 of the 3 written tests with LB vs. FC/TBL (24.7% vs. 14.7%), and 2 or 3 components of the written test (8.1% vs. 3.2%, respectively, P=0.006 for difference in number of failed tests). Conversely, 82.1% passed all 3 parts of the written test with FC/TBL vs. 67.2% with LB. (absolute difference 14.9%, 95% CI 4.8-24.0%).

We assessed the time students in the FC/TBL group spent watching podcast instruction from analytics for each podcast obtained from the Mediasite Enterprise Video Platform (http://www.sonicfoundry.com/mediasite/, Madison, WI). We used a logged internet protocol (IP) address assigned to a device (laptop, computer, or tablet on a network) as a unique identifier to represent a single student. An average of 63 students (range 47-95, 66.3% of 95 students enrolled) viewed each of the 23 podcasts spanning 9 hours of instruction. The average length of the 23 podcasts was 23.25 minutes and the average portion watched per podcast was 77.1%. We noted a general downward trend of the number of students watching at least some portion of the podcast as the 3-day TBL phase progressed. In addition, we noted a downward trend within each day for the assigned podcast viewing (Fig. 1). However, those that watched the podcasts generally maintained or increased the proportion of each podcast viewed, until the final podcast on acute ischemic stroke (Fig. 2).

When comparing the aggregate time each student spent on line watching each podcast vs. the real-time duration of the podcast, it appeared that students spent, on average, 100% of the podcast duration watching the podcasts. In other words, time spent watching the podcast approximated the run-time of the podcasts, even though the proportion of the podcast watched was less than complete. This suggests that students must have “rewound” portions of the podcasts that they watched to a moderate degree while watching the content.

Discussion

Given improvement in test scores, we advocate for and support the AHA decision to incorporate FC and TBL techniques in the educational program for the 2015 ACLS guidelines. Evaluation methods for mastery of ACLS course, per the AHA include both content (written) and performance assessments. Successful students must demonstrate team leadership and psychomotor skills in managing cardiac arrest and peri-arrest scenarios. Though evidence for content and construct validity for the written portion of ACLS assessment is largely lacking, and some evidence is to the contrary, both written and performance assessments have been shown to correlate moderately. A previous study advocated for dual evaluation methods as complementary, demonstrating different psychomotor and cognitive skills [8,9]. Given delivery of similar content in our course, improvements in test scores indicate further mastery of the required material. In the highest-stakes environment of cardiac arrest, demonstration of mastery at an enhanced level is certainly preferable.

Further, although scores improved marginally, we found that fewer students failed 1, 2, and 3 components of the written tests. Recognizing that the only required written assessment for ACLS is currently MC, we felt that additional cognitive assessment was necessary to judge mastery of a broader set of ACLS skills, including rhythm interpretation and fill-in-the blank clinical management.

Narrative student feedback from our TBL sessions was mixed, despite numerical course evaluations of small-group sessions rated an average of 4.7 on a five-point scale. Some students viewed the TBL exercise as “inefficient,” stating that discussion with classmates seemed unnecessary, with several stating they could have done this alone. Others valued the “team” learning, and wanted to dispense with individual exercise time. These preferences seem to defeat the purpose of TBL, which relies on redundancy of personal integration of material, then small-group discussion followed by large group validation. Many opined that, since the TBL exercise was not MC format, it did not prepare them for their future testing needs.

We chose to implement TBL using individual- and group-readiness-assurance tests (IRAT and GRAT), with a worksheet that required recall of ACLS algorithm steps in clinical management. These exercises were a distinct departure from the usual MC testing of our students. We believe that medical school testing in MC format has significant limitations. Recall that the United States Medical Licensing Examination (USMLE) Parts 1 and 2, and USMLE “shelf” exams for basic science topics and clinical clerkships, are given entirely in MC format. 1 student criticized the IRAT and GRAT exercises in class, and similar fill-in-the blank case management testing as, “guess what the instructor is thinking.” Given this student’s position 2 months from graduation, the fill-in-the blank clinical scenario management test is exactly designed to demonstrate the students’ ability to “think like a doctor,” rather than regurgitate facts by choosing the best of five MCs. This speaks perhaps to the need to substantially vary and alter the USMLE format. We believe in expansion of evaluation methods that require spontaneous generation of thought.

Additional time and costs are associated with the FC/TBL model. The ACLS course required an additional 30 hours of class preparation (podcasts + worksheet development) by the instructor. The podcast recording cost is estimated at $900 ($100 per hour charged for technician time and use of recording studio). Additional costs for post-production and hosting the podcasts were absorbed by the infrastructure of the Division of Educational Technology at the School of Medicine. The remaining 21 hours of instructor preparation cost in excess of $2,000.

We had technical problems with podcasts related to limited bandwidth when 95 students tried to view the podcasts simultaneously in the same geographic area. Many students said they went home to view.

Students frequently claim that they prefer podcasts to real-time instruction because they can both speed up the podcast, running at 1.5 or 2X speed, as well as review portions of podcasts that they need to see again. They view this as more efficient. We found that these 3 dynamics balanced each other out: students who watched the podcasts spent 100% of the real-time duration watching the content, even though they covered but 77.1% of each recording. Therefore, instruction by podcast does not appear to “save time” for medical students, as they spend the same time studying without exposure to all content.

The FC model involves first exposure to learning prior to class, so face-to-face classroom time can engage students in active learning [10]. There has been little, if any, record of the FC model for teaching ACLS. Therefore, we applied this model to an ACLS course and complemented it with TBL activities to promote group interaction, reflection and student engagement. Our findings suggest that FC/TBL is a suitable model for teaching ACLS as demonstrated through marginal improvements on student assessments and student feedback: “I felt the flipped classroom format was an excellent way to start the course--the worksheets stimulated a very helpful discussion,” and, “I loved that we went over the cases and the script over and over through podcasts, quizzes, worksheets, and verbally. I think it appealed to each different style of learner.”

Our study has limitations. For the FC component, our analytics show that up to one-third of students apparently did not watch the podcasts at all. Those that did watched, on average, did for 3-quarters of the real-time duration. This confounds may claim that the new format was an improvement over LB format. The improved performance may simply have been due to increased total instructional time devoted to the course. We specifically allotted time in the mornings to watch the day’s podcasts, and then scheduled TBL for the afternoons. Other FC designs allow students to view content completely asynchronously at their sole discretion.

We used a single instructor for the large group component of TBL. Other models, perhaps preferred, would have content experts facilitate small-group discussion. Our large classroom space did not support small-group discussion during TBL. Our ACLS class was pass/fail graded, and included required remediation as per AHA guidelines. Ultimately, almost all students passed the course and received a certification card. This “low stakes” educational environment may have bred complacency in study habits and effort. Furthermore, placement of the course at the end of 4th year, just before or after match day (depending on the academic year), may have negatively motivated students for course content mastery. The class forming the experimental group in 2015 had received iPads upon entry to medical school and was very technologically savvy by their 4th year. This may have enhanced the effectiveness of the FC model.

In conclusion, a FC/TBL format for ACLS marginally improved written test results for final-year medical students over traditional LB format, and significantly reduced the number of students who failed 1, 2, and 3 components of the written evaluation.

Article information

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Supplementary material

Fig. 1.

Downward trend within each day for the assigned podcast viewing by medical students during the advanced cardiac life support classroom in the University of California-Irvine School of Medicine, the United States of America.

Fig. 2.

Maintained or increased proportion of each of 23 podcasts viewed until final podcast on acute ischemic stroke for students during the advanced cardiac life support classroom in the University of California, Irvine School of Medicine, United States.

References

• 1. Subramanian A, Timberlake M, Mittakanti H, Lara M, Brandt ML. Novel educational approach for medical students: improved retention rates using interactive medical software compared with traditional lecture-based format. J Surg Educ 2012;2012:449-452. http://dx.doi.org/10.1016/j.jsurg.2012.05.013 Article
• 2. Littlewood KE, Shilling AM, Stemland CJ, Wright EB, Kirk MA. High-fidelity simulation is superior to case-based discussion in teaching the management of shock. Med Teach 2013;35:e1003-1010. ArticlePubMed
• 3. Center for research on learning and teaching. Active learning [Internet]. Ann Arbor, MI: University of Michigan; 2012 [cited 2016 Jan 1]. Available from: http://www.crlt.umich.edu/tstrategies/tsal
• 4. Thompson BM, Schneider VF, Haidet P, Levine RE, McMahon KK, Perkowski LC, Richards BF. Team-based learning at ten medical schools: two years later. Med Educ 2007;41:250-257. http://dx.doi.org/10.1111/j.1365-2929.2006.02684.x ArticlePubMed
• 5. Wayne DB, Butter J, Siddall VJ, Fudala MJ, Wade LD, Feinglass J, McGaghie WC. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med 2006;21:251-256. http://dx.doi.org/10.1111/j.1525-1497.2006.00341.x ArticlePubMedPMC
• 6. American Heart Association. 2015 ACLS Guideline Update [Internet]. Dalls, TX: American Heart Association; 2015 Available from: http://view.heartemail.org/?j=fe581176716c07757410&m=fe681570756c02757514&ls=fdbb1573706d07747d11717762&l=fe8713787760007572&s=fe3015777561047c741379&jb=ffcf14&ju=fe24107971630d7d751274&r=0
• 7. Sinz E, Navarro K, Soderberg ES. Advanced cardiovascular life support provider manual. Dalls, TX: American Heart Association; 2011.
• 8. Rodgers DL, Bhanji F, McKee BR. Written evaluation is not a predictor for skills performance in an advanced cardiovascular life support course. Resuscitation 2010;81:453-456. http://dx.doi.org/10.1016/j.resuscitation.2009.12.01 ArticlePubMed
• 9. Strom SL, Anderson CL, Yang L, Canales C, Amin A, Lotfipour S, McCoy CE, Langdorf MI. Correlation of simulation examination to written test scores for advanced cardiac life support testing: prospective cohort study. West J Emerg Med 2015;16:907-912. http://dx.doi.org/10.5811/westjem.2015.10.26974 ArticlePubMedPMC
• 10. Brame C. Flipping the classroom [Internet]. Nashville, TN: Vanderbilt University Center for Teaching; 2013 [cited 2016 Jan 1]. Available from: http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/

Appendix

Figure & Data

References

Citations

Citations to this article as recorded by  

• Team-based learning (TBL) in clinical disciplines for undergraduate medical students—a scoping review
  Irene Sterpu, Lotta Herling, Jonas Nordquist, Jerome Rotgans, Ganesh Acharya
  BMC Medical Education.2024;[Epub]     CrossRef
• Comparing the use of flipped classroom in combination with problem-based learning or with case-based learning for improving academic performance and satisfaction
  Bárbara Oliván-Blázquez, Alejandra Aguilar-Latorre, Santiago Gascón-Santos, Mª José Gómez-Poyato, Diana Valero-Errazu, Rosa Magallón-Botaya, Rachel Heah, Ana Porroche-Escudero
  Active Learning in Higher Education.2023; 24(3): 373.     CrossRef
• The effect of simulation‐based flipped classroom on acquisition of cardiopulmonary resuscitation skills: A simulation‐based randomized trial
  Eman Arafa Hassan, Suad Elsayed Abdelmotalb Elsaman
  Nursing in Critical Care.2023; 28(3): 344.     CrossRef
• Examination performance with flipped classroom as instructional strategy in the carbohydrate metabolism course unit at a Philippine medical school: Estimation of average treatment effect from observational data
  Michael Van Haute, Jan David Monzon
  Biochemistry and Molecular Biology Education.2023; 51(4): 428.     CrossRef
• The effects of flipped classrooms to improve learning outcomes in undergraduate health professional education: A systematic review
  Cho Naing, Maxine A. Whittaker, Htar Htar Aung, Dinesh Kumar Chellappan, Amy Riegelman
  Campbell Systematic Reviews.2023;[Epub]     CrossRef
• Flipped learning as an educational model in a cardiology residency program
  Carlos R Sierra-Fernández, Huipe-Dimas Alejandra, Sergio A Trevethan-Cravioto, Francisco J Azar-Manzur, López-Meneses Mauricio, Luis R Garnica-Geronimo
  BMC Medical Education.2023;[Epub]     CrossRef
• Effects of Prebriefing Using Online Team-Based Learning in Advanced Life Support Education for Nurses
  Eunkyoung Song, S. Barry Issenberg, Young Sook Roh
  CIN: Computers, Informatics, Nursing.2023; 41(11): 909.     CrossRef
• Best Practices and Educator Strategies for Facilitating a Flipped Classroom in Graduate Medical Education
  Alison Falck, Heather French, Rita Dadiz, Megan M. Gray, Amber Mendres-Smith, Bret Nolan, Kirsti Martin, Margarita M. Vasquez
  American Journal of Perinatology.2023;[Epub]     CrossRef
• The effect of flipped classroom in multiple clinical skills training for clinical interns on Objective Structured Clinical Examinations (OSCE)
  Weihao Zhang, Jiaoyang Gu, Fan Li, Feifei Feng, Huiqiang Chen, Xiaowei Xing, Lan Liu
  Medical Education Online.2022;[Epub]     CrossRef
• Team-Based Learning Among Health Care Professionals: A Systematic Review
  Tilak Joshi, Pravash Budhathoki, Anurag Adhikari, Ayusha Poudel, Sumit Raut, Dhan B Shrestha
  Cureus.2022;[Epub]     CrossRef
• Medical students’ self-regulation of learning in a blended learning environment: a systematic scoping review
  Rouba Ballouk, Victoria Mansour, Bronwen Dalziel, Jenny McDonald, Iman Hegazi
  Medical Education Online.2022;[Epub]     CrossRef
• Cardiopulmonary Resuscitation Training for Healthcare Professionals
  Kasper Glerup Lauridsen, Bo Løfgren, Lise Brogaard, Charlotte Paltved, Lone Hvidman, Kristian Krogh
  Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare.2022; 17(3): 170.     CrossRef
• Incorporating modified team-based learning into a flipped basic medical laboratory course: impact on student performance and perceptions
  Jing Shen, Hongyan Qi, Yingying Chen, Ruhuan Mei, Cencen Sun, Zhengyang Wang
  BMC Medical Education.2022;[Epub]     CrossRef
• Fail, flip, fix, and feed – Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis
  Manu Kapur, John Hattie, Irina Grossman, Tanmay Sinha
  Frontiers in Education.2022;[Epub]     CrossRef
• Impact of blended learning on learning outcomes in the public healthcare education course: a review of flipped classroom with team-based learning
  Hee Young Kang, Hae Ran Kim
  BMC Medical Education.2021;[Epub]     CrossRef
• To Flip or Not to Flip: Learning Style Preferences among Millennial Physician Assistant Students
  Katherine Schultz, Alicia Schaffer, Rebecca Rebman, Anthony Shanks
  Cureus.2021;[Epub]     CrossRef
• Flipping a biochemistry class within a medical curriculum: Impacts on perception, engagement, and attainment
  Hana. M. A. Fakhoury, Hanaa A. Fatoum, MHD Amer Aldeiry, Hawazen Alahmad, Joud Enabi, Sara Kayali, Yahya Bawahab, Emad M. Masuadi, Akef Obeidat, Colin James Lumsden
  Biochemistry and Molecular Biology Education.2021; 49(4): 625.     CrossRef
• Evaluating Usability in Blended Learning Programs Within Health Professions Education: a Scoping Review
  Anish K. Arora, Charo Rodriguez, Tamara Carver, Matthew Hacker Teper, Laura Rojas-Rozo, Tibor Schuster
  Medical Science Educator.2021; 31(3): 1213.     CrossRef
• Effects of three different first-aid training methods on knowledge retention of caregivers and teachers: a randomized and longitudinal cohort study in China
  F. Li, J.S. Zhang, X.Y. Sheng, J.L. wang, X.M. Shen, W.P. Xia, L.X. Shen, F. Jiang
  Public Health.2020; 178: 97.     CrossRef
• A unique flipped classroom approach shows promising results in physician assistant education
  Saarang Deshpande, David Ritzenthaler, Alec Sun, Nicole Rudert, Jensen Lewis
  Medical Teacher.2020; 42(3): 285.     CrossRef
• The flipped classroom: A meta-analysis of effects on student performance across disciplines and education levels
  Peter Strelan, Amanda Osborn, Edward Palmer
  Educational Research Review.2020; 30: 100314.     CrossRef
• Introducing early‐phase medical students to clinical paediatrics using simulation and a flipped‐classroom
  Penelope Uther, Kerri‐Anne Van Munster, Nancy Briggs, Susan O'Neill, Sean Kennedy
  Journal of Paediatrics and Child Health.2019; 55(9): 1107.     CrossRef
• Effect of a Flipped Classroom on Knowledge Acquisition and Retention in an Internal Medicine Residency Program
  Kelly L. Graham, Amy Cohen, Eileen E. Reynolds, Grace C. Huang
  Journal of Graduate Medical Education.2019; 11(1): 92.     CrossRef
• Flipping the Classroom in Health Care Higher Education
  Leda Evans, Melodee L. Vanden Bosch, Susan Harrington, Nancy Schoofs, Cynthia Coviak
  Nurse Educator.2019; 44(2): 74.     CrossRef
• Implementation of a “Flipped Classroom” for Neurosurgery Resident Education
  Fady Girgis, Jonathan P. Miller
  Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques.2018; 45(1): 76.     CrossRef
• Simulation training for medical emergencies in the dental setting using an inexpensive software application
  N. Kishimoto, N. Mukai, Y. Honda, Y. Hirata, M. Tanaka, Y. Momota
  European Journal of Dental Education.2018;[Epub]     CrossRef
• Academic outcomes of flipped classroom learning: a meta-analysis
  Kuo-Su Chen, Lynn Monrouxe, Yi-Hsuan Lu, Chang-Chyi Jenq, Yeu-Jhy Chang, Yu-Che Chang, Pony Yee-Chee Chai
  Medical Education.2018; 52(9): 910.     CrossRef
• Using integrated problem- and lecture-based learning teaching modes for imaging diagnosis education
  Jun-Yan Yue, Jie Chen, Wen-Guang Dou, Chang-Hua Liang, Qing-Wu Wu, Yi-Yong Ma, Zhi-Ping Zhu, Mei-Xia Li, Yan-Long Hu
  BMC Medical Education.2018;[Epub]     CrossRef
• Flipping the classroom: a national pilot curriculum for physiology in neonatal–perinatal medicine
  Heather French, Megan Gray, Maria Gillam-Krakauer, Elizabeth M. Bonachea, Melissa Carbajal, Allison Payne, Margarita M. Vasquez, Laura Rubinos, Alison Falck, Susan Izatt, Rita Dadiz
  Journal of Perinatology.2018; 38(10): 1420.     CrossRef
• Flipped-classroom training in advanced cardiopulmonary life support
  Jin Ho Beom, Ji Hoon Kim, Hyun Soo Chung, Su Mi Kim, Dong Ryul Ko, Junho Cho, Didier Lepelletier
  PLOS ONE.2018; 13(9): e0203114.     CrossRef
• Impact of multimodal methods to teach advanced cardiovascular life support principles in the core doctor of pharmacy curriculum
  Andrea S. Newsome, Susan E. Smith, William A. Hawkins, Michael Fulford, Beth B. Phillips
  JACCP: JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY.2018; 1(2): 95.     CrossRef
• Perceptions of Trainers on a Flipped Train–the-Trainers Course for Simulation
  Barış Sezer, Melih Elcin, Eylem Topbaş
  Bartın Üniversitesi Eğitim Fakültesi Dergisi.2018; 7(3): 958.     CrossRef
• A systematic review of the effectiveness of flipped classrooms in medical education
  Fei Chen, Angela M Lui, Susan M Martinelli
  Medical Education.2017; 51(6): 585.     CrossRef
• Effect of the Extended Inverted Classroom on clinical simulation for the resuscitation of trauma patients: Pilot study of student perceptions of learning
  Luis Carlos Domínguez, Diego Sierra, Juan José Pepín, Gonzalo Moros, Angélica Villarraga
  Colombian Journal of Anesthesiology.2017; 45: 4.     CrossRef
• Efecto del Aula Invertida Extendida a simulación clínica para la resucitación del paciente traumatizado: estudio piloto de las percepciones estudiantiles sobre el aprendizaje
  Luis Carlos Domínguez, Diego Sierra, Juan José Pepín, Gonzalo Moros, Angélica Villarraga
  Revista Colombiana de Anestesiología.2017; 45: 4.     CrossRef
• Effect of the Extended Inverted Classroom on clinical simulation for the resuscitation of trauma patients: Pilot study of student perceptions of learning☆
  Luis Carlos Domínguez, Diego Sierra, Juan José Pepín, Gonzalo Moros, Angélica Villarraga
  Colombian Journal of Anesthesiology.2017; 45: 4.     CrossRef
• Medical students’ satisfaction with the Applied Basic Clinical Seminar with Scenarios for Students, a novel simulation-based learning method in Greece
  Panteleimon Pantelidis, Nikolaos Staikoglou, Georgios Paparoidamis, Christos Drosos, Stefanos Karamaroudis, Athina Samara, Christodoulos Keskinis, Michail Sideris, George Giannakoulas, Georgios Tsoulfas, Asterios Karagiannis
  Journal of Educational Evaluation for Health Professions.2016; 13: 13.     CrossRef