Graphical abstract

Introduction

Background/rationale

Coronavirus disease 2019 (COVID-19) has spread dramatically worldwide, with more than 700 million confirmed cases and over 6 million deaths as of June 2023, according to the World Health Organization COVID-19 Dashboard [1]. Beginning in February 2020, the COVID-19 pandemic has led to training hospitals offering fewer or suspended opportunities for hands-on clinical participation by medical students to ensure their safety in the United States [2]. Medical education in Taiwan, accordingly, has become vastly different from what was traditionally offered prior to the pandemic, creating unfamiliarity and challenges as well as leading to a “new normal.”

Generally, the assessment tools implemented in Taiwan follow the 6 domains of competencies as established by the Accreditation Council for Graduate Medical Education model: (1) medical knowledge, (2) patient care, (3) professionalism, (4) communication, (5) interpersonal skills, and (6) practice-based learning and improvement [3]. While the pandemic has led to the segregation of medical students from regular hospital training owing to transmission concerns, the assessment of medical students has persisted to assist students in identifying and responding to their individual learning needs. Notably, technology has been adopted to ensure the continuation of medical education for curriculum delivery, case presentation, inpatient care, and outpatient consultation during the pandemic worldwide [2,4]. Most in-person training has transitioned to virtual learning to reduce the number of people physically getting together in one place. This has provided learning opportunities regardless of the whereabouts and safety of the medical students.

Major infectious disease epidemics have been shown to increase public fear, with decreased utilization of ambulatory services, inpatient care, and emergency care in Taiwan [5]. The diminishing patient population may also significantly hinder medical students’ learning experiences in most specialties. Despite fewer patients in most departments, certain specialties, such as emergency, pulmonary, and infectious Medicine, are heavily burdened by more critically ill patients. This dire situation has created another set of challenges for trainers and trainees. Many studies have highlighted the differences between male and female students in coping with stressful situations in the United States [6]. Based on previous articles, we argue that while COVID-19 poses a significant physical and mental health burden for health professionals and medical students, specialty and gender are 2 factors that appear to play a significant role in accommodating learning capabilities.

Objectives

In Taiwan, COVID-19 has undeniably limited traditional educational methods given the newly established governmental regulations on social distancing. Students are likely to be affected by risk mitigation in addition to workforce and resource utilization. The pandemic has disrupted the traditional education routines of future medical professionals. This study aimed to evaluate the impact of COVID-19 on the clinical performance of Taiwanese medical students by comparing students’ scores before and during the COVID-19 pandemic using standardized assessment tools and to explore the factors associated with the efficacy of medical education within the scoring systems.

Methods

Ethics statement

This study was approved by the Institutional Review Board of Taipei Veterans General Hospital (2022-01-021CC) and the requirement for informed consent was waived.

Study design

This is a 2-group comparative study examining students’ scores before and during the COVID-19 pandemic according to the Strengthening the Reporting of Observational Studies in Epidemiology statement available from: https://www.strobe-statement.org.

Clinical training background

The medical education program in Taiwan is a 6-year curriculum, in which the last 2 years are clinical years. Students receive both in-hospital rotations and lectures during their clinical experience. The Taipei Veterans General Hospital clerkship mandates that students rotate through various surgical, internal medicine, diagnostic medicine, psychiatry, and public health specialties. Attending physicians assess students’ monthly clinical performance using an online student passport, the electronic portfolio (e-portfolio) as shown in Fig. 1. Before COVID-19, medical students were divided into teams with one or 2 attending physicians as leaders, and they learned to work as front-line physicians for patient care, and assist attending physicians and resident physicians during their clinical practice. The Taipei Veterans General Hospital medical student education system was affected during COVID-19, and compromises were made, including an alternative for the delivery of medical education and testing, the redeployment of personnel, and decreased routine patient cases.

Setting

The study used a before-after (COVID-19) comparative study design that collected all medical students’ clinical scores from Taipei Veterans General Hospital student passports from 1/1/2018 to 6/31/2021. Scores before 2/1/2020 were placed in the pre-COVID-19 group, and scores from and after 2/1/2020 were placed in the midst-COVID-19 group. Each group was further categorized into COVID-19-affected specialties (pulmonary, infectious, and emergency medicine) and other specialties.

Participants

Taipei Veterans General Hospital medical students’ e-portfolio scores provided by attending physicians were collected and divided by gender. The scores were further divided into COVID-19-affected specialties (emergency, pulmonary, and infectious medicine) and other specialties. All 830 students received a clinical performance score from 0 to 100 at the end of each rotation in internal medicine, surgical medicine, and other specialties.

Variables

The scores of different specialties and pre- and midst-COVID-19 differences were used to assess the performance of the examinees in this study. The maximum score for each specialty was 100. Differences were compared between COVID-19-affected specialties and the other specialties, between pre- and midst-COVID-19 periods, according to students’ age, and according to the gender of students and attending physicians.

Data sources/measurement

The examiners scored the students’ performance using a computer program and the results were automatically processed. All variables were recorded in an Excel spreadsheet (Microsoft Corp.).

Bias

No selection bias was found in the study design.

Study size

Because all medical students were included, prior sample size estimation was not performed. Post-hoc power analysis for the score differences between pre-COVID-19 group and the midst-COVID-19 group showed a power (1-β error probability) of more than 0.9999 in both COVID-19-affected specialties and other specialties. Post hoc power analysis was done with an α of 0.05 and a 2-tailed test using G*Power ver. 3.1.9.4 (Heinrich-Heine-Universität Düsseldorf).

Statistical methods

This study analyzed the differences in performance among participants educated in different specialties before and during the COVID-19 pandemic. The chi-square test and 2-sample t-test were used to analyze demographic and score differences, respectively. Cohen’s d was utilized to report the effect size of differences. Generalized estimating equations (GEEs) were used to evaluate the impact of student’s age, student’s gender, attending physician’s gender, specialty attended, and the COVID-19 pandemic on students’ scores. Statistical analyses were performed using SAS ver. 9.3 (SAS Institute Inc.) and G*Power. Statistical significance was set at a P-value <0.05.

Results

Participants

Table 1 shows the baseline characteristics of the participants. This study included 1,322 students and 918 attending physicians. In total, 16, 944 clinical scores of these students were collected from the online passport system. Clinical scores were divided into 2 subgroups: pre-COVID-19 (n=1,434) and midst-COVID-19 (n=598). The other specialties group was divided into 10,494 scores in the pre-COVID-19 subgroup and 4,418 in the midst-COVID-19 subgroup.

Main results

Fig. 2 shows that in the COVID-19-affected specialties, the averages and standard deviations were 90.14±3.55 (n=1,434) for pre-COVID-19 group and 88.51±3.52 (n=598) for the midst-COVID-19 group (Cohen’s d=0.46, P<0.0001). In other specialties, the averages and standard deviations were 90.06±3.58 (n=10,494) for pre-COVID-19 group and 88.32±3.68 (n=4,418) for the midst-COVID-19 group (Cohen’s d=0.48, P<0.0001). The attending physicians were divided into groups of 699 male and 219 female physicians. The students comprised 837 males and 485 females. It was found that the male students’ scores (89.33–3.68) were significantly lower than the female students’ scores (89.99–3.66) (Cohen’s d=0.18, P=0.0017). Students’ scores were analyzed in terms of the attending physician’s gender. As shown in Fig. 3, there was a significant difference in the average scores given by male attending physicians to male and female students before the COVID-19 pandemic. Male students received an average of 89.61±3.65, while female students received an average of 90.32±3.94 (Cohen’s d=0.19, P<0.0001). The midst-COVID-19 male students received an average of 87.94–3.69 and the female students received an average of 88.61–3.79 (Cohen’s d=0.13, P<0.0001). Fig. 4 shows that within the female attending physician group, pre-COVID-19 male students received an average of 89.68–3.62, and female students received an average of 90.22–3.93 (Cohen’s d=0.10, P<0.0001). The midst-COVID-19 male students received 87.76–3.67 and the female students received 88.54–3.51 (Cohen’s d=0.22, P<0.0001).

GEE analysis was conducted to evaluate the impact of student’s age (R²=0.2509, unstandardized beta coefficient=-0.03, standard error [SE]=0.02, P=0.1278), student’s gender (female) (unstandardized beta coefficient=1.10, SE=0.20, P<0.0001), attending physician’s gender (female) (unstandardized beta coefficient=-0.19, SE=0.08, P=0.0184), COVID-19-affected specialties (unstandardized beta coefficient=0.26, SE=0.11, P=0.0145), and COVID-19 pandemic (unstandardized beta coefficient=-1.99, SE=0.13, P<0.0001) on students’ scores (Table 2). Raw response data are available in Dataset 1.

Discussion

Key results

In this study, we found that within COVID-19-affected specialties, the midst-COVID-19 score was significantly lower than the pre-COVID-19 score. Regarding the other specialties, the midst-COVID-19 score was also significantly lower than the midst-COVID-19 score. In terms of gender, male students’ scores were significantly lower than female students’ scores. GEE revealed that the COVID-19 pandemic, COVID-19-affected specialties, student’s gender, and attending physician’s gender were independent factors affecting student performance scores. Only student’s gender positively influenced the students’ scores. The COVID-19 pandemic negatively influenced students’ scores.

Interpretation

This study revealed a decline in the clinical performance of students trained in COVID-19-affected departments with similar or larger patient populations prior to the pandemic, in contrast to the learning-by-doing theory. The departments most affected by the viral pandemic were the emergency, pulmonary, and infectious medicine departments, primarily due to disease progression. This study consistently revealed: (1) a decline in students’ clinical scores during COVID-19 across specialties and genders, and (2) female students outperforming male students regardless of the COVID-19 pandemic. These results could be attributed to the reduced number of both inpatient and outpatient populations and changes in healthcare diversity during the pandemic. In addition, the elevated risk of nosocomial infection and high fatality rate in vulnerable groups may prevent patients from seeking routine medical care. The lack of patient diversity and cases has affected students’ clinical participation and limited their first-hand experiences. Emic descriptions have been shown to be more memorable because they portray higher imagery values, which in turn improve and solidify knowledge. However, a decline in first-hand experiences may lead to poorer memory of what is learned. For the sake of medical students’ safety, since the beginning of COVID-19, most medical schools have minimized direct patient care. For example, Taipei Veterans General Hospital implemented a transition from conventional education to virtual classes and limited student participation to reduce the risk of transmission. This study showed that students performed worse clinically after the start of the pandemic, and the difficulties encountered with virtual learning may, in part, be the reason for this decline. COVID-19 has affected all aspects of the medical industry. However, gender roles may also affect adaptive learning. Our study showed that female students consistently received higher scores in clinical settings than their male counterparts, regardless of the COVID-19 pandemic status. This might be because women perform better in interpersonal settings. The underlying gender characteristics could have played a role in the students’ clinical performance in this study.

Comparison with previous studies

Severely infected COVID-19 patients presented with acute respiratory distress syndrome and respiratory failure, thereby placing a high priority on infectious disease physicians and pulmonologists during the pandemic, as in China [7]. COVID-19 created concerns regarding the spread of contagion and led to public social distancing, which inevitably resulted in the deferment and cancellation of elective procedures in Taiwan [8]. Many studies have shown a reduction in hospital inpatient activity during the peak of COVID-19 infections in the United States [9,10]. One study reported a 39% decline in medical access after the COVID-19 outbreak in a community hospital in Taiwan [5]. The learning-by-doing theory states that students better adapt and learn through a hands-on approach to their environment in all countries [11]. As the flow of critical patients greatly increased, emergency departments became overwhelmed with potentially infected cases [12]. Attending and resident physicians were reallocated to critical areas to care for increased cases of acute patients in hospitals [13]. This led to a lack of teaching time, underscoring the need for instruction for medical students. In addition, courses have been altered from in-person teaching to mostly online learning, regardless of the number of patients in the departments, which in turn may interfere with students’ learning capabilities because of a lack of social interaction and lower motivation [2]. In-person lectures are either pre-recorded or live-streamed using online meeting applications. Technology allows the convenience of virtual learning for medical curriculum delivery, case presentations, virtual patient care, testing, and interviews to be delivered to groups or individuals, as reported in the United States [14]. Online teaching allows both group and individual settings, an easy process of students’ interaction in real life, and face-to-face sessions from any location. Although technology provides convenience, personal connections between faculty and students can be challenging in virtual courses. Students may report lack of interaction with instructors, response time deficiencies, and absence of traditional classroom socialization with their peers [2]. In addition, students may be prone to distractions because of stilted interactions. Many clinical practice studies have shown that women performed better under uncertainty and during stressful periods than men in clinical practice [6]. However, our previous study found that male students performed better than female students on standardized written examinations, while female students performed better than male students in various clerkships; additionally, men performed better with knowledge-based content, whereas women seemed more at ease in clinical environments [15].

Limitations/generalizability

While this study sheds light on the impact of COVID-19 on the clinical performance of Taiwanese medical students, several limitations must be considered. The findings may not be universally applicable, as the study is confined to the context of Taiwan, and differences in healthcare systems, educational structures, and pandemic responses could affect the generalizability of the results to other regions. It is essential to acknowledge that the absence of admission score data of the students limits the depth of the comparative analysis, potentially impacting the comprehensive interpretation of the results. The reliance on online standardized evaluation systems introduces a potential limitation, as these systems may not fully capture the complex and dynamic nature of clinical skills assessment. The study’s temporal scope, covering the period from 2018 to mid-2021, may not fully capture the evolving nature of the pandemic and the subsequent adaptations in medical education. Additionally, the categorization of specialties into COVID-19-affected and other specialties oversimplifies the intricacies within each category, potentially overlooking the unique challenges faced by specific medical disciplines. Furthermore, the study’s focus on specific variables, such as gender and the presence of female attending physicians, may not account for the multifaceted factors influencing medical student performance during the pandemic. Consequently, caution should be exercised when generalizing these findings to diverse global contexts and healthcare settings.

Suggestion

We believe that our experience can serve as a guide for future studies on avoiding the disruption of traditional education routines, investigating resilient medical education models, prioritizing the integration of online evaluation systems, and considering gender-related factors, for future medical professionals during pandemics.

Conclusions

COVID-19 has altered medical curricula worldwide. The present study concluded that COVID-19 has negatively affected medical students’ clinical performance, regardless of specialty. In addition, traditional in-person teaching has transitioned to virtual learning. The extended results showed that female students performed better than male students, both before and during the pandemic. This conclusion could be attributed to gender differences. Overall, this study shows that scores declined due to COVID-19, and the findings may be adapted to better-suited medical curricula that address medical education influenced by the pandemic.

Acknowledgments

Supplementary materials

Fig. 1.

E-portfolio. The students receive both in-hospital rotations and in-class lectures during the clinical years. Taipei Veterans General Hospital clerkship mandates that students rotate through various surgical specialties, internal medicine specialties, diagnostic medicine, psychiatry, and public health. The attending physicians assessed the students’ monthly clinical performances using an online student passport, the electronic portfolio.

Fig. 2.

Pre-COVID-19 and midst-COVID-19 score comparison. Within the COVID-19-affected specialties group, the pre-COVID-19 group scored 90.14–3.55, and the midst-COVID-19 group scored 88.51–3.52 (Cohen’s d=0.33, P<0.0001). Within the other specialties group, the pre-COVID-19 group scored 90.06–3.58, and the midst-COVID-19 group scored an average of 88.32–3.68 (Cohen’s d=0.34, P<0.0001). COVID-19, coronavirus disease 2019.

Fig. 3.

Male attending physician grading: male students versus female students. Pre-COVID-19 male students received an average of 89.61–3.65, and the female students received an average of 90.32–3.94 (Cohen’s d=0.13, P<0.0001). The midst-COVID-19 male students received an average of 87.9–3.69 and the female students received an average of 88.61–3.79 (Cohen’s d=0.13, P<0.0001). COVID-19, coronavirus disease 2019.

Fig. 4.

Female attending physician grading: male students versus female students. Pre-COVID-19 male students received an average of 89.68–3.62, and the female students received an average of 90.22–3.93 (Cohen’s d=0.10, P<0.0001). The midst-COVID-19 male students received an average of 87.76–3.67 and the female students received an average of 88.54–3.51 (Cohen’s d=0.15, P<0.0001). COVID-19, coronavirus disease 2019.

References

• 1. World Health Organization. WHO COVID-19 dashboard [Internet]. World Health Organization; 2023 [cited 2023 May 31]. Available from: https://covid19.who.int/
• 2. Kaul V, Gallo de Moraes A, Khateeb D, Greenstein Y, Winter G, Chae J, Stewart NH, Qadir N, Dangayach NS. Medical education during the COVID-19 pandemic. Chest 2021;159:1949-1960. https://doi.org/10.1016/j.chest.2020.12.026 ArticlePubMed
• 3. Lo WL, Lin YG, Pan YJ, Wu YJ, Hsieh MC. Faculty development program for general medicine in Taiwan: past, present, and future. Ci Ji Yi Xue Za Zhi 2014;26:64-67. https://doi.org/10.1016/j.tcmj.2014.05.002 ArticlePubMedPMC
• 4. Webster P. Virtual health care in the era of COVID-19. Lancet 2020;395:1180-1181. https://doi.org/10.1016/S0140-6736(20)30818-7 ArticlePubMedPMC
• 5. Jeng Y, Chen FH, Jen GH, Chen HC, Yen AM, Chen CD, Kuo HW, Wang ST, Hsu CY. Impact of COVID-19 pandemic on accessibility of Taiwanese medical care. Am J Manag Care 2021;27:e330-e335. https://doi.org/10.37765/ajmc.2021.88698 ArticlePubMed
• 6. Gorth DJ, Magee RG, Rosenberg SE, Mingioni N. Gender disparity in evaluation of internal medicine clerkship performance. JAMA Netw Open 2021;4:e2115661. https://doi.org/10.1001/jamanetworkopen.2021.15661 ArticlePubMedPMC
• 7. He H, Hu C, Xiong N, Liu C, Huang X. How to transform a general hospital into an “infectious disease hospital” during the epidemic of COVID-19. Crit Care 2020;24:145. https://doi.org/10.1186/s13054-020-02864-z ArticlePubMedPMC
• 8. Chatterji P, Li Y. Effects of the COVID-19 pandemic on outpatient providers in the United States. Med Care 2021;59:58-61. https://doi.org/10.1097/MLR.0000000000001448 ArticlePubMed
• 9. Oseran AS, Nash D, Kim C, Moisuk S, Lai PY, Pyhtila J, Sequist TD, Wasfy JH. Changes in hospital admissions for urgent conditions during COVID-19 pandemic. Am J Manag Care 2020;26:327-328. https://doi.org/10.37765/ajmc.2020.43837 ArticlePubMed
• 10. Birkmeyer JD, Barnato A, Birkmeyer N, Bessler R, Skinner J. The impact of the COVID-19 pandemic on hospital admissions in the United States. Health Aff (Millwood) 2020;39:2010-2017. https://doi.org/10.1377/hlthaff.2020.00980 ArticlePubMedPMC
• 11. Varghese C, Nongkynrih B, Mikkelsen B. Learning by doing: accelerate towards the NCD target in SDG through primary healthcare comment on “universal health coverage for non-communicable diseases and health equity: lessons from Australian primary healthcare”. Int J Health Policy Manag 2022;11:708-710. https://doi.org/10.34172/ijhpm.2021.96 ArticlePubMed
• 12. Hu B, Guo H, Zhou P, Shi ZL. Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol 2021;19:141-154. https://doi.org/10.1038/s41579-020-00459-7 ArticlePubMed
• 13. Ashokka B, Ong SY, Tay KH, Loh NH, Gee CF, Samarasekera DD. Coordinated responses of academic medical centres to pandemics: sustaining medical education during COVID-19. Med Teach 2020;42:762-771. https://doi.org/10.1080/0142159X.2020.1757634 ArticlePubMed
• 14. Durfee SM, Goldenson RP, Gill RR, Rincon SP, Flower E, Avery LL. Medical student education roadblock due to COVID-19: virtual radiology core clerkship to the rescue. Acad Radiol 2020;27:1461-1466. https://doi.org/10.1016/j.acra.2020.07.020 ArticlePubMedPMC
• 15. Huang CC, Huang CC, Yang YY, Lin SJ, Chen JW. The influence of gender on the communication skills assessment of medical students. Eur J Intern Med 2015;26:670-674. https://doi.org/10.1016/j.ejim.2015.06.017 ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by  

• The emergence of generative artificial intelligence platforms in 2023, journal metrics, appreciation to reviewers and volunteers, and obituary
  Sun Huh
  Journal of Educational Evaluation for Health Professions.2024; 21: 9.     CrossRef