Introduction

Background/rationale

The proportion of medical education delivered in the form of face-to-face lectures and other more traditional methods is in decline, due in part to a growing emphasis on self-directed learning (SDL) and the increasing availability of online and remote learning resources [1,2]. The coronavirus disease 2019 (COVID-19) pandemic has seen a dramatic acceleration of this evolution, with many clinical placements suspended and medical schools forced to move swathes of content delivery online [3,4]. However, arguably the most rapid and exponential growth in remotely available medical educational content has been independent of formal teaching institutions.

Free Open Access Meducation (FOAM) refers to a variety of sources of remote, freely accessible medical education. Typically, FOAM modalities include blogs, podcasts, and social media, but the definition can be expanded to other formats such as free online question banks. The term traditionally refers to both the online availability of medical information and the community contributing to and engaging with such data [5]. FOAM use in postgraduate populations has increased exponentially over the last decade [6-8]. However, while numerous studies have explored undergraduate use of individual FOAM resources, the extent to which medical students are aware of the FOAM movement and the proportion of independent study time they spend engaging with FOAM have yet to be thoroughly investigated. There is a particular deficit in understanding how students select and use resources independent of those created or signposted by educational institutions.

One significant study in this area suggested that although in-person lecture attendance remains a key method of learning, remote resources such as online question banks now comprise a similar proportion of medical student study time [9]. Resource choices appear to be significantly influenced by the purpose of use (e.g., learning new concepts versus revising previously encountered topics), with resources such as question banks and hand-written notes being utilized to a much greater extent than face-to-face delivery closer to exam time [9]. Those resources correspond to the format of many FOAM resources, which are often designed according to well-evidenced principles of knowledge attainment and retention, such as formative testing and spaced repetition [10,11].

Objectives

This large, cross-sectional, remotely-delivered study aimed to investigate patterns in medical students’ SDL behaviors, their SDL resource choices (including the use of FOAM), the factors motivating these choices, and the potential impact of the COVID-19 pandemic on these variables. Ultimately, we aimed to identify patterns of study behaviors in undergraduate medical students that educational institutions can use to help tailor teaching and curriculum design both during and after restrictions imposed by the COVID-19 pandemic.

Methods

Ethics statement

Ethical approval was sought and granted by the St. George’s, University of London Research Ethics Committee (REC reference 2020.0111). Informed consent was obtained in the first section of the online questionnaire. All participants consented to their anonymous data being used for this study and/or additional data analysis and publication in a peer-reviewed journal.

Study design

We employed a national cross-sectional mixed survey using a range of multiple-choice, ranked, and free-text questions. This article was described according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement [12].

Participants

In addition to the authors of this article, 29 medical student collaborators representing 33 different medical schools within the United Kingdom, were recruited and responsible for survey dissemination across all cohorts within their institute of study (Supplement 1). Collaborators were encouraged to advertise the survey for a minimum of 2 cycles, a minimum of 2 weeks apart. Methods of survey dissemination included formal university communication channels, newsletters, social media, and informal communication channels. Graduate and undergraduate medical students were invited to complete the online survey between April 30, 2020 and July 1, 2020.

Setting

Inclusion criteria and exclusion criteria: Medical students were included attending a university within the United Kingdom and consenting to the use of anonymous data for this study and/or additional data analysis and publication via a peer-reviewed journal. Students were excluded currently undertaking a pause in their studies or an intercalated degree and involving in the design or planning of the study.

Validity and reliability of the questionnaire: A draft survey was designed by the named authors. Similar previous research in the area was considered, and additional questions were added, directly addressing motivational factors and knowledge of FOAM [9]. The national survey comprised 16 items, including a mix of multiple-choice, ranked, and free-text response questions relating to basic demographic information and SDL behaviors (Supplement 2). The survey was piloted in a small group of medical students before being reviewed and launched in order to ensure that the questions were unambiguous and the phrasing did not generate responder bias. No issues were raised during the initial pilot of the survey. Participants accessed the survey via a link distributed by the collaborators. The questionnaire items consisted of diverse forms, including multiple-choice, ranked, and free-text responses. Therefore, a formal reliability test could not be undertaken.

The link took respondents to a Microsoft Form, which included both the informed consent form and the survey questionnaire itself. Students and collaborators distributing the survey were blinded to previous responses.

Variables

Independent study time according to group and study materials, frequency of use of study materials, factor determining students’ opinion of FOAM resources, and awareness of FOAM were variables.

Study size

A post hoc power calculation was conducted using G*Power ver. 3.1.9.4 (Heinrich-Heine-Universität Düsseldorf, Düsseldorf, Germany; http://www.gpower.hhu.de/). For the Mann-Whitney U-test evaluating study time allocated pre- and post-COVID of independent group, the power was calculated at 0.8505. Input values were as follows: two tails; parent distribution, Lapace; effect size, 0.2; alpha error probability, 0.05; and sample size, 168 and 1,396 (graduate students vs. undergraduate students). Effect size was set as low arbitrarily. If the comparison was for gender, power was 0.9933 (471 men versus 1,080 women). Effect size was set as low arbitrarily

For Wilcoxon signed-rank test of paired association, power was 0.8779. Input values were as follows: two tails; parent distribution, Lapace; effect size, 0.2 alpha error probability, 0.05; and sample size, 164 (graduate students). Effect size was set as low arbitrarily. If sample sizes were 1,396 (graduate entry group), 471 (men), 1,080 (women), powers were 1.0, 0.9996, and 1.0.

Qualitative variables

We asked respondents to report their understanding of the definition of FOAM, and later coded this variable dependent upon its consistency with the accepted definition [5].

Statistical methods

The statistical analysis was completed using IBM SPSS Statistics software for Mac ver. 26.0.0 (IBM Corp., Armonk, NY, USA). Descriptive statistics were used to analyze student demographics and study behaviors. Following an assessment of the data distribution, the Mann-Whitney U-test was used for between-group comparisons and Wilcoxon signed-rank test for paired associations. The Friedman test was used to compare differences in factors determining students’ use of FOAM resources by ranking them in order of importance. The Cochran Q test was used to assess changes in learning provisions over time.

Results

Raw response data are available from: Dataset 1.

Descriptive data of participants

In total, 1,626 responses were collected, of which 1,564 were included in the analysis. Of the 62 responses excluded, 61 were duplicates, and 1 did not meet the inclusion criteria as the respondent self-identified as an intercalating student. The median age of student responders was 21 years old (interquartile range, 3 years), 69.1% (n=1,080) were women and 30.1% (n=471) were men, while 0.8% (n=13) preferred not to declare their gender. The 1,396 students (89.3%) studied in undergraduate-entry programs and 168 students (10.7%) were in the graduate entry. All year groups were represented (Fig. 1). The student responders represented 37 UK medical schools (Supplement 1).

Independent study time

Before the pandemic, undergraduate-entry students reported spending more time studying independently than graduate-entry students (P<0.05). For all groups (men, women, undergraduates, and graduate students) the largest component of independent study time was allocated to university-provided information. This time spending, in combination with textbooks, represented approximately half of students’ study time. The other half of independent study time was dedicated to free or paid websites, podcasts, YouTube and other video resources, and online question banks. For all groups, the most used resources for independent study time were free websites and question banks.

When universities were shut to students because of the COVID-19 pandemic, students reported an increase in the mean weekly number of hours they spent studying independently irrespective of the route of entry (graduate-entry programs, P<0.001; undergraduate-entry programs, P<0.001). When comparing by gender, only women demonstrated a statistically significant increase (P<0.001 versus P=0.203). All cohorts except men reported an increase in the use of free websites and question banks (women, P<0.001; men, P=0.972; graduate-entry programs, P<0.05; undergraduate-entry programs, P<0.005), paid websites and question banks (women, P<0.001; men, P=0.531; graduate-entry programs, P<0.05; undergraduate-entry programs, P<0.001), and YouTube and other internet video resources (women, P<0.001; men, P=0.494; graduate-entry programs, P=0.005; undergraduate-entry programs, P<0.001). A significant increase was observed across all cohorts for apps, podcasts, and social media (women, P<0.001; men, P<0.05; graduate-entry programs, P<0.05; undergraduate-entry programs, P<0.001).

A consistent increase in resource use was not observed with regards to textbooks and journal articles, which showed a decrease in use by men students during COVID-19 (P<0.05). University information and personal notes, which also showed a decline in use by men students (P<0.001) as well as undergraduate-entry students overall (P<0.001).

Although undergraduate-entry students reported spending more time studying independently than graduate-entry students (P<0.05) before the pandemic, during COVID-19 lockdown, graduate-entry students surpassed the undergraduate program cohort in the number of hours spent studying independently (P<0.05). Although their independent study hours were similar before COVID-19 restrictions, women students demonstrated a much more significant increase in independent study time during COVID-19 than men (P<0.005) (Fig. 2).

Frequency of use of study materials

As depicted in Fig. 3, within the 7 days prior to survey completion, students reported using university information and personal notes most frequently (41.6% daily), followed by free websites and question banks (29.6% daily). Paid websites and question banks, as well as apps, podcasts, and social media were utilized least, with a majority of students reporting that they had not used these at all (54.2% and 55.6%, respectively).

Motivational factors

Fig. 4 illustrates that the most important factor determining students’ opinion of FOAM resources was accessibility (μrank=3.97). The least important was author credentials (Fig. 4). These factors were ranked similarly across men and women as well as undergraduate and graduate-entry program students as indicated in Fig. 5, with the exception being design and length of resources, which were ranked as third and fourth most important for graduates, respectively.

Awareness of FOAM

Despite students reporting a high usage of resources that can be characterized as FOAM, only 7.4% (n=116) had heard of the term, while the remainder were either unsure what FOAM was (7.2%, n=113) or had never heard of it previously (85.4%, n=1,335). Of those who provided a definition of FOAM, 86.6% of their statements concurred with the definition provided by Nickson and Cadogan [5].

Changes in learning provisions during COVID-19

To explore whether the timing of a student’s response to the survey had affected their experience of learning provisions, we compared the first half of responses to the second half of responses (i.e., May versus June 2020) using the Cochran Q test. There was no statistically significant difference in learning provisions between these groups (P>0.05).

Discussion

Key results

University-provided materials were the single resource type to which students dedicated the most independent learning time. However, this comprised only a minority of total independent study hours. Instead, respondents reported dedicating half of their learning time to both free and paid websites and question banks, video resources such as YouTube, and more novel educational modalities such as apps, podcasts, and social media. During the COVID-19 pandemic, with clinical placements terminated for the overwhelming majority of participants, students increased the number of hours they spent studying independently.

Interpretation

This increase in the independent study was driven by the increasing use of novel and online educational resources, a large proportion of which met the definition of FOAM [5]. Despite the shift of university materials online and towards formal online lectures, there was no consistently reported an increase in the use of university-provided materials. Similar trends were observed in the use of personal notes. Among male students and undergraduates, the amount of time dedicated to learning from such resources decreased. The amount of time allocated to sources beyond the university is in keeping with the current literature [9]. For example, a study of 522 medical students in 2011 reported that the vast majority believed that using sources other than those provided by their formal institutions was vital for passing examinations and that university-provided materials inadequately covered all of the material necessary [13].

A large study of Australian medical students also demonstrated an increasing reliance on question banks for study, many of which are subscription-only [9]. Previous studies have shown a correlation between the use of question books—the print forerunner to online question banks—and higher examination scores [14]. We found that overall students increased their use of paid websites during COVID-19 restrictions, but that a large subset (>50%) reported never using paid websites. This raises important questions about equitable access to learning materials as students move away from independent learning resources, such as textbooks, likely to be available via their educational institutions.

Together these findings paint an intriguing picture of students relying on university materials, increasingly supplemented by FOAM resources. Despite their growing reliance on FOAM resources, only 8% of participants had heard of the term “FOAM.” This phenomenon suggests that undergraduate medical students are accessing FOAM resources while being unaware of the philosophy underpinning the movement. This is reflected by students’ reported motivations for selecting particular learning resources: across all subgroups, students were most influenced by the accessibility and user-friendliness of resources. Students were least influenced by author credentials. This is in contrast to earlier studies, which found that peer recommendation was the most powerful motivating factor, in line with the collaborative and horizontal philosophy of FOAM [15]. We posit that as FOAM usage has become increasingly normalized among medical learners, there is declining recognition of its origins and traditional identity as a community of collaborators. Whether this changes as undergraduate learners become practicing physicians remains to be investigated, and therefore what this means for the FOAM movement is unclear.

Limitations

The limitations of this study include lack of questionnaire validation and the use of snowball sampling as a means of survey distribution; while increasing the number of participants, this may have impacted the representativeness of the sample. Unequal representation of different year groups at each university could have affected sub-cohort analysis results. Additionally, no responses were gathered from 5 UK medical schools, and there was an unequal representation of participants from the remaining 37 medical schools. Data from each participant were collected cross-sectionally. This non-sequential survey limits the interpretation of the results. For example, if a student completed the study on a particular week in which there was no university material provided, the time spent studying independently may have been much higher than the following week when the material was provided. Additionally, results may have been influenced by the proximity of survey completion to examinations, since previous research has demonstrated that students’ resource use changes during exam season [9]. A potential solution to this may have been to capture data from each student longitudinally over a series of weeks.

Generalizability

Due to the large and heterogeneous sample, the findings reported in this study can, to a great extent, be generalized to the broader population of UK medical students, especially those studying in undergraduate-entry programs, since the majority of our data was collected from this group. To a lesser extent, our findings could also potentially reflect the study behaviors of medical students studying in other countries, particularly courses run in developed and English-speaking nations (e.g., the United States, Canada, Australia), as evidenced by a recent study evaluating similar behaviors [9], as well as in other countries where medical schools follow a similar course structure to the MBBS (Bachelor of Medicine, Bachelor of Surgery) curriculum (e.g., India). However, generalizations at the international level should be made with caution, if at all, and must considerer factors influencing students’ ability to access SDL resources, including access to a quick and stable broadband connection, personal electronic devices, and financial affordability. In addition, language barriers associated with the use of such resources should be considered. One must also consider the unique nature of the year in which the data was collected. While students were asked to report pre-COVID study behaviors, novel stressors placed upon respondents may have had unforeseen effects on their self-reported answers. In order to evaluate the generalizability and reliability of our data, we recommend repeating the study once relative normality has returned to medical education provision within the United Kingdom.

Suggestion

Universities should recognize students’ growing transition towards online study materials. However, if FOAM resources are to be effectively integrated into the medical curricula of the 2020s and beyond, there is a pressing need to better understand how students are using FOAM to supplement traditional teaching methods. This movement will likely involve large prospective trials evaluating current student behaviors and the effectiveness of interventions that incorporate FOAM resources. There is an additional need to ensure equitable access for all students to the resources. They need to access reputable and high-quality FOAM and other remote materials. There is alsoa pressing need for investigating medical students’ ability to select, appraise, and use educational materials independent of their educational institution.

Conclusion

In this cross-sectional study of undergraduate medical students, we demonstrated a growing trend towards a relative increase in the use of learning resources independent of university provision. The COVID-19 pandemic has exaggerated these trends, while also contributing to the rise in the total time students have spent studying independently. The accessibility of such resources most strongly influences medical students’ decisions regarding resource choice, and students appear less influenced by factors such as peer recommendation. Alongside a lack of awareness of the FOAM movement, undergraduate medical students are interacting with online resources in a very different way from postgraduate populations. What this means for the future of the FOAM movement is unclear. What is clear, however, is that educators must appreciate the observed change in students’ independent learning behaviors. They must ensure equitable access to such materials and support students in making informed decisions about materials they use. Then,educators will be able to support students in choosing resources of sufficient quality to ensure that the students of today become the safe practitioners of tomorrow.

Article information

Authors’ contributions

Conceptualization: JB, KSR, AEC, EH, AR, GP, AK, TT, YZP, KK, JR. Data curation: JB, KSR, AEC, EH, AR, GP, AK, TT, YZP. Formal analysis: JB, KSR, AEC, AK, YZP. Funding acquisition: none. Methodology: JB, KSR, AEC, AK, YZP. Project administration: JB. Visualization: JB. Writing–original draft: JB, KSR, AEC, EH, AR, GP, AK, TT, YZP, KK, JR. Writing–review & editing: JB, KSR, AEC, EH, AR, GP, AK, TT, YZP, KK, JR.

Conflict of interest

Dr. Knight is the founder and lead editor of “PaediatricFOAM” (www.paediatricfoam.com). Dr. Round is an editor of “PaediatricFOAM” (www.paediatricfoam.com). No potential conflict of interest relevant to this article was reported.

Funding

No funding was received for this study.

Data availability

Data files are available from Harvard Dataverse: https://doi.org/10.7910/DVN/78UW2J

Dataset 1. Raw response data from 1,564 medical students in the United Kingdom to the survey questionnaire regarding study behaviors and Free Open Access Meducation.

jeehp-18-05-dataset1.xlsx

Acknowledgments

Supplementary materials

Fig. 1.

Year of study and MBBS (Bachelor of Medicine, Bachelor of Surgery) course details (undergraduate vs. graduate-entry program) of survey responders (n=1,564). One respondent (unspecified undergraduate) is not represented.

Fig. 2.

Comparison of the mean number of independent study hours using different study resources pre-coronavirus disease 2019 (COVID-19) versus during the COVID-19 lockdown for different student cohorts by gender (male vs. female) and course of study (undergraduates vs. graduate-entry program students).

Fig. 3.

Frequency of use of different study resources within the 7 days prior to survey completion, conducted during the coronavirus disease 2019 lockdown period, as indicated by the percentage of student responders.

Fig. 4.

Factors impacting students’ use of independent study sources ranked from most important (6) to least important (1) (n=1,546) determined using the Friedman test (χ2=1,802.690, P<0.001).

Fig. 5.

Factors impacting students’ use of independent study sources ranked from most important (6) to least important (1) for distinct student cohorts, including undergraduate-entry students (n=1,396) and graduate-entry program students (n=168), as well as female (n=1,080) and male (n=471) respondents as determined using the Friedman test.

References

• 1. Cardall S, Krupat E, Ulrich M. Live lecture versus video-recorded lecture: are students voting with their feet? Acad Med 2008;83:1174-1178. https://doi.org/10.1097/ACM.0b013e31818c6902 ArticlePubMed
• 2. Franklin DS, Gibson JW, Samuel JC, Teeter WA, Clarkson CW. Use of lecture recordings in medical education. Med Sci Educ 2011;21:21-28. https://doi.org/10.1007/bf03341590 Article
• 3. Taha MH, Abdalla ME, Wadi M, Khalafalla H. Curriculum delivery in medical education during an emergency: a guide based on the responses to the COVID-19 pandemic. MedEdPublish 2020;9:69. https://doi.org/10.15694/mep.2020.000069.1 ArticlePubMedPMC
• 4. Arandjelovic A, Arandjelovic K, Dwyer K, Shaw C. COVID-19: considerations for medical education during a pandemic. MedEdPublish 2020;9:87. https://doi.org/10.15694/mep.2020.000087.1 ArticlePubMedPMC
• 5. Nickson CP, Cadogan MD. Free Open Access Medical Education (FOAM) for the emergency physician. Emerg Med Australas 2014;26:76-83. https://doi.org/10.1111/1742-6723.12191 ArticlePubMed
• 6. Cadogan M, Thoma B, Chan TM, Lin M. Free Open Access Meducation (FOAM): the rise of emergency medicine and critical care blogs and podcasts (2002-2013). Emerg Med J 2014;31:e76-e77. https://doi.org/10.1136/emermed-2013-203502 ArticlePubMed
• 7. Burkholder TW, Bellows JW, King RA. Free Open Access Medical Education (FOAM) in emergency medicine: the global distribution of users in 2016. West J Emerg Med 2018;19:600-605. https://doi.org/10.5811/westjem.2018.3.36825 ArticlePubMedPMC
• 8. Barton J, Round J, Knight K. Paediatric Free Open Access Meducation (FOAM): behaviours, trends and implications. Postgrad Med J 2020;96:245-249. https://doi.org/10.1136/postgradmedj-2019-137204 ArticlePubMed
• 9. Wynter L, Burgess A, Kalman E, Heron JE, Bleasel J. Medical students: what educational resources are they using? BMC Med Educ 2019;19:36. BMC Med Educ 2019;19:36. https://doi.org/10.1186/s12909-019-1462-9 ArticlePubMedPMC
• 10. Kerfoot BP, Shaffer K, McMahon GT, Baker H, Kirdar J, Kanter S, Corbett EC Jr, Berkow R, Krupat E, Armstrong EG. Online “spaced education progress-testing” of students to confront two upcoming challenges to medical schools. Acad Med 2011;86:300-306. https://doi.org/10.1097/ACM.0b013e3182087bef ArticlePubMed
• 11. Lindsey RV, Shroyer JD, Pashler H, Mozer MC. Improving students’ long-term knowledge retention through personalized review. Psychol Sci 2014;25:639-647. https://doi.org/10.1177/0956797613504302 ArticlePubMed
• 12. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Group. STROBE statement [Internet]. Bern: University of Bern STROBE Group; 2008 [cited 2021 Mar 14]. Available from: https://www.strobe-statement.org/
• 13. Zhang J, Peterson RF, Ozolins IZ. Student approaches for learning in medicine: what does it tell us about the informal curriculum? BMC Med Educ 2011;11:87. https://doi.org/10.1186/1472-6920-11-87 ArticlePubMedPMC
• 14. Cooper AL, Elnicki DM. Resource utilisation patterns of third-year medical students. Clin Teach 2011;8:43-47. https://doi.org/10.1111/j.1743-498X.2010.00393.x ArticlePubMed
• 15. Judd T, Elliott K. Selection and use of online learning resources by first-year medical students: cross-sectional study. JMIR Med Educ 2017;3:e17. https://doi.org/10.2196/mededu.7382 ArticlePubMedPMC

Figure & Data

References

Citations

Citations to this article as recorded by  

• The state of health professions students’ self-directed learning ability during online study and the factors that influence it
  Xiaoyue Xu, Ziyi Li, Louisa Mackay, Na Li, Yaheng Zhang, Yujie Wu, Yang Zhang
  BMC Medical Education.2024;[Epub]     CrossRef
• Assessing medical students’ perception and educational experience during COVID-19 pandemic
  Ernest Z. Low, Niall J. O’Sullivan, Vidushi Sharma, Isabella Sebastian, Roisin Meagher, Dalal Alomairi, Ebraheem H. Alhouti, Claire L. Donohoe, Michael E. Kelly
  Irish Journal of Medical Science (1971 -).2023; 192(3): 1015.     CrossRef
• Medical students’ self-directed learning skills during online learning amid the COVID-19 pandemic in a Korean medical school
  Jihyun Si
  Korean Journal of Medical Education.2022; 34(2): 145.     CrossRef
• Advances in e-learning in undergraduate clinical medicine: a systematic review
  T. Delungahawatta, S. S. Dunne, S. Hyde, L. Halpenny, D. McGrath, A. O’Regan, C. P. Dunne
  BMC Medical Education.2022;[Epub]     CrossRef