INTRODUCTION

Originally developed for large business school classes, team-based learning (TBL) is a well-defined instructional strategy employed in numerous medical schools in the United States [1] and across the globe [2,3]. TBL uses a structured process that relies on individual and group accountability to promote mastery of factual knowledge, development of cognitive skills, and integration of information from multiple sources in addition to active problem solving and cohesive teamwork. TBL employs a three-phase strategy: (1) advance preparation: students independently study a preparatory assignment; (2) readiness assurance: students demonstrate mastery of the assignment through Individual and Group Readiness Assurance Tests (IRATs/GRATs); and (3) application: teams apply knowledge to problem-solving exercises, termed Group Application (GApp) exercises, in which teammates work to reach a consensus answer, followed by whole-class discussion and debate over the best solution to the problem [4]. In addition to RATs, individual accountability is promoted by peer evaluation of each team member’s contribution to team productivity.

Recently, Haidet et al. [5] proposed the core design elements of TBL. These include team formation, readiness assurance, immediate feedback, sequencing of in-class problem solving, four Ss (significant problem, same problem, specific choice, and simultaneous reporting), incentive structure, and peer review. This study will focus on the incentive structure of TBL, which is thought to be a critical component of the learning process. In the aforementioned conceptual model of TBL, grading individual performance motivates preparation prior to the TBL exercise and grading team performance will provide motivation to cohesively collaborate and engage with peers for maximal success [5]. Previous studies on TBL in medical education demonstrate improved learner engagement [2], learner satisfaction [6,7], emotional intelligence [8], and academic performance with particular benefit for lower-performing students [9]. To our knowledge, there is a void in the literature regarding incentive structure and its relation to TBL. While graded IRATs and GRATs promote thorough preparation, the cornerstone of each TBL module is the GApp exercise. In the 2009–2010 academic year, the study institution moved from a graded GApp exercise to an ungraded GApp exercise in the year two curriculum, eliminating team grades as a motivator for students to actively participate in group problem solving. This change provided the unique opportunity to assess students’ perceptions of graded versus ungraded GApps on the overall TBL experience as well as identify specific factors that contribute to student grading preference.

METHODS

Team-based learning at study site

At the study site preclinical instruction modalities include lecture, laboratory exercises, clinical case discussions, online/independent study modules, audience response sessions, and TBL. While lecture is the most frequent teaching method used by faculty, TBL modules heavily supplement the core curriculum. Student assessment is primarily accomplished through summative course examinations accounting for approximately 60%–95% of the overall course grades. Individual and group TBL scores, including peer evaluation, account for most of the remaining 5%–40%, with several courses including additional graded assessments.

At the beginning of each academic year, first and second year students are randomly sorted by faculty nito teams of 5–7, ensuring diverse backgrounds and experiences among team members. Students remain in these teams through all courses in an academic year and participate in >25 TBL sessions per year. Sessions last approximately 2–3 hours, with GApps accounting for approximately 80–120 minutes. The RAT and GApps are created by a faculty content expert and independently reviewed by at least one additional faculty member.

For each TBL module, advance assignments are given prior to the TBL session (phase 1) and may include textbook readings, lecture notes, journal articles, and independent study modules. TBL sessions begin with a 10-item multiple-choice IRAT covering material from the advance assignment. Immediately following the IRAT, each team takes the same RAT as a group. TBL facilitators provide immediate feedback on RAT performance and clarify with brief discussion all RAT answers, completing phase 2. The remainder of the TBL session consists of phase 3, the GApp exercise.

GApp exercises consist of clinically-based case scenarios paired with a series of multiple choice questions requiring extensive problem solving and critical reasoning. Questions focus on diagnostic and therapeutic decision-making with emphasis on basic and clinical science integration [10]. Teams are permitted to use reference materials while reaching consensus on GApp answer choices, but consultation across teams is not permitted. Following each question, teams simultaneously reveal answer choices. If all teams choose the same correct answer, the GApp proceeds to the next question. If teams disagree over the best answer, TBL facilitators lead class discussion/debate until resolution of important concepts is achieved.

The TBL strategy ensures that students receive immediate feedback on the RATs, are forced to reach team consensus on application exercises, and simultaneously report and defend their team decisions. High levels of team functioning as well as effective interpersonal communication are critical to strong performance in TBL modules. Individual and team scores are incorporated into each student’s course grade. For the population studied, an individual student’s TBL score consisted of 30% IRAT, 30% GRAT, and 40% GApp during year one and 40% IRAT, 60% GRAT, and 0% GApp during year two. During year one, peer evaluation occurs at the end of each of the four 10-week courses and serves as TBL grade multiplier. During year two, peer evaluations occur at the end of each academic term and are incorporated into term 1 and term 2 exam grades.

Measures

After a review of literature to assess intended outcomes of TBL, a 22-item multiple choice survey was developed to measure perceived impact of graded vs. ungraded GApp exercises on the student TBL experience (see Table 1 for survey questions) and reviewed by several faculty members at the study institution. Each item prompted students to consider the likelihood of specific TBL outcomes with graded or ungraded GApp exercises. Items were classified into 6 domains―general assessment, participation & communication, intra-team discussion, inter-team discussion, perceived effort, teamwork skills. Study participants were unaware of categories and items were randomly distributed throughout the survey. Answer choices for each item were as follows: (1) graded GApp exercises; (2) both graded and ungraded GApp exercises; and (3) ungraded GApp exercises. Response proportions were compared to the expected proportions (i.e., 33.33%) using a one sample chi-square analysis. Significance was set to P<0.002 using a Boneferroni correction. Here, a significant P-value indicates that there is a statistically significant association between students' perception of their TBL experience and grading category. Additionally, three open response questions (“Which Group Application exercise format do you prefer?”, “What is the reason(s) for the preference stated in the previous question?”, and “Which component(s) of TBL help you with remembering course material?”) were included to provide further insight into students’ subjective experiences in TBL with regard to GApp exercises as well as generating knowledge outcomes. Responses to open response questions were categorized for quantification during post hoc analysis of survey data.

Procedures

The population selected for this study participated in a TBL-supplemented preclinical curriculum with graded GApp exercises during Year One and ungraded GApps during year two at one United States Medical School. The study institution employs a graded curriculum (i.e., not pass/fail). With Institutional Review Board approval, the survey was completed at the end of the second year by consecutive classes in academic years 2009–2010 and 2010–2011. As a measure of advance preparatory effort, second year IRAT scores were compared for classes in which GApp exercises were graded (class of 2011) and ungraded (class of 2010). Only scores from TBL modules in which identical RATs were administered were included in analysis.

RESULTS

One hundred seventy-five out of 183 students (95.6% response rate) completed the survey. Response rates for the three open response questions were 35.5% (n=65), 32.2% (n=59), and 29.5% (n=54), respectively. Chi-square analyses were used to determine whether differences in graded versus ungraded GApp exercises impacted students TBL experience. Results showed statistically significant differences (P<0.05; Boneferroni correction P<0.002) on 20 of 22 survey items. Percentage of responses and chi-square results for graded, ungraded, or both graded and ungraded GApp exercises are reported in Table 1. Significant differences between grading categories were noted for (1) general assessment of TBL, (2) student perception of their own participation and communication during TBL, (3) the facilitation of intra-team discussion, (4) the facilitation of inter-team discussion, (5) student perceptions of their own effort, and (6) development of teamwork skills.

To uncover why students prefer graded or ungraded GApp exercises, survey results were further analyzed. Chi-square analysis of responses from only students selecting ‘ungraded’ for survey item 4 (i.e., those students who responded as ‘preferring ungraded’ GApp exercises; n=140), shows a significant association between ungraded GApp exercises and these 140 students’ perception of improved inter-team discussion (Table 2). On the other hand, chi-square analysis of responses from only students selecting ‘graded’ for survey item 4 (i.e., only those students who responded as ‘preferring graded’ GApp exercises; n=12), shows a significant association between graded GApp exercises and these 12 students’ perception of improved effort (Table 3). However, despite perceived differences in preparatory effort among these 12 students, IRAT data shows students are equally prepared coming into a TBL module regardless of GApp grade weight (Table 4).

Open response questions 1–3 provide additional insight into reasons for student GApp exercise preference (data shown in Tables 5–7, respectively). Notably, the percentages reported in Table 5 to open response question 1 (“Which Group Application exercise format do you prefer?”) corroborate well with those in Table 1 (item 4). Table 6 shows that students preferring ungraded GApp exercises in open ended question 1 (n=55) listed reduced stress, improved discussion, increased efficiency, and an improved learning environment in response to open response question 2 (“What is the reason(s) for the preference stated in the previous question?”), while those students preferring graded (n=4) listed improved effort/motivation as reasons for their preference. Together, these data indicate that a large majority of students prefer ungraded GApp exercises due to perceived decreases in stress leading to an improvement in the quality of group discussion and improved learning during TBL. Note also that a large majority of those students preferring graded GApp exercises listed group discussion as the most helpful TBL component for remembering course material (Table 7). Conversely, a minority of students prefer graded GApp exercises due to a perceived increase in effort leading to more motivation and better focus during TBL. Pertinent responses to three open response questions are included in the discussion section.

DISCUSSION

The past several decades have seen a paradigm shift in medical education. The call for learner-centered pedagogy has created numerous strategic innovations in curricular delivery. TBL is an innovative tool to supplement traditional lecture-based undergraduate medical curricula with active learning. GApp exercises challenge students to cooperatively analyze difficult clinical cases while incorporating important principals of basic science, population health, and medical ethics. Indeed, meaningful and well-constructed GApp exercises are the cornerstone of a TBL module and are largely responsible for TBL efficacy, which is noted for improved academic performance [11], professional development, emotional intelligence [8], and student satisfaction [2,7]. The current study investigates the incentive structure of TBL through a modification of conventional GApp exercises by examining student preference and student perceptions of TBL outcomes when GApp exercises are excluded from TBL grades.

Importantly, student survey data in the present study indicate the perceived effectiveness of GApp exercises in generating knowledge outcomes, developing teamwork skills, and preparing students for clinical clerkships is, in a large part, independent of grade weight. Moreover, IRAT grades did not differ significantly (Table 4) after the study institution removed grades from GApp exercises in the second year curriculum, demonstrating undiminished student advanced preparation or independent learning prior to TBL. Yet still, 83.8% of students polled prefer ungraded GApp exercises with only 7.2% preferring graded.

Of students who prefer ungraded GApp exercises, the majority cited a better learning environment during the TBL module with reduced feelings of stress and anxiety and improved group discussion. Data from open response questions further suggests that most students learned more from group discussion with ungraded GApp exercises due to the effective elimination of extraneous discussion for the sake of grades. Relevant responses include the following:

 “This year (with ungraded application exercises) was way less stressful and more beneficial to my learning.”
 “Application exercises test deeper understanding of materials. They sometimes go beyond what is ‘testable,’so being ungraded facilitates discussion without the burden/stress of grades.”
 “The ungraded format is much better because people spend less time arguing and more time learning.”
 “I feel more ideas were discussed and it made difficult concepts easier to understand because no one was out for points–only education.”
 “There is less stress so emphasis is placed on learning to learn instead of learning for a grade. It also reduces animosity in teams who get an answer wrong.”
 “This year (with ungraded GApp exercises) our group was interested not necessarily with arriving at the correct answer (though we deliberated thoroughly and seriously to have sound support for our answer choice), but we were instead most concerned with learning. People are more open to correction when not fighting for a grade.”

It is interesting, in light of findings that medical student stress peaks during the first two years of medical school [12], that the majority of second year medical students surveyed in this study indicate an overwhelming preference for ungraded GApp exercises due to reduced feelings of stress and anxiety. While stress is inherent and unavoidable in medical training and practice, it is during the first two years of medical school when students are uprooted from their friends and family and face increased scholastic workload, competition from peers, and high stakes standardized testing that psychological distress can overwhelm and disengagement coping strategies become prevalent. Overwhelming stress in students relying on unhealthy coping mechanisms may manifest as depression, substance abuse, decreased altruism/empathy, unprofessional conduct, and in a word, burnout [13].

There is a paucity of information about curricular contributions to medical student stress. In developing pedagogical innovations, medical educators must be cognizant of and sensitive to any unintended effect on the learning environment and student well-being. Although several studies have shown that students prefer TBL to other ‘learner-centered’ educational approaches [6,7], our data may suggest that in highly motivated students, such as medical students, extrinsic incentives to participate (i.e., grades) may have the adverse effect of unnecessarily increasing stress levels without concomitant increases in knowledge or other outcomes. Relevant insights from open response questions include the following:

 “Medical school is full of stress and the added stress of graded applications are unnecessary and do not significantly contribute to better learning.”
  “(With ungraded GApp exercises) I can think clearly. I feel better about speaking up and learn more…. People still take it seriously even though it isn’t graded.”
“(Ungraded GApp exercises) facilitates learning in an environment where it is acceptable to fail. Just because the application isn’t graded doesn’t mean students don’t prepare for it.”
 “Learning environment is amazing (with ungraded GApp exercises) and fosters excellent discussion because we still feel pressure to support our answers.”
 “I give full effort regardless of (GApp exercise) grade weight.”

While stress can arouse feelings of fear and anger, not all medical students find it unconstructive. Indeed, it can be a powerful motivator. A small minority of students in this study preferred graded GApp exercises despite any perceived increase in stress. These students felt motivated to put forth more effort if the GApp exercise was graded. Relevant data from open response questions from these students are as follows:

 “I enjoy the challenge of grades. Ungraded (application exercises) are too ‘low key.’ People tend to slack off more and ignore what is going in discussion. There is lots of irrelevant chatter.”
 “Without grading, GApp exercises may as well be a part of normal class time. People don’t try as hard. My group barely even cares to pay attention.”

While these students perceived an increase in effort with graded GApp exercises, knowledge outcomes depend on combined learning through IRAT, GRAT, and GApp exercises, and our IRAT data indicates that an individual’s mastery of the advance assignment is independent of whether GAPs are graded. Moreover, these students represent only a minority of those polled and ‘typical’ student perceptions do not indicate decreased effort when GApp exercises are not graded. Future studies will be necessary to quantify any potential reduction of effort.

A limitation of this study is that responses are from two classes of medical students from one institution with exposure to TBL during both the first and second year of their curriculum. The extensive exposure to TBL among our study population may lend this particular group of students to focus less on grades and more on learning. Results of this study should be interpreted in light of this and therefore may not be representative of all medical students. While much effort was taken to create a valid survey, expert reviewers were from the study institution only and outside review was not utilized. An additional limitation is that, except for the IRAT scores, our reported outcomes are largely based on students’ subjective impressions of their experience, as opposed to a more objective evaluation of their learning. That said, no marked trends in overall student performance during second year were noted when graded versus ungraded GApp exercises were used. In light of our findings, future studies should investigate the effectiveness of TBL modules when various portions of the module are ungraded.

In conclusion, medical students perceive a reduction in stress, an improved learning environment, and higher quality group discussion when GApp exercises are ungraded without sacrificing outcomes in knowledge acquisition or the development of teamwork skills. Students perceive TBL as equally effective regardless of GApp exercises grade weight, but prefer ungraded GApp exercises. Extrinsic incentives (i.e., grades) may not be necessary to ensure participation in active learning strategies and have the adverse effect of unnecessarily increasing stress levels. The use of ungraded GApps appears, therefore, to be a successful modification of TBL, making it more “student-friendly” while maintaining the goals of active learning and development of teamwork skills among medical students.

SUPPLEMENTARY MATERIAL

References

• 1. Thompson BM, Schneider VF, Haidet P, Levine RE, McMahon KK, Perkowski LC, Richards BF. Team-based learning at ten medical schools: two years later. Med Educ 2007;41:250-257. http://dx.doi.org/10.1111/j.1365-2929.2006.02684.x ArticlePubMed
• 2. Chung EK, Rhee JA, Baik YH, A OS. The effect of team-based learning in medical ethics education. Med Teach 2009;31:1013-1017. http://dx.doi.org/10.3109/01421590802590553 ArticlePubMed
• 3. Davidson LK. A 3-year experience implementing blended TBL: active instructional methods can shift student attitudes to learning. Med Teach 2011;33:750-753. http://dx.doi.org/10.3109/0142159X.2011.558948 ArticlePubMed
• 4. Michaelsen LK, Knight AB, Fink LD, editors. Team-based learning: a transformative use of small groups in college teaching. Sterling, VA: Stylus Publishing; 2004.
• 5. Haidet p, Levine RE, Parmelee DX, Crow S, Kennedy F, Kelly PA, Perkowski L, Michaelsen L, Richards BF. Perspective: Guidelines for reporting team-based learning activities in the medical and health sciences education literature. Acad Med 2012;87:292-299. http://dx.doi.org/10.1097/ACM.0b013e318244759e ArticlePubMed
• 6. Vasan NS, DeFouw DO, Compton S. A survey of student perceptions of team-based learning in anatomy curriculum: favorable views unrelated to grades. Anat Sci Educ 2009;2:150-155. http://dx.doi.org/10.1002/ase.91 ArticlePubMed
• 7. Deardorff AS, Moore JA, Borges NJ, Parmelee DX. Assessing first year medical student attitudes of effectiveness of team-based learning. Med Sci Educ 2010;20:67-72. http://www.iamse.org/artman/publish/article_533.shtml
• 8. Borges NJ, Kirkham K, Deardorff AS, Moore JA. Development of emotional intelligence in a team-based learning internal medicine clerkship. Med Teach 2012;34:802-806. http://dx.doi.org/10.3109/0142159X.2012.687121 ArticlePubMed
• 9. Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. The impact of team-based learning on medical students' academic performance. Acad Med 2010;85:1739-1745. http://dx.doi.org/10.1097/ACM.0b013e3181f52bed ArticlePubMed
• 10. McKee E. Risk analysis in human genetics: a team-based learning exercise [Internet] Washington, DC: MedEdPORTAL; 2011 Available from: http://www.mededportal.org/publication/8409 Article
• 11. Zingone MM, Franks AS, Guirguis AB, George CM, Howard-Thompson A, Heidel RE. Comparing team-based and mixed active-learning methods in an ambulatory care elective course. Am J Pharm Educ 2010;74:160. http://dx.doi.org/10.5688/aj7409160 ArticlePubMedPMC
• 12. Dahlin M, Joneborg N, Runeson B. Stress and depression among medical students: a cross-sectional study. Med Educ 2005;39:594-604. http://dx.doi.org/10.1111/j.1365-2929.2005.02176.x ArticlePubMed
• 13. Dyrbye LN, Thomas MR, Shanafelt TD. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med 2006;81:354-373. ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by  

• Lowering the Stakes: Quasi-Experimental, Mixed-Methods Evaluation of a Restructured Grading Approach in a Graduate Public Health Research Methods Course
  Delia L. Lang, Caroline M. Barry, Umedjon Ibragimov, Juan L. Rodriguez, Elizabeth Reisinger Walker
  Pedagogy in Health Promotion.2024; 10(4): 250.     CrossRef
• Implementation of Team-Based Learning for a Clinical Module of the Ethiopian Undergraduate Anesthesia Curriculum and Students’ Perspectives: A Pilot Cross-Sectional Study
  Hailemariam Mulugeta, Abebayehu Zemedkun
  Advances in Medical Education and Practice.2023; Volume 14: 1413.     CrossRef
• Use of Established Guidelines When Reporting on Interprofessional Team-Based Learning in Health Professions Student Education: A Systematic Review
  Annette W. Burgess, Deborah M. McGregor
  Academic Medicine.2022; 97(1): 143.     CrossRef
• Assessment for Learning with Ungraded and Graded Assessments
  Karly A. Pippitt, Kathryn B. Moore, Janet E. Lindsley, Paloma F. Cariello, Andrew G. Smith, Tim Formosa, Karen Moser, David A. Morton, Jorie M. Colbert-Getz, Candace J. Chow
  Medical Science Educator.2022; 32(5): 1045.     CrossRef
• Weekly team-based learning scores and participation are better predictors of successful course performance than case-based learning performance: role of assessment incentive structure
  Gonzalo A. Carrasco, Kathryn C. Behling, Osvaldo Lopez
  BMC Medical Education.2021;[Epub]     CrossRef
• Geographic Trends in Team-based Learning (TBL) Research and Implementation in Medical Schools
  Jimmy Ming Hong, Preman Rajalingam
  Health Professions Education.2020; 6(1): 47.     CrossRef
• Impact of readiness assurance process and faculty feedback on individual application exercises: a model for continuous assessment in physiology
  Kirtana Raghurama Nayak, Dhiren Punja, Chinmay Suryavanshi
  Advances in Physiology Education.2020; 44(4): 509.     CrossRef
• Introducing the Team Based Learning as an Approach to Reviving Interest in Biochemistry amongst Undergraduate Medical Students-An Exploratory Study
  Vanita Lal, Bharti Bhandari, Garima Gupta, Kuldeep Singh, Praveen Sharma
  Annals of the National Academy of Medical Sciences (India).2020; 53: 166.     CrossRef
• A Novel Grading Strategy for Team‐Based Learning Exercises in a Hands‐on Course in Molecular Biology for Senior Undergraduate Underrepresented Students in Medicine Resulted in Stronger Student Performance
  Gonzalo A. Carrasco, Kathryn C. Behling, Osvaldo J. Lopez
  Biochemistry and Molecular Biology Education.2019; 47(2): 115.     CrossRef
• First year medical student performance on weekly team-based learning exercises in an infectious diseases course: insights from top performers and struggling students
  Gonzalo A. Carrasco, Kathryn C. Behling, Osvaldo J. Lopez
  BMC Medical Education.2019;[Epub]     CrossRef
• Implementation of Team-Based Learning: a Tale of Two New Medical Schools
  Gonzalo A. Carrasco, Kathryn C. Behling, Osvaldo J. Lopez
  Medical Science Educator.2019; 29(4): 1201.     CrossRef
• An innovative addition to team-based-learning pedagogy to enhance teaching and learning: Students' perceptions of team exams
  Parto S. Khansari, Leanne Coyne
  Currents in Pharmacy Teaching and Learning.2018; 10(1): 90.     CrossRef
• From Anatomical Knowledge to Clinical Comprehension: a Peer-Oriented Learning Session to Help Medical Students Make the Leap
  Hector Lopez, Evan Goldman, John Gaughan, Sangita Phadtare
  Medical Science Educator.2017; 27(2): 177.     CrossRef
• The Effect of Graded Assessment on Medical Student Performance in TBL Exercises
  Kathryn C. Behling, Matthew M. Gentile, Osvaldo J. Lopez
  Medical Science Educator.2017; 27(3): 451.     CrossRef
• A systematic review of the published literature on team-based learning in health professions education
  Tyler Reimschisel, Anna L. Herring, Jennifer Huang, Tara J. Minor
  Medical Teacher.2017; 39(12): 1227.     CrossRef
• Student perceptions of independent versus facilitated small group learning approaches to compressed medical anatomy education
  Alexander Whelan, John J. Leddy, Sean Mindra, J.D. Matthew Hughes, Safaa El‐Bialy, Christopher J. Ramnanan
  Anatomical Sciences Education.2016; 9(1): 40.     CrossRef
• Embryology, Anatomy, and Radiology of Cervical Cysts and Cleft Lip/Palate: A Team-Based Learning Module for Medical Students
  Janine Prange-Kiel, Julie G. Champine, Alisa J. Winkler, Diane M. Twickler
  MedEdPORTAL.2016;[Epub]     CrossRef
• A comparison of the effectiveness of the team-based learning readiness assessments completed at home to those completed in class
  Jennifer M. Carbrey, Colleen O’Connor Grochowski, Joseph Cawley, Deborah L. Engle
  Journal of Educational Evaluation for Health Professions.2015; 12: 34.     CrossRef