Graphical abstract

Introduction

Background/rationale

Pass/fail grading in medical schools has received considerable attention because of its beneficial effect on students’ emotional well-being, without compromising their academic performance [1]. A contributor to these favorable academic outcomes might be students’ perceptions of their learning environment, which have been suggested to affect academic performance [2]. Since the pass/fail grading system helps students perceive the learning environment as supportive [3], these perceptions might contribute to their performance. Another potential factor associated with better performance is intrinsic motivation, which emerges from personal enjoyment or curiosity [4]. Unlike the detrimental effects of hierarchical grades, pass/fail grading has been suggested to foster intrinsic motivation [5]. Furthermore, students’ perceptions of their learning environment are correlated to intrinsic motivation [6] and are an essential antecedent for students’ motivation [7]. These results implied a cascading link between the grading system and academic performance in medical students, mediated by 2 factors, but further research is needed to explore this mechanism.

The known benefits of pass/fail grading might not simply be due to reducing the number of grades, but also to the characteristics of criterion-referenced assessments. Pass/fail grading implies a shift to a criterion-referenced standard setting through which students are compared against a clear standard and not against the performance of others. Therefore, the use of 3 or more categories based on a criterion, instead of only 2 categories (pass/fail), might yield positive results compared with relative rank-based tiers. Considering the value of criterion-referenced assessment, Yonsei University College of Medicine (YUCM) in Korea introduced a new grading system in 2014, changing from norm-referenced A–F grading to criterion-referenced honors/pass/fail grading. The new system, with fewer grades and criterion-referenced standards, might influence students’ learning in similar ways to what has occurred with pass/fail grading. However, there is a scarcity of relevant studies on this topic.

Objectives

This study aimed to investigate how grading systems affect academic performance based on students’ perceptions of their learning environment and motivation. Drawing upon the findings of previous studies, we propose a serial mediation model between honors/pass/fail grading and academic performance. We hypothesized that: (1) students’ perceptions of the learning environment would mediate the relationship between the grading system and academic performance; (2) intrinsic motivation would mediate the relationship between the grading system and academic performance; and (3) learning environment perceptions and intrinsic motivation would serially mediate the relationship between the grading system and academic performance.

Methods

Ethics statement

The Institutional Review Board (IRB) of Severance Hospital, Yonsei University Health Systems, approved this study using data obtained from normal educational practice in commonly accepted educational settings; thus, it was considered as an IRB-exempt study (IRB no., Y-2020-0201). Therefore, the requirement for informed consent was waived.

Study design

We adopted a cross-sectional study design to compare students’ survey responses and examination scores between before and after the grading system was changed.

Setting

In 2014, YUCM in Korea introduced a new grading system, moving from a norm-referenced assessment using A–F letter grading (A+, A, A-, B+, B, B-, C+, C, C-, D+, D, D-, F) to a criterion-referenced assessment using honors/pass/fail grading. Under the new system, the core mandatory courses that all students take use a 3-tiered grading system, and elective courses use a 2-tiered (pass/fail) grading system. This change was the first attempt among 40 medical schools in Korea and was an innovative endeavor in the highly competitive Korean medical education context. As there were concerns about the possibility of compromised academic performance due to the removal of ranks before introducing the criterion-referenced honors/pass/fail grading system, the relationship between the new grading system and academic performance was perceived as a critical issue.

Participants

This study included 238 second-year medical students from 2014 (n=127; A–F grading) and 2015 (n=111; honors/pass/fail grading). The former was the last cohort graded under the norm-referenced A–F grading system, and the latter was the first cohort to experience the new criterion-referenced 3-tiered grading system. The Department of Medical Education at YUCM conducted routine surveys on students at the end of each year to assess and ensure educational quality. Furthermore, upon completing the second academic year, corresponding to the end of the preclinical curriculum, students took the Basic Medical Science Examination as part of their requirements. Data from both the surveys and the examinations were used.

Variables

The demographic characteristics of the study participants, including age, gender, and medical school entrance type (undergraduate entry: 6-year program; graduate entry: 4-year program), were analyzed in this study. For learning-related characteristics, we measured learning environment perceptions, intrinsic motivation, and academic performance in medical students.

Data sources/measurements

Learning environment perceptions

Students’ perceptions of the learning environment were assessed using the Dundee Ready Education Environment Measure (DREEM) [8]. The DREEM comprises 50 items scored from 0 (strongly disagree) to 4 (strongly agree). The 9 negative statements are scored inversely. The DREEM measures students’ perceptions of learning, teachers, atmosphere, academic self, and social self. This scale can be analyzed in terms of the scores for individual items, 5 subscales, and all items, and a higher score signifies a more positive perception of the assessed construct. In this study, we used the total DREEM score as an indicator of overall learning environment perceptions. Cronbach’s α for the total scale was 0.864.

Intrinsic motivation

The Academic Motivation Scale (AMS) was used to assess intrinsic motivation for learning [9]. The AMS comprises 7 subscales that measure 3 types of intrinsic motivation (to know, accomplish things, and experience stimulation), 3 forms of extrinsic motivation (identified, introjected, and external regulation), and amotivation. Each AMS subscale comprises 4 items rated on a 7-point scale ranging from 1 (does not correspond at all) to 7 (corresponds exactly). In this study, we used the sum of the 3 intrinsic motivation subscale scores, with higher scores indicating a greater desire to engage in an activity based on intrinsic motivation. Cronbach’s α for this domain was 0.946.

Academic performance

The Basic Medical Science Examination scores, administered by the Korea Association of Medical Colleges, were used to measure students’ academic performance. This examination is a comprehensive and objective test of the medical knowledge of anatomy, biochemistry, physiology, pharmacology, microbiology, parasitology, and pathology. The examination included 260 multiple-choice items, and a score of 1 point was assigned for each correct answer. In this study, the percentage of correct scores was used for the analysis.

Raw and coding data are available at Dataset 1.

Bias

No selection bias was observed in this study.

Study size

The sample size was not estimated a priori because this study was conducted retrospectively using data collected from normal educational practices.

Statistical methods

All statistical analyses were performed using R software ver. 4.0.2 (https://www.r-project.org). A 2-way imputation method was implemented to impute the missing item scores for learning environment perceptions and intrinsic motivation [10]. Each of the 2 scales, DREEM and AMS, had 0.9% of missing values. The R package “miceadds” was used for 2-way imputation (Supplement 1). To compare the characteristics of the A–F grading and honors/pass/fail grading classes, we conducted the independent sample t-test and chi-square test for continuous and categorical variables, respectively. Pearson correlation analysis was performed to explore the relationship between continuous variables, while the point biserial correlation method was used to measure the associations between continuous and binary variables. Next, the serial mediation effect of perceptions of the learning environment and intrinsic motivation on the relationship between the grading system and academic performance was examined. Fig. 1 presents the conceptual model. The PROCESS macro (model 6) was used to examine the indirect effect based on a 95% bootstrapped confidence interval (CI) with 10,000 resamplings. If the interval did not contain 0, a significant indirect effect was considered to have been shown. Participants’ gender and type of medical school entrance were controlled in the model.

Results

Participants

Table 1 shows the demographic and learning-related characteristics of the participants. The mean age of the 2 groups did not significantly differ, and there were no statistical differences in gender or entrance type distribution. Regarding learning-related characteristics, the honors/pass/fail grading class perceived the learning environment more positively and scored significantly higher on intrinsic motivation. The academic performance of the groups was different, with the honors/pass/fail grading class demonstrating higher performance on the Basic Medical Science Examination.

Main results

Table 2 presents the correlation matrices for the study variables. All variables were positively correlated. Table 3 shows the coefficients in the hypothetical model. The honors/pass/fail grading class predicted better perceptions of the learning environment (unstandardized coefficient [β]=10.685, P<0.001). In addition, the grading system and perceptions of the learning environment were significant predictors of intrinsic motivation (β=3.453, P=0.033; β=0.242, P<0.001). Finally, the grading system and intrinsic motivation predicted academic performance (β=2.417, P=0.046; β=0.114, P=0.019), whereas learning environment perceptions did not. Thus, the honors/pass/fail grading class predicted more positive perceptions of the learning environment, which in turn predicted higher intrinsic motivation and better academic performance.

Table 4 presents the indirect effects estimated using bootstrapping. Learning environment perceptions and intrinsic motivation sequentially mediated the association between the grading system and academic performance (effect, 0.295; 95% bootstrapped CI, 0.021 to 0.665). However, there were no significant indirect effects when only learning environment perceptions or intrinsic motivation was used as a mediator (effect, 0.136; 95% bootstrapped CI, -0.492 to 0.733; effect, 0.395; 95% bootstrapped CI, -0.012 to 1.046, respectively). In addition, the effect of the grading system on performance was independent of learning environment perceptions or intrinsic motivation (β=2.417, P=0.046).

Discussion

Key results

This study sought to identify the mechanisms that might impact the academic performance of medical students during the transition from a norm-referenced A–F grading system to criterion-referenced honors/pass/fail grading. Our results showed a direct effect of the honors/pass/fail grading system on academic performance and a serial mediation effect in which the grading system indirectly influenced performance through perceptions of the learning environment and intrinsic motivation. Interestingly, the indirect effect of simple mediation via either mediator between the assessment system and student performance was not statistically significant.

Interpretation

In this case, a change in the grading system improved students’ perceptions of the learning environment, subsequently increasing intrinsic motivation and, in turn, enhancing academic performance. This close relationship between intrinsic motivation and learning environment perceptions, which has been noted in the literature and supported in this study, might be explained by self-determination theory, an established framework for understanding motivation [11]. This theory emphasizes that fulfilling basic psychological needs (autonomy, competence, and relatedness) is fundamental to building intrinsic motivation. When viewed through the lens of self-determination theory, changing the grading system from the norm-referenced 13-category system to criterion-referenced honors/pass/fail grading might have had certain advantages [12]: (1) the removal of relative rank-based grades might have allowed students to pursue learning for curiosity and interest, and thus have increased their autonomy relating to their studies; (2) the implementation of a structured system in which the students could be assessed as having achieved enough when they met specific criteria could have inspired a sense of competence; and (3) increased relatedness might have been facilitated by alleviating competition and fostering a collaborative climate.

Collectively, our results suggest that the grading system transition provided a favorable learning environment for the students, as captured by their more positive perceptions, and resulted in higher intrinsic motivation, which led to better academic performance.

Comparison with previous studies

Previous studies found benefits in learning environment perceptions [13] and intrinsic motivation [5] within pass/fail grading, and this study of the honors/pass/fail grading system reported these benefits as well. Furthermore, our study showed significant correlations between perceptions, intrinsic motivation, and academic performance in line with previous research [2,4,6,7]. However, perceptions of the learning environment were not identified as a valid variable for academic performance in the multivariable regression model of this study. These results imply that the independent effect of learning environment perceptions was diminished due to interactions with other variables. The results of the serial mediation analysis, which showed that learning environment perceptions and intrinsic motivation were not the sole mediators but sequential mediators, further confirmed their combined role in medical students’ performance.

Limitations/generalizability

These findings broaden our understanding of how a change in the grading system could affect the academic performance of medical students. Despite its strengths, this study had several limitations. First, it examined only one cohort of students for each grading system at a single center. Therefore, the results might not be generalizable to other cohorts, settings, or populations. Second, the use of self-report questionnaires was a limitation in ensuring the findings’ validity. A mixed approach involving quantitative and qualitative measures would be helpful in triangulating our findings and further clarifying the underlying processes and mechanisms. Third, we could not exclude the influence of factors other than students’ perceptions of the learning environment and motivation in mediating the relationship between the grading system and academic performance. Fourth, an issue not addressed in this study was whether honors/pass/fail grading differentially affected students’ academic performance and mental health. Inconsistent findings have been reported on this matter, with some studies supporting the elimination of honors grades for the sake of well-being [14], while the distinction/pass/fail grading system has been reported to be beneficial relating to perceived stress [1]. However, note that such inconsistencies could have stemmed from the use of different control groups, since the former study focused on the pass/fail grading system as the comparison group, whereas the latter study used traditional letter grades as control groups. Therefore, further studies are needed to understand the varying effects of the honors/pass/fail grading system on medical students’ learning and mental health. Moreover, research on student motivation and performance during clerkship and residency training is needed to illustrate the long-term effects of changes in the grading system.

Suggestions

A direct link between the grading system and students’ performance in our model might suggest an independent influence of the grading system or the presence of other mediating factors. Further studies are required to understand the comprehensive effects of honors/pass/fail grading and identify additional paths through which the grading system and medical students’ academic performance relate to each other. Although this study did not address extrinsic motivation, it would be necessary to understand its role in honors/pass/fail grading. An extensive meta-analysis found that intrinsic motivation and extrinsic incentives jointly affected performance and did not work in opposition to each other [15]. It was possible that an honors grade served as an external reward, and favorable academic performance in our study could have been affected by both intrinsic and extrinsic motivation. Thus, there might be various paths linking honors/pass/fail grading to academic performance that need to be further explained.

Conclusion

Our results indicated that a change from a norm-referenced A–F grading to an honors/pass/fail grading system was associated with more positive perceptions of the learning environment and higher intrinsic motivation in the early years of medical education. The removal of rank-based evaluations might have fostered a positive learning environment that enhanced students’ intrinsic motivation for learning, which could be related to the fact that academic performance did not decline.

Acknowledgments

Supplementary materials

Fig. 1.

Conceptual model of the serial mediation effect of learning environment perceptions and intrinsic motivation on the relationship between the grading system and academic performance.

References

• 1. Jacobs JL, Samarasekera DD, Shen L, Rajendran K, Hooi SC. Encouraging an environment to nurture lifelong learning: an Asian experience. Med Teach 2014;36:164-168. https://doi.org/10.3109/0142159X.2013.852168 ArticlePubMed
• 2. Ahmed Y, Taha MH, Al-Neel S, Gaffar AM. Students’ perception of the learning environment and its relation to their study year and performance in Sudan. Int J Med Educ 2018;9:145-150. https://doi.org/10.5116/ijme.5af0.1fee ArticlePubMedPMC
• 3. Dyrbye L, Shanafelt T. A narrative review on burnout experienced by medical students and residents. Med Educ 2016;50:132-149. https://doi.org/10.1111/medu.12927 ArticlePubMed
• 4. Taylor G, Jungert T, Mageau GA, Schattke K, Dedic H, Rosenfield S, Koestner R. A self-determination theory approach to predicting school achievement over time: the unique role of intrinsic motivation. Contemp Educ Psychol 2014;39:342-358. https://doi.org/10.1016/j.cedpsych.2014.08.002 Article
• 5. Chamberlin K, Yasue M, Chiang IC. The impact of grades on student motivation. Act Learn High Educ 2023;24:109-124. https://doi.org/10.1177/146978741881972 Article
• 6. Zalts R, Green N, Tackett S, Lubin R. The association between medical students’ motivation with learning environment, perceived academic rank, and burnout. Int J Med Educ 2021;12:25-30. https://doi.org/10.5116/ijme.5ff9.bf5c ArticlePubMedPMC
• 7. Orsini C, Binnie VI, Wilson SL. Determinants and outcomes of motivation in health professions education: a systematic review based on self-determination theory. J Educ Eval Health Prof 2016;13:19. https://doi.org/10.3352/jeehp.2016.13.19 ArticlePubMedPMC
• 8. Roff SU, McAleer S, Harden RM, Al-Qahtani M, Ahmed AU, Deza H, Groenen G, Primparyon P. Development and validation of the Dundee ready education environment measure (DREEM). Med Teach 1997;19:295-299. https://doi.org/10.3109/01421599709034208 Article
• 9. Vallerand RJ, Pelletier LG, Blais MR, Briere NM, Senecal C, Vallieres EF. The Academic Motivation Scale: a measure of intrinsic, extrinsic, and amotivation in education. Educ Psychol Meas 1992;52:1003-1017. https://doi.org/10.1177/0013164492052004025 Article
• 10. Sijtsma K, van der Ark LA. Investigation and treatment of missing item scores in test and questionnaire data. Multivariate Behav Res 2003;38:505-528. https://doi.org/10.1207/s15327906mbr3804_4 ArticlePubMed
• 11. Ryan RM, Deci EL. Self-determination theory: basic psychological needs in motivation, development, and wellness. The Guilford Press; 2017. https://doi.org/10.1521/978.14625/28806
• 12. Ryan RM, Deci EL. Intrinsic and extrinsic motivation from a self-determination theory perspective: definitions, theory, practices, and future directions. Contemp Educ Psychol 2020;61:101860. https://doi.org/10.1016/j.cedpsych.2020.101860 Article
• 13. Zgheib NK, Dimassi Z, Arawi T, Badr KF, Sabra R. Effect of targeted curricular reform on the learning environment, student empathy, and hidden curriculum in a medical school: a 7-year longitudinal study. J Med Educ Curric Dev 2020;7:2382120520953106. https://doi.org/10.1177/2382120520953106 ArticlePubMedPMC
• 14. Smith JF Jr, Piemonte NM. The problematic persistence of tiered grading in medical school. Teach Learn Med 2023;35:467-476. https://doi.org/10.1080/10401334.2022.2074423 ArticlePubMed
• 15. Cerasoli CP, Nicklin JM, Ford MT. Intrinsic motivation and extrinsic incentives jointly predict performance: a 40-year meta-analysis. Psychol Bull 2014;140:980-1008. https://doi.org/10.1037/a0035661 ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by