Graphical abstract

Introduction

Background

The medical educational activities carried out by medical school faculty members are rapidly changing quantitatively and qualitatively. The recent shift of the medical education paradigm to competency-based medical education has required medical schools to organize and operate an educational system that enables individual students to successfully demonstrate their achievement of learning outcomes [1]. Therefore, the role of faculty members has expanded from traditional instructors to curriculum designers, mentors, advisors, and coaches so that learners can demonstrate the intended outcomes throughout the entire curriculum [2,3]. In addition, coronavirus disease 2019 challenged faculty members and students to an online learning environment, which led to a change from in-person education to online or hybrid education, and from large-scale lectures to small-group or individual-based learning [4,5].

Medical school faculty members struggle to carry out multiple duties: research, clinical practice, education, and public service. Despite increasing demands for engagement in education, education still ranks lowest in priority among their duties. This is mainly due to the fact that evaluation of educational performance is less systematic and its standards are less demanding than the evaluation of research performance and clinical practice performance. In order to be promoted, medical faculty members spend more time caring for patients and producing research results, rather than participating in education [6-8].

Efforts are being made to analyze the medical education activities of medical faculty members and to reward them appropriately for their dedication in education. The Association of American Medical Colleges (AAMC) developed a metrics system to measure faculty members’ effort and contributions to medical education and presented a list of educational activities that faculty members could conduct. Additionally, they proposed the concept of the relative value unit, which refers to the value of the weight of the unit of activities relative to the mission or policy of the medical school, to measure educational activities [9].

In Korea, a study has measured the efforts and contribution of medical faculty members in education at a single medical school [10]. The evaluation items and weights vary depending on the medical school’s mission; however, the items for fundamental medical education activities might be similar. Therefore, it is necessary to investigate the common evaluation items and weighting methods of medical education activities.

Objectives

The aim of this study was to investigate the current status of evaluation of educational performance of medical faculty members through an analysis of promotion regulations of medical schools in Korea. Specifically, promotion regulations of each medical school were classified through content analysis, and contents of individual medical school’s promotion regulations were reorganized according to the classification.

Methods

Ethics statement

Ethical approval was not required for this study. This study did not include a clinical trial and did not collect any personal data. Information that was publicly available on websites was collected and analyzed.

Study design

This study used mixed methods. The strategy of this study was type 1 mixed methods [11,12], as this study connected the results of a qualitative study (phase 1) with the result of a quantitative study (phase 2) (Fig. 1).

Phase 1: Qualitative study

Data collection

In August 2022, we searched the websites of medical schools and their universities in Korea to identify publicly accessible promotion regulations with provisions on medical educational activities. Data were collected from the promotion regulations of 22 out of 40 medical schools (Fig. 1).

Personal characteristics of the research team

The research team consisted of experts on medical education. J.P., a PhD in educational measurement and evaluation, and H.W.J., an MD, PhD in medicine, were professors in the Department of Medical Education at a school of medicine with several years of experience in the development of the faculty member evaluation systems.

Data analysis

Data were analyzed through deductive content analysis. This study defined and categorized the educational activities of faculty members recognized in the promotion regulations of each medical school according to the categories suggested in the AAMC framework to improve the correspondent validity and compared the results between the AAMC and our results [9]. The AAMC framework defines 4 educational activity categories: teaching (T), development of education products (D), education administration and service (A), and scholarship in education (S), and 17 main activities and 69 sub-activities. We added the student affairs (SA) category because many medical schools in Korea recognized work related to student affairs, such as student counseling and career coaching, as educational activities. Educational activities that did not fit into any of these 5 categories were classified as “others.” Two researchers (J.P. and H.W.J.) iteratively read and reviewed the contents in the collected promotion regulations, and codes were assigned to all documents. according to these 6 categories and their sub-activities. We also explored scoring methods of educational activities and factors for adjusting the weight of educational activities. The coding structure that emerged from the data was critically discussed between the researchers until consensus was achieved.

Phase 2: Quantitative study

Variables

The coverage rate of educational activities in each category and the inclusion rate of individual educational activities in promotion regulations were analyzed. The type of institution, the year of establishment, and the number of faculty members at each medical school were variables for the basic characteristics of medical schools.

Data sources/measurement

Main data were the contents of individual medical school’s promotion regulations which were reorganized according to the classification based on the results of the qualitative research (Dataset 1). Data on medical schools’ basic characteristics were collected from the database of the Korean Association of Medical Colleges. Scoring methods and quality evaluation of medical educational activities is available at Dataset 2.

Bias

No selection bias was identified. This study included all medical schools that met the study’s eligibility criteria.

Study size

There are a total of 40 medical schools in Korea, and a survey was conducted on all schools, of which 22 met the research conditions. Therefore, no sample size was estimated for quantitative analysis.

Statistical methods

The quantitative variables were presented as frequency and percentage. Frequency, descriptive, and correlation analyses were performed. Correlations between the characteristics of medical schools and categories of educational activities were assessed using Spearman’s or Pearson’s correlation analysis. All data were processed in IBM SPSS ver. 27.0 (IBM Corp.).

Results

General characteristics of medical schools

Sixteen were private medical schools and six were public medical schools. Eight medical schools had 101–200 faculty members, seven had 201–300 faculty members, and seven had more than 300 faculty members. Ten medical schools were established after 1980, and the remaining 12 were established before 1980 (Table 1).

Educational activities specified in the promotion regulations of Korean medical schools

Classifying medical educational activities stipulated in the promotion regulation of medical schools, the researchers yielded 6 categories and 20 activities with 57 sub-activities; teaching (5 activities and 16 sub-activities), development of education products (3 activities and 9 sub-activities), education administration and service (4 activities and 16 sub-activities), scholarship in education (5 activities and 12 sub-activities), student affairs (3 activities and 4 sub-activities), and others (Table 2). The “others” category had unique activities, such as donations, fundraising, medical students’ pass rate on the Korean Medical Licensing Examination, the pass rate on the specialty examination, disciplinary results, and so forth.

The coverage rate of educational activities in each category was the highest in the development of education products category (2.4/3, 78.8%). The second highest was teaching category (3.6/5, 71.8%), and the lowest was scholarship in education category (0.8/5, 16.4%) (Table 2). The inclusion rate of individual educational activities in promotion regulations was the highest in lecture activities, development of personnel, and direction of education components (21/22. 95.5%) (Table 2).

Correlation and comparison between categories of educational activities and characteristics of medical schools

Private schools had a strong positive correlation with a longer history and the inclusion of medical educational activities in the promotion regulation in the following categories: teaching, scholarship in education, and student affairs. Private medical schools tended to include more educational activities than public medical schools (Fig. 2A, Supplement 1), and this tendency was prominent for scholarship and student affairs. The number of activities in the education administration and service category increased as the number of faculty members increased (Fig. 2B). The “scholarship in education” and “student affairs” had a moderate correlation with the “established year of school.” The correlation between individual activities and characteristics of medical school were shown in Table 3 [13] (Supplement 2).

Evaluation of educational activities

Adjustment factors for the weight of educational activities

Four factors were identified to adjust the weight of medical educational activities; the characteristics of the target subjects, the characteristics of the faculty members, the number of faculty members involved in the activity, and the difficulty of the activity. Each factor, especially the difficulty of the activity, had various weighting methods. Fourteen medical educational activities were evaluated with more than one of these 4 weighting factors. The faculty members number was considered in evaluating lecture activities, small-group activities, and development of educational materials. The faculty members’ characteristics were considered only in evaluating the development of personnel (Table 4).

The scoring method and the quality evaluation

Most medical schools chose a criterion-referenced evaluation, with a scoring method based on raw data or grade scores within the range. For 14 medical educational activities, several medical schools displayed an upper limit on the points that could be received. There were 2 types of penalties: receiving negative points and rejecting performance (conditional approval). Examples of point deduction included failure to submit a lecture syllabus before the lecture, and the examples of rejection of performance included failure to meet the attendance time or frequency, such as participation in faculty member development seminars and committees. There were 3 types of evidence utilized to evaluate the quality of educational performance. Quality evaluation was conducted for 7 educational activities, among which lecture activities were evaluated most frequently (Table 5).

Discussion

Key results

This study provided an overview of the current evaluation status of medical school faculty members’ educational performance by identifying medical education activities and their assessment methods along with weighting factors through an analysis using mixed methods in Korea. The inclusion in the promotions regulations and the weighting and scoring method for the specific activity differed from school to school, which implied that medical schools demanded various educator roles from their faculty members depending on their mission and policies.

Interpretation

While almost all medical schools included lecture activities in their promotion regulations, laboratory activities and clinical practice activities were included in relatively a small number of medical schools. Considering that many faculty members spend a considerable amount of time on them, effort seems necessary to reflect them more actively in the promotion regulations. The high inclusion rate of the development of personnel in promotion regulations seemed to be related to the standard of the Korean accreditation of medical schools such as “at least 50% of full-time faculty members must participate in medical education training or education-related faculty member development programs for at least 3 hours a year” [14].

Activities in the scholarship in education category, except for presentations in education, were rarely included in the regulations. Besides publishing papers, there are many activities related to scholarship in education, which include writing education research reports, making abstract-based presentations at educational academic meetings, giving symposium lectures, and so on. Medical schools should strive to ensure that all educational activities of faculty members are included in the promotion regulations, so that faculty members dedicated to medical education can be properly recognized and evaluated for their effort [6-8].

Among 4 adjustment factors for the weight of educational activities, the difficulty of the activity was the most frequently considered, and the number of items in each factor varied among medical schools. The AAMC recommended implementing quantitative and qualitative weights differently in stage 1, and completing the weights based on the mission of the school in stage 2 [9]. When establishing standards for evaluating faculty members’ educational performance, the weight of educational activities must be determined by reflecting the mission and policy of medical schools so that faculty members can accept the evaluation standards and devote themselves to educational activities consistent with the direction of the mission and policy.

Most educational activities were evaluated with a scoring system in which the more faculty members did, the more points they got. The penalty, either point deduction or conditional approval, was given when the activities were not properly carried out, or when the additional tasks necessary to complete the activities were not met.

Limitations/suggestions

First, the data of this study were based on the promotion regulations published on the websites of medical schools or universities. There was a possibility that some universities/medical schools did not disclose the details of promotion regulations on the web.

Second, the small number of study subjects made further statistics other than correlation analysis impossible. In addition, caution may be required when interpreting the result of correlation analysis with a small sample number. Therefore, it would be appropriate to conduct a more comprehensive study using surveys and in-depth interviews through representative organizations such as the Korea Association of Medical Colleges.

Conclusion

Rapidly changing medical education calls for more efforts of faculty members to meet the needs of learners and society. In order for faculty members to devote themselves actively and passionately to education, they must receive reasonable rewards. This study analyzed the status of medical education activities reflected in the promotion regulations of medical schools in 2022 in Korea. This study provides essential data for improving the reward system for medical faculty members’ educational efforts.

Article information

Authors’ contributions

Two authors equally contributed to conceptualization, data collection and analysis, drafting the article, and reviewing and editing the article.

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Funding

This work was supported by the Soonchunhyang University Research Fund.

Data availability

Data files are available from Harvard Dataverse: https://doi.org/10.7910/DVN/FL823U

Dataset 1. Educational activities in regulations.

jeehp-20-07-dataset1.xlsx

Dataset 2. Scoring methods and quality evaluation of medical educational activities.

jeehp-20-07-dataset2.xlsx

Acknowledgments

Supplementary materials

Fig. 1.

Data collection and analysis process.

Fig. 2.

Comparison of the average number of the educational activities included in each category according to the type of medical schools (A) and the number of faculty (B). Development: development of education products. Administration: education administration and service, scholarship.

References

• 1. Shumway JM, Harden RM; Association for Medical Education in Europe. AMEE guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach 2003;25:569-584. https://doi.org/10.1080/0142159032000151907 ArticlePubMed
• 2. Meeuwissen SN, Stalmeijer RE, Govaerts M. Multiple-role mentoring: mentors’ conceptualisations, enactments and role conflicts. Med Educ 2019;53:605-615. https://doi.org/10.1111/medu.13811 ArticlePubMedPMC
• 3. Na SJ, Ji YG, Lee DH. Application of Bloom’s taxonomy to formative assessment in real-time online classes in Korea. Korean J Med Educ 2021;33:191-201. https://doi.org/10.3946/kjme.2021.199 ArticlePubMedPMC
• 4. Stojan J, Haas M, Thammasitboon S, Lander L, Evans S, Pawlik C, Pawilkowska T, Lew M, Khamees D, Peterson W, Hider A, Grafton-Clarke C, Uraiby H, Gordon M, Daniel M. Online learning developments in undergraduate medical education in response to the COVID-19 pandemic: a BEME systematic review: BEME guide no. 69. Med Teach 2022;44:109-129. https://doi.org/10.1080/0142159X.2021.1992373 ArticlePubMed
• 5. Han ER, Yeo S, Kim MJ, Lee YH, Park KH, Roh H. Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review. BMC Med Educ 2019;19:460. https://doi.org/10.1186/s12909-019-1891-5 ArticlePubMedPMC
• 6. Thomas PA, Diener-West M, Canto MI, Martin DR, Post WS, Streiff MB. Results of an academic promotion and career path survey of faculty at the Johns Hopkins University School of Medicine. Acad Med 2004;79:258-264. https://doi.org/10.1097/00001888-200403000-00013 ArticlePubMed
• 7. LeMaire SA, Trautner BW, Ramamurthy U, Green SY, Zhang Q, Fisher WE, Rosengart TK. An academic relative value unit system for incentivizing the academic productivity of surgery faculty members. Ann Surg 2018;268:526-533. https://doi.org/10.1097/SLA.0000000000002921 ArticlePubMed
• 8. Zuckerman MD, Lin S, Alsalmi F, Li-Sauerwine S. Narrative review of clinical productivity and teaching in emergency medicine. Cureus 2021;13:e14309. https://doi.org/10.7759/cureus.14309 ArticlePubMedPMC
• 9. Nutter DO, Bond JS, Coller BS, D'Alessandri RM, Gewertz BL, Nora LM, Perkins JP, Shomaker TS, Watson RT. Measuring faculty effort and contributions in medical education. Acad Med 2000;75:199-207. https://doi.org/10.1097/00001888-200002000-00025 ArticlePubMed
• 10. Kim HB, Myung SJ, Yu HG, Chang JY, Shin CS. Influences of faculty evaluating system on educational performance of medical school faculty. Korean J Med Educ 2016;28:289-294. https://doi.org/10.3946/kjme.2016.34 ArticlePubMedPMC
• 11. Kaur N, Vedel I, El Sherif R, Pluye P. Practical mixed methods strategies used to integrate qualitative and quantitative methods in community-based primary health care research. Fam Pract 2019;36:666-671. https://doi.org/10.1093/fampra/cmz010 ArticlePubMedPMC
• 12. Johnson RB, Onwuegbuzie AJ, Turner LA. Towards a definition of mixed method research. J Mix Methods Res 2007;1:112-33. https://doi.org/10.1177/1558689806298224 Article
• 13. Senthilnathan S. Usefulness of correlation analysis [Internet]. SSRN; 2019 [cited 2022 Aug 10]. Available from: https://ssrn.com/abstract=3416918
• 14. Korean Institute of Medical Education and Evaluation. Basic medical education accreditation standards [Internet]. Korean Institute of Medical Education and Evaluation; 2022 [cited 2022 Aug 10]. Available from: https://kimee.or.kr/medical-education/criteria/

Figure & Data

References

Citations

Citations to this article as recorded by