Graphical abstract

Introduction

Background

Advancements in the medical field directly impact national health levels. Within this framework, the caliber of medical school education is widely acknowledged as a crucial element in global health policies. The production of well-trained medical professionals is vital for bolstering a country’s health infrastructure and enhancing the provision of medical services. In light of this significant duty, Korea has been actively standardizing and enhancing medical school education by implementing accreditation standards since 2000 [1]. The Korean Institute of Medical Education and Evaluation (KIMEE) is at the forefront, serving as the principal body responsible for managing the evaluation and accreditation of medical education in Korea [2]. The Accreditation Standards of KIMEE 2019 (ASK2019) were developed to improve the medical school education standard and update the educational content to align with international benchmarks, notably the World Federation for Medical Education Global Standards for Quality Improvement [3]. There are 9 evaluation domains, 36 sub-areas, 92 basic standards, and 51 high-quality standards in ASK 2019. Compared to previous evaluation standards, although the total number of items has mainly remained consistent, there has been a noticeable expansion in evaluation areas and divisions, resulting in a more refined and comprehensive assessment framework [4]. The accreditation and evaluation process for medical schools in Korea includes a self-study by the school, collection, and review of self-study and student reports, an on-site visit, and a final decision on accreditation status. While numerous studies have indicated that the evaluation and accreditation system has positively influenced various aspects, such as curriculum development and the enhancement of student performance [5], a comprehensive analysis and understanding of these systems’ specific operational processes and outcomes still need to be improved.

Meta-evaluation has been an effective tool for assessing the accreditation process, as it critically examines the evaluation process itself [6]. In Korea, meta-evaluation methods have been used to assess university certification evaluations across various disciplines [7,8]. The KIMEE has conducted a meta-evaluation from 2011 until the second cycle of the third round of evaluation and certification of Korean medical schools.

Past meta-evaluations proved that the standards used to evaluate medical school accreditation were effective in promoting quality improvement and accountability in healthcare. However, they needed to catch up in capturing certain qualitative aspects that could not be measured with quantitative methods, indicating a notable gap in assessment. In light of this, a comprehensive meta-evaluation of the newly implemented ASK2019 was essential.

Objectives

This study aimed to perform a comprehensive meta-evaluation of medical school accreditation and evaluation based on ASK2019 to identify areas for improvement and potential future directions. To achieve this goal, the study systematically collected and analyzed the insights and perceptions of a diverse group of stakeholders. This group included members of the self-assessment committee, the local visiting committee, administrative staff, and medical professors, all within the operational framework of ASK2019.

Methods

Ethics statement

The Hanyang University Institutional Review Board (IRB no., HYUIRB 202307-011) approved this study, classifying it as low-risk and exempting it from the requirement for written consent because it utilizes anonymous email surveys that participants returned after giving consent.

Study design

This cross-sectional, national-level study involving stakeholders from 40 medical schools was conducted using an online survey.

Setting

From September 2022 to February 2023, over 5 months, letters requesting cooperation were sent to 40 medical schools in Korea, facilitated by the coordination and recommendation of the KIMEE. To collect survey responses, the KIMEE sent these letters to medical professors and administrative staff at the 40 medical schools. Surveys were also distributed to the site visit committee members and the self-evaluation committee via the email addresses registered with the KIMEE, ensuring a thorough data collection process. The research team acquired and analyzed anonymized data from the KIMEE.

Participants

The survey participants comprised 396 individuals, including professors from 40 national medical schools, members of self-evaluation committees with experience in medical school accreditation processes, and site visit committee members from the KIMEE who have participated in the evaluation and accreditation of medical schools. Additionally, administrative staff and faculty members from each medical school were part of the cohort. Participation in the survey was voluntary.

Variables

The variables were derived from Daniel Stufflebeam’s CIPP (Context, Input, Process, Product) model [9] and were finalized after consulting with experts on the validity of each item. Context evaluation is the Overall Perception of Accreditation, which corresponds to understanding the necessity and contribution of accreditation, alongside assessing the positive outcomes and the reliability, fairness, and objectivity of KIMEE. Input evaluation involves assessing the resources, strategies, and plans before the accreditation process begins. This procedure aligns with evaluating the appropriateness and clarity of accreditation standards and the feasibility of implementing these standards. In-process evaluation evaluates the procedures and methods used during the accreditation process, including addressing challenges in preparing self-evaluation study reports. Finally, product evaluation looks at the outcomes of the accreditation process, including the appropriateness of resources allocated to site visits, perceptions of the site visit committee members, and the accuracy of site visit evaluation results.

Data sources/measurement

The research team developed the survey items collaboratively and subsequently refined through iterative discussions with the KIMEE’s Quality Improvement Committee. Experts experienced in accreditation to ensure the survey’s validity and reliability (Supplement 1). The survey encompassed various aspects of accreditation, such as its necessity, contribution to medical education, and perceptions of the KIMEE’s credibility, fairness, and objectivity. Additionally, it assessed the clarity and applicability of the accreditation standards and procedures. Responses were collected using a 6-point Likert scale for all items, and multiple-choice questions were employed to assess the positive outcomes of accreditation and the challenges encountered in preparing self-evaluation study reports. Context, input, process, and product evaluation criteria for the ASK2019 program are available in Supplement 2.

Bias

To minimize selection bias, we conducted surveys with one representative from each group: the self-evaluation study committee, the site visit committee, administrative staff, and faculty members from all 40 medical schools in South Korea.

Study size

Using G*Power (Heinrich-Heine-Universität Düsseldorf) [10] and assuming a medium effect size (F=0.25) [11] at a 0.05 significance level, at least 280 participants were needed for one-way analysis of variance (ANOVA) to ensure a statistical power of 0.95. Considering potential dropouts, this study recruited 300 participants.

Statistical methods

Data analysis was conducted using IBM SPSS ver. 25.0 (IBM Corp.). The collected data comprised basic statistics, including mean and standard deviation. The t-test, one-way ANOVA, and the chi-square test were utilized to compare perceptions of medical education accreditation among 4 distinct groups: self-evaluation study committee members, site visit evaluators, general medical faculty, and medical school administrative staff.

Results

Participants

After excluding missing data, the study included 332 participants. This cohort consisted of 172 self-evaluation study committee members, 28 site visit evaluators, 69 general professors from medical schools, and 64 administrative staff members from medical schools (Table 1).

Context evaluation: overall perception of accreditation response data of the survey are available at Dataset 1.

Necessity of accreditation

Statistically significant differences were observed in the perceived necessity of accreditation among the 4 groups (Table 2). The most significant difference was noted between the site visit committee members and medical school faculty members.

Contribution to accreditation and evaluation

As shown in Table 3, there was no significant difference in the responses between the self-evaluation study committee members and medical school professors regarding how much the evaluation and accreditation process contributed to improving the quality of medical education at their respective medical schools. Both groups exhibited predominantly positive attitudes.

Positive outcomes of accreditation

No significant differences were observed among the groups regarding the positive outcomes of accreditation (Table 4). Members of the self-evaluation study committee and administrative staff predominantly perceived the accreditation process as beneficial, with improvements in areas previously identified as deficient during the self-evaluation study being the most frequent indication.

Reliability, fairness, and objectivity of the KIMEE

Significant differences were observed in how members of the self-evaluation study committee and administrative staff perceived the reliability, fairness, and objectivity of the KIMEE, as shown in Table 5. Across all 3 aspects, administrative staff held a more favorable opinion of the KIMEE than the self-evaluation study committee members.

Input evaluation: perception of accreditation standards

Appropriateness and clarity of accreditation standards

Significant differences were observed between the self-evaluation study committee members and the site visit committee members regarding opinions on accreditation standards. These differences pertained to the adequacy of basic criteria, excellence criteria, and the clarity of the standards (Table 6). Notably, site visit evaluators had more positive perceptions than self-assessment committee members for all 3 items.

Feasibility of implementing accreditation standards

Regarding the feasibility of implementing evaluation and accreditation standards in medical education, notable differences were observed in the average responses among self-evaluation study committee members, site visit committee members, and administrative staff (Table 7). Specifically, the disparity between the perceptions of administrative staff and self-evaluation study committee members was the most marked for perceived feasibility.

Process evaluation: perceptions of the accreditation process

Appropriateness of medical education accreditation procedures and methods

Significant differences were observed in the average perceived suitability of the current accreditation processes between self-evaluation study committee members and administrative staff, except for the interim evaluation report (Table 8). Administrative staff held a more favorable opinion of the appropriateness of medical education accreditation procedures and methods than self-evaluation study committee members.

Challenges encountered in the preparation of self-evaluation study reports

The distribution of challenges faced while preparing self-evaluation study reports varied among different stakeholder groups in medical schools (Table 9). Self-evaluation study committee members reported that the most significant difficulty was a need for a clearer understanding of accreditation standards (23.8%), while administrative staff primarily found duplication of content (35%) to be the most challenging aspect.

Product evaluation: perceptions of site visit evaluations

Appropriateness of resources allocated to site visits

Site visit committee members had more positive perceptions than self-evaluation study committee members regarding the appropriateness of resource allocation for site visits (Table 10). However, there was no significant difference between the 2 groups in terms of cost appropriateness.

Perceptions of the site visit committee members

There was no significant difference between the self-evaluation study committee members and the site visit committee members regarding the professionalism of the site visit committee (Table 11).

Perceptions of site visit evaluation results

Regarding the accuracy of the final evaluation report, a significant difference was observed between the self-evaluation study committee members and the site visit committee members (Table 12). Site visit committee members had more positive perceptions than self-evaluation study committee members.

Discussion

Key results

This study aimed to explore the diverse perspectives of key stakeholders engaged in medical education accreditation in South Korea. These stakeholders included members of the self-evaluation study committee, site visit committee members, administrative staff, and medical professors. During the meta-evaluation process, a cross-sectional survey was administered. The findings revealed that members of the self-evaluation study committee and medical professors generally held positive views regarding the role of accreditation in enhancing medical education.

Interpretation/comparison with previous studies

It is consistent with results from the post-second cycle meta-evaluation, which showed that medical school professors and staff involved in accreditation preparation acknowledged its importance in identifying and addressing institutional weaknesses [12].

However, this study highlighted several challenges, particularly for medical schools preparing for accreditation. These institutions encountered resource constraints during the data collection, analysis, and reporting phases, which are integral to meeting the established criteria and procedures [13,14]. Although the significance of accreditation in enhancing quality is widely acknowledged, there was a marked discrepancy in perceptions between the site visit committee members and the administrative staff. Addressing the divergence in viewpoints between evaluators and medical schools is an urgent issue that demands immediate attention to facilitate a smooth accreditation preparation process. Considering the critical role that administrative staff play in this preparation, it is essential to improve their working conditions to allow them to focus effectively on their duties. Furthermore, it is worth considering offering compensation for their significant contributions to the accreditation process.

The study also revealed a general skepticism among both site visit committee members and self-evaluation study committee members toward the essential and excellence criteria of accreditation, possibly due to a lack of clear understanding and guidance. Although the overall impact of accreditation on improving the quality of medical education was viewed positively, the responses concerning the basic and excellence criteria from self-evaluation study committee members and site visit evaluators were less favorable than those from the post-2nd cycle meta-evaluation.

Nevertheless, the self-evaluation study committee members noted a significant shift in their perception of the site visit evaluators’ professionalism, with a more positive outlook in this study than in the post-2nd cycle evaluation. Similarly, the accuracy and reliability of the reporting process were viewed more favorably.

To streamline and strengthen the medical education accreditation process, it is essential to broaden support by providing additional technical, financial, and human resources, thereby alleviating the burden on faculty members. The KIMEE should provide clear and detailed guidelines for accreditation to facilitate systematic preparation. Regular training sessions are important to improve stakeholders’ understanding and preparedness for the accreditation process. Promoting better collaboration and communication between medical schools and accreditation bodies is vital to align their goals. Introducing incentives can mitigate the perceived burden of accreditation, increase motivation, and foster a positive outlook [15-17]. Providing consistent feedback and continually refining standards and procedures are crucial to enhancing the effectiveness of the process. Finally, adopting a data-driven approach for ongoing meta-evaluation of the process will enable informed, evidence-based improvements.

The meta-evaluation reveals critical implications for Korea’s medical education accreditation for enhanced stakeholder communication and collaboration, simplification of accreditation procedures, and improved support for administrative staff. Addressing skepticism with more explicit guidelines and training is critical for a positive accreditation perception. These steps are vital for fostering a transparent, efficient accreditation process that uplifts medical education standards in Korea.

Limitations and generalizability

Using surveys can result in response bias because stakeholders may offer socially desirable responses or withhold their genuine opinions due to discomfort with the survey process. Furthermore, gathering data at a single time point can overlook the dynamic nature of perceptions, particularly when accreditation standards change. Reliance on self-reported surveys may also lead to bias, as participants might need to accurately report their behaviors or perceptions.

Suggestions

This study emphasizes the need for longitudinal research to understand the long-term effects of medical school accreditation on medical education quality and outcomes. It also points out the significance of analyzing resource allocation strategies to improve the efficiency of the accreditation process. It calls for an examination of the clarity and suitability of accreditation standards. Finally, the study stresses the importance of fostering an environment and culture that prioritizes continuous quality improvement in medical education by incorporating lessons learned from the accreditation process into continuous professional development.

Conclusion

The study sheds light on various stakeholders’ multifaceted perceptions of accreditation in medical education. Although there is a broadly positive attitude toward the essential nature and benefits of accreditation, differing opinions on the evaluation process, criteria, and the accrediting organization indicate a necessity for improved communication, the harmonization of expectations, and ongoing refinement of accreditation methods. These results highlight the significance of a cooperative approach to accreditation, one in which input from all parties is considered and integrated, fostering a transparent, equitable, and efficient process that ultimately elevates the standard of medical education.

Acknowledgments

Supplementary materials

References

• 1. Meng KH. The history and implications of the Medical Education Accreditation System in Korea: implementation and activities in early stages. Korean Med Educ Rev 2020;22:1-8. https://doi.org/10.17496/kmer.2020.22.1.1 Article
• 2. Lim KY. A proposal for the future of medical education accreditation in Korea. J Educ Eval Health Prof 2020;17:32. https://doi.org/10.3352/jeehp.2020.17.32 ArticlePubMedPMC
• 3. Kim YC. Korean Institute of Medical Education and Evaluation Presidential Address: the role of KIMEE as a medical education accreditation agency during the coronavirus disease 2019 pandemic. J Educ Eval Health Prof 2021;18:2. https://doi.org/10.3352/jeehp.2021.18.2 ArticlePubMedPMC
• 4. Choi H, Ahn Y. The ASK 2019 in the accreditation of medical school. Med J Chosun Univ [Internet]. 2019 [cited 2024 Feb 10];44:1-7. Available from: https://db.koreascholar.com/Article/Detail/370827
• 5. Alenezi S, Al-Eadhy A, Barasain R, AlWakeel TS, AlEidan A, Abohumid HN. Impact of external accreditation on students’ performance: Insights from a full accreditation cycle. Heliyon 2023;9:e15815. https://doi.org/10.1016/j.heliyon.2023.e15815 ArticlePubMedPMC
• 6. Pillai KN, Srinivas G. A study of the post‐accreditation scenario in the north eastern region of India: a meta‐evaluation of the National Assessment and Accreditation Council processes and procedures. Qual High Educ 2006;12:95-106. https://doi.org/10.1080/13538320600685222 Article
• 7. Kim EM, Kim YJ, Lee SH, Namgung JY, Lee JI, Kim J. The study of developing the meta evaluation criteria for the Accreditation of Dental Education in Korea. J Korean Dent Assoc 2022;60:143-152. https://doi.org/10.22974/jkda.2022.60.3.002 Article
• 8. Cho OK, Choi K. The development of the meta-evaluation criteria for the Accreditation Evaluation System of Engineering Education in Korea. J Eng Educ Res 2013;16:28-41. https://doi.org/10.18108/jeer.2013.16.3.28 Article
• 9. Stufflebeam DL. The CIPP model for evaluation. In: Kellaghan T, Stufflebeam DL, editors. International Handbook of Educational Evaluation. Springer; 2003. p. 31-32. https://doi.org/10.1007/978-94-010-0309-4_4
• 10. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39:175-191. https://doi.org/10.3758/bf03193146 ArticlePubMed
• 11. Schafer T, Schwarz MA. The meaningfulness of effect sizes in psychological research: differences between sub-disciplines and the impact of potential biases. Front Psychol 2019;10:813. https://doi.org/10.3389/fpsyg.2019.00813 ArticlePubMedPMC
• 12. Yoo HH, Kim MK, Yoon YS, Lee KM, Lee JH, Hong SJ, Huh JS, Park WK. Change of accreditation standards of medical schools by the Korean Institute of Medical Education and Evaluation from 2000 to 2019. J Educ Eval Health Prof 2020;17:2. https://doi.org/10.3352/jeehp.2020.17.2 ArticlePubMedPMC
• 13. Al-Shareef AS, AlQurashi MA, Al Jabarti A, Alnajjar H, Alanazi AA, Almoamary M, Shirah B, Alqarni K. Perception of the accreditation of the National Commission for Academic Accreditation and assessment at different health colleges in Jeddah, Saudi Arabia. Cureus 2023;15:e43871. https://doi.org/10.7759/cureus.43871 ArticlePubMedPMC
• 14. Jung H, Jeon WT, An S. Is accreditation in medical education in Korea an opportunity or a burden? J Educ Eval Health Prof 2020;17:31. https://doi.org/10.3352/jeehp.2020.17.31 ArticlePubMedPMC
• 15. Bigdeli S, Arabshahi SK, Sohrabi Z, Zazoly AZ. Exploring the challenges of educational accreditation of teaching hospitals and providing the solutions. J Educ Health Promot 2021;10:144. https://doi.org/10.4103/jehp.jehp_894_20 ArticlePubMedPMC
• 16. Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. Int J Qual Health Care 2008;20:172-183. https://doi.org/10.1093/intqhc/mzn005 ArticlePubMed
• 17. Rink LC, Oyesanya TO, Adair KC, Humphreys JC, Silva SG, Sexton JB. Stressors among healthcare workers: a summative content analysis. Glob Qual Nurs Res 2023;10:23333936231161127. https://doi.org/10.1177/23333936231161127 ArticlePubMedPMC

Figure & Data

References

Citations

Citations to this article as recorded by  

• The new placement of 2,000 entrants at Korean medical schools in 2025: is the government’s policy evidence-based?
  Sun Huh
  The Ewha Medical Journal.2024;[Epub]     CrossRef