Journal of Educational Evaluation for Health Professions

Research articles

An innovative resident-driven mortality case review curriculum to teach and drive system-based practice improvements in the United States: Nila S. Radhakrishnan, Margaret C. Lo, Rohit Bishnoi, Subhankar Samal, Robert Leverence, Eric Rosenberg, Zareen Zaidi; J Educ Eval Health Prof. 2018;15:31. Published online December 26, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.31; Funded: National Center for Advancing Translational Sciences

22,596 View
260 Download
3 Web of Science
4 Crossref

Purpose
Traditionally, the morbidity and mortality conference (M&MC) is a forum where possible medical errors are discussed. Although M&MCs can facilitate identification of opportunities for systemwide improvements, few studies have described their use for this purpose, particularly in residency training programs. This paper describes the use of M&MC case review as a quality improvement activity that teaches system-based practice and can engage residents in improving systems of care.
Methods
Internal medicine residents at a tertiary care academic medical center reviewed 347 consecutive mortalities from March 2014 to September 2017. The residents used case review worksheets to categorize and track causes of mortality, and then debriefed with a faculty member. Selected cases were then presented at a larger interdepartmental meeting and action items were implemented. Descriptive statistics and thematic analysis were used to analyze the results.
Results
The residents identified a possible diagnostic mismatch at some point from admission to death in 54.5% of cases (n= 189) and a possible need for improved management in 48.0% of cases. Three possible management failure themes were identified, including failure to plan, failure to communicate, and failure to rescue, which accounted for 21.9%, 10.7 %, and 10.1% of cases, respectively. Following these reviews, quality improvement initiatives proposed by residents led to system-based changes.
Conclusion
A resident-driven mortality review curriculum can lead to improvements in systems of care. This novel type of curriculum can be used to teach system-based practice. The recruitment of teaching faculty with expertise in quality improvement and mortality case analyses is essential for such a project.

Citations

Citations to this article as recorded by

Implementation of Hospital Mortality Reviews: A Systematic Review
Moritz Sebastian Schönfeld, Martin Härter, Ann Sophie Schröder, Katrin Kokartis, Hans-Jürgen Bartz, Levente Kriston
Journal of Patient Safety.2024; 20(2): 138. CrossRef
One health system’s morbidity and mortality conference survey: a step on the journey to organizational high reliability
Steven E. Raper, Patricia G. Sullivan, Lawrence S. Levin
International Journal of Surgery: Global Health.2023;[Epub] CrossRef
Defining a High-Quality and Effective Morbidity and Mortality Conference
Brendin R. Beaulieu-Jones, Spencer Wilson, Daniel S. Howard, Gordana Rasic, Ben Rembetski, Erica A. Brotschi, Luise I. Pernar
JAMA Surgery.2023; 158(12): 1336. CrossRef
Implementation of morbidity and mortality conference in a community hospital NICU and narrative review
Christina Chan, Christine Pazandak, Dimitrios Angelis
Frontiers in Pediatrics.2023;[Epub] CrossRef

Comparison of standard-setting methods for the Korean Radiological Technologist Licensing Examination: Angoff, Ebel, bookmark, and Hofstee: Janghee Park, Duck-Sun Ahn, Mi Kyoung Yim, Jaehyoung Lee; J Educ Eval Health Prof. 2018;15:32. Published online December 26, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.32; Funded: National Health Personnel Licensing Examination Board

18,850 View
247 Download
11 Web of Science
8 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to compare the possible standard-setting methods for the Korean Radiological Technologist Licensing Examination, which has a fixed cut score, and to suggest the most appropriate method.
Methods
Six radiological technology professors set standards for 250 items on the Korean Radiological Technologist Licensing Examination administered in December 2016 using the Angoff, Ebel, bookmark, and Hofstee methods.
Results
With a maximum percentile score of 100, the cut score for the examination was 71.27 using the Angoff method, 62.2 using the Ebel method, 64.49 using the bookmark method, and 62 using the Hofstee method. Based on the Hofstee method, an acceptable cut score for the examination would be between 52.83 and 70, but the cut score was 71.27 using the Angoff method.
Conclusion
The above results suggest that the best standard-setting method to determine the cut score would be a panel discussion with the modified Angoff or Ebel method, with verification of the rated results by the Hofstee method. Since no standard-setting method has yet been adopted for the Korean Radiological Technologist Licensing Examination, this study will be able to provide practical guidance for introducing a standard-setting process.

Citations

Citations to this article as recorded by

Setting standards for a diagnostic test of aviation English for student pilots
Maria Treadaway, John Read
Language Testing.2024;[Epub] CrossRef
The challenges inherent with anchor-based approaches to the interpretation of important change in clinical outcome assessments
Kathleen W. Wyrwich, Geoffrey R. Norman
Quality of Life Research.2023; 32(5): 1239. CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef
Comparison of the validity of bookmark and Angoff standard setting methods in medical performance tests
Majid Yousefi Afrashteh
BMC Medical Education.2021;[Epub] CrossRef
Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
Journal of Educational Evaluation for Health Professions.2021; 18: 25. CrossRef
Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination
Mi Kyoung Yim, Sujin Shin
Journal of Educational Evaluation for Health Professions.2020; 17: 14. CrossRef
Performance of the Ebel standard-setting method for the spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisting of multiple-choice questions
Jimmy Bourque, Haley Skinner, Jonathan Dupré, Maria Bacchus, Martha Ainslie, Irene W. Y. Ma, Gary Cole
Journal of Educational Evaluation for Health Professions.2020; 17: 12. CrossRef
Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination
Janghee Park, Mi Kyoung Yim, Na Jin Kim, Duck Sun Ahn, Young-Min Kim
Journal of Educational Evaluation for Health Professions.2020; 17: 28. CrossRef

Brief Report

Integrated clinical experience with concurrent problem-based learning is associated with improved clinical reasoning among physical therapy students in the United States: Brad Warren Willis, Anita Sethi Campbell, Stephen Paul Sayers, Kyle Gibson; J Educ Eval Health Prof. 2018;15:30. Published online December 25, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.30; Funded: Department of Anesthesiology, University of Missouri

19,290 View
398 Download
4 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Clinical reasoning (CR) is a key learning domain for physical therapy educators and a core skill for entry-level practitioners. Integrated clinical experience (ICE) and problem-based learning (PBL) have each been reported to improve interpersonal and social domains, while promoting knowledge acquisition and CR. Unfortunately, studies monitoring CR during ICE with concurrent PBL in physical therapy education are sparse. We hypothesized that ICE with concurrent PBL would be associated with improved self-reported CR in third-year student physical therapists (PTs) in the United States. The Self-Assessment of Clinical Reflection and Reasoning (SACRR) survey was administered to 42 student PTs at the beginning and end of their third and final year of didactic training. Between the pretest and posttest analyses, the participants completed faculty-led ICE and PBL coursework for 16 weeks. The overall SACRR score and 26 individual item scores were examined. The Wilcoxon rank-sum test and paired t-test were used, with statistical significance accepted at P< 0.05. Significant improvements were observed in the overall SACRR score (P< 0.001), including 6 of the 26 survey items centered around decision-making based on experience and evidence, as well as self-reflection and reasoning. ICE with PBL was associated with improved self-assessed CR and reflection in third-year student PTs in the United States. Monitoring the impact of curricular design on CR may improve educators’ ability to enhance cognitive and psychomotor skills, which underscores the importance of increasing the explicit use of theoretical frameworks and teaching techniques for coping with uncertainty as a way of enhancing entry-level training.

Citations

Citations to this article as recorded by

What is complexity of hospital-based physiotherapy from the perspective of physiotherapists themselves? A grounded theory study
Lieven de Zwart, Niek Koenders, Rudi Steenbruggen, Ria Nijhuis-van der Sanden, Thomas J Hoogeboom
BMJ Open.2023; 13(4): e069368. CrossRef
What methods are being used to teach clinical reasoning in physical therapy education and are they effective? A systematic review of literature
Melissa Cencetti, Laurie Brogan, Anthony F. Carusotto, Kristen Karnish
Physiotherapy Quarterly.2023; 31(3): 20. CrossRef
Renovación metodológica y evaluación como plataforma para el desarrollo de competencias de razonamiento profesional
Máximo Escobar Cabello, Iván Sanchez Soto
Investigación en Educación Médica.2020; 9(34): 76. CrossRef

Research articles

The relationship of examinees’ individual characteristics and perceived acceptability of smart device-based testing to test scores on the practice test of the Korea Emergency Medicine Technician Licensing Examination: Eun Young Lim, Mi Kyoung Yim, Sun Huh; J Educ Eval Health Prof. 2018;15:33. Published online December 27, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.33; Funded: Korea Health Personnel Licensing Examination Institute, Hallym University

18,997 View
233 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
Smart device-based testing (SBT) is being introduced into the Republic of Korea’s high-stakes examination system, starting with the Korean Emergency Medicine Technician Licensing Examination (KEMTLE) in December 2017. In order to minimize the effects of variation in examinees’ environment on test scores, this study aimed to identify any associations of variables related to examinees’ individual characteristics and their perceived acceptability of SBT with their SBT practice test scores.
Methods
Of the 569 candidate students who took the KEMTLE on September 12, 2015, 560 responded to a survey questionnaire on the acceptability of SBT after the examination. The questionnaire addressed 8 individual characteristics and contained 2 satisfaction, 9 convenience, and 9 preference items. A comparative analysis according to individual variables was performed. Furthermore, a generalized linear model (GLM) analysis was conducted to identify the effects of individual characteristics and perceived acceptability of SBT on test scores.
Results
Among those who preferred SBT over paper-and-pencil testing, test scores were higher for male participants (mean± standard deviation [SD], 4.36± 0.72) than for female participants (mean± SD, 4.21± 0.73). According to the GLM, no variables evaluated— including gender and experience with computer-based testing, SBT, or using a tablet PC—showed a statistically significant relationship with the total score, scores on multimedia items, or scores on text items.
Conclusion
Individual characteristics and perceived acceptability of SBT did not affect the SBT practice test scores of emergency medicine technician students in Korea. It should be possible to adopt SBT for the KEMTLE without interference from the variables examined in this study.

Citations

Citations to this article as recorded by

Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
Evaluation of Student Satisfaction with Ubiquitous-Based Tests in Women’s Health Nursing Course
Mi-Young An, Yun-Mi Kim
Healthcare.2021; 9(12): 1664. CrossRef

Evaluation of team-based learning in a doctor of physical therapy curriculum in the United States: Donald H. Lein, John D. Lowman, Christopher A. Eidson, Hon K. Yuen; J Educ Eval Health Prof. 2017;14:3. Published online February 28, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.3; Funded: University of Alabama at Birmingham

42,590 View
443 Download
7 Web of Science
6 Crossref

Abstract PDF

Purpose
The purpose of this retrospective study was to evaluate students’ academic outcomes after implementation of the team-based learning (TBL) approach in patient/client management courses in an entry-level doctor of physical therapy (DPT) curriculum.
Methods
The research design of this study involved comparing written and practical exam scores from DPT student cohorts taught with the traditional instructional methods (lecture-based) to those of students from subsequent cohorts taught using the TBL approach in two patient/client management courses: basic skills and cardiopulmonary. For this comparison, the exams used, the number of contact hours and labs, and the instructors who taught these courses remained the same during the transition between these two instructional methods (traditional vs. TBL). The average of all individual course exam scores was used for data analysis.
Results
In both courses, there were no meaningful differences in the mean exam scores among students across years of cohorts receiving the same instructional method, which allowed clustering students from different years of cohorts in each course receiving the same instructional method into one group. For both courses, the mean exam score was significantly higher in the TBL group than in the traditional instruction group: basic skills course (P<0.001) and cardiopulmonary course (P<0.001).
Conclusion
Student cohorts taught using the TBL approach academically outperformed those who received the traditional instructional method in both entry–level DPT patient/client management courses.

Citations

Citations to this article as recorded by

Comparison of the impact of team-based learning and lecture-based learning on nursing students' core competencies: A systematic review and meta-analysis
Xin Gao, Di Yan, Ya Zhang, Xiang Ruan, Tingyu Kang, Ruotong Wang, Qi Zheng, Siju Chen, Jinxia Zhai
Nurse Education in Practice.2024; 76: 103945. CrossRef
Team-Based Learning Among Health Care Professionals: A Systematic Review
Tilak Joshi, Pravash Budhathoki, Anurag Adhikari, Ayusha Poudel, Sumit Raut, Dhan B Shrestha
Cureus.2022;[Epub] CrossRef
Enhanced numeracy skills following team-based learning in United States pharmacy students: a longitudinal cohort study
Rob Edwin Carpenter, Leanne Coyne, Dave Silberman, Jody Kyoto Takemoto
Journal of Educational Evaluation for Health Professions.2022; 19: 29. CrossRef
Call for Consistency: the Need to Establish Gross Anatomy Learning Objectives for the Entry-Level Physical Therapist
Melissa A. Carroll, Mary Tracy-Bee, Alison McKenzie
Medical Science Educator.2021; 31(3): 1193. CrossRef
Collaborative student-faculty research to support PhD research education
Mary J. Dyck, Nancy L. Novotny, John Blakeman, Crystal Bricker, Ashley Farrow, Janet LoVerde, Sandra D. Nielsen, Brenda Johnson
Journal of Professional Nursing.2020; 36(3): 106. CrossRef
A flexible, group-based assessment strategy for Historically Black College and University pharmacy students
Munder Zagaar, Linh D. Nguyen, JaRyce Echols, Hanan Loubani
Currents in Pharmacy Teaching and Learning.2020; 12(9): 1129. CrossRef

Attitudes to proposed assessment of pharmacy skills in Korean pharmacist licensure examination: Joo Hee Kim, Ju-Yeun Lee, Young Sook Lee, Chul-Soon Yong, Nayoung Han, Hye Sun Gwak, Jungmi Oh, Byung Koo Lee, Sukhyang Lee; J Educ Eval Health Prof. 2017;14:6. Published online March 27, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.6; Funded: Korean National Health Personnel Licensure Examination Board

46,896 View
374 Download
2 Web of Science
3 Crossref

Abstract PDF

Purpose
The survey aimed to obtain opinions about a proposed implementation of pharmacy skills assessment in Korean pharmacist licensure examination (KPLE).
Methods
A 16-question survey was distributed electronically to 2,738 people including 570 pharmacy professors of 35 pharmacy schools, 550 preceptors from 865 practice sites and 1,618 students who graduated in 2015. The survey solicited responses concerning the adequacy of the current KPLE in assessing pharmacy knowledge/skills/attitudes, deficiencies of pharmacy skills testing in assessing the professional competencies necessary for pharmacists, plans for pharmacy skills tests in the current KPLE, and subject areas of pharmacy practice.
Results
A total of 466 surveys were returned. The current exam is not adequate for assessing skills and attitudes according to 42%–48% of respondents. Sixty percent felt that skills test is necessary to assess qualifications and professional competencies. Almost two-thirds of participants stated that testing should be implemented within 5 years. More than 60% agreed that candidates should be graduates and that written and skills test scores can be combined for pass-fail decisions. About 70% of respondents felt that the test should be less than 2 hours in duration. Over half of the respondents thought that the assessor should be a pharmacy faculty member with at least 5 years of clinical experience. Up to 70% stated that activities related to patient care were appropriate and practical for the scope of skills test.
Conclusion
Pharmacy skills assessment was supported by the majority of respondents.

Citations

Citations to this article as recorded by

A scoping review of the methods and processes used by regulatory bodies to determine pharmacists’ readiness for practice
Eimear Ni Sheachnasaigh, Cathal Cadogan, Judith Strawbridge, Laura J. Sahm, Cristin Ryan
Research in Social and Administrative Pharmacy.2022; 18(12): 4028. CrossRef
Development of a Platform to Align Education and Practice: Bridging Academia and the Profession in Portugal
Filipa Alves da Costa, Ana Paula Martins, Francisco Veiga, Isabel Ramalhinho, José Manuel Sousa Lobo, Luís Rodrigues, Luiza Granadeiro, Matilde Castro, Pedro Barata, Perpétua Gomes, Vítor Seabra, Maria Margarida Caramona
Pharmacy.2020; 8(1): 11. CrossRef
Selection of Tasks for Assessment of Pharmacy Clinical Performance in Korean Pharmacist Licensure Examination: Results of an Expert Survey
Nayoung Han, Ju-Yeun Lee, Hye Sun Gwak, Byung Koo Lee, Young Sook Lee, Sukhyang Lee, Chul-Soon Yong, Joo Hee Kim, Jung Mi Oh, v
Korean Journal of Clinical Pharmacy.2017; 27(3): 119. CrossRef

Brief report

Smart device-based testing for medical students in Korea: satisfaction, convenience, and advantages: Eun Young Lim, Mi Kyoung Yim, Sun Huh; J Educ Eval Health Prof. 2017;14:7. Published online April 24, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.7; Funded: National Health Personnel Licensing Examination Board of Korea

33,716 View
290 Download
7 Web of Science
9 Crossref

Abstract PDF

The aim of this study was to investigate respondents’ satisfaction with smart device-based testing (SBT), as well as its convenience and advantages, in order to improve its implementation. The survey was conducted among 108 junior medical students at Kyungpook National University School of Medicine, Korea, who took a practice licensing examination using SBT in September 2015. The survey contained 28 items scored using a 5-point Likert scale. The items were divided into the following three categories: satisfaction with SBT administration, convenience of SBT features, and advantages of SBT compared to paper-and-pencil testing or computer-based testing. The reliability of the survey was 0.95. Of the three categories, the convenience of the SBT features received the highest mean (M) score (M= 3.75, standard deviation [SD]= 0.69), while the category of satisfaction with SBT received the lowest (M= 3.13, SD= 1.07). No statistically significant differences across these categories with respect to sex, age, or experience were observed. These results indicate that SBT was practical and effective to take and to administer.

Citations

Citations to this article as recorded by

Development and application of a mobile-based multimedia nursing competency evaluation system for nursing students: A mixed-method randomized controlled study
Soyoung Jang, Eunyoung E. Suh
Nurse Education in Practice.2022; 64: 103458. CrossRef
Effects of School-Based Exercise Program on Obesity and Physical Fitness of Urban Youth: A Quasi-Experiment
Ji Hwan Song, Ho Hyun Song, Sukwon Kim
Healthcare.2021; 9(3): 358. CrossRef
Development, Application, and Effectiveness of a Smart Device-based Nursing Competency Evaluation Test
Soyoung Jang, Eunyoung E. Suh
CIN: Computers, Informatics, Nursing.2021; 39(11): 634. CrossRef
Evaluation of Student Satisfaction with Ubiquitous-Based Tests in Women’s Health Nursing Course
Mi-Young An, Yun-Mi Kim
Healthcare.2021; 9(12): 1664. CrossRef
How to Deal with the Concept of Authorship and the Approval of an Institutional Review Board When Writing and Editing Journal Articles
Sun Huh
Laboratory Medicine and Quality Assurance.2020; 42(2): 63. CrossRef
Evaluation of usefulness of smart device-based testing: a survey study of Korean medical students
Youngsup Christopher Lee, Oh Young Kwon, Ho Jin Hwang, Seok Hoon Ko
Korean Journal of Medical Education.2020; 32(3): 213. CrossRef
Presidential address: Preparing for permanent test centers and computerized adaptive testing
Chang Hwi Kim
Journal of Educational Evaluation for Health Professions.2018; 15: 1. CrossRef
Journal Metrics of Infection & Chemotherapy and Current Scholarly Journal Publication Issues
Sun Huh
Infection & Chemotherapy.2018; 50(3): 219. CrossRef
The relationship of examinees’ individual characteristics and perceived acceptability of smart device-based testing to test scores on the practice test of the Korea Emergency Medicine Technician Licensing Examination
Eun Young Lim, Mi Kyoung Yim, Sun Huh
Journal of Educational Evaluation for Health Professions.2018; 15: 33. CrossRef

Research article

The job competency of radiological technologists in Korea based on specialists opinion and questionnaire survey: Chang-Seon Lim, Yang-Sub Lee, Yong-Dae Lee, Hyun-Soo Kim, Gye-Hwan Jin, Seong-Youl Choi, Yera Hur; J Educ Eval Health Prof. 2017;14:9. Published online May 11, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.9; Funded: National Health Personnel Licensing Examination Board of Korea

35,591 View
314 Download
2 Web of Science
2 Crossref

Abstract PDF

Purpose
Although there are over 40,000 licensed radiological technologists (RTs) in Korea, job competency standards have yet to be defined. This study aims to clarify the job competency of Korean RTs. Methods: A task force team of 11 professional RTs were recruited in order to analyze the job competency of domestic and international RTs. A draft for the job competency of Korean RTs was prepared. A survey was then conducted sampling RTs and the attitudes of their competencies were recorded from May 21 to July 30, 2016. Results: We identified five modules of professionalism, patient management, health and safety, operation of equipment, and procedure management and 131 detailed job competencies for RTs in Korea. “Health and safety” had the highest average score and “professionalism” had the lowest average score for both job performance and importance. The content validity ratios for the 131 subcompetencies were mostly valid. Conclusion: Establishment of standard guidelines for RT job competency for multidisciplinary healthcare at medical institutions may be possible based on our results, which will help educators of RT training institutions to clarify their training and education.

Citations

Citations to this article as recorded by

Digital skills of therapeutic radiographers/radiation therapists – Document analysis for a European educational curriculum
B. Barbosa, I. Bravo, C. Oliveira, L. Antunes, J.G. Couto, S. McFadden, C. Hughes, P. McClure, A.G. Dias
Radiography.2022; 28(4): 955. CrossRef
Stakeholder Engagement in Competency Framework Development in Health Professions: A Systematic Review
Breanna Lepre, Claire Palermo, Kylie J. Mansfield, Eleanor J. Beck
Frontiers in Medicine.2021;[Epub] CrossRef

Opinion

Proposal for improving the system of physical therapy education and the Korean physical therapist licensing examination based on a comparison of the systems in World Confederation for Physical Therapy member countries: Min-Hyeok Kang, Tae-Hwan Lee, Sang-Min Cha, Jae-Seop Oh, Tae-Sik Lee, Tae-Young Oh, Suhn-Yeop Kim, Hyoung-Soo Lee, Gyu-Wan Lee, Ki-Song Kim; J Educ Eval Health Prof. 2017;14:10. Published online May 24, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.10; Funded: National Health Personnel Licensing Examination Board of Korea

50,407 View
290 Download
1 Web of Science
2 Crossref

PDF

Citations

Citations to this article as recorded by

Enhancing the Autonomy of Physical Therapy in Korea and Its Significance for the National Healthcare System: Facing the Challenges of a Super-aging Society
Ki-song Kim
Physical Therapy Korea.2023; 30(2): 87. CrossRef
Importance and utilization frequency of essential competencies of Korean physical therapists
Junghyun Choi, Taeyoung Oh, Jae Seop Oh, Wootaek Lim, Jeonhyeong Lee, Seul Ki Han, Yun Sang Park, Hyeok Gyu Kwon, Chang Sik Ahn
Journal of Educational Evaluation for Health Professions.2020; 17: 24. CrossRef

Research articles

Establishment of a dental license regulation authority is required in Korea: results of the Delphi technique: Jin-Woo Choi, Kack-Kyun Kim, Jihyun Lee, Dong-Ju Choi, Kyung-Nyun Kim; J Educ Eval Health Prof. 2017;14:11. Published online May 29, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.11; Funded: Korea Health Personnel Licensing Examination Institute

38,573 View
263 Download
2 Web of Science
2 Crossref

Abstract PDF

Purpose
In addition to dental education, a system for the evaluation and management of dental licensing and certification is required to meet the growing societal demand for more competent dentists. In this study, the Delphi technique was used to gather opinions from a variety of professionals on the problems of and remedies for the dental license management system in Korea.
Methods
Delphi surveys were conducted from April 2016 to October 2016 in South Korea. A variety of dental professionals were included and categorized into 3 groups according to their expertise as follows: the basic dentistry group, the clinical dentistry group, and the policy group. The Delphi technique was conducted in 3 rounds of e-mail surveys, each with different questions that probed with increasing depth on the dental license management system. In each successive round, the responses were categorized, scored on a Likert scale, and statistically analyzed.
Results
After categorizing the results of the first survey and ranking the results of the second survey using the Delphi technique, regulation by a licensing authority was found to be the most critical issue. This was followed by the license renewal system, continuing education, a tiered licensure system, improvement of foreign license approval, and utilization of retirees, in decreasing order of importance. The third Delphi survey showed a similar ranking, with regulation by a licensing authority being the major concern. Opinions regarding the dental license management system were provided as open-ended responses. The responses of the 3 groups showed statistically significant differences in the scores for the issue of regulation by a licensing authority. After re-grouping into the dentistry group and the policy group, the issue received a significantly higher score in the dentistry group.
Conclusion
The quality of dental treatment should be managed to protect patients and dental professionals. For this purpose, the establishment of an independent license regulation authority along with legislative changes is required.

Citations

Citations to this article as recorded by

O‐HEALTH‐EDU: A viewpoint into the current state of Oral Health Professional education in Europe: Part 1: Programme‐level data
Jonathan Dixon, James Field, Sibylle Vital, Maria van Harten, Valerie Roger‐Leroi, Julia Davies, Maria‐Cristina Manzanares‐Cespedes, Ilze Akota, Denis Murphy, Corrado Paganelli, Gabor Gerber, Barry Quinn, Stephanie Tubert‐Jeannin
European Journal of Dental Education.2024;[Epub] CrossRef
Design, delivery and effectiveness of health practitioner regulation systems: an integrative review
Kathleen Leslie, Ivy Lynn Bourgeault, Anne-Louise Carlton, Madhan Balasubramanian, Raha Mirshahi, Stephanie D. Short, Jenny Carè, Giorgio Cometto, Vivian Lin
Human Resources for Health.2023;[Epub] CrossRef

Junior doctors’ preparedness to prescribe, monitor, and treat patients with the antibiotic vancomycin in an Australian teaching hospital: Cameron J Phillips, Ross A McKinnon, Richard J Woodman, David L Gordon; J Educ Eval Health Prof. 2017;14:13. Published online June 7, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.13; Funded: National Health and Medical Research Council of Australia

39,196 View
287 Download
12 Web of Science
10 Crossref

Abstract PDF

Purpose
We aimed to assess the preparedness of junior doctors to use vancomycin, and to determine whether attending an educational session and being provided pocket guidelines were associated with self-reported confidence and objective knowledge.
Methods
This was a 2-component cross-sectional study. A 60-minute educational session was implemented and pocket guidelines were provided. Preparedness was evaluated by a self-reported confidence survey in the early and late stages of each training year, and by continuing medical education (CME) knowledge scores.
Results
Self-confidence was higher among those later in the training year (n=75) than in those earlier (n=120) in the year for all questions. In the late group, vancomycin education was associated with higher self-confidence regarding the frequency of therapeutic drug monitoring (P=0.02) and dose amendment (P=0.05); however, the confidence for initial monitoring was lower (P<0.05). Those with pocket guidelines were more confident treating patients with vancomycin (P<0.001), choosing initial (P=0.01) and maintenance doses (P<0.001), and knowing the monitoring frequency (P=0.03). The 85 respondents who completed the knowledge assessment scored a mean±standard deviation of 8.55±1.55 on 10 questions, and the interventions had no significant effect.
Conclusion
Attending an educational session and possessing pocket guidelines were associated with preparedness, as measured by higher self-reported confidence using vancomycin. High knowledge scores were attained following CME; however attending an educational session or possessing pocket guidelines did not significantly increase the knowledge scores. Our findings support providing educational sessions and pocket guidelines to increase self-confidence in prescribing vancomycin, yet also highlight the importance of evaluating content, format, and delivery when seeking to improve preparedness to use vancomycin through education.

Citations

Citations to this article as recorded by

Assessment and evaluation of prescribing competences: A systematic review and recommendations
John C. McLachlan, Asa Auta, Aderonke Ajiboye
British Journal of Clinical Pharmacology.2024;[Epub] CrossRef
Awareness, perception, and barriers of healthcare providers toward the revised consensus guideline for therapeutic monitoring of vancomycin
Sarah S. Alghanem, Abdullah Albassam, Nwayer Al-Rashidi, Zainab Bin Haidar
Saudi Pharmaceutical Journal.2023; 31(6): 955. CrossRef
Implementation of antimicrobial stewardship to reduce antimicrobial drug resistance
Mohammed S Razzaque
Expert Review of Anti-infective Therapy.2021; 19(5): 559. CrossRef
Are vancomycin dosing guidelines followed? A mixed methods study of vancomycin prescribing practices
Jane E. Carland, Sophie L. Stocker, Melissa T. Baysari, Crystal Li, Jacqueline Själin, Maria A. Moran, Sarah Tang, Indy Sandaradura, Tania Elhage, Timothy Gilbey, Kenneth M. Williams, Deborah J. E. Marriott, Richard O. Day
British Journal of Clinical Pharmacology.2021; 87(11): 4221. CrossRef
Evaluation of Eight-Item Vancomycin Prescribing Confidence Questionnaire Among Junior Doctors
Lu Liu, Zhehan Jiang, Ana Xie, Weimin Wang
Frontiers in Medicine.2021;[Epub] CrossRef
The knowledge and perceptions regarding antibiotic stewardship of the interns rotating at the Bloemfontein Academic Complex
Anke Archer, Marna Blom, Renette de Lange, Esther Jansen van Vuuren, Theunis E Kellerman, Samantha Potgieter, Gina Joubert
South African Family Practice.2021;[Epub] CrossRef
Ist die Fachrichtung assoziiert mit der Selbstsicherheit im Umgang mit rationaler Antibiotikaverordnung?
F. Schneider, C. M. Schulz, M. May, G. Schneider, M. Jacob, H. Mutlak, M. Pawlik, M. Zoller, M. Kretzschmar, C. Koch, M. G. Kees, M. Burger, S. Lebentrau, A. Novotny, M. Hübler, T. Koch, M. Heim
Der Anaesthesist.2020; 69(3): 162. CrossRef
The association of the anesthesiologist’s academic and educational status with self-confidence, self-rated knowledge and objective knowledge in rational antibiotic application
Frederick Schneider, Christian M. Schulz, Matthias May, Gerhard Schneider, Christian Ernst, Matthias Jacob, Kai Zacharowski, Thomas Hachenberg, Maren Schmidt, Moritz Kretzschmar, Bernhard Graf, Martin G. Kees, Michael Pawlik, Michael Sander, Christian Koc
BMC Research Notes.2020;[Epub] CrossRef
Rational application of antibiotics—The influence of anaesthetists’ gender on self‐confidence and knowledge
Markus Heim, Frederick Schneider, Haitham Mutlak, Christian M. Schulz, Thomas Hachenberg, Michael Zoller, Jens Soukup, Michael Pawlik, Julika Schön, Martin Kees, Matthias Hübler, Michael Sander, Christian Ernst, Matthias Jacob, Matthias May
Acta Anaesthesiologica Scandinavica.2019; 63(8): 1037. CrossRef
Sustained improvement in vancomycin dosing and monitoring post-implementation of guidelines: Results of a three-year follow-up after a multifaceted intervention in an Australian teaching hospital
Cameron J. Phillips, Ross A. McKinnon, Richard J. Woodman, David L. Gordon
Journal of Infection and Chemotherapy.2018; 24(2): 103. CrossRef

Evaluation of an undergraduate occupational health program in Iran based on alumni perceptions: a structural equation model: Semira Mehralizadeh, Alireza Dehdashti, Masoud Motalebi Kashani; J Educ Eval Health Prof. 2017;14:16. Published online July 26, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.16; Funded: Semnan University of Medical Sciences

31,733 View
311 Download
2 Web of Science
1 Crossref

Abstract PDF

Purpose
Evaluating educational programs can improve the quality of education. The present study evaluated the undergraduate occupational health program at the Semnan University of Medical Sciences in Semnan, Iran, with a focus on the associations between alumni perceptions of the learning environment and the outcomes of the occupational health program. Methods: A cross-sectional questionnaire survey was conducted among alumni of the undergraduate occupational health program. We asked alumni to rate their perceptions of the items using a 4-point Likert scale. The associations between alumni perceptions of the educational program and curriculum, faculty, institutional resources, and learning outcomes were modeled and described using structural equation modeling procedures. Results: A descriptive analysis of alumni perceptions indicated low evaluations for the administrative system, practical and research-based courses, and the number of faculty members. We found that a structural model of the evaluation variables of curriculum, faculty qualifications, and institutional resources significantly predicted undergraduate educational outcomes. The curriculum had direct and indirect effects on learning outcomes, mediated by faculty. Conclusion: The findings of our study highlight the usefulness of the structural equation modeling approach for examining links between variables related to the learning process and learning outcomes. Surveys of alumni can provide data for reassessing the learning environment in the light of the professional competencies needed for occupational health graduates.

Citations

Citations to this article as recorded by

Integrated-Based Curriculum of Pharmaceutical Dosage Forms (ICPDF): What Factors Affect the Learning Outcome Attainment?
Anis Yohana Chaerunisaa, Akhmad Habibi, Muhaimin Muhaimin, Mailizar Mailizar, Tommy Tanu Wijaya, Ahmad Samed Al-Adwan
International Journal of Environmental Research and Public Health.2023; 20(5): 4272. CrossRef

Review article

Overview and current management of computerized adaptive testing in licensing/certification examinations: Dong Gi Seo; J Educ Eval Health Prof. 2017;14:17. Published online July 26, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.17; Funded: Hallym Institute for Applied Psychology

38,835 View
371 Download
9 Web of Science
10 Crossref

Abstract PDF

Computerized adaptive testing (CAT) has been implemented in high-stakes examinations such as the National Council Licensure Examination-Registered Nurses in the United States since 1994. Subsequently, the National Registry of Emergency Medical Technicians in the United States adopted CAT for certifying emergency medical technicians in 2007. This was done with the goal of introducing the implementation of CAT for medical health licensing examinations. Most implementations of CAT are based on item response theory, which hypothesizes that both the examinee and items have their own characteristics that do not change. There are 5 steps for implementing CAT: first, determining whether the CAT approach is feasible for a given testing program; second, establishing an item bank; third, pretesting, calibrating, and linking item parameters via statistical analysis; fourth, determining the specification for the final CAT related to the 5 components of the CAT algorithm; and finally, deploying the final CAT after specifying all the necessary components. The 5 components of the CAT algorithm are as follows: item bank, starting item, item selection rule, scoring procedure, and termination criterion. CAT management includes content balancing, item analysis, item scoring, standard setting, practice analysis, and item bank updates. Remaining issues include the cost of constructing CAT platforms and deploying the computer technology required to build an item bank. In conclusion, in order to ensure more accurate estimations of examinees’ ability, CAT may be a good option for national licensing examinations. Measurement theory can support its implementation for high-stakes examinations.

Citations

Citations to this article as recorded by

Validation of the cognitive section of the Penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT-CCNB)
Akira Di Sandro, Tyler M. Moore, Eirini Zoupou, Kelly P. Kennedy, Katherine C. Lopez, Kosha Ruparel, Lucky J. Njokweni, Sage Rush, Tarlan Daryoush, Olivia Franco, Alesandra Gorgone, Andrew Savino, Paige Didier, Daniel H. Wolf, Monica E. Calkins, J. Cobb S
Brain and Cognition.2024; 174: 106117. CrossRef
The current utilization of the patient-reported outcome measurement information system (PROMIS) in isolated or combined total knee arthroplasty populations
Puneet Gupta, Natalia Czerwonka, Sohil S. Desai, Alirio J. deMeireles, David P. Trofa, Alexander L. Neuwirth
Knee Surgery & Related Research.2023;[Epub] CrossRef
Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study
Ioannis Tsaousis, Georgios D. Sideridis, Hannan M. AlGhamdi
Journal of Psychoeducational Assessment.2021; 39(8): 954. CrossRef
Accuracy and Efficiency of Web-based Assessment Platform (LIVECAT) for Computerized Adaptive Testing
Do-Gyeong Kim, Dong-Gi Seo
The Journal of Korean Institute of Information Technology.2020; 18(4): 77. CrossRef
Transformaciones en educación médica: innovaciones en la evaluación de los aprendizajes y avances tecnológicos (parte 2)
Veronica Luna de la Luz, Patricia González-Flores
Investigación en Educación Médica.2020; 9(34): 87. CrossRef
Introduction to the LIVECAT web-based computerized adaptive testing platform
Dong Gi Seo, Jeongwook Choi
Journal of Educational Evaluation for Health Professions.2020; 17: 27. CrossRef
Computerised adaptive testing accurately predicts CLEFT-Q scores by selecting fewer, more patient-focused questions
Conrad J. Harrison, Daan Geerards, Maarten J. Ottenhof, Anne F. Klassen, Karen W.Y. Wong Riff, Marc C. Swan, Andrea L. Pusic, Chris J. Sidey-Gibbons
Journal of Plastic, Reconstructive & Aesthetic Surgery.2019; 72(11): 1819. CrossRef
Presidential address: Preparing for permanent test centers and computerized adaptive testing
Chang Hwi Kim
Journal of Educational Evaluation for Health Professions.2018; 15: 1. CrossRef
Updates from 2018: Being indexed in Embase, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreci
Sun Huh
Journal of Educational Evaluation for Health Professions.2018; 15: 36. CrossRef
Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination
Dong Gi Seo, Myeong Gi Kim, Na Hui Kim, Hye Sook Shin, Hyun Jung Kim
Journal of Educational Evaluation for Health Professions.2018; 15: 26. CrossRef

Research article

Sequential Objective Structured Clinical Examination based on item response theory in Iran: Sara Mortaz Hejri, Mohammad Jalili; J Educ Eval Health Prof. 2017;14:19. Published online September 8, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.19; Funded: Tehran University of Medical Sciences

29,344 View
264 Download
2 Web of Science
1 Crossref

Abstract PDF

Purpose
In a sequential objective structured clinical examination (OSCE), all students initially take a short screening OSCE. Examinees who pass are excused from further testing, but an additional OSCE is administered to the remaining examinees. Previous investigations of sequential OSCE were based on classical test theory. We aimed to design and evaluate screening OSCEs based on item response theory (IRT).
Methods
We carried out a retrospective observational study. At each station of a 10-station OSCE, the students’ performance was graded on a Likert-type scale. Since the data were polytomous, the difficulty parameters, discrimination parameters, and students’ ability were calculated using a graded response model. To design several screening OSCEs, we identified the 5 most difficult stations and the 5 most discriminative ones. For each test, 5, 4, or 3 stations were selected. Normal and stringent cut-scores were defined for each test. We compared the results of each of the 12 screening OSCEs to the main OSCE and calculated the positive and negative predictive values (PPV and NPV), as well as the exam cost.
Results
A total of 253 students (95.1%) passed the main OSCE, while 72.6% to 94.4% of examinees passed the screening tests. The PPV values ranged from 0.98 to 1.00, and the NPV values ranged from 0.18 to 0.59. Two tests effectively predicted the results of the main exam, resulting in financial savings of 34% to 40%.
Conclusion
If stations with the highest IRT-based discrimination values and stringent cut-scores are utilized in the screening test, sequential OSCE can be an efficient and convenient way to conduct an OSCE.

Citations

Citations to this article as recorded by

Utility of eye-tracking technology for preparing medical students in Spain for the summative objective structured clinical examination
Francisco Sánchez-Ferrer, J.M. Ramos-Rincón, M.D. Grima-Murcia, María Luisa Sánchez-Ferrer, Francisco Sánchez-del Campo, Antonio F. Compañ-Rosique, Eduardo Fernández-Jover
Journal of Educational Evaluation for Health Professions.2017; 14: 27. CrossRef

Technical report

Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination: In Sook Park, Yeon Ok Suh, Hae Sook Park, So Young Kang, Kwang Sung Kim, Gyung Hee Kim, Yeon-Hee Choi, Hyun-Ju Kim; J Educ Eval Health Prof. 2017;14:20. Published online September 11, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.20; Funded: National Health Personnel Licensing Examination Board of Korea

31,577 View
262 Download
3 Web of Science
4 Crossref

Abstract PDF

Purpose
The purpose of this study was to improve the quality of items on the Korean Nursing Licensing Examination by developing and evaluating case-based items that reflect integrated nursing knowledge.
Methods
We conducted a cross-sectional observational study to develop new case-based items. The methods for developing test items included expert workshops, brainstorming, and verification of content validity. After a mock examination of undergraduate nursing students using the newly developed case-based items, we evaluated the appropriateness of the items through classical test theory and item response theory.
Results
A total of 50 case-based items were developed for the mock examination, and content validity was evaluated. The question items integrated 34 discrete elements of integrated nursing knowledge. The mock examination was taken by 741 baccalaureate students in their fourth year of study at 13 universities. Their average score on the mock examination was 57.4, and the examination showed a reliability of 0.40. According to classical test theory, the average level of item difficulty of the items was 57.4% (80%–100% for 12 items; 60%–80% for 13 items; and less than 60% for 25 items). The mean discrimination index was 0.19, and was above 0.30 for 11 items and 0.20 to 0.29 for 15 items. According to item response theory, the item discrimination parameter (in the logistic model) was none for 10 items (0.00), very low for 20 items (0.01 to 0.34), low for 12 items (0.35 to 0.64), moderate for 6 items (0.65 to 1.34), high for 1 item (1.35 to 1.69), and very high for 1 item (above 1.70). The item difficulty was very easy for 24 items (below −2.0), easy for 8 items (−2.0 to −0.5), medium for 6 items (−0.5 to 0.5), hard for 3 items (0.5 to 2.0), and very hard for 9 items (2.0 or above). The goodness-of-fit test in terms of the 2-parameter item response model between the range of 2.0 to 0.5 revealed that 12 items had an ideal correct answer rate.
Conclusion
We surmised that the low reliability of the mock examination was influenced by the timing of the test for the examinees and the inappropriate difficulty of the items. Our study suggested a methodology for the development of future case-based items for the Korean Nursing Licensing Examination.

Citations

Citations to this article as recorded by

Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study
Kyunghee Kim, So Young Kang, Younhee Kang, Youngran Kweon, Hyunjung Kim, Youngshin Song, Juyeon Cho, Mi-Young Choi, Hyun Su Lee
Journal of Educational Evaluation for Health Professions.2023; 20: 18. CrossRef
Automating assessment of Design exams: A case study of novelty evaluation
Nandita Bhanja Chaudhuri, Debayan Dhar, Pradeep G. Yammiyavar
Expert Systems with Applications.2021; : 116108. CrossRef
Levels, antecedents, and consequences of critical thinking among clinical nurses: a quantitative literature review
Yongmi Lee, Younjae Oh
Journal of Educational Evaluation for Health Professions.2020; 17: 26. CrossRef
Factors Influencing the Success of the National Nursing Competency Examination taken by the Nursing Diploma Students in Yogyakarta
Yulia Wardani
Jurnal Ners.2020; 14(2): 172. CrossRef

First
Prev
Page of 9
Next
Last

Funded articles