Skip Navigation
Skip to contents

JEEHP : Journal of Educational Evaluation for Health Professions

OPEN ACCESS
SEARCH
Search

Article category

Page Path
HOME > Article category > Article category
441 Article category
Filter
Filter
Article category
Keywords
Publication year
Authors
Funded articles
Research article
An innovative resident-driven mortality case review curriculum to teach and drive system-based practice improvements in the United States  
Nila S. Radhakrishnan, Margaret C. Lo, Rohit Bishnoi, Subhankar Samal, Robert Leverence, Eric Rosenberg, Zareen Zaidi
J Educ Eval Health Prof. 2018;15:31.   Published online December 26, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.31
  • 18,171 View
  • 254 Download
AbstractAbstract PDFSupplementary Material
Purpose
Traditionally, the morbidity and mortality conference (M&MC) is a forum where possible medical errors are discussed. Although M&MCs can facilitate identification of opportunities for systemwide improvements, few studies have described their use for this purpose, particularly in residency training programs. This paper describes the use of M&MC case review as a quality improvement activity that teaches system-based practice and can engage residents in improving systems of care.
Methods
Internal medicine residents at a tertiary care academic medical center reviewed 347 consecutive mortalities from March 2014 to September 2017. The residents used case review worksheets to categorize and track causes of mortality, and then debriefed with a faculty member. Selected cases were then presented at a larger interdepartmental meeting and action items were implemented. Descriptive statistics and thematic analysis were used to analyze the results.
Results
The residents identified a possible diagnostic mismatch at some point from admission to death in 54.5% of cases (n= 189) and a possible need for improved management in 48.0% of cases. Three possible management failure themes were identified, including failure to plan, failure to communicate, and failure to rescue, which accounted for 21.9%, 10.7 %, and 10.1% of cases, respectively. Following these reviews, quality improvement initiatives proposed by residents led to system-based changes.
Conclusion
A resident-driven mortality review curriculum can lead to improvements in systems of care. This novel type of curriculum can be used to teach system-based practice. The recruitment of teaching faculty with expertise in quality improvement and mortality case analyses is essential for such a project.
Brief Report
Integrated clinical experience with concurrent problem-based learning is associated with improved clinical reasoning among physical therapy students in the United States  
Brad Warren Willis, Anita Sethi Campbell, Stephen Paul Sayers, Kyle Gibson
J Educ Eval Health Prof. 2018;15:30.   Published online December 25, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.30
  • 18,052 View
  • 368 Download
  • 1 Citations
AbstractAbstract PDFSupplementary Material
Clinical reasoning (CR) is a key learning domain for physical therapy educators and a core skill for entry-level practitioners. Integrated clinical experience (ICE) and problem-based learning (PBL) have each been reported to improve interpersonal and social domains, while promoting knowledge acquisition and CR. Unfortunately, studies monitoring CR during ICE with concurrent PBL in physical therapy education are sparse. We hypothesized that ICE with concurrent PBL would be associated with improved self-reported CR in third-year student physical therapists (PTs) in the United States. The Self-Assessment of Clinical Reflection and Reasoning (SACRR) survey was administered to 42 student PTs at the beginning and end of their third and final year of didactic training. Between the pretest and posttest analyses, the participants completed faculty-led ICE and PBL coursework for 16 weeks. The overall SACRR score and 26 individual item scores were examined. The Wilcoxon rank-sum test and paired t-test were used, with statistical significance accepted at P< 0.05. Significant improvements were observed in the overall SACRR score (P< 0.001), including 6 of the 26 survey items centered around decision-making based on experience and evidence, as well as self-reflection and reasoning. ICE with PBL was associated with improved self-assessed CR and reflection in third-year student PTs in the United States. Monitoring the impact of curricular design on CR may improve educators’ ability to enhance cognitive and psychomotor skills, which underscores the importance of increasing the explicit use of theoretical frameworks and teaching techniques for coping with uncertainty as a way of enhancing entry-level training.

Citations

Citations to this article as recorded by  
  • Renovación metodológica y evaluación como plataforma para el desarrollo de competencias de razonamiento profesional
    Máximo Escobar Cabello, Iván Sánchez Soto
    Investigación en Educación Médica.2020; (34): 76.     CrossRef
Research articles
The effects of an empathy role-playing program for operating room nursing students in Iran  
Negin Larti, Elaheh Ashouri, Akram Aarabi
J Educ Eval Health Prof. 2018;15:29.   Published online December 13, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.29
  • 23,610 View
  • 418 Download
  • 16 Citations
AbstractAbstract PDFSupplementary Material
Purpose
The purpose of this study was to investigate the effects of a role-playing training program conducted among operating room nursing students on empathetic communication with patients through measurements of empathy scores.
Methods
This study was carried out among 77 operating room nursing students from the first to the fourth years studying at the School of Nursing and Midwifery of Isfahan University of Medical Sciences in the academic year 2017–2018. The intervention administered to the experimental group included a 12-hour training program on expressing empathy to patients that incorporated roleplaying. The Jefferson Scale of Empathy-Health Profession Student version was completed by the participants before, immediately after, and 1 month after the intervention. A comparative analysis of these 3 time points was conducted.
Results
No significant difference was found in the total pre-intervention mean empathy scores before the intervention between the control group and the experimental group (P= 0.50). However, the total mean empathy scores in the experimental group immediately after and 1 month after the intervention were higher than those in the control group (P< 0.001).
Conclusion
Empathy training through a role-playing technique was effective at improving the empathy scores of operating room nursing students, and this finding also underscores the fact that empathy can be promoted by education. Changing the educational curriculum of operating room nursing students is suggested in order to familiarize them with the concept of empathy in the operating room.

Citations

Citations to this article as recorded by  
  • Responding to crises: rewilding accounting education for the Anthropocene
    Lisa Powell, Nicholas McGuigan
    Meditari Accountancy Research.2022;[Epub]     CrossRef
  • Operating room nurses' experience about patient cares for laparotomy surgeries: A phenomenological study
    Maryam Bastami, Behzad Imani, Moghadari Koosha
    Journal of Family Medicine and Primary Care.2022; 11(4): 1282.     CrossRef
  • Analysis of the Effect of Holistic Nursing in the Operating Room Based on PDCA and Evidence-Based Nursing in the Otorhinolaryngology Operating Room: Based on a Retrospective Case-Control Study
    Leina Bian, Jianhua Li, Wang Li, Xiaoyan Hu, Ming Dai, Yuvaraja Teekaraman
    Contrast Media & Molecular Imaging.2022; 2022: 1.     CrossRef
  • The Role of Establishing Neurosurgical Specialist Nurse Working Group in the Recovery and Prevention of Negative Psychological Emotion after Meningioma Surgery
    Zhen Luo, Qiaoyu Yang, Min Tang, Chaofeng Fan, Yuvaraja Teekaraman
    Contrast Media & Molecular Imaging.2022; 2022: 1.     CrossRef
  • Use of Alternative Methodologies in Veterinary Medicine Learning and Acceptance of Students
    Pablo-Jesús Marín-García, Lola Llobat
    Education Sciences.2022; 12(7): 476.     CrossRef
  • First-year nursing students’ initial contact with the clinical learning environment: impacts on their empathy levels and perceptions of professional identity
    Qinghua Wang, Xiaohong Cao, Tianjiao Du
    BMC Nursing.2022;[Epub]     CrossRef
  • Effectiveness of simulation-based interventions at improving empathy among healthcare students: A systematic review and meta-analysis
    Joelle Yan Xin Chua, Emily Ang, Siew Tiang Lydia Lau, Shefaly Shorey
    Nurse Education Today.2021; 104: 105000.     CrossRef
  • Experience in organizing the training of doctors to provide emergency care in a simulation center
    S. I. Chistyakov, A. Yu. Smorkalov, O. V. Gorokh, A. A. Pevnev
    Virtual Technologies in Medicine.2021; (1): 26.     CrossRef
  • Compassion toward others and self-compassion predict mental and physical well-being: a 5-year longitudinal study of 1090 community-dwelling adults across the lifespan
    Ellen E. Lee, Tushara Govind, Marina Ramsey, Tsung Chin Wu, Rebecca Daly, Jinyuan Liu, Xin M. Tu, Martin P. Paulus, Michael L. Thomas, Dilip V. Jeste
    Translational Psychiatry.2021;[Epub]     CrossRef
  • The impact of communication education provided with creative drama method on midwifery undergraduate
    Arzu Kul Uçtu, Nazan Karahan
    European Journal of Midwifery.2021; 5(September): 1.     CrossRef
  • Decline in empathy levels and sex differences in medical students from the Caribbean
    Eugenia Smirna González-Díaz, María Guadalupe Silva-Vetri, Patricia Díaz-Calzadilla, Aracelis Calzadilla-Núñez, Alejandro Reyes-Reyes, Víctor Patricio Díaz-Narváez
    Revista de la Facultad de Medicina.2021; 69(4): e86227.     CrossRef
  • Outcomes of Randomized Clinical Trials of Interventions to Enhance Social, Emotional, and Spiritual Components of Wisdom
    Ellen E. Lee, Katherine J. Bangen, Julie A. Avanzino, BaiChun Hou, Marina Ramsey, Graham Eglit, Jinyuan Liu, Xin M. Tu, Martin Paulus, Dilip V. Jeste
    JAMA Psychiatry.2020; 77(9): 925.     CrossRef
  • Family-centered practice in nursing education
    Tahani Casameni Montiel, Andrea Ancer Leal, Nicole Baltazar
    Nursing.2020; 50(7): 61.     CrossRef
  • Comparing Empathy Levels in Doctor of Pharmacy Students and Exemplary Pharmacist Preceptors
    Charlene R. Williams, Philip T. Rodgers, Jacqueline E. McLaughlin, Thomas A. Angelo, Greene Shepherd
    American Journal of Pharmaceutical Education.2020; 84(3): 7497.     CrossRef
  • Assessing the effect of empathy-enhancing interventions in health education and training: a systematic review of randomised controlled trials
    Rachel Winter, Eyad Issa, Nia Roberts, Robert I Norman, Jeremy Howick
    BMJ Open.2020; 10(9): e036471.     CrossRef
  • Effects of an Educational Film About Body Donors on Students' Empathy and Anxiety Levels in Gross Anatomy
    Julia Iaconisi, Friederike Hasselblatt, Benjamin Mayer, Michael Schoen, Tobias Maria Böckers, Anja Böckers
    Anatomical Sciences Education.2019; 12(4): 386.     CrossRef
Comparison of the level of cognitive processing between case-based items and non-case-based items on the Interuniversity Progress Test of Medicine in the Netherlands  
Dario Cecilio-Fernandes, Wouter Kerdijk, Andreas Johannes Bremers, Wytze Aalders, René Anton Tio
J Educ Eval Health Prof. 2018;15:28.   Published online December 12, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.28
  • 17,649 View
  • 185 Download
  • 3 Citations
AbstractAbstract PDFSupplementary Material
Purpose
It is assumed that case-based questions require higher-order cognitive processing, whereas questions that are not case-based require lower-order cognitive processing. In this study, we investigated to what extent case-based and non-case-based questions followed this assumption based on Bloom’s taxonomy.
Methods
In this article, 4,800 questions from the Interuniversity Progress Test of Medicine were classified based on whether they were case-based and on the level of Bloom’s taxonomy that they involved. Lower-order questions require students to remember or/and have a basic understanding of knowledge. Higher-order questions require students to apply, analyze, or/and evaluate. The phi coefficient was calculated to investigate the relationship between whether questions were case-based and the required level of cognitive processing.
Results
Our results demonstrated that 98.1% of case-based questions required higher-level cognitive processing. Of the non-case-based questions, 33.7% required higher-level cognitive processing. The phi coefficient demonstrated a significant, but moderate correlation between the presence of a patient case in a question and its required level of cognitive processing (phi coefficient= 0.55, P< 0.001).
Conclusion
Medical instructors should be aware of the association between item format (case-based versus non-case-based) and the cognitive processes they elicit in order to meet the desired balance in a test, taking the learning objectives and the test difficulty into account.

Citations

Citations to this article as recorded by  
  • What faculty write versus what students see? Perspectives on multiple-choice questions using Bloom’s taxonomy
    Seetha U. Monrad, Nikki L. Bibler Zaidi, Karri L. Grob, Joshua B. Kurtz, Andrew W. Tai, Michael Hortsch, Larry D. Gruppen, Sally A. Santen
    Medical Teacher.2021; 43(5): 575.     CrossRef
  • Aménagement du concours de première année commune aux études de santé (PACES) : entre justice sociale et éthique confraternelle en devenir ?
    R. Pougnet, L. Pougnet
    Éthique & Santé.2020; 17(4): 250.     CrossRef
  • Knowledge of dental faculty in gulf cooperation council states of multiple-choice questions’ item writing flaws
    Mawlood Kowash, Hazza Alhobeira, Iyad Hussein, Manal Al Halabi, Saif Khan
    Medical Education Online.2020;[Epub]     CrossRef
Corrigendum
Funding information of the article entitled “Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination”
Dong Gi Seo, Jeongwook Choi
J Educ Eval Health Prof. 2018;15:27.   Published online December 4, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.27    [Epub ahead of print]
Corrects: J Educ Eval Health Prof 2018;15(0):14
  • 16,310 View
  • 164 Download
PDF
Research article
Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination  
Dong Gi Seo, Myeong Gi Kim, Na Hui Kim, Hye Sook Shin, Hyun Jung Kim
J Educ Eval Health Prof. 2018;15:26.   Published online October 18, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.26
  • 19,401 View
  • 272 Download
  • 2 Citations
AbstractAbstract PDFSupplementary Material
Purpose
This study aimed to identify the best way of developing equivalent item sets and to propose a stable and effective management plan for periodical licensing examinations.
Methods
Five pre-equated item sets were developed based on the predicted correct answer rate of each item using linear programming. These pre-equated item sets were compared to the ones that were developed with a random item selection method based on the actual correct answer rate (ACAR) and difficulty from item response theory (IRT). The results with and without common items were also compared in the same way. ACAR and the IRT difficulty were used to determine whether there was a significant difference between the pre-equating conditions.
Results
There was a statistically significant difference in IRT difficulty among the results from different pre-equated conditions. The predicted correct answer rate was divided using 2 or 3 difficulty categories, and the ACAR and IRT difficulty parameters of the 5 item sets were equally constructed. Comparing the item set conditions with and without common items, including common items did not make a significant contribution to the equating of the 5 item sets.
Conclusion
This study suggested that the linear programming method is applicable to construct equated-item sets that reflect each content area. The suggested best method to construct equated item sets is to divide the predicted correct answer rate using 2 or 3 difficulty categories, regardless of common items. If pre-equated item sets are required to construct a test based on the actual data, several methods should be considered by simulation studies to determine which is optimal before administering a real test.

Citations

Citations to this article as recorded by  
  • Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
    Sun Huh
    Journal of Educational Evaluation for Health Professions.2022; 19: 2.     CrossRef
  • Reading Comprehension Tests for Children: Test Equating and Specific Age-Interval Reports
    Patrícia Silva Lúcio, Fausto Coutinho Lourenço, Hugo Cogo-Moreira, Deborah Bandalos, Carolina Alves Ferreira de Carvalho, Adriana de Souza Batista Kida, Clara Regina Brandão de Ávila
    Frontiers in Psychology.2021;[Epub]     CrossRef
Brief report
Benefits of focus group discussions beyond online surveys in course evaluations by medical students in the United States: a qualitative study  
Katharina Brandl, Soniya V. Rabadia, Alexander Chang, Jess Mandel
J Educ Eval Health Prof. 2018;15:25.   Published online October 16, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.25
  • 20,873 View
  • 301 Download
  • 2 Citations
AbstractAbstract PDFSupplementary Material
In addition to online questionnaires, many medical schools use supplemental evaluation tools such as focus groups to evaluate their courses. Although some benefits of using focus groups in program evaluation have been described, it is unknown whether these inperson data collection methods provide sufficient additional information beyond online evaluations to justify them. In this study, we analyze recommendations gathered from student evaluation team (SET) focus group meetings and analyzed whether these items were captured in open-ended comments within the online evaluations. Our results indicate that online evaluations captured only 49% of the recommendations identified via SETs. Surveys to course directors identified that 74% of the recommendations exclusively identified via the SETs were implemented within their courses. Our results indicate that SET meetings provided information not easily captured in online evaluations and that these recommendations resulted in actual course changes.

Citations

Citations to this article as recorded by  
  • Student evaluations of teaching and the development of a comprehensive measure of teaching effectiveness for medical schools
    Constantina Constantinou, Marjo Wijnen-Meijer
    BMC Medical Education.2022;[Epub]     CrossRef
  • Mentoring as a transformative experience
    Wendy A. Hall, Sarah Liva
    Mentoring & Tutoring: Partnership in Learning.2021; 29(1): 6.     CrossRef
Case report
Dental students’ learning attitudes and perceptions of YouTube as a lecture video hosting platform in a flipped classroom in Korea  
Chang Wan Seo, A Ra Cho, Jung Chul Park, Hag Yeon Cho, Sun Kim
J Educ Eval Health Prof. 2018;15:24.   Published online October 11, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.24
  • 26,377 View
  • 360 Download
  • 10 Citations
AbstractAbstract PDFSupplementary Material
Purpose
The aim of this study was to confirm the applicability of YouTube as a delivery platform of lecture videos for dental students and to assess their learning attitudes towards the flipped classroom model.
Methods
Learning experiences after using the YouTube platform to deliver preliminary video lectures in a flipped classroom were assessed by 69 second-year students (52 males, 17 females) at Dankook University College of Dentistry, Korea, who attended periodontology lectures during 2 consecutive semesters of the 2016 academic year. The instructor uploaded the lecture videos to YouTube before each class. At the end of the second semester, the students were surveyed using a questionnaire devised by the authors.
Results
Of the students, 53 (76.8%) always watched the lecture before the class, 48 (69.6%) used their smartphones, and 66 (95.7%) stated that they watched the lectures at home. The majority of the students replied that the video lectures were easier to understand than face to face lectures (82.6%) and that they would like to view the videos again after graduation (73.9%).
Conclusion
Our results indicate that YouTube is an applicable platform to deliver video lectures and to expose students to increased learning opportunities.

Citations

Citations to this article as recorded by  
  • Learning of paediatric dentistry with the flipped classroom model
    Nuria E. Gallardo, Antonia M. Caleya, Maria Esperanza Sánchez, Gonzalo Feijóo
    European Journal of Dental Education.2022; 26(2): 302.     CrossRef
  • Effects of Video Length on a Flipped English Classroom
    Zhonggen Yu, Mingle Gao
    SAGE Open.2022; 12(1): 215824402110684.     CrossRef
  • An Evaluation of the Usefulness of YouTube® Videos on Crown Preparation
    Syed Rashid Habib, Aleshba Saba Khan, Mohsin Ali, Essam Abdulla Abutheraa, Ahmad khaled alkhrayef, Faisal Jibrin Aljibrin, Nawaf Saad Almutairi, Ammar A. Siddiqui
    BioMed Research International.2022; 2022: 1.     CrossRef
  • Diş Hekimliği Eğitiminde Öğrencilerin Uzaktan Eğitim ve E-Öğrenme Algıları: Zorluklar ve Fırsatlar
    Ayşe TORAMAN, Ebru SAĞLAM, Serhat KÖSEOĞLU
    JOURNAL OF BIOTECHNOLOGY AND STRATEGIC HEALTH RESEARCH.2022;[Epub]     CrossRef
  • Social media as a learning tool: Dental students’ perspectives
    Mona T. Rajeh, Shahinaz N. Sembawa, Afnan A. Nassar, Seba A. Al Hebshi, Khalid T. Aboalshamat, Mohammed K. Badri
    Journal of Dental Education.2021; 85(4): 513.     CrossRef
  • Social Media Usage among Dental Undergraduate Students—A Comparative Study
    Eswara Uma, Pentti Nieminen, Shani Ann Mani, Jacob John, Emilia Haapanen, Marja-Liisa Laitala, Olli-Pekka Lappalainen, Eby Varghase, Ankita Arora, Kanwardeep Kaur
    Healthcare.2021; 9(11): 1408.     CrossRef
  • Does forced-shift to online learning affect university brand image in South Korea? Role of perceived harm and international students’ learning engagement
    Umer Zaman, Murat Aktan, Hasnan Baber, Shahid Nawaz
    Journal of Marketing for Higher Education.2021; : 1.     CrossRef
  • Flipped Classroom Experiences in Clinical Dentistry – A Strategic Mini-Review
    Abdullah Aljabr
    The Open Dentistry Journal.2021; 15(1): 717.     CrossRef
  • Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
    Sun Kim, A Ra Cho, Chul Woon Chung
    Journal of Educational Evaluation for Health Professions.2021; 18: 28.     CrossRef
  • Attitudes toward Social Media among Practicing Dentists and Dental Students in Clinical Years in Saudi Arabia
    Khalid Aboalshamat, Sharifah Alkiyadi, Sarah Alsaleh, Rana Reda, Sharifa Alkhaldi, Arwa Badeeb, Najwa Gabb
    The Open Dentistry Journal.2019; 13(1): 143.     CrossRef
Research articles
Agreement between 2 raters’ evaluations of a traditional prosthodontic practical exam integrated with directly observed procedural skills in Egypt  
Ahmed Khalifa Khalifa, Salah Hegazy
J Educ Eval Health Prof. 2018;15:23.   Published online September 27, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.23
  • 22,444 View
  • 194 Download
  • 1 Citations
AbstractAbstract PDFSupplementary Material
Purpose
This study aimed to assess the agreement between 2 raters in evaluations of students on a prosthodontic clinical practical exam integrated with directly observed procedural skills (DOPS).
Methods
A sample of 76 students was monitored by 2 raters to evaluate the process and the final registered maxillomandibular relation for a completely edentulous patient at Mansoura Dental School, Egypt on a practical exam of bachelor’s students from May 15 to June 28, 2017. Each registered relation was evaluated from a total of 60 marks subdivided into 3 score categories: occlusal plane orientation (OPO), vertical dimension registration (VDR), and centric relation registration (CRR). The marks for each category included an assessment of DOPS. The marks of OPO and VDR for both raters were compared using the graph method to measure reliability through Bland and Altman analysis. The reliability of the CRR marks was evaluated by the Krippendorff alpha ratio.
Results
The results revealed highly similar marks between raters for OPO (mean= 18.1 for both raters), with close limits of agreement (0.73 and −0.78). For VDR, the mean marks were close (mean= 17.4 and 17.1 for examiners 1 and 2, respectively), with close limits of agreement (2.7 and −2.2). There was a strong correlation (Krippendorff alpha ratio, 0.92; 95% confidence interval, 0.79– 0.99) between the raters in the evaluation of CRR.
Conclusion
The 2 raters’ evaluation of a clinical traditional practical exam integrated with DOPS showed no significant differences in the evaluations of candidates at the end of a clinical prosthodontic course. The limits of agreement between raters could be optimized by excluding subjective evaluation parameters and complicated cases from the examination procedure.

Citations

Citations to this article as recorded by  
  • In‐person and virtual assessment of oral radiology skills and competences by the Objective Structured Clinical Examination
    Fernanda R. Porto, Mateus A. Ribeiro, Luciano A. Ferreira, Rodrigo G. Oliveira, Karina L. Devito
    Journal of Dental Education.2022;[Epub]     CrossRef
Learning through multiple lenses: analysis of self, peer, nearpeer, and faculty assessments of a clinical history-taking task in Australia  
Kylie Fitzgerald, Brett Vaughan
J Educ Eval Health Prof. 2018;15:22.   Published online September 18, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.22
  • 21,848 View
  • 267 Download
  • 1 Citations
AbstractAbstract PDFSupplementary Material
Purpose
Peer assessment provides a framework for developing expected skills and receiving feedback appropriate to the learner’s level. Near-peer (NP) assessment may elevate expectations and motivate learning. Feedback from peers and NPs may be a sustainable way to enhance student assessment feedback. This study analysed relationships among self, peer, NP, and faculty marking of an assessment and students’ attitudes towards marking by those various groups.
Methods
A cross-sectional study design was used. Year 2 osteopathy students (n= 86) were invited to perform self and peer assessments of a clinical history-taking and communication skills assessment. NPs and faculty also marked the assessment. Year 2 students also completed a questionnaire on their attitudes to peer/NP marking. Descriptive statistics and the Spearman rho coefficient were used to evaluate relationships across marker groups.
Results
Year 2 students (n= 9), NPs (n= 3), and faculty (n= 5) were recruited. Correlations between self and peer (r= 0.38) and self and faculty (r= 0.43) marks were moderate. A weak correlation was observed between self and NP marks (r= 0.25). Perceptions of peer and NP marking varied, with over half of the cohort suggesting that peer or NP assessments should not contribute to their grade.
Conclusion
Framing peer and NP assessment as another feedback source may offer a sustainable method for enhancing feedback without overloading faculty resources. Multiple sources of feedback may assist in developing assessment literacy and calibrating students’ self-assessment capability. The small number of students recruited suggests some acceptability of peer and NP assessment; however, further work is required to increase its acceptability.

Citations

Citations to this article as recorded by  
  • History-taking level and its influencing factors among nursing undergraduates based on the virtual standardized patient testing results: Cross sectional study
    Jingrong Du, Xiaowen Zhu, Juan Wang, Jing Zheng, Xiaomin Zhang, Ziwen Wang, Kun Li
    Nurse Education Today.2022; 111: 105312.     CrossRef
Opinion
How to evaluate learning in a flipped classroom
Sun Kim
J Educ Eval Health Prof. 2018;15:21.   Published online September 13, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.21
  • 19,840 View
  • 249 Download
  • 2 Citations
PDFSupplementary Material

Citations

Citations to this article as recorded by  
  • The role of web-based flipped learning in EFL learners’ critical thinking and learner engagement
    Ya Pang
    Frontiers in Psychology.2022;[Epub]     CrossRef
  • Qualitative Data Requirements in the Divayana Evaluation Model
    Dewa Gede Hendra Divayana, Ni Ketut Widiartini, I Gede Ratnaya
    International Journal of Qualitative Methods.2022; 21: 160940692211348.     CrossRef
Software report
Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece  
Kyung (Chris) Tyek Han
J Educ Eval Health Prof. 2018;15:20.   Published online August 17, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.20
  • 27,663 View
  • 223 Download
  • 5 Citations
AbstractAbstract PDFSupplementary Material
Computerized adaptive testing (CAT) technology is widely used in a variety of licensing and certification examinations administered to health professionals in the United States. Many more countries worldwide are expected to adopt CAT for their national licensing examinations for health professionals due to its reduced test time and more accurate estimation of a test-taker’s performance ability. Continuous improvements to CAT algorithms promote the stability and reliability of the results of such examinations. For this reason, conducting simulation studies is a critically important component of evaluating the design of CAT programs and their implementation. This report introduces the principles of SimulCAT, a software program developed for conducting CAT simulation studies. The key evaluation criteria for CAT simulation studies are explained and some guidelines are offered for practitioners and test developers. A step-by-step tutorial example of a SimulCAT run is also presented. The SimulCAT program supports most of the methods used for the 3 key components of item selection in CAT: the item selection criterion, item exposure control, and content balancing. Methods for determining the test length (fixed or variable) and score estimation algorithms are also covered. The simulation studies presented include output files for the response string, item use, standard error of estimation, Newton-Raphson iteration information, theta estimation, the full response matrix, and the true standard error of estimation. In CAT simulations, one condition cannot be generalized to another; therefore, it is recommended that practitioners perform CAT simulation studies in each stage of CAT development.

Citations

Citations to this article as recorded by  
  • Preliminary Development of an Item Bank and an Adaptive Test in Mathematical Knowledge for University Students
    Fernanda Belén Ghio, Manuel Bruzzone, Luis Rojas-Torres, Marcos Cupani
    European Journal of Science and Mathematics Education.2022; 10(3): 352.     CrossRef
  • Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study
    Ioannis Tsaousis, Georgios D. Sideridis, Hannan M. AlGhamdi
    Journal of Psychoeducational Assessment.2021; 39(8): 954.     CrossRef
  • Exploring Counselor‐Client Agreement on Clients’ Work Capacity in Established and Consultative Dyads
    Uma Chandrika Millner, Diane Brandt, Leighton Chan, Alan Jette, Elizabeth Marfeo, Pengsheng Ni, Elizabeth Rasch, E. Sally Rogers
    Journal of Employment Counseling.2020; 57(3): 98.     CrossRef
  • Development of a Computerized Adaptive Testing for Internet Addiction
    Yong Zhang, Daxun Wang, Xuliang Gao, Yan Cai, Dongbo Tu
    Frontiers in Psychology.2019;[Epub]     CrossRef
  • Updates from 2018: Being indexed in Embase, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreci
    Sun Huh
    Journal of Educational Evaluation for Health Professions.2018; 15: 36.     CrossRef
Research article
A novel tool for evaluating non-cognitive traits of doctor of physical therapy learners in the United States  
Marcus Roll, Lara Canham, Paul Salamh, Kyle Covington, Corey Simon, Chad Cook
J Educ Eval Health Prof. 2018;15:19.   Published online August 17, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.19
  • 26,703 View
  • 341 Download
  • 6 Citations
AbstractAbstract PDFSupplementary Material
Purpose
The primary aim of this study was to develop a survey addressing an individual’s non-cognitive traits, such as emotional intelligence, interpersonal skills, social intelligence, psychological flexibility, and grit. Such a tool would provide beneficial information for the continued development of admissions standards and would help better capture the full breadth of experience and capabilities of applicants applying to doctor of physical therapy (DPT) programs.
Methods
This was a cross-sectional survey study involving learners in DPT programs at 3 academic institutions in the United States. A survey was developed based on established non-proprietary, non-cognitive measures affiliated with success and resilience. The survey was assessed for face validity, and exploratory factor analysis (EFA) was used to identify subgroups of factors based on responses to the items.
Results
A total of 298 participants (90.3%) completed all elements of the survey. EFA yielded 39 items for dimensional assessment with regression coefficients < 0.4. Within the 39 items, 3 latent constructs were identified: adaptability (16 items), intuitiveness (12 items), and engagement (11 items).
Conclusion
This preliminary non-cognitive assessment survey will be able to play a valuable role in DPT admissions decisions following further examination and refinement.

Citations

Citations to this article as recorded by  
  • Predictors of Success on the National Physical Therapy Examination in 2 US Accelerated-Hybrid Doctor of Physical Therapy Programs
    Breanna Reynolds, Casey Unverzagt, Alex Koszalinski, Roberta Gatlin, Jill Seale, Kendra Gagnon, Kareaion Eaton, Shane L. Koppenhaver
    Journal of Physical Therapy Education.2022; 36(3): 225.     CrossRef
  • Grit, Resilience, Mindset, and Academic Success in Physical Therapist Students: A Cross-Sectional, Multicenter Study
    Marlena Calo, Belinda Judd, Lucy Chipchase, Felicity Blackstock, Casey L Peiris
    Physical Therapy.2022;[Epub]     CrossRef
  • Predicting graduate student performance – A case study
    Jinghua Nie, Ashrafee Hossain
    Journal of Further and Higher Education.2021; 45(4): 524.     CrossRef
  • Examining Demographic and Preadmission Factors Predictive of First Year and Overall Program Success in a Public Physical Therapist Education Program
    Katy Mitchell, Jennifer Ellison, Elke Schaumberg, Peggy Gleeson, Christina Bickley, Anna Naiki, Severin Travis
    Journal of Physical Therapy Education.2021; 35(3): 203.     CrossRef
  • Doctor of Physical Therapy Student Grit as a Predictor of Academic Success: A Pilot Study
    Rebecca Bliss, Erin Jacobson
    Health Professions Education.2020; 6(4): 522.     CrossRef
  • Personality-oriented job analysis to identify non-cognitive factors predictive of performance in a doctor of physical therapy program in the United States
    Maureen Conard, Kristin Schweizer
    Journal of Educational Evaluation for Health Professions.2018; 15: 34.     CrossRef
Brief report
The implementation and evaluation of an e-Learning training module for objective structured clinical examination raters in Canada  
Karima Khamisa, Samantha Halman, Isabelle Desjardins, Mireille St. Jean, Debra Pugh
J Educ Eval Health Prof. 2018;15:18.   Published online August 6, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.18
  • 21,920 View
  • 240 Download
  • 4 Citations
AbstractAbstract PDFSupplementary Material
Improving the reliability and consistency of objective structured clinical examination (OSCE) raters’ marking poses a continual challenge in medical education. The purpose of this study was to evaluate an e-Learning training module for OSCE raters who participated in the assessment of third-year medical students at the University of Ottawa, Canada. The effects of online training and those of traditional in-person (face-to-face) orientation were compared. Of the 90 physicians recruited as raters for this OSCE, 60 consented to participate (67.7%) in the study in March 2017. Of the 60 participants, 55 rated students during the OSCE, while the remaining 5 were back-up raters. The number of raters in the online training group was 41, while that in the traditional in-person training group was 19. Of those with prior OSCE experience (n= 18) who participated in the online group, 13 (68%) reported that they preferred this format to the in-person orientation. The total average time needed to complete the online module was 15 minutes. Furthermore, 89% of the participants felt the module provided clarity in the rater training process. There was no significant difference in the number of missing ratings based on the type of orientation that raters received. Our study indicates that online OSCE rater training is comparable to traditional face-to-face orientation.

Citations

Citations to this article as recorded by  
  • Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review
    Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
    Journal of Educational Evaluation for Health Professions.2021; 18: 11.     CrossRef
  • Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia
    Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
    Journal of Educational Evaluation for Health Professions.2021; 18: 23.     CrossRef
  • No observed effect of a student-led mock objective structured clinical examination on subsequent performance scores in medical students in Canada
    Lorenzo Madrazo, Claire Bo Lee, Meghan McConnell, Karima Khamisa, Debra Pugh
    Journal of Educational Evaluation for Health Professions.2019; 16: 14.     CrossRef
  • ОБ’ЄКТИВНИЙ СТРУКТУРОВАНИЙ КЛІНІЧНИЙ ІСПИТ ЯК ВИМІР ПРАКТИЧНОЇ ПІДГОТОВКИ МАЙБУТНЬОГО ЛІКАРЯ
    M. M. Korda, A. H. Shulhai, N. V. Pasyaka, N. V. Petrenko, N. V. Haliyash, N. A. Bilkevich
    Медична освіта.2019; (3): 19.     CrossRef
Research article
Examiner seniority and experience are associated with bias when scoring communication, but not examination, skills in objective structured clinical examinations in Australia  
Lauren Chong, Silas Taylor, Matthew Haywood, Barbara-Ann Adelstein, Boaz Shulruf
J Educ Eval Health Prof. 2018;15:17.   Published online July 18, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.17
  • 23,995 View
  • 258 Download
  • 12 Citations
AbstractAbstract PDFSupplementary Material
Purpose
The biases that may influence objective structured clinical examination (OSCE) scoring are well understood, and recent research has attempted to establish the magnitude of their impact. However, the influence of examiner experience, clinical seniority, and occupation on communication and physical examination scores in OSCEs has not yet been clearly established.
Methods
We compared the mean scores awarded for generic and clinical communication and physical examination skills in 2 undergraduate medicine OSCEs in relation to examiner characteristics (gender, examining experience, occupation, seniority, and speciality). The statistical significance of the differences was calculated using the 2-tailed independent t-test and analysis of variance.
Results
Five hundred and seventeen students were examined by 237 examiners at the University of New South Wales in 2014 and 2016. Examiner gender, occupation (academic, clinician, or clinical tutor), and job type (specialist or generalist) did not significantly impact scores. Junior doctors gave consistently higher scores than senior doctors in all domains, and this difference was statistically significant for generic and clinical communication scores. Examiner experience was significantly inversely correlated with generic communication scores.
Conclusion
We suggest that the assessment of examination skills may be less susceptible to bias because this process is fairly prescriptive, affording greater scoring objectivity. We recommend training to define the marking criteria, teaching curriculum, and expected level of performance in communication skills to reduce bias in OSCE assessment.

Citations

Citations to this article as recorded by  
  • Comparing Entrustable Professional Activity Scores Given by Faculty Physicians and Senior Trainees to First-Year Residents
    Steven J Katz, Dennis Wang
    Cureus.2022;[Epub]     CrossRef
  • Association entre les performances cliniques des étudiants et leur réussite aux Épreuves classantes nationales informatisées : une étude de cohorte rétrospective monocentrique
    L. Azoyan, Y. Lombardi, M.C. Renaud, A. Duguet, S. Georgin-Lavialle, F. Cohen-Aubart, G. Ibanez, O. Steichen
    La Revue de Médecine Interne.2022;[Epub]     CrossRef
  • Bias in Medical School Clerkship Grading: Is It Time for a Change?
    Rachel A. Russo, Dana M. Raml, Anna J. Kerlek, Martin Klapheke, Katherine B. Martin, Jeffrey J. Rakofsky
    Academic Psychiatry.2022;[Epub]     CrossRef
  • How biased are you? The effect of prior performance information on attending physician ratings and implications for learner handover
    Tammy Shaw, Timothy J. Wood, Claire Touchie, Debra Pugh, Susan M. Humphrey-Murto
    Advances in Health Sciences Education.2021; 26(1): 199.     CrossRef
  • The role of training in student examiner rating performance in a student-led mock OSCE
    Jian Hui Koo, Kim Yao Ong, Yun Ting Yap, Kum Ying Tham
    Perspectives on Medical Education.2021; 10(5): 293.     CrossRef
  • Does objective structured clinical examination examiners’ backgrounds influence the score agreement?
    Oscar Gilang Purnajati, Rachmadya Nur Hidayah, Gandes Retno Rahayu
    The Asia Pacific Scholar.2021; 6(2): 48.     CrossRef
  • Ethnic and gender bias in objective structured clinical examination: A critical review
    IrisC. I. Chao, Efrem Violato, Brendan Concannon, Charlotte McCartan, Sharla King, MaryRoduta Roberts
    Education in the Health Professions.2021; 4(2): 37.     CrossRef
  • Tutor–Student Partnership in Practice OSCE to Enhance Medical Education
    Eve Cosker, Valentin Favier, Patrice Gallet, Francis Raphael, Emmanuelle Moussier, Louise Tyvaert, Marc Braun, Eva Feigerlova
    Medical Science Educator.2021; 31(6): 1803.     CrossRef
  • Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia
    Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
    Journal of Educational Evaluation for Health Professions.2021; 18: 23.     CrossRef
  • Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review
    Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
    Journal of Educational Evaluation for Health Professions.2021; 18: 11.     CrossRef
  • Is There Variability in Scoring of Student Surgical OSCE Performance Based on Examiner Experience and Expertise?
    Claire L. Donohoe, Frank Reilly, Suzanne Donnelly, Ronan A. Cahill
    Journal of Surgical Education.2020; 77(5): 1202.     CrossRef
  • Insights into student assessment outcomes in rural clinical campuses
    Boaz Shulruf, Gary Velan, Lesley Forster, Anthony O’Sullivan, Peter Harris, Silas Taylor
    BMC Medical Education.2019;[Epub]     CrossRef

JEEHP : Journal of Educational Evaluation for Health Professions