Department of Medical Education, Seoul National University College of Medicine, Seoul, Korea
© 2019, Korea Health Personnel Licensing Examination Institute
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Item | Author | Category | Context | Input | Process | Product | |
---|---|---|---|---|---|---|---|
Proposal by Stuffelbeam, founder of CIPP Model | Stufflebeam and Shinkfield [7] (1985) | Criteria | Define institution situation; learner identification and demand inspection; search for demand satisfaction opportunity; problem diagnosis and determination of objective appropriacy | System capability, solution program strategy, design procedure of strategy execution, budget, schedule check | Flaw check or prediction of procedures in progress or the execution process; information provision for preplanned sequential decisions; report and judgment of events and activities regarding the execution | Collection of technology/judgment regarding achievements; linkage with information on the objective, situation, input, process; value and advantage analysis | |
Material collection method | Utilization of systems analysis, survey study, literature survey, public hearings, interviews, diagnostic assessment, Delphi technique | Available human and material resources, resolution strategy, design procedure, possibility and economic analysis, literature survey, pilot program survey, advocacy groups, pilot attempt | procedural disorder identification and accidental disorder awareness, detailed information acquisition for scheduled decision-making, describe the actual process, continuous interaction with the program operation staff, and observation or their activities | Operational definition and measurement of the performance standards and collection of the judgments by the interested parties, qualitative/quantitative analysis | |||
Purpose | Necessary for decision-making regarding education objective and purpose when education begins (provide standard of change plans, performance judgment): use in the decision-making for education planning | Necessary for support resource, resolution strategy, and design procedure selection (provide the basis for change activity composition, execution process judgment basis): use in the decision-making for education structuralization | Necessary for program planning, procedure, and improvement; necessary for actual situation basis provision in performance analysis: use in the decision-making for education execution | Operational definition and measurement of the performance standard; collection of judgments by interested parties regarding the performance; qualitative/quantitative analysis: use in the decision-making for recycling | |||
Institution | Jung and Moon [26] (2013) | Criteria | Service demand and situation, service objective domain | Budget, human resource management, facility and resource environment, service operation and content | Service activity, service satisfaction, service evaluation | Service application, service performance | |
School | Shin et al. [19] (2018) | Criteria | Demand analysis, objective setting | Execution plan (human resource, procedure, support system, etc.), performance detail | Program activity, program management and evaluation | Program performance (achievement, satisfaction, effectiveness) | |
Nursing | Kim and Son [27] (2017) | Criteria | Intention and necessity | contents of hospital introduction, senior nurses’ working experience | Composition and facilitation | Usefulness of the program, feeling involvement through activities, global satisfaction | |
Material collection method | Questionnaire | Questionnaire | Questionnaire | Questionnaire | |||
Medical health professions | Ashghali-Farahani et al. [14] (2018) | Criteria | Inappropriate infrastructure; unknown duties | Biomedical approach; incomprehensive curriculum; lack of professional NICU nursing mentors; inappropriate admission process of NICU students; lack of NICU skill labs | More emphasize on theoretical education; the overlap of credits with each other and the inconsistency among the mentor; ineffective assessment | Preferring routine work instead of professional job; tendency to leave the job; clitical incompetency of graduates; dissatisfaction of graduates | |
Material collection method | Semi-structured interview; open question | ||||||
Target of evaluation | NICU student, NICU graduate nurse, neonatologist, faculty member, nurse | ||||||
Neyazi et al. [10] (2016) | Criteria | Goals, organization and management area | Interest and understanding of students towards field and labor market; faculty members; research and educational spaces and equipment | Student research activity; educational courses and programs, teaching and learning process; student progress evaluation; evaluated factors for graduates | Efficiency of research and educational programs, teaching and learning process to increase knowledge and job performance of graduates | ||
Material collection method | Researcher-made questionnaires inspired from the CIPP model and internal evaluation literatures | ||||||
Target of evaluation | Students, graduates | ||||||
Al-Khathami [11] (2012) | Criteria | Achievement of program goals; barriers to achieve goals, objectives, and needs | Alternative procedural design for: contents, academic sessions, hospital sessions, half day release sessions | Process involved in to learning activities; trainers; theoretical sessions; clinical sessions | Overall impression about the program; barriers to achieve goals, objectives, and needs; assessment tools; enjoyment; satisfaction | ||
Material collection method | Questionnaire (quantitative, qualitative) | ||||||
Target of evaluation | Trainee | ||||||
Yarmohammadian and Mohebbi [15] (2015) | Criteria | Human specialists and scientific services for needs of the local community | Head of department, faculty, | Activities of group manager, students, administrators | - | ||
students, curriculum, funding, training facilities | of library; scientific research and teaching–learning activities of faculty | ||||||
Material collection method | Questionnaire | ||||||
Target of evaluation | Directorates, faculty members, students, and library staff | ||||||
Neyazi et al. [12] (2016) | Criteria | Goals, management, and organization area | Facility and spaces | Educational courses and programs, learning and teaching process; administration and financial; program evaluation | Graduates | ||
Material collection method | Questionnaire | ||||||
Target of evaluation | Department head, faculty members, and library staff | ||||||
Rooholamini et al. [9] (2017) | Criteria | Perceptions of learning; perceptions of teachers; academic self-perceptions; perceptions of the environment; social self-perceptions | Content of curriculum | The process of learning; process of teaching | Students’ performance; the process of teaching and learning | ||
Material collection method | Review of current evidence on integration; consultation with experts; Modified Dundee Ready Education Environment Measure (DREEM) | A researcher made questionnaire | 1: researcher–made questionnaires for evaluating the quality of each integrated course; 2: researcher–made questionnaires for evaluating the quality of early clinical exposure | 1: learner centered integrated basic science, portfolios; 2: brainstorming (students); 3: semi-structured interview (professors of basic sciences) | |||
Target of evaluation | Students, faculty and administrators | Faculties and curriculum committee | First and second year medical students | First and second year medical students; professors of basic sciences | |||
Lee et al. [8] (2019) | Criteria | Goals, necessity or needs | Available input resources (human and material resources); educational strategy | Implementation according to plan; evaluation of the program by students | Goal achievement; satisfaction of the curriculum | ||
Material collection method | Questionnaire, FGI, meeting minutes, syllabus, curriculum | Questionnaire, FGI, meeting minutes, time table | Questionnaire, FGI, meeting minutes, syllabus | Questionnaire, FGI, meeting minutes, grades | |||
Target of evaluation | Students, faculty |
Item | Context | Input | Process | Product |
---|---|---|---|---|
Keywords extracted by content analysis (frequency) | Goals (6); necessity or needs (5); infrastructure (2); organization (2); management (2); intention (1); duties (1); barriers to achieve goals (1) | Material resources, facilities (6); human resource (6); contents (5); curriculum (3); funding (2); academic approach (1); admission process of students (1); interest and understanding of students (1); educational and strategy (1); implement plan (1) | Educational and service process (7); program evaluation (4); educational courses and programs (3); student progress evaluation (2); composition (1); facilitation (1); administration and financial (1); service satisfaction (1) | Global satisfaction (5); students’ and service achievement (4); program performance (4); efficiency of research and educational programs to increase knowledge and performance (1); barriers to achieve goals (1); the process of teaching and learning (1) |
CIPP, Context, Input, Process, and Product; NICU, newborn intensive care unit; FGI, focus group interview.