Indian medical students’ perspectives on problem-based learning experiences in the undergraduate curriculum: One size does not fit all

Article information

J Educ Eval Health Prof. 2013;10.11
Publication date (electronic) : 2013 October 31
doi : https://doi.org/10.3352/jeehp.2013.10.11
1Department of Physiology, School of Medical Sciences and Research, Sharda University, Greater Noida, Uttar Pradesh, India
2Department of Physiology, Kasturba Medical College, Manipal, Manipal University, Manipal, Karnataka, India
*Corresponding email: drmanjunatha@gmail.com
Received 2013 August 13; Accepted 2013 October 29.

Abstract

Purpose:

Problem-based learning (PBL) is a well-established method for facilitating the learning of basic science concepts in the context of clinical cases. Relevant evidence is lacking regarding PBL’s effectiveness and acceptability as perceived by students accustomed to classical traditional teaching in India. Hence, this study gathered students’ opinions on PBL versus Traditional teaching methods to generate a foundation for institutional policymaking and ultimately, changes in the policy of regulatory bodies.

Methods:

A total of 773 first year medical students admitted from 2007-2010 in Kasturba Medical College Manipal, Manipal University, India were asked to respond to a 15-item questionnaire evaluating their preferences for PBL or traditional methods such as lectures after undergoing a systematically conducted PBL session in physiology. Their responses were analyzed with an unpaired t-test. Their comments were also collected.

Results:

PBL scored significantly higher for most items in the questionnaire for “learning efficiency” and “student-teacher relationship”. The students’ comments highlighted the importance of a trained tutor/facilitator to enhance the learning process.

Conclusion:

Our students are willing to adapt to the PBL method, although they recognize certain benefits of traditional pedagogy. For learning efficiency and the student-teacher relationship, the students feel that neither method holds an advantage. We recommend that the future medical curriculum in India be a hybrid form of PBL and traditional methods with specific training on the unfamiliar PBL approach for both faculty and students.

INTRODUCTION

In many developing countries such as India, the undergraduate medical curriculum is still divided into pre-clinical and clinical phases, with limited integration. In recent years, one of the major innovations in medical education has been problem-based learning (PBL). PBL, as used in medical education has specific purpose, features and outcomes and has been used as a driving force to generate curriculum reform through the concentration on a single curriculum element [1]. PBL involves the use of clinical problems to motivate students to identify and apply research concepts and information to realistic situations, work collaboratively, and communicate effectively. PBL is student-centered, encouraging students to become more thoughtful problem-solvers. It promotes life-long habits of active learning: the most effective technique for learning, applying, integrating, and retaining information. It is now a well-established method of facilitating basic science education intended for clinical application [2,3]. In India, PBL has made forays into several medical schools of repute, which are experimenting with this method as an adjunct to traditional teaching, though it has not been widely implemented.

Students’ experiences of teaching and learning contexts are a function of both their prior experiences and the present context. In order to improve their learning outcomes, we need to consider both the context and their experiences of that context. It is within this framework that educators must evaluate the technique’s effectiveness and assess whether PBL serves the overall learning objectives. The PBL revolution has spawned a growing body of research that attempts to evaluate the effectiveness of the approach [4]; however, relevant evidence is not available regarding PBL’s effectiveness and acceptability as perceived by students in Indian conditions of classical traditional teaching, which prompted this study. The study aims at capturing perceptions of students comparing PBL vis-a-vis the traditional methods with respect to determinants of learning like information gathering and skills like teamwork. We believe that the present study will help us to make recommendations that will form the basis of policies at the institutional level and ultimately by regulatory bodies.

METHODS

Subjects

The participants were first year medical undergraduate (UG) students admitted from 2007-2010 in Kasturba Medical College Manipal, Manipal University, India. A total of 773 students participated in a systematically conducted PBL session in physiology during the middle of their academic year after having experienced at least 4 months of traditional teaching in physiology.

Implementation of PBL

The students were initially briefed about the principles, methodology, and practice of a proper PBL session. They were then divided into groups of 10 and a tutor/facilitator from the faculty was assigned to each group. Thereafter, PBL was conducted in 2 sessions. The first session was for one hour, wherein the students introduced themselves and elected a leader and a scribe. Then they were given the first part of a clinical case comprising history, symptoms, and signs. They went through the case details slowly and thoroughly, ensured they understood the meaning of difficult and novel terms and after detailed discussion, identified their learning needs and distributed the work to be done among themselves. They were free to meet amongst themselves later for further discussion. The second session, held one week later, lasted for 2 hours. In the first hour, the students shared their knowledge and understanding, asked for further information related to the case, and discussed the problem again at length until they reached a consensus. In the second hour they presented their cases to the other groups, stressing mainly the methodology the group used, the way they approached the problem, the questions they generated, and finally their understanding of the pathophysiological aspects of the case and the learning outcomes they achieved. After each presentation, the members of other groups were free to ask questions or to add relevant comments.

Questionnaire

At the end of the session, the objective of this research study was explained to the students and they were invited to participate. Informed verbal consent was obtained from all those who volunteered. They were asked to fill in a 15-item questionnaire evaluating their preferences for PBL or traditional pedagogy as they had experienced it in the physiology class. The questionnaire was adapted and modified from those in existing literature (Appendix 1). An open-ended qualitative section captured qualitative quotes from the respondents, which were thematically analyzed. In addition, students were asked to mention where they had experienced at least 10 years of their schooling before entering medical school: India or abroad.

Analysis

Responses to the items in the questionnaire were scored as follows: traditional much better (1), traditional better (2), both the same (3) PBL better (4), or PBL much better (5). Univariate analysis was performed and the scores were compared using the one-sample t-test and Wilcoxon signed rank tests. Results across groups (at least 10 years of schooling in India vs abroad; category “I” vs. “A”) were compared using the twosample t-test. After considering the results of the first analysis, it was felt worthwhile to further study the data in detail. Since no significant differences between the “I” and the “A” groups were found (data not shown), the data was pooled for further analysis. Responses were recoded as traditional much better (2), traditional better (1), both the same (0), PBL better (1) and PBL much better (2). For each item of the questionnaire, the mean values for the responses in the traditional groups (1 and 2) were compared with the mean values for the responses in the PBL groups (1 and 2) using the unpaired two-sample ttest. The difference between the means was calculated and significance was determined.

RESULTS

Quantitative results

Out of the 800 students recruited for the study, 773 returned a completed questionnaire. Of these, 159 students were from category “A”, whereas 614 students were from “I”. For most of the items, the students rated PBL to be better than the traditional method. PBL scored highest for instilling the capacity for teamwork, preparing students to face clinical postings, development of reasoning, independent thinking, curiosity, and a questioning attitude, and finally, for generating good interpersonal relationships amongst students. Irrespective of their schooling, the majority of the students believed that the overall value of PBL was greater than that of the traditional method of teaching (Fig. 1). Table 1 shows the results of data pooled from both the ‘I’ and the ‘A’ groups (scoring pattern 0, 1, 2). There was a highly significant difference between the mean scores for PBL and traditional methods for most of the items of the questionnaire except for “learning efficiency”, “understanding principles”, and “student-teacher relationships”. Though not significant, scores for learning efficiency were higher for PBL. As far as understanding of principles was concerned, the majority of the students were of the opinion that the traditional method was better (44.02% of the “I” group and 46.09% of the “A” group), while the rest were more or less equally divided between “both the same” and “PBL better”, corresponding with the mean score in favor of the traditional method (Fig. 2). Moreover, even with detailed analysis of the pooled data, the mean scores for the traditional method were higher (Table 1), suggesting a tendency for a slight preference towards the traditional method (P=0.051). In case of student-teacher relationships, the majority of students from the “A” group believed that it was similar for both the modes of teaching, though amongst students of the “I” group, approximately a third believed PBL was better, another third felt that both had equal value, and the remaining third opined that the traditional mode was better (Fig. 3). Pooled data analysis found marginally higher mean scores for the traditional method, but these were not statistically significant.

Fig. 1

Percentage of students favoring problem-based learning with respect to various items in Kasturba Medical College Manipal, Manipal University, India. I: at least 10 years of schooling in India before entering medical school; A: at least 10 years of schooling abroad before entering medical school. PBL, problem-based learning.

Mean scores of all students’ preference to 15 items, based on a 0, 1, 2 scoring system on problem-based learning experiences in the undergraduate curriculum in Kasturba Medical College Manipal, Manipal University, India.

Fig. 2

Preference of students with respect to understanding principles on problem-based learning experiences in the undergraduate curriculum in Kasturba Medical College Manipal, Manipal University, India. I: at least 10 years of schooling in India before entering medical school; A: at least 10 years of schooling abroad before entering medical school. PBL, problem-based learning.

Fig. 3

Preference of students with respect to student-teacher relationships on problem-based learning experiences in the undergraduate curriculum in Kasturba Medical College Manipal, Manipal University, India. I: at least 10 years of schooling in India before entering medical school; A: at least 10 years of schooling abroad before entering medical school. PBL, problem-based learning.

Qualitative assessment

Comments from students captured interesting observations ranging from questioning the “expertise” of teachers in using PBL methods, the environment, and the size of the group, to a sense of ownership and better self-esteem gained after the experience of PBL. They were classified into various themes.

Teamwork: Some students complained that all group members did not participate equally, and they felt that this hindered their learning process.

Assessment: Some students commented that the participation and the performance of the group members during the PBL process should be formally assessed and this assessment should play a vital role in the final exam/evaluation/grading.

Value for time: Some students thought PBL was a waste of time as it did not help them to prepare for the exams. Others commented that it would help them to become better clinicians.

Role of tutors: Some students felt pressured and unable to think with an open mind as they thought the tutor was too strict. Some specific comments were as follows: first, there is no need for tutors in a PBL class; second, our tutors were too strict. It hampered my learning; third, tutors need more training to conduct PBLs.

DISCUSSION

Acquisition of knowledge, information gathering, understanding of general principles, and learning efficiency

The majority of our students perceived that as far as acquisition of knowledge and information gathering in physiology was concerned, PBL was better than traditional teaching (Table 1). Present study addressed only student perceptions and did not directly test the level of acquired knowledge. Students also felt that the traditional method was superior to PBL for understanding general principles and concepts (Fig. 2). Problem- based learning students were also found to have the same perceived level of anatomy knowledge as the students of more traditional educational approaches in eight medical schools in the Netherlands [5]. PBL has been documented to have an adverse effect on some students who have to adopt a shift in their mindset, as they now need to explore rather than merely receive content knowledge [6]. Based on this, there could have been differences between the “A’’ and “I” groups, as, unlike students in group “A”, the “I” students had probably never been exposed to a PBL approach, and we expected to find a preference for PBL in the “A” group students. However, contrary to our expectations, we found no significant differences between the groups. It appears that the preference for traditional learning is not shaped by prior exposure to PBL and is not merely due to the apprehension associated with a need for a shift in mindset. The drawback of PBL is the purported stress that students feel about the perceived danger of PBL leading to gaps in understanding of general principles and concepts, further complicated by such incorrect understanding not being rejected but being corroborated by other naive students and untrained/inexperienced tutors [6]. This would tilt the balance in favor of traditional teaching wherein the experienced faculty teachers are traditionally expected to ensure that all the necessary and important principles and concepts have been taught.

The amount of basic science knowledge that is sufficient to equip an undergraduate to successfully and confidently function as a medical practitioner is difficult to determine. Moreover, studies indicate that PBL does not impact knowledge acquisition, but impacts application of knowledge [7]. As far as learning efficiency was concerned, there was no significant difference between preference of students for PBL vs. preference for the traditional method (Table 1). Our results support a recent review which reported no unequivocal evidence in favor of PBL enhancing learning [7]. Our study was based on students’ perceptions; though their knowledge and understanding was not assessed by any evaluative process, the fact that they self-evaluated their knowledge and its application as a part of their learning objectives is valuable in itself.

Personal enjoyment, satisfaction, motivation, and stimulation of interest in the subject

PBL students consistently find their course more enjoyable and demonstrate better interpersonal skills compared with traditionally trained students [8]. Making learning fun and enjoyable is an essential method of driving individual and group learning [9]. We obtained encouraging results as students felt that PBL provided more personal enjoyment and satisfaction than the traditional method of teaching. PBL also scored higher for motivation and stimulating interest.

Interpersonal relationships and teamwork

Through their interactions, students learn skills that are highly relevant to their future work as doctors, such as teamwork, leadership, and delegation [10]. Ultimately, graduates of PBL curricula demonstrate equivalent or superior professional competencies compared with graduates of more traditional curricula [10]. PBL was rated better for enhancing interpersonal relationships and for instilling the capacity for teamwork (Table 1) implying that students perceived these benefits of PBL.

Student-teacher relationship and the role of tutors in PBL

PBL requires tutors to function as facilitators rather than acting as providers of information. More than the tutors’ subject- matter expertise and their ability to explain concepts in a way that is easily understood by students, the ability of tutors to communicate informally with students has a greater impact on learning at each of the PBL phases. This informal communication also creates a less threatening learning environment promoting a free exchange of ideas [11]. We found no significant differences in the number of students who preferred the traditional method versus PBL (Table 1, Fig. 3). However, as mentioned in the results, students wrote some interesting comments regarding the role of the tutor. Tutor expertise has a significant effect on student learning outcomes [12]. In a PBL setting, the boundaries between the facilitator and student are noticeably reduced, providing opportunities for the student to be empowered in raising pertinent questions challenging existing issues in relation to a PBL problem [13]. New attitudes and skills may be required of the teaching faculty so that they are willing and competent to deal with the PBL method. It is debatable whether the perceptions of our students were affected by the subject expertise of the tutors or their training, but hints from answers to the open-ended questions about serious concerns regarding the role played by some tutors indicate the need for tutors to understand the philosophy and methodology of PBL so that they can manage the learning process better.

Development of reasoning, curiosity and a questioning attitude, and independent thinking

The hallmark of any PBL approach is to generate questions in an effort to systematically analyze and solve the problem, and, not unexpectedly, the PBL method of teaching has been shown to be an effective instructional tool to foster critical thinking and problem solving skills among medical students [10]. Our study adds importance to such knowledge, as the students themselves perceived and rated PBL higher than traditional teaching for development of reasoning, curiosity and a questioning attitude, and independent thinking, all the characteristics needed for successful and efficient patient care and for lifelong, independent, self-directed learning.

Preparation for clinical postings

PBL was perceived by students to help them prepare for clinical postings (Fig. 1). Relevant comments indicated that PBL sessions made them ‘feel like a doctor.’ However, one study reported that students in their fifth year of medical school feel better prepared for their internship if they have been taught with the traditional method of teaching [14]. It is possible that the differences observed in that study and ours might be due to the smaller number of students in their study (n=100) or the fact that they were in their fifth year of study. In any case, it is interesting to note that even first year students perceived that PBL would help them to prepare for subsequent clinical postings.

Assessment

Many students suggested that they should be marked by the tutors for the PBL sessions and these marks should play a vital role in the overall final assessment and grading. Some complained that there was not equal participation from all of the group members. Inclusion of marks in the final assessment could improve student participation. The students also commented that some of the teachers were too strict, which hindered their ability to think freely. These students might have been even worse off if they were further pressured by the stress of scoring marks. PBL would then defeat the very purpose of stimulating interest and independent thinking. Student participation is a key ingredient in the success of a PBL program [15], and it would be worthwhile to collect more information on the assessment process and its impact on the overall participation and future performance of the students.

Compatibility with sociocultural values, past experiences, and future needs

Delivery of instruction in PBL involves active peer teachinglearning in an open communication style. This may pose an apparent serious conflict with the Asian communication style generally and culturally dominated by reticence [16]. We specifically compared both the groups of students (A and I) because in India, the education in schools is still very teachercentric although in countries like the USA, UK, and Malaysia, from where most of our foreign and non-resident Indian (NRI) students have had their major part of education, there is a little more freedom and a little more stress on questioning. It is known that various factors like the role and training of facilitators, the way the curriculum is implemented and resourced, and the extent to which assessment drives the learning process can impact whether or not the learning of basic sciences can be accomplished by the PBL process [17]. However, wedid not find appreciable differences between the two groups, and overall, most of the students preferred the PBL mode of teaching.

Recommendations

A PBL curriculum has several advantages over a conventional curriculum, but our study suggests that it may not necessarily have an added advantage over the traditional method in some domains of teaching and learning physiology. These domains must be further investigated. Some students perceived that PBL was not better for understanding general principles and concepts. These concepts may be first taught in the traditional way, and the actual PBL could start after the students acquire basic physiology knowledge. Hence, our recommendation to medical schools experimenting with the implementation of PBL is to incorporate certain aspects of conventional teaching. We believe that the future medical curriculum of our institution should be a hybrid of PBL and conventional curricula involving trained facilitators. Strong support from administrators and careful training of both faculty and students is required for successful implementation in medical schools, and we agree that this is necessary to reap the benefits of the PBL method.

Limitations

The PBL exercise in our setup was done in two sessions and may not be representative of all kinds of PBL exercises, as they vary among institutions. These data have been obtained from a single medical school in South India; this school may not be representative of the whole country, as India is a country of varied cultures. Moreover, perceptions may differ amongst males and females, but gender comparisons were not performed in the study.

In conclusion, the present study revealed several positive and negative aspects of the students’ experiences with PBL. The students’ perceptions did not differ significantly based on where their primary education had been. Overall, they seemed willing to adapt to the PBL method of learning, while at the same time highlighting the perceived merits of traditional teaching. Interestingly, the data showed that, regarding learning efficiency, student-teacher relationships, and understanding principles, neither method held an advantage statistically. However, more students felt that the traditional method was better for understanding principles. We conclude that PBL should be introduced into the curriculum as part of a hybrid system initially, and further studies must be performed, addressing the items for which the students showed no significant preference.

Notes

CONFLICT OF INTEREST

No potential conflict of interest relevant to this article was reported

SUPPLEMENTARY MATERIAL

Audio recording of the abstract.

ACKNOWLEDGMENTS

We wish to acknowledge Upali Nanda for her valuable ideas and input and for helping us with the statistics.

References

1. Schmidt HG. Problem-based learning: rationale and description. Med Educ 1983;17:11–16. http://dx.doi.org/10.1111/j.1365-2923.1983.tb01086.x.
2. Norman GR, Schmidt HG. The psychological basis of problembased learning: a review of the evidence. Acad Med 1992;67:557–565. http://dx.doi.org/10.1097/00001888-199209000-00002.
3. Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med 1993;68:550–563. http://dx.doi.org/10.1097/00001888-199307000-00015.
4. Sanson-Fisher RW, Lynagh MC. Problem-based learning: a dissemination success story? Med J Aust 2005;183:258–260. https://www.mja.com.au/journal/2005/183/5/problem-based-learningdissemination-success-story.
5. Prince KJ, van Mameren H, Hylkema N, Drukker J, Scherpbier AJ, van der Vleuten CP. Does problem-based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy. Med Educ 2003;37:15–21. http://dx.doi.org/10.1046/j.1365-2923.2003.01402.x.
6. Bernstein P, Tipping J, Bercovitz K, Skinner HA. Shifting students and faculty to a PBL curriculum: attitudes changed and lessons learned. Acad Med 1995;70:245–247. http://dx.doi.org/10.1097/00001888-199503000-00019.
7. Hartling L, Spooner C, Tjosvold L, Oswald A. Problem-based learning in pre-clinical medical education: 22 years of outcome research. Med Teach 2010;32:28–35. http://dx.doi.org/10.3109/01421590903200789.
8. Nandi PL, Chan JN, Chan CP, Chan P, Chan LP. Undergraduate medical education: comparison of problem-based learning and conventional teaching. Hong Kong Med J 2000;6:301–306. http://www.hkmj.org/article_pdfs/hkm0009p301.pdf.
9. DiCarlo SE. Too much content, not enough thinking, and too little fun! Adv Physiol Educ 2009;33:257–264. http://dx.doi.org/10.1152/advan.00075.2009.
10. Onyon C. Problem-based learning: a review of the educational and psychological theory. Clin Teach 2012;9:22–26. http://dx.doi.org/10.1111/j.1743-498X.2011.00501.x.
11. Wood DF. ABC of learning and teaching in medicine: problembased learning. BMJ 2003;326:328. http://dx.doi.org/10.1136/bmj.326.7384.328.
12. Savin-Baden M. Disciplinary differences or modes of curriculum practice? Who promised to deliver what in problem-based learning? Biochem Mol Biol Educ 2003;31:338–343. http://dx.doi.org/10.1002/bmb.2003.494031050263.
13. Tucker CM, Herman KC. Using culturally sensitive theories and research to meet the academic needs of low-income African American children. Am Psychol 2002;57:762–773. http://dx.doi.org/10.1037/0003-066X.57.10.762.
14. Millan LP, Semer B, Rodrigues JM, Gianini RJ. Traditional learning and problem-based learning: self-perception of preparedness for internship. Rev Assoc Med Bras 2012;58:594–599. http://dx.doi.org/10.1590/S0104-42302012000500018.
15. Wiznia D, Korom R, Marzuk P, Safdieh J, Grafstein B. PBL 2.0: enhancing problem-based learning through increased student participation. Med Educ Online 2012;17:17375. http://dx.doi.org/10.3402/meo.v17i0.17375. Epub 2012 Jun 11.
16. Gwee MC. Globalization of problem-based learning (PBL): crosscultural implications. Kaohsiung J Med Sci 2008;24((3 Suppl)):S14–S22. http://dx.doi.org/10.1016/S1607-551X(08)70089-5.
17. Khoo HE. Implementation of problem-based learning in Asian medical schools and students’ perceptions of their experience. Med Educ 2003;37:401–409. http://dx.doi.org/10.1046/j.1365-2923.2003.01489.x.

Appendices

Appendix 1. PBL Evaluation Form

For each of the following "outcomes" please compare PBL as you have experienced it with the traditional teaching methods using the following scale: 1=Traditional much better, 2=Traditional generally better, 3=Both the same, 4=PBL generally better, 5=PBL much better. Choose one: The major part of my schooling was IN INDIA / ABROAD.

Article information Continued

Fig. 1

Percentage of students favoring problem-based learning with respect to various items in Kasturba Medical College Manipal, Manipal University, India. I: at least 10 years of schooling in India before entering medical school; A: at least 10 years of schooling abroad before entering medical school. PBL, problem-based learning.

Fig. 2

Preference of students with respect to understanding principles on problem-based learning experiences in the undergraduate curriculum in Kasturba Medical College Manipal, Manipal University, India. I: at least 10 years of schooling in India before entering medical school; A: at least 10 years of schooling abroad before entering medical school. PBL, problem-based learning.

Fig. 3

Preference of students with respect to student-teacher relationships on problem-based learning experiences in the undergraduate curriculum in Kasturba Medical College Manipal, Manipal University, India. I: at least 10 years of schooling in India before entering medical school; A: at least 10 years of schooling abroad before entering medical school. PBL, problem-based learning.

Table 1.

Mean scores of all students’ preference to 15 items, based on a 0, 1, 2 scoring system on problem-based learning experiences in the undergraduate curriculum in Kasturba Medical College Manipal, Manipal University, India.

Traditional better PBL better P-value
Number of students Mean score Number of students Mean score
Acquisition of knowledge 80 0.84 110 0.93 NS
Information gathering 49 0.61 135 1.19 <0.0001
Understanding general principles 112 0.98 87 0.57 NS
Learning efficiency 85 0.87 106 0.91 NS
Personal enjoyment and satisfaction 55 0.67 138 1.12 <0.001
Motivation level 58 0.48 139 1.02 <0.001
Stimulating interest in the subject 49 0.73 137 1.16 0.002
Interpersonal relationships 24 0.21 154 1.34 <0.001
Teamwork 14 0.14 158 1.49 <0.001
Student-teacher relationship 109 0.54 116 0.54 NS
Development of reasoning 34 0.50 146 1.34 <0.001
Curiosity and a questioning attitude 34 0.38 145 1.20 <0.001
Developing independent thinking 47 0.40 144 0.78 <0.001
Preparation for clinical subjects 25 0.32 151 1.39 < 0.001
Overall value 56 0.79 127 0.99 0.006

For each of the following "outcomes" please compare PBL as you have experienced it with the traditional teaching methods using the following scale: 1=Traditional much better, 2=Traditional generally better, 3=Both the same, 4=PBL generally better, 5=PBL much better. Choose one: The major part of my schooling was IN INDIA / ABROAD.

Traditional system better Both the same PBL better
1 2 3 4 5
1 Acquisition of knowledge
2 Information gathering
3 Understanding general principles
4 Learning efficiency
5 Personal enjoyment and satisfaction
6 Motivation level
7 Stimulating interest in the subject
8 Interpersonal relationships
9 Teamwork
10 Student-teacher relationships
11 Development of reasoning
12 Curiosity and a questioning attitude
13 Developing independent thinking
14 Preparation for clinical subjects
15 Overall value

Any Comments: Please write on the back side of this sheet.