|Year : 2019 | Volume
| Issue : 1 | Page : 13-18
Comparative evaluation of structured clinical case examination with traditional long-case examination for clinical competence assessment
Department of General Medicine, S. B. K. S. Medical Institute and Research Centre, Sumandeep Vidyapeeth, Vadodara, Gujarat, India
|Date of Web Publication||20-Jun-2019|
Dr. Hetal Pandya
Professor and Head, Department of General Medicine, Dhiraj Hospital, S. B. K. S. Medical Institute and Research Centre, Piparia, Vadodara, Gujarat
Source of Support: None, Conflict of Interest: None
Background: The clinical examinations are of key importance in competency assessment for medical education. However, in traditional long-case examination (TLCE), the product of history taking and physical examination is assessed, and the process is not observed. We suggest a structured clinical case examination (SCCE) with the aim to develop the best possible evaluation method for clinical competencies of undergraduate medical students. Methodology: A new assessment method SCCE based on principles of (1) direct observation, (2) objectivity, and (3) structured format was developed. This cross-sectional study was done during the internal ward ending examination of 3rd-year clinical batch. Students' views about TLCE were taken. The examination with SCCE and TLCE methods was taken by four subject faculties for two subgroups A and B, respectively. Feedback was taken from students and examiners in preformed questionnaire. Faculties' views regarding feasibility, objectivity, process, content, impact on assessment, and learning were noted through focus group discussion. Results: On quantitative analysis of results of both methods, SCCE has found to have a better categorization of the students in competent, average competent, and noncompetent groups, while TLCE group has shown clustering of students (up to 66%) in the average group. Almost all students strongly agreed that SCCE method is better than traditional method on almost all parameters. All examiners found this new method feasible, doable, and more objective. Conclusions: The proposed SCCE method fulfills almost all characteristics for ideal clinical competency assessment method. It can be easily implemented as an assessment method in clinical subjects.
Keywords: Clinical case examination, objective structured clinical examination, objective structured long examination record, traditional long-case examination
|How to cite this article:|
Pandya H. Comparative evaluation of structured clinical case examination with traditional long-case examination for clinical competence assessment. J Integr Health Sci 2019;7:13-8
|How to cite this URL:|
Pandya H. Comparative evaluation of structured clinical case examination with traditional long-case examination for clinical competence assessment. J Integr Health Sci [serial online] 2019 [cited 2019 Jul 20];7:13-8. Available from: http://www.jihs.in/text.asp?2019/7/1/13/260829
| Introduction|| |
The current evaluation system for undergraduate medical students is primarily divided into two types of examinations – formative and summative in India as per the Medical Council of India rules and regulations. Formative es mainly done in parts as theory examination, term/ward ending practical/clinical examination, and viva voce. The practical clinical examinations are of key importance in the assessment of clinical competence. They play a very important role in the certification of candidates before they are allowed to practice medicine. The traditional method used for clinical examination includes a long case, one or two short cases, and viva voce. In a “long-case examination,” the product of history taking and physical examination is assessed, and the process is not observed. The discussion frequently focuses around the theoretical aspects of differential diagnosis, planning of investigations, and management. The approach and the expectations of different examiners may be different. This leads to subjectivity and inconsistencies in marking. The examination has rightly been referred to as “luck of the draw.” Communication skills which are necessary to acquire information are also not assessed. The existing long case thus fails to meet the criteria of validity, reliability, and objectivity.
Gleeson has attempted to remodel and improve the long-case examination. He has developed objective structured long examination record (OSLER) as a more valid, reliable, and objective tool for the assessment of clinical competence. The OSLER including 10 essential items seems to be a practical and useful method to increase the validity, reliability, and objectivity of a long-case examination. The OSLER is both examiner and candidate friendly., OSLER is also practical in terms of organizational logistics. However, this method could not able to gain wide acceptance because of practical and feasibility issues, especially in India. Although found useful by most of assessors and students in a study by Bhaleroo, many assessors were concerned about the consumption of significant workforce and time resources, while 50% of students found it stressful. Objective structured clinical examination (OSCE) has been suggested as an alternative and has been used worldwide for the assessment of clinical skills since its introduction by Harden and Gleeson in 1979. Because of many lacunae such as time constraint, logistical problems, maintaining uniform difficulty levels, shortage of observers, observer fatigue, and time-consuming, this “very objective” method of evaluation – OSCE/objective structured practical examination – has not yet become a very popular method of undergraduate evaluation. This very objective method of clinical competency assessment was also not found to replace long-case examination, rather used as an adjuvant method.
With due consideration to the pros and cons of the evaluation methods in practice, we suggest a structured clinical case examination (SCCE). It is an amalgamation of the older (conventional) method of clinical case examination where the student is asked to examine one patient and then is questioned on it and the relatively more recent OSLER and OSCE methods. The final aim is to develop the best possible evaluation method for clinical competencies of undergraduate medical students. The objectives of the study were (1) to evaluate the proposed SCCE method for its logistics and objectivity and (2) to compare this new assessment method with the current assessment method (traditional long-case examination [TLCE]).
| Methodology|| |
A new assessment method SCCE was developed based on principles of (1) direct observation, (2) objectivity, and (3) structured format for clinical competency assessment in ward ending examination in the Department of Medicine, S. B. K. S. Medical Institute and Research Centre, Sumandeep Vidyapeeth. The method is described in detail below and SCCE evaluation sheet is attached as [Table 1].
|Table 1: Structured clinical case examination evaluation sheet, H. Pandya|
Click here to view
Structured clinical case examination
In this method (SCCE), the students were given clinical case of respiratory, per abdomen system randomly. The cases are almost similar difficulty level. The students had taken a detailed history and done clinical examination of the patient in given time limit of 30 min as per the current long-case examination pattern. The students were assessed by a panel of four examiners. The SCCE evaluation sheet/checklist for examiners was prepared by an investigator in advance. This evaluation sheet was divided into major compartments such as history taking, general examination, systemic examination, differential diagnosis, and treatment. Each compartment was further fragmented into small components to cover various minor aspects, direct skill observation as in OSCE, and even some subjective cognitive and communication skill evaluation by examiner too. The marking for each component and its weightage was decided in advance as per the seniority level of students, syllabus, knowledge areas/domains to be tested, and importance of these minute components [format of SCCE evaluation sheet, [Table 1].
After obtaining ethics committee approval and permission from the Dean of College, the evaluation sheet was validated two senior faculties – one subject expert and one expert in field of medical education. Third-year undergraduate medical students, during their third clinical posting in the department of medicine, were selected for the study. The study design used was case–control type. At the end of their clinical terms, two subbatches were randomly evaluated by any one of these described assessment methods such as (1) TLCE – control group and (2) SCCE – case group. The sample size of SCCE group was 36 and that of TLCE group was 34. Both subbatches were of the same seniority level and were taught the same syllabus by the same teachers, and examinations were also taken by the same set of examiners. The examiners were briefed about the SCCE method and evaluation sheet before the examination.
We informed the students of SCCE batch about the evaluation method adopted for their batch and objectives of this research study just before the examination. Written informed consent of all participants inclusive of examiners was taken. Their views about TLCE were taken in feedback form before the examination. The examination with SCCE evaluation form was taken by four assistant professors of the department. After completion of examination, the feedback about this new method was taken from students through questionnaire. Focus group discussion (FGD) was arranged with all four examiners and their views regarding feasibility, objectivity, process, content, impact on assessment, and learning were noted.
| Observations and Results|| |
As this SCCE method is proposed as modification of traditional way of assessing clinical competency in most of the medical colleges in India, first of all we have taken the views of students on long-case examination as they already had experienced it many times. Merits and demerits described by enrolled students of 3rd-year and 6th-semester batches are listed in [Table 2]. The students were more concerned about time management, teacher dominated, bias, issues related to patients and instruments, and piecemeal assessment. The faculties were more concerned about nonuniformity, subjectivity, and poor performance of students.
Quantitative assessment of students' performance by both methods (structured clinical case examination with traditional long-case examination)
The performance of students in the form of marks obtained in SCCE examination was compared with the result of counterpart subbatch, whose examination was taken with TLCE. Both batches were taught a similar syllabus in the same manner, and the assessment was also done in the same manner (with a similar type of cases, preexamination case history taking) except for the suggested modification in examination method for the study group. The total marks for the examination taken by both methods (SCCE and TLCE) were 50. The mean mark of SCCE method was 25.36 ± 8.40 and that of TLCE was 26.85 ± 3.70. When applied Student's t-test, P = 0.345, suggesting nonsignificant difference. However, mark distribution obtained by SCCE was widely distributed, while marks obtained by TLCE were consistent and narrowly distributed, marks obtained through SCCE method ranged from 6 to 44, while that by TLCE method was from 20 to 35.
For further comparative analysis, the result was categorized in below average (<25 marks), average (25–30 marks), good (30–35 marks), and excellent (more than 35 marks). With the use of SCCE method, 15 students (42%) got less than passing marks (25 marks), 11 students (30%) got marks between 25 and 30, 5 students (14%) were in high mark group of 30–35 marks, and 5 students (14%) got more than 35 marks each. The mark distribution of TLCE group has shown a cluster of students (up to 66%) in the average group (25–30 marks). Only 7 students (20%) were below average and 3 students (1%) got marks in good category. Statistical difference was found in the mark distribution ability of both methods as P = 0.02 by Chi-square method [Table 3].
|Table 3: Categorization of marks obtained by structured clinical case examination and traditional long-case examination method|
Click here to view
Qualitative analysis of student feedback
Feedback of students and faculties regarding their perception for SCCE method was taken immediately after the examination using feedback questionnaire.
[Table 4] shows the frequency distribution of students' responses as per Likert scale. Almost all students strongly agreed that SCCE method is better than traditional method on almost all parameters.
Qualitative analysis of faculty feedback
The feedback of the four faculties of department who had conducted the ward ending clinical examination of both case and control groups was taken through FGD immediately after the examination [Figure 1].
|Figure 1: (a) Frequency distribution in traditional long-case examination group. (b) Frequency distribution in structured clinical case examination group|
Click here to view
The major points discussed in FGD were feasibility, objectivity, process of examination, content of evaluation sheet and impact on learning. The inferences from FGD with faculties are:
- All examiners found this new method feasible and doable in ward ending examinations in the department of medicine
- Their views about objectivity – Uniform format was used, and an equal number of questions were asked to all students. The whole-case examination was divided into its vital sections with the use of evaluation sheet. Each section had structured question pattern and predefined marking for each question. All these have led to much-reduced subjectivity
- Regarding the process of conducting SCCE, they found it almost similar to the previously used method for the same. The addition of evaluation sheet led to systemic assessment of all components of case examination. Although they found it time-consuming, it worth it
- All faculties strongly agreed about the appropriateness of the content of evaluation sheet. According to them, all essential components of clinical case examination were included in structured way with appropriate weightage. Hence, there was no chance of skipping or overemphasis on only one or two components. The inclusion of communication skill assessment was the most appreciated part of evaluation as it is the very crucial skills to be acquired by the medical professionals in the current situation. The faculties appreciated the inclusion of direct observation of clinical/practical skills as part of SCCE method. As said by one of the faculties – “It is the best part of this examination, which is most crucial, yet we usually missed it often till now”
- During discussion about the impact of such evaluation on learning of clinical skills/competencies, one of the examiners described it as “Almost perfect method” for clinical competency assessment. As per another teacher, “if students and teaching faculties are aware that they will be assessed in this manner, the students will learn and teachers will teach accordingly.” Moreover, it will definitely improve the clinical skills in medical graduates, in which they are lacking a lot at present.
| Discussion|| |
In medical education, long- and short-case examination is used for clinical skill/competency assessment all over the world since many decades because of its proven benefits. However, over the time, many lacunae such as subjectivity, teacher dominance, and nonuniformity were recognized and reported. One of the main stakeholders of this examination system, namely the students, also reported many issues with it like inadequate time for history taking and also for assessment, noncooperative patients, lack of seriousness on part of examiners, various biases, draw of luck in getting cases, and unstructured viva going haywire [Table 1]. It is up most important to gain expertise in clinical competencies during medical course study for future successful medical practice. Long- and short-case examination is the only assessment method used to certify the same. As this traditional method was found inadequate, other methods such as OSLER and OSCE were developed and are in practice since many years now.
With gaining experiences about these methods, many pitfalls and feasibility issues surfaced with very objective line of assessment tool. Hence, TLCE still continues to dominate the clinical competency examination at most of the institutes in India. There is always been search for the assessment tool, which can assess “shows how” level of Miller's pyramid effectively and yet practical too. Here, we had tried to plug the loopholes and make it more full proof by adding structured format, predefined marking system, and direct observation of clinical skills (DOCS) through designing evaluation sheet for examiners.
When marks obtained by both methods were compared, the difference in mean was negligible. However, wide range of marks and more standard deviation of the mean for SCCE method suggest its ability to differentiate between highly competent, average competent, and noncompetent students. Compartmentalization of whole-case assessment and mark division as per their weightage gave better differentiating power to this method compared to traditional examination.
Students are the main stakeholders and beneficiaries; their opinion about new method is crucial. Most of the students voted in favor of the implementation of SCCE method in their examination. They agreed that this method assesses all components of case examination and addition of communication skill assessment is helpful. Only 55% of students agreed that DOCS is better with SCCE method. The compulsion to have mini-direct observation of procedural skills (DOPS) at all component levels was one of the key features of this method, but students could not able to recognize it as they were usually asked to demonstrate one or another clinical sign during traditional method too but at the discretion of examiner.
Faculty or examiner perception about the new method is the most important feedback as they are the actual end users. All examining faculties found SCCE way of examination feasible and doable. The acceptance by end users is the upmost important parameter for successful implementation of any new method or strategy. Uniformity of question pattern and their weightage made SCCE objectively structured tool as per them. The process of conducting examinations with SCCE evaluation sheet seems appropriate but time-consuming to all. In author's view, it is necessary and justifiable to give enough time to evaluate clinical competencies as these are vital skills to become competent and successful medical practitioner. Addition of compulsory mini-DOPS (or can be called as DOCS) and assessment of communication skills can make our current clinical case examination near ideal for clinical skill examination.
Limitations of the study
Quantitative analysis and comparison of result obtained was done with another set of students due to feasibility and time management issue, which might have lead to some inter participant variability. But we tried to minimize it by comparing the result of study group with the same level students, who were taught same syllabus by same teachers and also assessed by same set of examiners.
| Conclusions|| |
A great concerned is raised for lack of clinical skills in newly passed medical graduates. If clinical skills are assessed properly, learning for the same will improve itself. The proposed SCCE method fulfills almost all characteristics for ideal clinical competency assessment, without compromising between objectivity, validity, and reliability on the one hand and practicality on the other hand in the given setting; however, larger samples may be needed to confirm its reliability. It can be easily implemented as an assessment method in clinical subjects.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Stokes J F. 'The clinical examination: assessment of clinical skills'. Publisher: Dundee, Association for the Study of Medical Education 1974.
Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979;13:41-54.
Gleeson F. Assessment of clinical competence using the objective structured long examination record (OSLER). Med Teach 1997;19:7-14.
Bhaleroo P. Perception of medical teachers and students regarding objective structured long examination record as assessment tool – A prospective study. Int J Res Med Sci 2017;5:1679-82.
[Table 1], [Table 2], [Table 3], [Table 4]