|Year : 2020 | Volume
| Issue : 1 | Page : 28-32
Critical appraisal of different university assessment papers in pharmacology according to the three Rs criteria
Monica Jain, Kopal Sharma, Shivankan Kakkar
Department of Pharmacology, SMS Medical College, Jaipur, Rajasthan, India
|Date of Submission||12-Apr-2019|
|Date of Acceptance||23-Dec-2019|
|Date of Web Publication||27-Mar-2020|
Dr. Kopal Sharma
Department of Pharmacology, SMS Medical College, Jaipur, Rajasthan
Source of Support: None, Conflict of Interest: None
Background: In this study, we had attempted to evaluate pharmacology examination papers of various Indian medical universities according to the three Rs criteria (i.e., reduce, refine, and replace). This could provide an insight into the deficiencies in the present assessment pattern and aid in necessary improvements.
Materials and Methods: The past 10-year (2010–2019) university assessment papers for the second professional medical students in pharmacology subject from Rajasthan (Group A) were evaluated and compared with the same years' assessment papers from different states outside Rajasthan (Group B). Each question paper was scored. For “Reduce” criteria, a maximum score of 40 was given. Similarly, the maximum score for “Refine” category was 40. As “Replace” is an overlapping criterion, the paper could score a maximum of 20 scores only. A question paper could score a maximum of 100 score if it fulfills all the parameters for the given criterion. Thereafter based on the scores, grades were given to the papers.
Results: There was a statistically significant difference in the scores of both Group A and Group B (P = 0.037). The mean score for Group A was greater than the Group B. Papers in both the groups need improvement in terms of the three Rs criteria.
Conclusion: University question papers in pharmacology from Rajasthan universities as well as outside Rajasthan need improvement. It is only through the revisions in the pattern of assessment we would be able to train the students to achieve different competencies as per the new Medical Council of India curriculum.
The following core competencies are addressed in this article: Medical knowledge, Patient care, Interpersonal and communication skills, Systems-based practice.
Keywords: New Medical Council of India curriculum, pharmacology, three Rs criteria, university question papers
|How to cite this article:|
Jain M, Sharma K, Kakkar S. Critical appraisal of different university assessment papers in pharmacology according to the three Rs criteria. Int J Acad Med 2020;6:28-32
|How to cite this URL:|
Jain M, Sharma K, Kakkar S. Critical appraisal of different university assessment papers in pharmacology according to the three Rs criteria. Int J Acad Med [serial online] 2020 [cited 2020 Sep 26];6:28-32. Available from: http://www.ijam-web.org/text.asp?2020/6/1/28/281452
| Introduction|| |
We as facilitators of education have often pondered over the questions like what should be the approach to assessment. The answer can be simple as is the case with school education in India. It is because the school years focus on knowledge and knowledge evaluation only. The situation could be more challenging as is the case with a college education, which covers not only knowledge but also the practical domain of learning. More so, the answer becomes all together complicated in medical education which is not only science but also the art of applying knowledge in clinical settings.
Written assessments in a medical college, especially during pre- and paraclinical years, should ensure that whatever learned is translated into practice. Traditionally speaking, it was the Bloom's taxonomy that outlined the domains of learning. Bloom's taxonomy has three distinct domains of learning that are cognitive (knowledge), psychomotor (practical skills), and affective (attitudes). Here, it is important to note that competency in one domain does not automatically render the student master in its other domains. While designing an assessment, it is pertinent to include all the domains.
In education, we use the terms assessment and evaluation interchangeably, but there is a discernible difference between both the terms. Assessment is for student learning, and in contrast, evaluation is for an educational program, though assessment and evaluation both involve passing a judgment of learning.
In this study, we have evaluated the pharmacology university papers of II MBBS examination according to the three Rs criteria proposed by Manikandan and Gitanjali. Pharmacology has been an ever-evolving science with the introduction of new molecules to new devices, from the traditional system of medicine to modern medicine, and targeted drug delivery has been regarded as a miracle to humankind. There has been scholarly advancement in the education process as well. It is imperative for any medical undergraduate to understand the concepts of pharmacology as he/she will be the future prescriber of medicines. Here comes the dual responsibility, of assessors to train and assess the students and of the students to gain clinical and practical skills rather than cramming the subject by rote.
| Materials and Methods|| |
In this study, the past 10 years' (2010–2019) university assessment papers for the second professional medical students of the regular batch in pharmacology subject were evaluated. Only 10 years' papers were included as they were available easily from the universities.
Assessment papers from the Medical Council of India (MCI) recognized colleges from Rajasthan (Western part of India) and all other different geographical regions (Eastern, Northern, and Southern) of India. As the present study falls under exemption umbrella protocol, it had approval from the institutional review board.
All the papers were evaluated in terms of three Rs criteria as proposed by Manikandan and Gitanjali. Although the concept of three Rs was introduced by WMS Russell and RL Burch which was in context to animal experiments. However, Manikandan and Gitanjali reintroduced this concept in their research work for the assessment of pharmacology teaching and training of Indian medical graduate.
All the papers were scored by single researcher. As the researcher involved in scoring had affiliations with the Rajasthan University so to prevent bias in the analysis step, all the assessment papers were number tagged by a person not involved in the research. This resulted in hiding of the details such as university name and year from the assessment papers.
An arbitrary scoring system was designed to score all the papers. [Table 1] depicts the different criterions and parameters that were used to evaluate the assessment papers. Reduce and refine criterions had two parameters, while replace had only one parameter. If 25% of the assessment paper follows the given parameter of each of the three criterions, then a minimum score of 5 was given. If 26%–50% of the assessment paper follows the given parameter, then a minimum score of 10 was given. Similarly, for all of the three given criterions, a maximum score of 20 could be given to a paper if 76%–100% of the paper follows the given parameter. The total score was obtained by adding all the subscores for the three criterions.
|Table 1: Different criteria, parameters, and their scoring scheme for evaluating the question papers|
Click here to view
In this manner, an assessment paper could score a maximum score of 40 score for “Reduce” criteria. Similarly, the maximum score for “Refine” category was 40. As “Replace” is an overlapping criterion as what is reduced needs to be replaced so in “Replace” category, the paper could score a maximum of 20 scores only. In this manner, each assessment paper could score a maximum of 100 (40 + 40 + 20) score if it fulfills all the parameters for the three criterions [Table 1].
Based on the above score, each question paper was graded as described below. If the total score was:
- 75% and above; grade: exceptionally good
- 51%–74%; grade: acceptable
- ≤50%; grade: needs improvement.
After the completion of the scoring process, the number tags were removed, and for statistical evaluation, all the assessment papers were grouped into two categories: Group A: assessment papers from Rajasthan state and Group B: assessment papers from outside Rajasthan state.
Statistical analysis was conducted with IBM Statistical Package for the Social Sciences for Windows version 17.0 software (IBM Corp., Armonk, NY, USA). Unpaired Student's t-test was used to compare the means. For all the analysis, P < 0.05 was considered statistically significant.
| Results|| |
The papers from Rajasthan universities (Group A) had a mean score of 50.50 ± 9.26 in contrast to the assessment papers from universities outside Rajasthan (Group B) which had a mean score of 42.50 ± 6.35. The unpaired t-test revealed that there was a statistically significant difference (P = 0.037) in the mean scores of both Group A and Group B [Table 2]. Further analysis of the papers in both the groups unveiled that all the assessment papers in both the groups need further improvement in terms of the three Rs criteria [Table 3].
| Discussion|| |
With the implementation of a new curriculum in pharmacology as laid by the MCI, the pharmacology education has been reformed by the introduction of various innovations in the form of case-based learning, objective-structured practical examination, and integrated teaching to name a few in the list. Ideally, written assessments should be able to categorize students as pass or fail, rank students, measure their improvement over time, provide feedback to students as well as teachers, and to maintain the quality of the educational program. The best of settings would assess students reliably limiting the luck factor. A clear distinction shall be achieved between the one who is very studious/regular and the other who has just studied a month before the examination.
In this scenario, the assessment or the way the students are evaluated in terms of their learning further drives learning in the students. Hence, it becomes mandatory to evaluate the present assessment examination being undertaken by the students to judge whether the learning objectives are fulfilled appropriately or not.
The result of the analysis of question papers in the pharmacology of different universities revealed that the year-wise scores for the two groups based on different three Rs criterions were not satisfactory.
In the coverage of recall questions and direct hits under the reduce criteria, the papers could score only a maximum score of 15 which is appreciably a low score. Similar results are also mirrored in the previous study of Sindhu et al., depicting a higher number of recall questions in the papers. On the contrary, a previous study analyzing different universities' papers in Bangladesh revealed that the mean percentage of recall questions was low.
Versatility or the ability to adapt to the community health-care needs should be the key quality of a skilled medical practitioner. We as facilitators of education can impart and test this skill by problem-based clinical scenarios or case-based clinical questions. This will give them the ability to discern the actual pharmacotherapeutics. Case-based learning in pharmacology imparts critical reasoning expertise to the students so that they can perform better in clinics later on.
None of the universities' question paper scored well in refine criteria. Only eight papers had case-based or problem-based questions or questions involving rationality for prescribing a particular drug for a given condition. This result is congruent with another previous study delineating that the essential parts of the cognitive domain including recall, understanding, and cognitive domain were missing in parts or completely in almost all the papers analyzed.
With the meticulous integration of all the three Rs criteria, we can strive to achieve the ideal question papers in pharmacology which is the real challenge. These three Rs proposed by Manikandan et al. although are similar to animal ethics, each hold a unique place in designing assessment for the students that really address the needs of the new MCI curriculum (competency based) and also the learners. Although the papers from Rajasthan and outside universities could not score high on all the parameters, this study forms the concrete on which the model papers in pharmacology could be assessed for their accuracy and content validity.
Further studies are required to evaluate papers of other medical universities in India so that general conclusions can be drawn and future assessments for the students can be improved. Based on this study, it is recommended that all the written assessments should have a component of immediate recall, case-based questions, rationality for prescribing drugs for a disease, and multiple-choice questions so that they meet the criteria of accreditation, and at the same time, enhance learning. In addition, each question paper should also be provided with an answer key prior to judging the students so that the pattern of marking scheme remains unbiased for all the students. The strength of the study was that it provided insight into the assessment pattern of different geographically scattered colleges in India and also highlighted the deficiencies in them in terms of three Rs criteria. The limitation of our study was that assessment papers from countries other than India were not evaluated. Furthermore, we have evaluated the papers based on three Rs criteria of Manikandan, which were applicable in the Indian scenario that might not be apt in other countries' scenario.
| Conclusion|| |
Assessment should be meticulously planned to incorporate all the elements of learning and to meet the unmet needs so far. University assessment papers in pharmacology from Rajasthan Universities as well as outside Rajasthan need improvement. The future of health of any country resides in the hands budding clinicians. Effective training and prudent assessment of Indian medical graduates is the need of the hour.
Ethical conduct of research
Permission obtained from Ethics Committee of the institute (SMS Medical College, Jaipur). The authors declare that they followed applicable EQUATOR Network (http://www.equator-network.org/) research reporting guidelines.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Bloom BS, Krathwohl DR. Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. Handbook 1: Cognitive domain. New York: Longmans; 1956.
Linn RL, Miller MD. Measurement and Assessment in Teaching. 9th
ed. Upper Saddle River, NJ: Prentice Hall; 2005.
Manikandan S, Gitanjali B. The three Rs of written assessment: The JIPMER experience. J Pharmacol Pharmacother 2016;7:115-9.
] [Full text]
Badyal DK. Evolution of pharmacology education in India: Past and future. Indian J Pharmacol 2018;50:159-68. [Full text]
van dear Vleuten CP, Schuwirth LW. How to design a useful test: The principles of assessment. In: Swanwick T, editor. Understanding Medical Education: Evidence, Theory and Practice. 1st
ed. West Sussex: Wiley-Blackwell and ASME; 2010.
Verma M, Chhatwal J, Singh T. Reliability of essay type questions-effect of structuring. Assess Educ Princ Policy Pract 1997;4:265-70.
Buckwalter JA, Schumacher R, Albright JP, Cooper RR. Use of an educational taxonomy for evaluation of cognitive performance. J Med Educ 1981;56:115-21.
Sindhu S, Singh HK, Saiman MT, Verma VK. Comparative evaluation of pharmacology question papers in medical colleges affiliated to different universities in North India along with recommendation. J Res Med Educ Ethics 2011;1:180-6.
Chowdhury DK, Saha D, Talukder MH, Habib MA, Islam AS, Ahmad MR, et al
. Evaluation of pharmacology written question papers of MBBS professional examinations. Bangladesh J Med Educ 2017;8:12-7.
Ten Cate O, Regehr G. The power of subjectivity in the assessment of medical trainees. Acad Med 2019;94:333-7.
Camiah S. Current educational reforms in nursing in the United Kingdom and their impact on the role of nursing lecturers in practice: A case study approach. Nurse Educ Today 1998;18:368-79.
[Table 1], [Table 2], [Table 3]