Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 
  • Users Online: 887
  • Home
  • Print this page
  • Email this page


 
 Table of Contents  
ORIGINAL ARTICLE
Year : 2020  |  Volume : 6  |  Issue : 1  |  Page : 28-32

Critical appraisal of different university assessment papers in pharmacology according to the three Rs criteria


Department of Pharmacology, SMS Medical College, Jaipur, Rajasthan, India

Date of Submission12-Apr-2019
Date of Acceptance23-Dec-2019
Date of Web Publication27-Mar-2020

Correspondence Address:
Dr. Kopal Sharma
Department of Pharmacology, SMS Medical College, Jaipur, Rajasthan
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/IJAM.IJAM_21_19

Rights and Permissions
  Abstract 

Background: In this study, we had attempted to evaluate pharmacology examination papers of various Indian medical universities according to the three Rs criteria (i.e., reduce, refine, and replace). This could provide an insight into the deficiencies in the present assessment pattern and aid in necessary improvements.
Materials and Methods: The past 10-year (2010–2019) university assessment papers for the second professional medical students in pharmacology subject from Rajasthan (Group A) were evaluated and compared with the same years' assessment papers from different states outside Rajasthan (Group B). Each question paper was scored. For “Reduce” criteria, a maximum score of 40 was given. Similarly, the maximum score for “Refine” category was 40. As “Replace” is an overlapping criterion, the paper could score a maximum of 20 scores only. A question paper could score a maximum of 100 score if it fulfills all the parameters for the given criterion. Thereafter based on the scores, grades were given to the papers.
Results: There was a statistically significant difference in the scores of both Group A and Group B (P = 0.037). The mean score for Group A was greater than the Group B. Papers in both the groups need improvement in terms of the three Rs criteria.
Conclusion: University question papers in pharmacology from Rajasthan universities as well as outside Rajasthan need improvement. It is only through the revisions in the pattern of assessment we would be able to train the students to achieve different competencies as per the new Medical Council of India curriculum.
The following core competencies are addressed in this article: Medical knowledge, Patient care, Interpersonal and communication skills, Systems-based practice.

Keywords: New Medical Council of India curriculum, pharmacology, three Rs criteria, university question papers


How to cite this article:
Jain M, Sharma K, Kakkar S. Critical appraisal of different university assessment papers in pharmacology according to the three Rs criteria. Int J Acad Med 2020;6:28-32

How to cite this URL:
Jain M, Sharma K, Kakkar S. Critical appraisal of different university assessment papers in pharmacology according to the three Rs criteria. Int J Acad Med [serial online] 2020 [cited 2020 May 25];6:28-32. Available from: http://www.ijam-web.org/text.asp?2020/6/1/28/281452


  Introduction Top


We as facilitators of education have often pondered over the questions like what should be the approach to assessment. The answer can be simple as is the case with school education in India. It is because the school years focus on knowledge and knowledge evaluation only. The situation could be more challenging as is the case with a college education, which covers not only knowledge but also the practical domain of learning. More so, the answer becomes all together complicated in medical education which is not only science but also the art of applying knowledge in clinical settings.

Written assessments in a medical college, especially during pre- and paraclinical years, should ensure that whatever learned is translated into practice. Traditionally speaking, it was the Bloom's taxonomy that outlined the domains of learning. Bloom's taxonomy has three distinct domains of learning that are cognitive (knowledge), psychomotor (practical skills), and affective (attitudes). Here, it is important to note that competency in one domain does not automatically render the student master in its other domains. While designing an assessment, it is pertinent to include all the domains.[1]

In education, we use the terms assessment and evaluation interchangeably, but there is a discernible difference between both the terms. Assessment is for student learning, and in contrast, evaluation is for an educational program,[2] though assessment and evaluation both involve passing a judgment of learning.

In this study, we have evaluated the pharmacology university papers of II MBBS examination according to the three Rs criteria proposed by Manikandan and Gitanjali.[3] Pharmacology has been an ever-evolving science with the introduction of new molecules to new devices, from the traditional system of medicine to modern medicine, and targeted drug delivery has been regarded as a miracle to humankind.[4] There has been scholarly advancement in the education process as well. It is imperative for any medical undergraduate to understand the concepts of pharmacology as he/she will be the future prescriber of medicines. Here comes the dual responsibility, of assessors to train and assess the students and of the students to gain clinical and practical skills rather than cramming the subject by rote.


  Materials and Methods Top


In this study, the past 10 years' (2010–2019) university assessment papers for the second professional medical students of the regular batch in pharmacology subject were evaluated. Only 10 years' papers were included as they were available easily from the universities.

Inclusion criteria

Assessment papers from the Medical Council of India (MCI) recognized colleges from Rajasthan (Western part of India) and all other different geographical regions (Eastern, Northern, and Southern) of India. As the present study falls under exemption umbrella protocol, it had approval from the institutional review board.

All the papers were evaluated in terms of three Rs criteria as proposed by Manikandan and Gitanjali.[3] Although the concept of three Rs was introduced by WMS Russell and RL Burch which was in context to animal experiments. However, Manikandan and Gitanjali reintroduced this concept in their research work for the assessment of pharmacology teaching and training of Indian medical graduate.[3]

Scoring

All the papers were scored by single researcher. As the researcher involved in scoring had affiliations with the Rajasthan University so to prevent bias in the analysis step, all the assessment papers were number tagged by a person not involved in the research. This resulted in hiding of the details such as university name and year from the assessment papers.

An arbitrary scoring system was designed to score all the papers. [Table 1] depicts the different criterions and parameters that were used to evaluate the assessment papers. Reduce and refine criterions had two parameters, while replace had only one parameter. If 25% of the assessment paper follows the given parameter of each of the three criterions, then a minimum score of 5 was given. If 26%–50% of the assessment paper follows the given parameter, then a minimum score of 10 was given. Similarly, for all of the three given criterions, a maximum score of 20 could be given to a paper if 76%–100% of the paper follows the given parameter. The total score was obtained by adding all the subscores for the three criterions.
Table 1: Different criteria, parameters, and their scoring scheme for evaluating the question papers

Click here to view


In this manner, an assessment paper could score a maximum score of 40 score for “Reduce” criteria. Similarly, the maximum score for “Refine” category was 40. As “Replace” is an overlapping criterion as what is reduced needs to be replaced so in “Replace” category, the paper could score a maximum of 20 scores only. In this manner, each assessment paper could score a maximum of 100 (40 + 40 + 20) score if it fulfills all the parameters for the three criterions [Table 1].

Grading

Based on the above score, each question paper was graded as described below. If the total score was:

  • 75% and above; grade: exceptionally good
  • 51%–74%; grade: acceptable
  • ≤50%; grade: needs improvement.


After the completion of the scoring process, the number tags were removed, and for statistical evaluation, all the assessment papers were grouped into two categories: Group A: assessment papers from Rajasthan state and Group B: assessment papers from outside Rajasthan state.

Statistical analysis

Statistical analysis was conducted with IBM Statistical Package for the Social Sciences for Windows version 17.0 software (IBM Corp., Armonk, NY, USA). Unpaired Student's t-test was used to compare the means. For all the analysis, P < 0.05 was considered statistically significant.


  Results Top


The papers from Rajasthan universities (Group A) had a mean score of 50.50 ± 9.26 in contrast to the assessment papers from universities outside Rajasthan (Group B) which had a mean score of 42.50 ± 6.35. The unpaired t-test revealed that there was a statistically significant difference (P = 0.037) in the mean scores of both Group A and Group B [Table 2]. Further analysis of the papers in both the groups unveiled that all the assessment papers in both the groups need further improvement in terms of the three Rs criteria [Table 3].
Table 2: Year-wise scores of questions papers in two groups

Click here to view
Table 3: Grading of the papers according to the mean scores

Click here to view



  Discussion Top


With the implementation of a new curriculum in pharmacology as laid by the MCI, the pharmacology education has been reformed by the introduction of various innovations in the form of case-based learning, objective-structured practical examination, and integrated teaching to name a few in the list.[4] Ideally, written assessments should be able to categorize students as pass or fail, rank students, measure their improvement over time, provide feedback to students as well as teachers, and to maintain the quality of the educational program.[5] The best of settings would assess students reliably limiting the luck factor. A clear distinction shall be achieved between the one who is very studious/regular and the other who has just studied a month before the examination.[6]

In this scenario, the assessment or the way the students are evaluated in terms of their learning further drives learning in the students. Hence, it becomes mandatory to evaluate the present assessment examination being undertaken by the students to judge whether the learning objectives are fulfilled appropriately or not.[7]

The result of the analysis of question papers in the pharmacology of different universities revealed that the year-wise scores for the two groups based on different three Rs criterions were not satisfactory.

In the coverage of recall questions and direct hits under the reduce criteria, the papers could score only a maximum score of 15 which is appreciably a low score. Similar results are also mirrored in the previous study of Sindhu et al., depicting a higher number of recall questions in the papers.[8] On the contrary, a previous study analyzing different universities' papers in Bangladesh revealed that the mean percentage of recall questions was low.[9]

Versatility or the ability to adapt to the community health-care needs should be the key quality of a skilled medical practitioner. We as facilitators of education can impart and test this skill by problem-based clinical scenarios or case-based clinical questions. This will give them the ability to discern the actual pharmacotherapeutics.[10] Case-based learning in pharmacology imparts critical reasoning expertise to the students so that they can perform better in clinics later on.[11]

None of the universities' question paper scored well in refine criteria. Only eight papers had case-based or problem-based questions or questions involving rationality for prescribing a particular drug for a given condition. This result is congruent with another previous study delineating that the essential parts of the cognitive domain including recall, understanding, and cognitive domain were missing in parts or completely in almost all the papers analyzed.[9]

With the meticulous integration of all the three Rs criteria, we can strive to achieve the ideal question papers in pharmacology which is the real challenge. These three Rs proposed by Manikandan et al. although are similar to animal ethics, each hold a unique place in designing assessment for the students that really address the needs of the new MCI curriculum (competency based) and also the learners. Although the papers from Rajasthan and outside universities could not score high on all the parameters, this study forms the concrete on which the model papers in pharmacology could be assessed for their accuracy and content validity.

Further studies are required to evaluate papers of other medical universities in India so that general conclusions can be drawn and future assessments for the students can be improved. Based on this study, it is recommended that all the written assessments should have a component of immediate recall, case-based questions, rationality for prescribing drugs for a disease, and multiple-choice questions so that they meet the criteria of accreditation, and at the same time, enhance learning. In addition, each question paper should also be provided with an answer key prior to judging the students so that the pattern of marking scheme remains unbiased for all the students. The strength of the study was that it provided insight into the assessment pattern of different geographically scattered colleges in India and also highlighted the deficiencies in them in terms of three Rs criteria. The limitation of our study was that assessment papers from countries other than India were not evaluated. Furthermore, we have evaluated the papers based on three Rs criteria of Manikandan, which were applicable in the Indian scenario that might not be apt in other countries' scenario.


  Conclusion Top


Assessment should be meticulously planned to incorporate all the elements of learning and to meet the unmet needs so far. University assessment papers in pharmacology from Rajasthan Universities as well as outside Rajasthan need improvement. The future of health of any country resides in the hands budding clinicians. Effective training and prudent assessment of Indian medical graduates is the need of the hour.

Ethical conduct of research

Permission obtained from Ethics Committee of the institute (SMS Medical College, Jaipur). The authors declare that they followed applicable EQUATOR Network (http://www.equator-network.org/) research reporting guidelines.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Bloom BS, Krathwohl DR. Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. Handbook 1: Cognitive domain. New York: Longmans; 1956.  Back to cited text no. 1
    
2.
Linn RL, Miller MD. Measurement and Assessment in Teaching. 9th ed. Upper Saddle River, NJ: Prentice Hall; 2005.  Back to cited text no. 2
    
3.
Manikandan S, Gitanjali B. The three Rs of written assessment: The JIPMER experience. J Pharmacol Pharmacother 2016;7:115-9.  Back to cited text no. 3
[PUBMED]  [Full text]  
4.
Badyal DK. Evolution of pharmacology education in India: Past and future. Indian J Pharmacol 2018;50:159-68.  Back to cited text no. 4
  [Full text]  
5.
van dear Vleuten CP, Schuwirth LW. How to design a useful test: The principles of assessment. In: Swanwick T, editor. Understanding Medical Education: Evidence, Theory and Practice. 1st ed. West Sussex: Wiley-Blackwell and ASME; 2010.  Back to cited text no. 5
    
6.
Verma M, Chhatwal J, Singh T. Reliability of essay type questions-effect of structuring. Assess Educ Princ Policy Pract 1997;4:265-70.  Back to cited text no. 6
    
7.
Buckwalter JA, Schumacher R, Albright JP, Cooper RR. Use of an educational taxonomy for evaluation of cognitive performance. J Med Educ 1981;56:115-21.  Back to cited text no. 7
    
8.
Sindhu S, Singh HK, Saiman MT, Verma VK. Comparative evaluation of pharmacology question papers in medical colleges affiliated to different universities in North India along with recommendation. J Res Med Educ Ethics 2011;1:180-6.  Back to cited text no. 8
    
9.
Chowdhury DK, Saha D, Talukder MH, Habib MA, Islam AS, Ahmad MR, et al. Evaluation of pharmacology written question papers of MBBS professional examinations. Bangladesh J Med Educ 2017;8:12-7.  Back to cited text no. 9
    
10.
Ten Cate O, Regehr G. The power of subjectivity in the assessment of medical trainees. Acad Med 2019;94:333-7.  Back to cited text no. 10
    
11.
Camiah S. Current educational reforms in nursing in the United Kingdom and their impact on the role of nursing lecturers in practice: A case study approach. Nurse Educ Today 1998;18:368-79.  Back to cited text no. 11
    



 
 
    Tables

  [Table 1], [Table 2], [Table 3]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
Materials and Me...
Results
Discussion
Conclusion
References
Article Tables

 Article Access Statistics
    Viewed413    
    Printed28    
    Emailed0    
    PDF Downloaded0    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]