Shakoori TA et al., Success of a journal

The Egyptian Journal of Medical Education

www.EJME.org 2019;1(1) eISSN 2090-2816

Original Article

How a Journal Club Course Changed Learners’ Approach in a Medical Doctorate Program- an Analysis of the PhD Scholars’ Perspective.

Tania Ahmed Shakoori* , Zeeshan Sarwar , Muhammad Najam'u Saqib , Muhammad Saqib Saeed , Usman Mahboob , Syed Asghar Naqi , Ian Willis , Janet Strivens

Department of Biomedical Sciences, King Edward Medical University, Lahore, Pakistan.

Department of Surgery, King Edward Medical University, Lahore, Pakistan.

Department of Pulmonology, King Edward Medical University, Lahore, Pakistan.

Department of Medical Education, Khyber Medical University, Peshawar, Pakistan.

Quality Enhancement Cell, King Edward Medical University, Lahore, Pakistan.

Center for Higher Education Studies, University of Liverpool, UK.

1,5

2,5

ABSTRACT

Background/Aim: A structured journal club course that counts towards the final assessment was introduced in a medical doctorate program course at King Edward Medical University, Pakistan. This study aims to explore its effects on the learning approach of PhD scholars. Methods: A qualitative study comprising of a focus group discussion, involving eight out of the total of thirteen PhD scholars was conducted in 2017. Focus group discussions were transcribed verbatim and codes and themes were identified using grounded theory. The findings were triangulated with data from teaching feedback forms of the course from all the thirteen PhD scholars from 2015 to 2017. Results: The participants described an overall positive effect of the course on their learning, including development of critical and scientific thinking. They also reported improvement in their clinical and teaching practice. They attributed these to the interactive nature of the course, role playing as author while presenting the assigned paper in journal club, help from the study guide, interdisciplinary peer learning and the awareness that they were being graded on their performance. Conclusions: A semester long structured journal club course that contributes to the final assessment may be an effective tool to teach medical doctorate students about the research process and critical appraisal skills.

Article Info

https://doi.org/10.33328/ejme.2019.002

*Corresponding author: +923454303059; Dept. of Biomedical Sciences, King Edward Medical University, Nelagumbad, Lahore 54000, Pakistan; drtaniashakoori@yahoo.com, drtaniashakoori@kemu.edu.pk

Received : 16 December, 2018

Accepted: 24 December 2018

Published : 9 January 2019

Keywords : learning, thinking, qualitative research, focus group discussion.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

http://creativecommons.org/licenses/by/4.0/

INTRODUCTION

Previous studies report the benefits of running structured regular journal clubs [1-5]. However, in most cases formats are not standardized [6] and there are very few studies reporting a detailed design of their journal clubs [2, 5, 7]. Furthermore there is no reported case of a formal journal club course, as part of a medical PhD curriculum, that counts towards the final assessment system. Such a course was introduced as part of the ‘revamped’ curriculum of ‘Medical Sciences PhD Program’ at King Edward Medical University (KEMU), Lahore, Pakistan, in 2015.

Even though research methodology and critical appraisal workshops are offered to medical doctors in Pakistan as part of their post graduate trainings, these are limited in duration and the awarding of completion certificates is only dependent on attendance and not assessment. This journal club course was introduced in the PhD curriculum to train the scholars in critical appraisal and the clinical research process. Its effects were analyzed after 2 years of successful running of the course.

METHODS

Ethical approval was obtained from the Institutional Review Board of KEMU (No 435/RC/KEMU). In December 2015 PhD classes commenced at KEMU under a ‘revamped curriculum’. A qualitative study was designed to determine effects of a ’journal club course’ in this curriculum on students’ approach to learning.

At the time of the preparation of this manuscript, this course was successfully completed by two batches of medical PhD scholars between 2015 and 2017 (six women and seven men). It is pertinent to note that these scholars included clinical as well as medical basic sciences’ professionals and had training in medical education as most had completed certificate courses in medical teaching or masters in medical education.

Amongst the salient features of the program was a 2.5 credit hours’ worth ‘Journal Club Course’. This course was composed of small group discussions on critical evaluation of published research articles, much like a regular journal club. However, the focus was not on keeping up to date with the latest medical advances but on learning critical appraisal skills, research methodology, statistical analysis and academic writing. Furthermore the course differed from a regular clinical journal club in that the student performance was evaluated on the basis of a presentation they gave on their assigned research paper and a reflective assignment. The marks obtained by the student were given weightage in the final assessment of the PhD program.

Each article was discussed as group and evaluated with help of standard checklists in class. One selected student prepared a power point presentation of the research article as if he/she was the principal author himself/herself, and received oral and written feedback from the attendees [figure 1] Finally the presenting student submitted a written reflective assignment using the given checklist [figure 2] as a guide. The assignment was graded, and detailed written feedback was provided to the student.

Eight scholars were able to participate in the Focus Group Discussion (FGD). A grounded theory approach was used for analysis of transcribed data. Quotes from all the transcripts of the FGD were regrouped as indicators for the identified themes in a tabulated form for analysis. These data were triangulated with the written feedback that had been collected anonymously from all students after each semester ended [figures 3, 4].

RESULTS

A total of 60 codes were generated in the 1st cycle of coding, and 14 categories in the 2nd cycle to identify the patterns in the data. Five main themes were identified [table 1]. These included ‘effect on learning’, ’effect on practice’, ‘role of assessment’, ‘role of written assignment’ and ‘role of study guide’. The theme ‘effect on learning’ comprised of categories ‘inspired deep learning’, ‘critical thinking’, ‘hands-on learning’, ‘better understanding of research’, ‘peer learning’ and ‘lateral Integration’. The theme ‘Effect on practice’ included the categories: effect on teaching practice; effect on clinical practice; improved presentation skills; improved publication track and change in thought processes.

1- Effect on learning

The participants described how the course inspired deep learning. The majority of the interviewees reported that they previously only scan-read research articles but now attempted to critically appraise them, noticed details and tried to interpret charts and graphs.

“We used to read the abstract and conclusion of the articles only... Journal club has equipped us to explore the mind of the authors. We can now analyze the flaws and gaps in his mind and see how we can fill these.”

One participant however mentioned how he still does not read the entire article deeply, due to lack of time. “If I think of my practices and attitudes about reading journal articles before and now, I see no change. The reason is that I don’t have much time.”

The interviewees also commented on a change in their thought process and how they viewed knowledge. They described how they realized that textbooks cannot be considered exclusive and irrefutable sources of knowledge, that evidence comes from research articles and even that can only be accepted after critical evaluation.

“Previously, we used to study books first then see the patients. Now we have this new understanding that diseases don’t study books. Exceptions are there and we need to see them and keep our eyes and minds open”.

One participant described development of scientific/ critical thinking through an anecdote:

“. .. I advised my colleague who was prescribing the new drug, not to blindly follow American guidelines regarding this drug. I suggested that we design a study and see if our local patients treated with the drug suffered from decompensated liver failure .. .and only then decide if we should prescribe this medication.”

Two participants were however skeptical about the perception that the critical appraisal exercises of journal club had a role in the development of this scientific thinking. One argued that this improvement was part of the natural evolution of a clinician as they do through their training and gain more experience.

“With every passing moment, our acumen to treat the patient improves. It’s a natural phenomenon. This could be the reason of change in our thought process as well. How can you know if this improvement in thought process occurred due to clinical practice or critical appraisal exercises in journal club?”

The PhD scholars also talked about how they have a better understanding of the research process after attending the journal club course, even though they had attended many didactic lectures and workshops on research methodology as part of their respective post graduate trainings.

“We got a better understanding of study designs. ..plus I got a better understanding of biostatistics through this journal club.”

They attributed the clarity of concepts gained from the course to the format which required them to defend the research in presentations as if they were the authors.

“The role given to us was that we are authors. That made a difference. We had to defend it (the research paper) and we had to own it. It was presented as our own project; that was the major difference.”

The participants discussed in detail how going through actual research articles gave them better understanding of concepts like study design and statistical tests as compared to lectures on the same topics.

“The more exposure we had to journal club the more clarity we got about research. Instead of spending our time on theoretical discussion of ‘how to conduct a case-control study’, we practically applied it in journal club and studied different scenarios.”

The participants appreciated the fact that there was a successful collaborative learning atmosphere despite the fact that the students ranged from senior professors to junior assistant professors and belonged to different specialties.

“For PhD, I actually prefer an interdisciplinary class. The good thing about all participants was that they took off their caps (of professorial cadre) and participated in the classroom as colleagues.”

2- Effect on Practice

All the PhD scholars in the cohort were involved in classroom and bedside teaching. They commented on how they had implemented some of the teaching techniques they experienced as students of the course into their own respective teaching practices. Those who did not have a journal club in their wards previously felt motivated to start one or had already started one.

“In our clinical wards, we didn’t have a journal club due to other commitments, but now we have added this to our teaching program.”

Some of participants were already running Journal Clubs in their department. They described how they had improved the format after exposure to this course. “..We have made it interactive now.” “Yes, there was a journal club before … but it was just for the sake of doing it. .. Now, ..focus is also on study design, no. of participants, its flaws etc.”

Most of the enrolled clinical scholars were active clinical researchers. They mentioned how defending the research papers of other authors during the journal club trained them in better conducting and presenting their own work at different clinical forums and conferences.

“We learnt from our journal club presentations, how to present in a conference in the form of power point slides and how to present in front of a learned audience.”

Figure 1: Checklist used to grade the Students research paper presentation.

3- Role of study guide

The participants talked about how they referred to the study guide while preparing their journal presentations and assignments and even when teaching the research process to their own trainees. “The study guide given to us at the start was like a measuring tool and using it we were able to organize our thoughts in better way.”

4- Role of written assignment

There was difference in agreement with regards to the importance of the written assignment. Most thought it added to their learning. “While writing the assignment, I read the paper in greater depth... So this written assignment was complementary to the oral presentation and the class discussion in enhancing my understanding.” One participant had a contrasting opinion about the utility of the assignment. “For me the written assignment was not very useful because for me it was simply writing down what I learnt here… I would rather not do it.”

Figure 2: A page from the study guide for the journal club course showing assessment criteria for the written assignment.

5- Role of assessment

The participants also commented that they concentrated more and worked harder because it was part of their summative assessment. “… We had to present like an author and we were being assessed on it and were also defending it that’s why we studied in depth.”

Results of Surveys

As can be seen in figure 4, the students’ response was overwhelmingly positive. The feedback for all questions asked in survey was either ‘excellent’ or ‘good’.

Figure 3: Evaluation of teaching in Journal Club course by the students.

DISCUSSION

It has been suggested that a regular, structured journal club with mandatory attendance best achieves its learning objectives [1, 4, 8, 9]. These features were a part of our journal club course and likely contributed to its success. In addition to adopting the best practices suggested in literature for running a successful journal club, the course contributed to the final assessment of the PhD program. There is substantial evidence that assessment and testing enhances memory retention and factual recall which is independent of the act of repeatedly going over the material as part of test preparation [10]. It has further been demonstrated that learning motivation of students increases as the weightage of the subject within the assessment system increases [11]. We argue that testing can also facilitate deep learning and reflection if the system of assessment was designed accordingly. The assessment of the journal club course was based largely on a reflective assignment which was graded on the basis of criteria that had already been provided to the students in their ‘study guide’. The criteria gave more weight to evidence of reflection [figure 2]. Furthermore the course had significant weight (11.1%) in the calculation of the final Cumulative Grade Point Average (CGPA) of the PhD scholars. The PhD scholars were very aware of this fact and this added to their motivation in learning. They ‘were being assessed on it’ so they ‘studied in depth’. Our findings thus further validate the concept that if the system rewards deep learning, critical thinking and reasoning, then the successful learner will aim for that [12, 13] .

Table 1: Themes identified along with their categories (where applicable) and codes

One feature of the journal club the students appreciated was presenting the papers as authors. They described how ‘putting themselves in the researcher’s shoes’ not only motivated them to study the article deeply, but also encouraged them to consider all the practical and logistic aspects of the research project as well. Thus this role-playing as the ‘author presenting his/her work’ forced them to look at the research and its clinical application from all angles. This is in line with published research which describes how role playing can help students make clearer connections between different concepts, and give them the ability to apply that information in different contexts [14]. Thus we saw many of the students from the cohort make research proposals, based on the questions that were raised during the discussion of their assigned journal club papers in class. We believe that the kind of reflection, which role-playing as an author triggered, could not have happened otherwise.

The format of the course complements the five stages of learning proposed by Taylor and Hamdy in their multi-theories model of adult learning [15]. According to this model learning begins with a ‘dissonance phase’ when the ‘learner’s existing knowledge is challenged and found to be incomplete’. During the ‘refinement phase’, the learner refines the new information into a series of new concepts. This is followed by the ‘organization phase’ where the learner restructures their ideas into schemata which make sense. ‘Reflection in action’ is key in this stage. We feel that the interactive discussion of the journal club, facilitated by an approachable facilitator in a comfortable learning environment, provided fertile ground for the learners to pass through the learning phases of ‘dissonance’, ‘refinement’ and ‘organization’. There was adequate time to discuss and reflect on the concepts during discussions. Furthermore the checklists provided in the study guide were frequently referred to during discussions and acted as ‘scaffolding’ to support formation of ideas in the ‘organization phase’ of learning. The ‘feedback phase’ corresponded to the feedback given in the presentation and written assignment stages of the journal club course.

Finally as the presenter prepared his/her reflective assignment, he/she continued with further organization of his/her thoughts based on feedback he/she received during the presentation stage and further reflected, all the time motivated by a desire to get good grades. Arguably these stages were most intense for the learner who was assigned the article and was being assessed, however the learning experience of the rest of the class cannot be underestimated. Knowing that they would go through the same cycle when their turn came, the rest of learners were highly motivated and engaged in discussions in all stages of each session. The last phase of the learning cycle or ‘consolidation’ inevitably continued as each student witnessed the next learner make his/her presentation and received feedback on it. It is also reasonable to extrapolate that this ‘reflection on action’ continued as the PhD scholar started working on the research synopsis for his/her PhD thesis and also as they guided their own research students as supervisors.

There is some evidence from the data that the course challenged if not changed some of the students’ epistemological beliefs related to medical evidence and research. This was an unexpected finding as it was not one of the original objectives when the course was first designed and executed. As an example, the students described how they changed their views on books as irrefutable sources of knowledge. According to literature, restricting to textbooks as the only sources of learning may stifle the development of scientific thinking and the critical appraisal skills [16]. It is interesting how when this cohort developed these skills they arrived at this same conclusion. The students narrated incidents from their previous educational experiences in post graduate training where they were discouraged from questioning in general and referring to anything beyond their medical textbooks. One participant captured this traditional learning atmosphere with the following verses of a famous poem by poet and philosopher Dr. Muhammad Iqbal “In their hearts no desire (to reflect) ever arises; when it does by chance, it dies young” [17].

Figure 4: Feedback of PhD Batches 2015-2018 and 2016-2019 on Teaching in Journal Club Course

The students further commented how they now truly understand that ‘diseases don’t read books’ so they have to think of exceptions when making diagnoses. One student commented that after learning to critically evaluate research articles she understood that she could not ‘trust journal articles blindly’ or apply the findings to her patients automatically. The participants described how they try to pass this belief on to their own trainees by saying to them “do not follow books. Books are only guidelines. If your minds are working, explore new things.’ Previous studies have shown that educational interventions may lead to domain specific epistemic changes in university students from diverse disciplines [18-20]. It was interesting to see the same phenomenon in our cohort of full time clinical professionals enrolled in a PhD program. With such changes in learning behavior and thinking process, change in practice was inevitable. The students described how they had self reportedly improved as clinical supervisors and even clinicians. They reported how they now reviewed research proposals, papers and theses more critically and confidently. They also reported replicating the journal club in the teaching program of their clinical trainees. Some of the participants (physicians) in the cohort reported improved clinical reasoning and attributed it to the training in critical appraisal and scientific thinking received in the course. Two surgeons disagreed and proposed that the improved clinical reasoning and application of scientific thought to patent care may be part of natural evolution of a practicing clinician. The difference in the two perceptions may be attributed to factors related to the individuals learning styles and factors specific to their specializations.

CONCLUSIONS

A semester long structured journal club course that contributed to the final assessment may be an effective tool to teach medical doctorate students about the research process and critical appraisal skills. In light of our findings it is recommended that such a course should made an essential part of PhD curriculum for doctors.

ACKNOWLEDGEMENT

Prof. Dr. Shakila Zaman whose teaching style inspired the format of the course.

Prof Dr. Faisal Masud, Ex VC KEMU for supporting development of this non-traditional course.

DISCLAIMER

The article has not been previously presented or published, and is not part of a PhD or thesis project.

CONFLICT OF INTEREST

There are no financial, personal, or professional conflicts of interest to declare.

REFERENCES

Hohmann E, Tetsworth K. Teaching residents: critical appraisal of the literature using a journal club format. Postgrad Med J. 2016; 92 (1093). doi:10.1136/postgradmedj-2015-133921
Marrone KA, Emens LA, Azad NS, Grossman SA. A Novel Approach to Journal Club Designed to Immerse Trainees in a Peer-Review Process to Critically Evaluate Oncology Manuscripts. Med Sci Educ. 2016;26(3):273-4. https://doi.org/10.1007/s40670-016-0276-1
Alavi-Moghaddam M, Yazdani S, Mortazavi F, Chichi S, Hosseini-Zijoud S-M. Evidence-based Medicine versus the Conventional Approach to Journal Club Sessions: Which One Is More Successful in Teaching Critical Appraisal Skills? Chonnam Med J. 2016; 52(2):107-11. doi: 10.4068/cmj.2016.52.2.107
Wilson OJ, Deighton K. A Semester-long Journal Club Improves Self-perceived Critical Appraisal Skills in Undergraduate Sport and Exercise Science Students. Delivering Excellence in Higher Education 2016.
Johnson A, Thornby K-A, Ferrill M. Critical Appraisal of Biomedical Literature with a Succinct Journal Club Template: The ROOTs Format. Hosp Pharm. 2017; 52(7):488-95. https://doi.org/10.1177/0018578717721104
Ahmadi N, McKenzie ME, MacLean A, Brown CJ, Mastracci T, McLeod RS. Teaching evidence based medicine to surgery residents-is journal club the best format? A systematic review of the literature. J Surg Educ. 2012; 69(1):91-100. https://doi.org/10.1016/j.jsurg.2011.07.004
Kovacevic MP, Baroletti S, Galal SM, Szumita PM. Journal Club Standardization Tool: Helping participants get the JIST. Curr Pharm Teach Learn. 2017. https://doi.org/10.1016/j.cptl.2017.11.012
Deenadayalan Y, Grimmer‐Somers K, Prior M, Kumar S. How to run an effective journal club: a systematic review. J Eval Clin Pract. 2008; 14(5):898-911. doi: 10.1111/j.1365-2753.2008.01050.x
Aronson JK. Journal Clubs: 2. Why and how to run them and how to publish them. BMJ Evid Based Med. 2017; 22(6):232-4. http://dx.doi.org/10.1136/ebmed-2017-110861
Ebbert JO, Montori VM, Schultz HJ. The journal club in postgraduate medical education: a systematic review. ‎Med Teach. 2001; 23(5):455-61. https://doi.org/10.1080/01421590120075670
Larsen DP, Butler AC, Roediger III HL. Test-enhanced learning in medical education. Med Educ. 2008; 42(10):959-66. https://doi.org/10.1111/j.1365-2923.2008.03124.x
Wormald BW, Schoeman S, Somasunderam A, Penn M. Assessment drives learning: an unavoidable truth? Anat Sci Educ. 2009; 2(5):199-204.
Biggs J, Kember D, Leung DY. The revised two-factor study process questionnaire: R-SPQ-2F. ‎Br J Educ Psychol. 2001; 71(1):133-49. https://doi.org/10.1348/000709901158433
Pavey J, Donoghue D. The use of role play and VLEs in teaching Environmental Management. Planet. 2003; 10(1):7-10. https://doi.org/10.11120/plan.2003.00100007
Taylor DC, Hamdy H. Adult learning theories: Implications for learning and teaching in medical education: AMEE Guide No. 83. ‎Med Teach. 2013; 35(11):e1561-e72. doi:10.3109/0142159X.2013.828153
Duncan DB, Lubman A, Hoskins SG. Introductory biology textbooks under-represent scientific process. J Microbiol Biol Edu: JMBE. 2011;12(2):143. doi:10.1128/jmbe.v12i2.307
Iqbal M. Kulyat-e-Iqbal: Urdu [Iqbal’s Collected Works: Urdu]. Lahore, Pakistan: Mahmood Siddiqullah; 1972.
Valanides N, Angeli C. Effects of instruction on changes in epistemological beliefs. Contemp Educ Psychol. 2005; 30(3):314-30. https://doi.org/10.1016/j.cedpsych.2005.01.001
Kienhues D, Bromme R, Stahl E. Changing epistemological beliefs: The unexpected impact of a short‐term intervention. ‎Br J Educ Psychol. 2008; 78(4):545-65. doi:10.1348/000709907X268589
Muis KR, Duffy MC. Epistemic climate and epistemic change: Instruction designed to change students' beliefs and learning strategies and improve achievement. Journal of Educational Psychology. 2013; 105(1):213. doi:10.1037/a0029690