Clinician engagement in ongoing health service quality improvement (QI) is an important component of improving clinical outcomes. Integrating QI in undergraduate and postgraduate medical education to equip clinicians with the knowledge and skills to improve care is increasingly acknowledged as important. A number of international medical accreditation bodies now include QI as a core competency, necessitating medical education programmes to develop authentic curricula in this area.1–4 The New Zealand and Australian Curriculum Framework for Junior Doctors includes clinical audit as a core element of all pre-vocational training.5,6 Like many other professional colleges, clinical audit is also a mandatory requirement set out by the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG).7

A growing number of universities worldwide are offering QI learning opportunities to medical learners, although there is significant heterogeneity in educational content, teaching methods, students targeted and learning outcomes.8 Undergraduate curricula vary from a few lectures (the majority involving fewer than 10 contact hours and often consisting of a single session) to the undertaking of a QI project.9 Postgraduate curricula targeting junior doctors often involve multiple encounters within existing core rotations, standalone sessions or elective rotations with participation in QI projects.8,10 The primary objective in most QI curricula is learning the principles and methods of QI, with common educational content including QI science and skills, safety incident analysis and systems thinking.8 Assessments of QI curricula have reported improvements in learner’s knowledge and self-efficacy, with lesser emphasis on changes in behaviour and clinical process.8 Incorporating experiential learning that is fully integrated into day-to-day clinical work has been suggested as a way to improve generalisability to future practice settings and lead to change in clinical practice.11

To date, the most promising form of experiential learning in undergraduate QI curricula combines classroom learning with practical projects.12 The QI projects commonly include elements of root-cause or safety incident analysis, a chart audit and reflective practice.13 Students are assigned to interprofessional QI teams14,15 or to community settings,16-18 or undertake mentored projects.19 Difficulties in achieving a balance between both didactic and experiential learning exist amidst competing educational demands of medical students. However, the QI project approach is consistent with adult learning principles, where learners are engaged in actual QI practice and the projects have the potential for improving clinical processes of care.13

Typical QI projects require iterative cycles of implementation and evaluation. However, longitudinal learning attachments and integrating QI learning into day-to-day clinical work are often not feasible.11 The competing educational demands on medical students require projects to be kept simple and workload appropriate.20 The aim of this paper was to describe how we incorporate experiential QI learning at the University of Auckland by integrating a clinical audit project into the Year 6 obstetrics and gynaecology clinical attachment.

Methods

At the University of Auckland, we provide classroom and web-based QI science and skills learning during the six-year undergraduate medical curriculum. Since 2004, we have also incorporated experiential QI learning by integrating a clinical audit project into final-year medical students’ obstetrics and gynaecology (O&G) clinical attachment across nine hospitals.

The improvement sciences theory and skills learning commences in Year 3 where the focus is on patient safety. Medical students are allocated into interprofessional teams with third-year nursing, pharmacy and optometry students for a two-day module which consists of face-to-face lectures covering teamwork in healthcare, understanding healthcare disparities, error and violation, human factors (such as practitioner fatigue) and measuring intervention impacts (include revenge effects). The module culminates in conducting a root cause analysis following an adverse event, and the process of open disclosure. Students in Year 4 have six hours of lectures on the critical appraisal of epidemiological evidence. In Year 5, students complete an eight-hour online self-directed module on clinical audit and improvement frameworks, sampling and measurement strategies, and QI science tools to enable them to display, describe and learn from clinical data.

In Year 6, students apply their improvement science skills to conduct a clinical audit project on a topic of their choice during their clinical attachment in O&G. The Year 6 QI programme is designed to help students view a health service through the lens of QI dimensions and the concepts of harm, waste and variation. Starting with the research evidence for best practice, students follow the audit cycle (see Figure 1).

Figure 1: The clinical audit cycle.

Figure credit: Health Quality and Safety Commission, New Zealand.  

Working with a clinical supervisor, the students identify an area for potential improvement and set an evidence-based standard of care. They conduct part of a simple clinical audit to measure current practice and investigate reasons for any deviation from the standard. Students put into practice basic approaches to measurement in QI, including identifying appropriate measures and sampling methods, collecting, analysing and interpreting data, and describing and displaying variation. Students work alone or in pairs and are allocated one academic half-day per week to complete the project. Students are encouraged to discuss their findings with the relevant stakeholders to inform their interpretation of the findings and to develop a suggested action plan. Although the QIPs are carried out during a short timeframe, Year 6 students do learn QI skills beyond conducting a clinical audit. Students are expected to discuss their findings with the relevant service providers, interpret their findings by describing the potential root causes and how they fit in the context of the service, and provide change ideas and recommendations of possible solutions that the service could implement. They also identify areas for further investigation. The students present to the local clinical team for discussion. We are aware of many service changes that have been made by clinicians as a result of the student presentations over the years. We then encourage future students to re-audit to check if the change/s has resulted in any improvement, thus closing the loop. The projects are formally assessed based on a written report and an oral presentation to their peers, QI faculty and where possible the clinical team.

Results

As at July 2016, over 1,250 projects have been completed, with two publications arising from this work so far incorporating multiple audits from multiple sites,21–22 and a thesis dissertation. The projects cover a broad range of obstetrics and gynaecology topics. They range from being condition-specific (eg, adherence to clinical indications for elective caesarean section before 39 weeks; prophylactic Anti-D administration for Rhesus (D) negative pregnant women) to general processes of care and practice (eg, adherence to handwashing policies; screening for family violence on admission). The topics chosen most often by students over the last 14 years include post-operative thromboprophylaxis (n=39 projects), induction of labour (n=53), colposcopy (n=98) and antenatal screening (for infections, congenital anomalies, gestational diabetes, anaemia, etc.) (n>100). A clinical audit project example is provided in Table 1.

Table 1: Example of a clinical audit project.

‍

A formal evaluation of the QI Programme was conducted in 2014 when a shorter clinical rotation in O&G (five weeks to four) was to be implemented. Details of the formal evaluation are reported elsewhere.23 This evaluation was funded by a University of Auckland Learning Enhancement Grant. In summary, the QI programme was seen by clinicians as a valuable way for Year 6 medical students to apply their knowledge and skills into practice, by performing a hands-on project within the constraints and context of a busy women’s health service. Additionally, most clinicians agreed the QIP has provided value to the O&G service and the organisation. Students reported gaining useful insights into QI and agreed that the skills learnt would be important for their future.

However, the evaluation revealed that there was significant variation in data collection and in programme implementation across the eight teaching sites. In addition, challenges were faced by students in selecting a topic that fulfilled project criteria, was relevant to the clinical supervisor and/or the service, and was achievable in the time frame. The findings of the evaluation led to the development and implementation of project-wide and site-specific solutions to ensure learning outcomes continue to be met. This has been largely achieved by developing new resources and making processes more efficient. All resources are now easily accessible to students and supervisors through the university education online portal, such as Year 5 teaching resources, a how-to guide, all previous project reports and a discussion board of ‘tips and hints’ from previous students. A fifteen minute ‘QI Project Overview’ video was developed, highlighting the key components of the clinical audit, the process to follow, timelines and expectations. A ‘topic selection form’ was developed to help ensure students select an appropriate topic and methodology (eg, standard, sample size, variables, etc.) and requires signoff from the supervisor. A ‘written report template’ was created to ensure reports are concise and relevant to stakeholders. Clinical supervisors at each hospital keep an updated list of potential topics, and have created a repository for written reports accessible to hospital clinicians. Hospital QI staff established processes to enable quicker and easier access to medical records.

In advance of the attachment, a welcome email was sent to students providing a link to a university education online portal. The key process steps are summarised below:

• Week 1: Orientation and topic selection
  
• Students and supervisors watch “QI project overview” video, and review available resources.
• Students receive an encrypted memory stick (to protect patient privacy) for data collection.
• Students meet with supervisor, identify topic and complete Topic Selection Form.
• Week 2–3: Conduct project
  
• Supervisor signs off Topic Selection Form with specific feedback to students.
• Students collect and analyse data.
• Students discuss findings with stakeholders and brainstorm barriers and enablers.
• Week 4: Complete project
  
• Students prepare and submit three-page written report.
• Students prepare and deliver a 15-minute oral presentation and answer questions.

The QI curriculum meets the University of Auckland graduate learning outcome for students to identify feasible strategies to improve health that incorporate the broader determinants of health at community and population level.24 It also meets the Australasian Medical Council (AMC) accreditation standard regarding QI, which requires students to be able to describe a systems approach to improving the quality and safety of healthcare.3 The 2005 AMC report noted “the team was impressed especially by the root cause analysis of sentinel events undertaken in Year 3 in a multidisciplinary training environment, and the Year 6 quality assurance project.”

We are committed to disseminating what we have learned through the delivery of our QI teaching programme. Recent efforts have included presentations at local faculty medical education rounds, and facilitation of a QI teaching workshop at the 2014 AMEE (An International Association for Medical Education) conference in Milan, Italy, by two of the authors (MRW and BK). We have appointed a part-time research coordinator (RP-J) to assist with resource development, research outputs and collaboration across all sites.

Discussion

Performing a hands-on project within the constraints and context of a busy O&G service is a feasible and effective method of teaching QI and provides value to the health service. Experiential QI learning, as described here, enhances clinical teaching and training. By integrating clinical audit projects within clinical attachments, medical schools can equip future clinicians with the necessary knowledge and skills to incorporate QI into their daily practice and improve clinical practice.

The Institute of Medicine’s ‘Learning Healthcare System’ report described an approach for integrating clinical research and clinical medicine.25 This has subsequently evolved into a broader concept of a learning health system where all stakeholders can securely, effectively and efficiently contribute, share and use data to create knowledge and improve health outcomes. We plan to support a learning network of clinical supervisors and university academics with a view to enhancing QI training, benchmarking service performance across regional hospitals, sharing ideas for improvement and engaging in academic research.

The importance of developing capability of clinical trainers in QI, transforming them into teachers of QI, has been previously highlighted as a way forward in improving QI in medical education.13 The learning network will facilitate this. We are building faculty capacity by involving more clinicians in student projects, and providing support from the academic department and hospital QI staff to guide them in supervising students to learn the required skills. Specifically, when clinicians first become involved in the project, we provide one-on-one tutoring in clinical audit methodology and mentor them with assessment, by co-evaluating student reports and presentations. We then provide ongoing feedback on the topics, helping them appropriately frame the topic as audit (not research). Informal feedback has been that clinicians feel more confident as time goes on to be able to provide students with adequate support and guidance. As they start using the students’ findings to drive implementation of solutions to improve service, they become more aware of the utility of the student projects, and thus motivated to become more involved in future projects. An unexpected finding in our formal evaluation of the QI programme was that clinicians felt they were learning QI skills from the students. This is important as the New Zealand Medical Council requires all clinicians to perform an annual clinical audit of their own medical practice in order to recertify.26

Together with clinician supervisors, we are also working towards documenting changes in clinical processes and patient outcomes that occurred as a result of student projects. Future research could look at whether focus on specific topics could impact the quality of care delivered in O&G, as was shown in improved health outcomes for patients with diabetes at community practices where medical students and their preceptors completed QI projects.16

Although students are encouraged to learn and apply QI principles beyond conducting a clinical audit, we acknowledge that it is challenging to achieve improvements in service and patient outcomes using small projects conducted in short blocks. That being said, we have worked closely with clinicians over the years to direct the student reports to the clinicians who have governance over the areas studied, and to engage with hospital-wide QI departments, in order to facilitate using the findings to improve service. One way to overcome this could be the team-based approach adopted by some postgraduate programmes in which individual junior doctors work on one part of an overall project, building on work done by their predecessors and a quality initiative over time.27,28

Our QI programme is undergoing continuous improvement. We continue to look for ways to modify the clinical audit project in order to reduce student workload. Our supervisors encourage students to choose only one standard, one or two measures, and a few stratifying variables; we are considering assessing the project proposal only, rather than the completed project; and we are exploring with other departments to collaborate with O&G to enable the clinical audit to be conducted over more than one placement, which would extend the project timeframe. We also want to engage clinicians at more sites to improve knowledge translation, for example, getting the findings from student projects back out to stakeholders who can use them to make a difference in their services. Clinicians who see improvements in patient outcomes will be more likely to seek out continuing QI education for themselves and to supervise future students.

Another challenge we are addressing is that of assessment. At the moment individual students are assessed for their QI competency by evaluating their group project presentation. There may be more effective ways of assessing competency and we may add other objective measures of student knowledge such as the QIKAT-R,29 or modify our assessment scheme to a validated tool.30 We plan to survey postgraduate trainees to assess retention of QI knowledge and skills, and investigate whether the projects increased QI capability so that learners were able to improve health service performance in their future roles. The survey will also gauge whether involvement in these projects enhanced exposure to rigorous research methods and structured reporting. As previously reported, this may potentially inspire health professionals to be engaged with academic medicine during their career.31

Medical schools have an integral role to play in ensuring future healthcare professionals are equipped with QI knowledge, skills and attitudes so that they can contribute towards improved clinical practice and patient outcomes. We provide a combination of didactic and experiential QI learning opportunities. The literature notes numerous benefits of experiential learning, although learning real-life QI practices is comparatively new. Incorporating clinical audit projects within short clinical attachments should be considered a feasible and effective method of QI education to medical students. The authors are happy to be contacted to share resources, consult with faculty or facilitate workshops.

‍