Sustaining multidisciplinary team training in New Zealand hospitals: a qualitative study of a national simulation-based initiative | OPEN ACCESS
View Article PDF
Healthcare today is predominantly delivered by teams.1 There is burgeoning literature on teamwork, team competence and interprofessional learning, with convincing evidence that failures in teamwork and communication can lead to bad outcomes for patients.2–5 This suggests that team training should be incorporated into everyday practice.6–8 In New Zealand, multidisciplinary team training is not yet ‘business as usual’.
Evidence is emerging that the use of simulation can generate greater improvements in teamwork skills than team-based training delivered didactically.9 Internationally, there are a number of simulation-based team training initiatives such as PROMPT,10,11 TeamSTEPPS12,13 and a Harvard insurer-funded simulation programme for operating theatre teams.14 We set out to implement simulation-based team training for operating theatre teams in New Zealand.
We set out to develop simulations of surgical cases that equally engaged all members of the multidisciplinary surgical team. While technologies to simulate anaesthetic tasks were readily available, we were unable to source surgical models that we could integrate with an anaesthesia simulator to enable surgeons to perform procedures such as incision, resection, suturing or haemorrhage control. We therefore created these models working with a medical special effects company (MedicFX). The course underwent an extensive pilot with 20 full surgical teams from two large hospitals. Each team participated in a full-day course. Participant evaluations were very positive and we demonstrated improved scores for teamwork in the workplace.15,16 Two further courses in an operating theatre suite tested the feasibility of running the simulations in situ.
With funding from New Zealand’s national no-fault accident insurer, the Accident Compensation Corporation (ACC),17 we developed NetworkZ,18 a simulation-based multi-disciplinary team training programme for operating theatre teams, and established it in New Zealand public hospitals (see Table 1). The programme is described on the website www.networkz.ac.nz. Implementation of NetworkZ began in 2017 and has sequentially rolled out across New Zealand public hospitals over four years. At the time of writing, 1,082 participants had attended a NetworkZ course, and 279 local DHB health practitioners had begun training to be a NetworkZ instructor. Participants reflect the full range of operating room roles; consultant surgeons (15%), consultant anaesthetists (15%), surgical trainees (7%), anaesthetic trainees (5%), nurses (38%), anaesthetic technicians (13%) and other staff such as healthcare assistants (7%).
Table 1: The NetworkZ programme.
We know of no other national programme for multidisciplinary operating theatre teams that integrates a high fidelity anaesthesia simulator such as the Laerdal 3G simulator with surgical models on which surgeons can perform operative procedures.
In a qualitative study in the first Cohort of five district health boards (DHBs) Jowsey and colleagues19 identified local factors associated with implementation challenges and successes, which informed our approach to establishing NetworkZ in later groups of DHBs. As the implementation phase will be completed by the end of 2020, our attention is now on the sustainability of the programme.
Theoretical framework
We defined sustainability of NetworkZ in terms of the maintenance of programme activities into the future.20 As our theoretical framework we used Rogers’ Diffusion of Innovation theory,21 which explains how new practices become embedded in organisations or populations. Rogers proposes that influential early adopters are important, as are the preceding conditions, such as decision-maker characteristics and communication behaviour, and features of the intervention itself. Rogers proposes that the key features of interventions that promote positive attitudes towards an intervention are; (1) the perceived relative advantage of the intervention, (2) compatibility of the intervention with existing structures, (3) (low) intervention complexity, (4) the ability to test out the intervention prior to full implementation and (5) observability of the interventions’ impact. These attributes in turn influence adoption and discontinuation decisions.21
In the present study we explored the perspectives on long-term sustainability of NetworkZ with DHB staff involved in its local delivery or establishment. Using the theoretical lens of Rogers’ Diffusion of Innovation theory21 to interpret the data, we aimed to identify the elements of the NetworkZ programme that promoted positive attitudes, and elements that posed challenges for sustainability.
Method
In this qualitative study we undertook semi-structured interviews of staff involved with delivering or establishing the NetworkZ programme and used deductive thematic analysis22,23 of the data based on Diffusion of Innovation theory.21 The study is part of a larger programme of evaluation of NetworkZ, registered with ANZCTR (ACTRN12617000017325) and approved by the NZ Health and Disability Ethics Committee (16/NTB/143).24
Interview
The semi-structured interview guide focused on programme strengths, impact, implementation experiences and needs for ongoing sustainability (Appendix). Minor changes to the interview guide were made iteratively to prompt participants to elaborate further on topics relevant to the project.
All interviews were conducted by one member of the research team (JL) who has a doctorate in psychology, and has been trained in interview techniques. She introduced herself to participants as a researcher on the NetworkZ evaluation team.
Sample population
At the start of the rollout, all 20 DHBs were divided into five groups (which we call cohorts), representing similar population catchments. Cohort 2, the focus of this study, included DHBs from each size grouping. The two largest DHBs in Cohort 2 were in large metropolitan centres each with multiple hospital sites and their own local simulation centre.
Potential interview participants were selected from existing lists of people involved in course delivery (instructors) and those who were involved in the establishment of NetworkZ through their role as managers (managers). Management roles included clinical director, chief medical officer, nurse manager, operating room manager, charge nurse, project manager or quality specialist. Instructor roles included course instructor, convenor and simulation technicians. In order to preserve confidentiality, we refer to these people in the findings as either managers or instructors. Potential interviewees were approached via email and invited to participate in a telephone or face-to-face interview. Interviews were conducted between December 2018 and March 2019.
Sampling
We used purposive sampling to recruit potential interviewees in proportion to the size of each DHB, NetworkZ role and professional group. Interviewees were recruited until coding saturation was achieved.
Data management and analysis
Interviews were transcribed verbatim and de-identified. The interview data were initially coded using semantic thematic analysis,23 focusing on what was said (rather than what was implied). Codes were then deductively developed into themes aligned to Diffusion of Innovation Theory’s five main factors. Coding was conducted in QSR Nvivo 12 Pro software25 by author 1 and reviewed by author 2.
Results
Interviews were conducted between December 2018 and April 2019. Forty-nine people were invited via email to participate and 27 agreed. Of those who declined, reasons given were leaving their role, not knowing enough about NetworkZ or lack of time to participate. Interview length ranged from 8 to 51 minutes. Interviewee characteristics are outlined in Table 2.
Table 2: Interviewee characteristics.
Thematic overview
Elements of the programme that influenced attitudes to NetworkZ, and its ongoing sustainability are presented as themes using the Diffusion of Innovations framework (relative advantage, compatibility with existing systems, complexity of course delivery and observability of programme impacts) and subthemes specific to the interview data (Figure 1). Of note, a theme around trialability, one of the elements of the Diffusions of Innovations framework, did not emerge.
Figure 1: Thematic map.
Theme 1: relative advantage
Interviewees perceived the key strengths of NetworkZ to be: multidisciplinary focus, in-situ delivery, realism of the courses and the relevance of the teamwork and communication focus, and generalisability of the course to other settings.
The concurrent training of all members of the multidisciplinary team was seen as an advantage of the courses. Interviewees valued the uniqueness of having a course for a full theatre team, the opportunity to act in their own role, and to listen and support each other in the debrief after the scenarios.
“I’ve done a lot of courses for anaesthetists with pretend surgeons. I can’t think of anything like this where it was so multidisciplinary.” [Instructor, Anaesthetist]
“Something simple that came up between the nurses and the surgeons in terms of when to do the count. And I can’t remember, the surgeons and the nurses each getting a shared insight into what each other’s priorities are at that time and how they can easily adjust or not to make it work for both of them.” [Instructor, Anaesthetist]
“The simulation process allows them to reflect on their degree of situational awareness they have, how they work safer as a team than as individuals in the team.” [Manager, Other]
The delivery of the courses in-situ using usual equipment and setup was described as facilitating engagement from participants by improving scenario realism (also discussed later), allowing for the troubleshooting of local systems and making access to simulation training possible for DHBs that did not have a local simulation centre.
“What happens in the simulation centre is everybody’s stressed. And everybody’s in an unfamiliar environment and so nobody performs to their best anyway. And besides, it goes back to the whole thing of, well actually if you’re stressing your own systems then you can improve your systems.” [Manager, Anaesthetist]
“It’s also checking the processes and the policies and things and how they work in real life situations. So, I think that’s what, one of the major benefits of doing in situ simulation is. Is that you’re actually trying it out and seeing what works and if it doesn’t work you can change things.” [Instructor, Anaesthetic Technician]
The realism of scenarios was described as a key strength that assisted with buy in to the programme, and active engagement during the course scenarios. Realism was described as particularly important for eliciting surgeon buy-in to the course.
“At the start of the session there’s varying degrees of engagement. And then once you actually run the scenarios which are quite realistic, and people actually do their role and they feel like it was kind of like a normal day in theatre.” [Instructor, Surgeon]
“I was involved with one of the scenarios and you’re actually sort of physically sweating… pretend scenarios where there’s no blood or anything is very hard to take seriously.” [Manager, Nurse]
“We practically had exactly the same scenarios happen with the guy with the amputated leg. We actually had pretty had much a very similar scenario happen about a year and a half ago.” [Instructor, Nurse]
The programme’s goals were described as highly relevant to the teamwork and communication priorities of instructors and managers alike.
“A lot of our issues are around communication breakdowns. So, you need any tools that we have that can help highlight and break down the hierarchical barriers.” [Instructor, Anaesthetic Technician]
“Our operating theatres should be a positive place for people to work, where there’s less of the hierarchical stuff of the past, where people can communicate and I think that’s going to lead to better patient outcomes.” [Instructor, Surgeon]
As described above, NetworkZ was viewed as uniquely valuable relative to other team training interventions currently available in New Zealand due to its multidisciplinary participation, in-situ delivery, realism and relevance to clinical practice. As such, interviewees felt that it could be useful for improving teamwork, local crisis response processes and patient safety in other areas of the hospital, such as emergency departments, intensive care units, post-operative care suites and in other surgical specialties such as obstetrics and gynaecology.
“We’ve got specialties that are really keen to do the programme but there are no scenarios for them.” [Instructor, Anaesthetic Technician]
“I’d like to see NetworkZ go into the different areas of the hospital, that we shouldn’t be the only ones that benefit from the training.” [Instructor, Anaesthetic Technician]
Theme 2: compatibility with existing systems
Strong compatibility of a programme with existing systems and values can increase the likelihood that it will be adopted and sustained.21 While the focus of NetworkZ was described as compatible with personal beliefs about the importance of great teamwork, the delivery of the courses was at times seen as challenging with available local resources.
Interviewees described resourcing of instructor time and theatre time as a key challenge. Some interviewees described friction between the ongoing delivery of the NetworkZ courses, and “time and money”, arising from busy staff workloads, fully booked theatres and the desire to fill theatres in order to maximise funding and reduce hospital deficits.
“It was a significant challenge for the DHB to be able to deliver that and maintain production.” [Manager, Surgeon]
“It’s been difficult to get that time allocated to the technical team so they’ve got time to go set up the day before.” [Instructor, Anaesthetist]
“We made a decision in the end that the only way we were going to make this work was to fit it into the programme of educational activities that we have once a month for an afternoon.” [Instructor, Anaesthetist]
As described in the quote above, a number of DHBs utilised pre-existing half-days set aside for education to run courses so that NetworkZ training had a negligible impact on their theatre lists. The use of non-operating education days minimised disruption to theatres but created an opportunity cost for instructors who missed other important education sessions that were happening at the same time. The time, cost and theatre space requirements of NetworkZ were larger than for many other courses held at these hospitals.
Theme 3: complexity of course delivery
Programme complexity can be a deterrent to programme adoption and sustainability.21 Instructing on local courses was described as a complex, and as described above, a resource-intensive experience. Interviewees referred to the “extra” tasks involved in setting up the courses and making them run smoothly on the day as often unaccounted for within the local resourcing budget.
“Not just an educational thing but all this. It goes where the manikin’s stored and who can touch it and who looks after it, the nurses can be freed up and contacting the surgeons to agree to it and all that sort of stuff.” [Instructor, Anaesthetist]
Tasks included recruiting staff, setting up the simulation equipment and organising course materials. For anaesthetic technicians and nurses, standard rostering practices did not offer instructors time to complete these tasks. For clinicians there was an option to use non-clinical time for this type of work, but interviewees noted there were many competing demands on this time. Thus, complexity in part reflected limited time to do the job as an “extra-curricular” instructor. Complexity was also generated by the multiple roles involved in delivering the courses, and the desire to ensure the courses were well run and were realistic for participants.
“It took us a long time to work out the different roles. [Instructor, Nurse]
There’s that pressure to run them smoothly on a, usually on a compressed timeframe.” [Instructor, Anaesthetist]
A number of interviewees also said that the complexity of course delivery meant they did not feel confident to provide optimal instructor training to their own local instructors. As such, they indicated a desire for ongoing national ‘expert’ support to deliver the instructors courses so that future staff were able to learn the content as it was originally intended. Similarly, many felt that national expertise would be important for troubleshooting, designing new scenarios and maintaining quality in the future. While people varied in their confidence about future delivery, some noted that they were likely to also need technical support to be able to deliver the programme beyond the initial period of support.
Interviewees in large DHBs reported challenges in course coordination and maintaining momentum when they did not have staff dedicated to support NetworkZ; those with dedicated staffing said this was critical to ensuring smooth running of the programme.
Theme 4: observability of programme impact
Observable benefits encourage adoption and continued support for programmes.19 The following sections discuss the improvements observed in (1) teamwork and communication, and (2) reduction in latent safety issues. Interviewees observed improved teamwork and communication at their DHB following NetworkZ training. Some interviewees referred generically to a sense of improved teamwork and communication skills, or improved teamwork behaviour.
“Having everyone work together during these simulations, it’s just really improving the communication between the groups and everyone’s starting to work together. It’s helping people get a better idea of everybody’s roles and what they’re capable of doing.” [Instructor, Anaesthetic Technician]
“Communication certainly seems to have improved in the fact that folk aren’t working in the little individual teams like the anaesthetics working independently from the scrub side of things. So there seems to be more teamwork.” [Instructor, Nurse]
Other observations about teamwork changes included better sharing of information between team members, improved confidence to speak up, better knowledge about their team members and reduced hierarchy.
“The briefings in the morning, they’re more clear. They’re more organised. So, calling each other by name.” [Instructor, Surgeon]
“Concepts like closed loop communication or introducing each other at the start or needing to have pre-briefings or sort of pauses in the middle of crises and stuff like that, that’s all just becoming a lot more familiar and expected and asked for.” [Instructor, Anaesthetist]
The delivery of the NetworkZ courses in local theatres provided opportunities to identify latent safety threats. Interviewees reported identifying gaps in staff knowledge of local crisis systems and equipment, and problems with those systems and equipment. Further training sessions were then held to respond to these gaps.
“People didn’t know how to use the defibrillator well and so it kind of brought out where the weaknesses were as a team.” [Instructor, Nurse]
“There was an issue with the Belmont [rapid infusion device] but there was some misunderstanding about how to set it up and of course that then generated an education session so the techs were more familiar with it.” [Instructor, Anaesthetist]
Other local improvements generated by the courses included changes to the process of ordering bloods in an emergency, the purchase of a second defibrillator and improved medication storage. Managers were generally supportive and hopeful about the programme impact. However, some felt they had not yet received sufficient information on course attendance or programme impact.
Discussion
Interviewees described the relative advantages of NetworkZ as multidisciplinary involvement, in-situ delivery, scenario realism, relevance to teamwork and communication in the operating theatre, and potential for translation beyond theatre to other areas in the DHB. While the training was compatible with local instructor and DHB interest in improved teamwork and perceived safety, it presented challenges due to pressure on staff time and operating theatre access. The perceived complexity of delivery suggested that ongoing dedicated expert support would be required. While those involved as instructors knew about the reported observable impacts of NetworkZ, this information was not always conveyed to senior management, who were the ones responsible for providing the time and resources required for course delivery.
Relative advantages
Overall interviewees described the programme as having several “relative advantages” over other teamwork or communication programmes they had attended, many of which did not involve established surgical teams working together in their own environment using realistic surgical models. These relative advantages drove engagement, and motivation to expand the programme to other hospital departments. Yet these advantages also made the programme more complex, requiring substantial resourcing. Decisions to cut programme costs or complexity need to be weighed up against the possibility that these will undercut programme benefits or stakeholder buy in. While interviewees spoke about a number of ways that the NetworkZ programme improved upon other available training programmes, funders will also need to view this programme as offering greater benefit than other available programmes. Continuing medical education resources are commonly allocated to passive forms of education such as conferences, yet these activities have only small effects on physician performance and outcomes.26,27 Sustaining a programme such as NetworkZ into the future will require a rethink of the way practitioners and institutions use their budgets for continuing medical education. Recognition by regulatory bodies of the need to improve teamwork and communication through multidisciplinary activities would support the sustainability of such initiatives.
Compatibility with existing systems
Programme adaptation to align with local context is another core component of most sustainability approaches.20,29 Other simulation training programmes have also noted challenges to sustainability and scalability, including challenges with financing, resourcing, recruiting and upskilling trainers.28 Our findings revealed a range of local ‘solutions’ to resourcing constraints. For example, some DHBs have chosen to run courses for half a day instead of a full day, and most have integrated the courses with existing time scheduled for education. Support from managers of each professional group can also assist in overcoming resourcing constraints.29,19 Further, national budget streams specifically designed for health workforce development could reduce some of the resourcing costs for DHBs, yet these currently do not have obvious mechanisms to facilitate or incentivise the pooling of training funds from different professional groups to enable the delivery of multidisciplinary training programmes.
Complexity of course delivery
The physical resources, instructor skill-levels and staff time required to deliver NetworkZ pose challenges for sustainability, due to the complexity of scheduling multidisciplinary participation, multiple roles and skillsets needed to run the simulation smoothly. Interviewees expressed the view that ongoing support would be needed beyond the current period of ACC funding. Declines in skill and fidelity are common for health interventions,29 and interviewees requested ongoing input from key experts to maintain the quality of NetworkZ. Other team simulation training programmes (eg, PROMPT) also use experts, rather than peers to train new instructors.10
Recruiting participants from all the professional groups that make up a team creates its own complexities and can be one of the biggest barriers to the delivery of simulation-based team training.30 Education has traditionally been delivered in professional silos, and multidisciplinary training is a paradigm shift creating challenges in timetabling, motivation to attend and funding streams. Leadership support is important for overcoming recruitment challenges,30 and adequate resourcing may assist here too. For large DHBs, dedicated staff time for a head instructor, or to cover recruitment, course coordination and timetabling was reportedly key to ensuring smooth delivery of the programme.
The existing delivery of NetworkZ would not have been possible without substantial funding from ACC. To sustain NetworkZ, ongoing funding will be required, either from a national source or local sites, or a combination of the two. Diverse sources of funding are likely to be more sustainable than a single funder.31
Observability of programme impacts
For managers, evidence of programme impact was of key interest. Ward et al32 similarly noted that research, clinical and patient experience, and local evidence was important for enabling the implementation of the TeamSTEPPS curriculum in rural hospitals. Likewise, a systematic review of sustainability approaches identified that building evidence was a key strategy of most sustainability models and approaches,20 to ensure decision makers are informed of the theoretical and empirical data justifying expenditure on the programme. Ultimately, as noted by some of the managers interviewed, the most convincing evidence for ongoing delivery of the programme would be improved patient outcomes. We are collecting evidence on patient outcomes, as well as teamwork and communication at a national level,24 but results will not be available before 2022. Thus collating short-term evidence that can be fed back to senior managers is also important. Documenting the improvements to local theatre systems, equipment and training may offer concrete, easy to collect, immediate evidence of programme impact for senior managers.
Strengths and limitations
A key strength of this work is sampling from multiple DHBs of different sizes from very large to very small, incorporating a range of perspectives from these varying contexts.
Limitations of the study include potential bias in data collection and analysis due to the researchers’ vested interest in the success of NetworkZ. The extent to which our findings can be generalised to other countries, other stakeholders not involved in establishing or running NetworkZ in their local DHBs and other similar training programmes remains to be tested.
One of the themes of Diffusion of Innovations theory that did not arise was ‘trialability’ of the intervention. The intervention was extensively trialled prior to obtaining funding for the programme. When signing up to the intervention, each DHB committed to training instructors, providing time for the course, and accepting responsibility for maintenance and depreciation of the gifted simulator. A trial run of the programme was therefore not feasible. However, as noted earlier, each DHB worked out for themselves how to implement the programme.
Future research
In this study we identified programme complexity as a potential challenge. The extent to which programme gains could be achieved by delivering parts of the course in a less resource-intensive workshop format or delivering some courses in purpose-built simulation facilities is yet to be tested. The surgical models, however, seem to be a valued, key component of the experience, allowing surgical participants to actively participate in the simulation. While studies suggest that repeated training is one way to increase the sustainability of simulation training benefits, other opportunities to sustain health programme benefits have yet to be explored,20 such as the value of incorporating the discussion of communication and teamwork concepts into the debriefing of real-life trauma cases. Resourcing instructors’ time arose as an issue in sustainability of the programme. The sustainability of unpaid instructor involvement in programmes such as NetworkZ has not, to our knowledge, been explored and may be a particular challenge for complex train-the-trainer programmes, particularly where these roles are part of organisational service but are not explicitly reimbursed. Optimal approaches to maintaining engaged instructors in sufficient numbers is a potential area for workforce development research.
Conclusion
NetworkZ invokes positive attitudes in local stakeholders, but its complexity and resource requirements pose a risk without adequate resourcing. Existing workforce development programmes are predominantly uni-professional, and the multi-professional requirements of NetworkZ pose additional complexity as existing systems for staff development within institutions, and discipline-specific continuing professional development are not set up to support multidisciplinary training. Team training programmes such as NetworkZ have the potential to yield important benefits for healthcare and require adequate resourcing. This will require a commitment from funders, institutions and professional bodies to building effective healthcare teams and better outcomes for patients.
Appendix
Semi-structured interview guide
Indicative interview questions and prompts
1. First I’d like to talk about your experience of the NetworkZ programme so far:
a. What’s been working well? What have been your highlights? What makes the course special, or different from other courses you’ve been involved in? (What do you see as the key strengths? Can you think of a time that illustrates X?)
2. What got you involved in NetworkZ in the first place? What do you value about being involved in the programme?
3. What positive impacts has the NetworkZ programme had in your DHB? (Are there any impacts that have come from delivering the course in-situ?)
4. In regards to your experience of the implementation of NetworkZ:
a. What’s been working well? What things were done in your DHB that you would recommend to other DHBs starting out? How have you overcome any challenges along the way?
5. It is intended that NetworkZ could become business as usual in the future. What is already happening to facilitate long-term sustainability of the programme in your DHB?
6. I’d like you to think about your aspirations for the NetworkZ programme, what benefits would you like to see the NetworkZ programme achieve for staff and patients? (What ideas do you have for improve the programmes’ impact on teamwork and patient safety?)
7. What do you see as the optimal way of delivering NetworkZ in your own DHB over the longer term? Pprompt for frequency, integrated vs siloed from other training (development of future scenarios, who should deliver the training)
8. What needs to happen to make NetworkZ reflect your vision for an ideal programme? What resources would be required? What support would be needed? From whom?(this question was dropped in the second iteration as it was largely covered in other questions)
9. Thinking about the longer term, what else can be done to ensure the programme is sustainable? How would the courses be delivered and adapted over time? Who would be involved? What options might exist for funding?
10. Is there anything else you would like to tell me about in relation to the NetworkZ programme?
1Additions and deletions made over the course of the interviews are detailed in parentheses.
Summary
Abstract
Aim
Healthcare is delivered by teams, but the training of healthcare staff is commonly undertaken in professional silos. This study investigated local perspectives on the sustainability of NetworkZ, a New Zealand national simulation-based multi-disciplinary operating room team training programme.
Method
Local course instructors and managers were invited to participate in semi-structured interviews. Diffusion of innovations theory was utilised to frame deductive thematic analysis of interview data.
Results
Twenty-seven people participated. Interviewees described valuing NetworkZ for its multi-disciplinary orientation, in-situ delivery, scenario realism, relevance to teamwork and communication and potential for generalisability to other settings. Interviewees also identified NetworkZ as generating improvements in teamwork and crisis management. NetworkZ was described as complex, due to multidisciplinary participation and the multiple roles and skillsets of instructors needed to run simulations smoothly, making the programme resource intensive to deliver.
Conclusion
NetworkZ is appreciated as a valuable and unique programme for developing important teamwork and communication skills. Its sustainability is dependent on adequate resourcing and funding.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1. Lingard L. Rethinking competence in the context of teamwork. In: Hodges B, Lingard L, eds. The Question of Competence: Reconsidering Medical Education in the Twenty-first Century. New York: Cornell University Press 2012:42–69.
2. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004; 13:330–34.
3. Lingard L, Reznick R, Espin S, et al. Team communications in the operating room: talk patterns, sites of tension, and implications for novices. Acad Med. 2002; 77(3):232–37.
4. Varpio L, Hall P, Lingard L, Schryer, C F. Interprofessional communication and medical error: a reframing of research questions and approaches. Acad Med. 2008; 83(10):S76–S81.
5. Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med. 2004; 79(2):186–94.
6. Hodges B, Lingard L. The question of competence: reconsidering medical education in the twenty first century. Ithaca and London: Cornell University Press 2012.
7. Institute of Medicine CoQoHCiA. To err is human: building a safer health system. Washington (DC): National Academies Press 2000.
8. McPherson K, Headrick L, Moss F. Working and learning together: good quality care depends on it, but how can we achieve it? Qual Health Care. 2001; 10:ii46–ii53.
9. Harvey EM, Echols SR, Clark R, Lee, E. Comparison of Two TeamSTEPPS® Training Methods on Nurse Failure-to-Rescue Performance. Clin Simul Nurs.2014; 10(2):e57–e64.
10. PROMPT Practical Obstetric Multi-Professional Training PROMPT Maternity Foundation; [Available from: www.promptmaternity.org/training/ accessed 27 September 2019].
11. Shoushtarian M, Barnett M, McMahon F, Ferris, J. Impact of introducing Practical Obstetric Multi-Professional Training (PROMPT) into maternity units in Victoria, Australia. BJOG. 2014; 121(13):1710–18.
12. Agency for Healthcare Research and Quality. TeamSTEPPS [Available from: http://www.ahrq.gov/teamstepps/index.html accessed 27 September 2019].
13. Clancy CM, Tornberg DN. TeamSTEPPS: assuring optimal teamwork in clinical settings. Am J Med Qual. 2007; 22(3):214–17.
14. Arriaga AF, Gawande AA, Raemer DB, et al. Pilot testing of a model for insurer-driven, large-scale multicenter simulation training for operating room teams. Ann Surg. 2014; 259(3):403–10.
15. Weller JM, Cumin D, Torrie J, et al. Multidisciplinary operating room simulation-based team training to reduce treatment errors: a feasibility study in New Zealand hospitals. NZ Med J 2015; 128(1418):40–51.
16. Weller JM, Cumin D, Civil ID, et al. Improved scores for observed teamwork in the clinical environment following a multidisciplinary operating room simulation intervention. NZ Med J 2016; 129(1439):59–67.
17. Accident Compensation Corporation. ACC 2018 [Available from: www.acc.govt.nz, accessed 18 October 2019].
18. NetworkZ. NetworkZ - Better outcomes in acute care 2019 [Available from: http://www.networkz.ac.nz/, accessed 18 October 2019].
19. Jowsey T, Beaver P, Long J, et al. Towards a safer culture: implementing multidisciplinary simulation-based team training in New Zealand operating theatres, a framework analysis. BMJ Open. 2019; 9:e027122.
20. Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018; 13(1):27.
21. Rogers EM. Diffusion of Innovations. New York: Free Press 2003.
22. Nowell LS, Norris JM, White DE, et al. Thematic analysis: Striving to meet the trustworthiness criteria. Int J Qual Methods. 2017; 16:1–13.
23. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006; 3(2):77–101.
24. Weller JM, Long JA, Beaver P, et al. Evaluation of the effect of multidisciplinary simulation-based team training on patients, staff and organisations: protocol for a stepped-wedge cluster mixed methods study of a national, insurer-funded initiative for surgical teams in New Zealand public hospitals. BMJ Open. 2020; 10:e032997
25. NVivo qualitative data analysis Software [program]. Version 12 version. Southport, United Kingdom, 2018.
26. Forsetlund L, Bjørndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009(2).
27. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof 2007; 27(1):6–15.
28. Delisle M, Pradarelli JC, Panda N, et al. Methods for scaling simulation-based teamwork training. BMJ Qual Saf 2020; 29(2):98–102.
29. Stirman SW, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012; 7:17–17.
30. Cumin D, Boyd MJ, Webster CS, Weller JM. A systematic review of simulation for multidisciplinary team training in operating rooms. Simul Healthc. 2013; 8(3):171–9.
31. Whelan J, Love P, Pettman T, et al. Cochrane Update: Predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health. 2014; 36(2):347–51.
32. Ward MM, Baloh J, Zhu X, Stewart, GL. Promoting action on research implementation in health services framework applied to TeamSTEPPS implementation in small rural hospitals. Health Care Manage Rev. 2017; 42(1):2–13.
For the PDF of this article,
contact nzmj@nzma.org.nz
Sustaining multidisciplinary team training in New Zealand hospitals: a qualitative study of a national simulation-based initiative | OPEN ACCESS
View Article PDF
Healthcare today is predominantly delivered by teams.1 There is burgeoning literature on teamwork, team competence and interprofessional learning, with convincing evidence that failures in teamwork and communication can lead to bad outcomes for patients.2–5 This suggests that team training should be incorporated into everyday practice.6–8 In New Zealand, multidisciplinary team training is not yet ‘business as usual’.
Evidence is emerging that the use of simulation can generate greater improvements in teamwork skills than team-based training delivered didactically.9 Internationally, there are a number of simulation-based team training initiatives such as PROMPT,10,11 TeamSTEPPS12,13 and a Harvard insurer-funded simulation programme for operating theatre teams.14 We set out to implement simulation-based team training for operating theatre teams in New Zealand.
We set out to develop simulations of surgical cases that equally engaged all members of the multidisciplinary surgical team. While technologies to simulate anaesthetic tasks were readily available, we were unable to source surgical models that we could integrate with an anaesthesia simulator to enable surgeons to perform procedures such as incision, resection, suturing or haemorrhage control. We therefore created these models working with a medical special effects company (MedicFX). The course underwent an extensive pilot with 20 full surgical teams from two large hospitals. Each team participated in a full-day course. Participant evaluations were very positive and we demonstrated improved scores for teamwork in the workplace.15,16 Two further courses in an operating theatre suite tested the feasibility of running the simulations in situ.
With funding from New Zealand’s national no-fault accident insurer, the Accident Compensation Corporation (ACC),17 we developed NetworkZ,18 a simulation-based multi-disciplinary team training programme for operating theatre teams, and established it in New Zealand public hospitals (see Table 1). The programme is described on the website www.networkz.ac.nz. Implementation of NetworkZ began in 2017 and has sequentially rolled out across New Zealand public hospitals over four years. At the time of writing, 1,082 participants had attended a NetworkZ course, and 279 local DHB health practitioners had begun training to be a NetworkZ instructor. Participants reflect the full range of operating room roles; consultant surgeons (15%), consultant anaesthetists (15%), surgical trainees (7%), anaesthetic trainees (5%), nurses (38%), anaesthetic technicians (13%) and other staff such as healthcare assistants (7%).
Table 1: The NetworkZ programme.
We know of no other national programme for multidisciplinary operating theatre teams that integrates a high fidelity anaesthesia simulator such as the Laerdal 3G simulator with surgical models on which surgeons can perform operative procedures.
In a qualitative study in the first Cohort of five district health boards (DHBs) Jowsey and colleagues19 identified local factors associated with implementation challenges and successes, which informed our approach to establishing NetworkZ in later groups of DHBs. As the implementation phase will be completed by the end of 2020, our attention is now on the sustainability of the programme.
Theoretical framework
We defined sustainability of NetworkZ in terms of the maintenance of programme activities into the future.20 As our theoretical framework we used Rogers’ Diffusion of Innovation theory,21 which explains how new practices become embedded in organisations or populations. Rogers proposes that influential early adopters are important, as are the preceding conditions, such as decision-maker characteristics and communication behaviour, and features of the intervention itself. Rogers proposes that the key features of interventions that promote positive attitudes towards an intervention are; (1) the perceived relative advantage of the intervention, (2) compatibility of the intervention with existing structures, (3) (low) intervention complexity, (4) the ability to test out the intervention prior to full implementation and (5) observability of the interventions’ impact. These attributes in turn influence adoption and discontinuation decisions.21
In the present study we explored the perspectives on long-term sustainability of NetworkZ with DHB staff involved in its local delivery or establishment. Using the theoretical lens of Rogers’ Diffusion of Innovation theory21 to interpret the data, we aimed to identify the elements of the NetworkZ programme that promoted positive attitudes, and elements that posed challenges for sustainability.
Method
In this qualitative study we undertook semi-structured interviews of staff involved with delivering or establishing the NetworkZ programme and used deductive thematic analysis22,23 of the data based on Diffusion of Innovation theory.21 The study is part of a larger programme of evaluation of NetworkZ, registered with ANZCTR (ACTRN12617000017325) and approved by the NZ Health and Disability Ethics Committee (16/NTB/143).24
Interview
The semi-structured interview guide focused on programme strengths, impact, implementation experiences and needs for ongoing sustainability (Appendix). Minor changes to the interview guide were made iteratively to prompt participants to elaborate further on topics relevant to the project.
All interviews were conducted by one member of the research team (JL) who has a doctorate in psychology, and has been trained in interview techniques. She introduced herself to participants as a researcher on the NetworkZ evaluation team.
Sample population
At the start of the rollout, all 20 DHBs were divided into five groups (which we call cohorts), representing similar population catchments. Cohort 2, the focus of this study, included DHBs from each size grouping. The two largest DHBs in Cohort 2 were in large metropolitan centres each with multiple hospital sites and their own local simulation centre.
Potential interview participants were selected from existing lists of people involved in course delivery (instructors) and those who were involved in the establishment of NetworkZ through their role as managers (managers). Management roles included clinical director, chief medical officer, nurse manager, operating room manager, charge nurse, project manager or quality specialist. Instructor roles included course instructor, convenor and simulation technicians. In order to preserve confidentiality, we refer to these people in the findings as either managers or instructors. Potential interviewees were approached via email and invited to participate in a telephone or face-to-face interview. Interviews were conducted between December 2018 and March 2019.
Sampling
We used purposive sampling to recruit potential interviewees in proportion to the size of each DHB, NetworkZ role and professional group. Interviewees were recruited until coding saturation was achieved.
Data management and analysis
Interviews were transcribed verbatim and de-identified. The interview data were initially coded using semantic thematic analysis,23 focusing on what was said (rather than what was implied). Codes were then deductively developed into themes aligned to Diffusion of Innovation Theory’s five main factors. Coding was conducted in QSR Nvivo 12 Pro software25 by author 1 and reviewed by author 2.
Results
Interviews were conducted between December 2018 and April 2019. Forty-nine people were invited via email to participate and 27 agreed. Of those who declined, reasons given were leaving their role, not knowing enough about NetworkZ or lack of time to participate. Interview length ranged from 8 to 51 minutes. Interviewee characteristics are outlined in Table 2.
Table 2: Interviewee characteristics.
Thematic overview
Elements of the programme that influenced attitudes to NetworkZ, and its ongoing sustainability are presented as themes using the Diffusion of Innovations framework (relative advantage, compatibility with existing systems, complexity of course delivery and observability of programme impacts) and subthemes specific to the interview data (Figure 1). Of note, a theme around trialability, one of the elements of the Diffusions of Innovations framework, did not emerge.
Figure 1: Thematic map.
Theme 1: relative advantage
Interviewees perceived the key strengths of NetworkZ to be: multidisciplinary focus, in-situ delivery, realism of the courses and the relevance of the teamwork and communication focus, and generalisability of the course to other settings.
The concurrent training of all members of the multidisciplinary team was seen as an advantage of the courses. Interviewees valued the uniqueness of having a course for a full theatre team, the opportunity to act in their own role, and to listen and support each other in the debrief after the scenarios.
“I’ve done a lot of courses for anaesthetists with pretend surgeons. I can’t think of anything like this where it was so multidisciplinary.” [Instructor, Anaesthetist]
“Something simple that came up between the nurses and the surgeons in terms of when to do the count. And I can’t remember, the surgeons and the nurses each getting a shared insight into what each other’s priorities are at that time and how they can easily adjust or not to make it work for both of them.” [Instructor, Anaesthetist]
“The simulation process allows them to reflect on their degree of situational awareness they have, how they work safer as a team than as individuals in the team.” [Manager, Other]
The delivery of the courses in-situ using usual equipment and setup was described as facilitating engagement from participants by improving scenario realism (also discussed later), allowing for the troubleshooting of local systems and making access to simulation training possible for DHBs that did not have a local simulation centre.
“What happens in the simulation centre is everybody’s stressed. And everybody’s in an unfamiliar environment and so nobody performs to their best anyway. And besides, it goes back to the whole thing of, well actually if you’re stressing your own systems then you can improve your systems.” [Manager, Anaesthetist]
“It’s also checking the processes and the policies and things and how they work in real life situations. So, I think that’s what, one of the major benefits of doing in situ simulation is. Is that you’re actually trying it out and seeing what works and if it doesn’t work you can change things.” [Instructor, Anaesthetic Technician]
The realism of scenarios was described as a key strength that assisted with buy in to the programme, and active engagement during the course scenarios. Realism was described as particularly important for eliciting surgeon buy-in to the course.
“At the start of the session there’s varying degrees of engagement. And then once you actually run the scenarios which are quite realistic, and people actually do their role and they feel like it was kind of like a normal day in theatre.” [Instructor, Surgeon]
“I was involved with one of the scenarios and you’re actually sort of physically sweating… pretend scenarios where there’s no blood or anything is very hard to take seriously.” [Manager, Nurse]
“We practically had exactly the same scenarios happen with the guy with the amputated leg. We actually had pretty had much a very similar scenario happen about a year and a half ago.” [Instructor, Nurse]
The programme’s goals were described as highly relevant to the teamwork and communication priorities of instructors and managers alike.
“A lot of our issues are around communication breakdowns. So, you need any tools that we have that can help highlight and break down the hierarchical barriers.” [Instructor, Anaesthetic Technician]
“Our operating theatres should be a positive place for people to work, where there’s less of the hierarchical stuff of the past, where people can communicate and I think that’s going to lead to better patient outcomes.” [Instructor, Surgeon]
As described above, NetworkZ was viewed as uniquely valuable relative to other team training interventions currently available in New Zealand due to its multidisciplinary participation, in-situ delivery, realism and relevance to clinical practice. As such, interviewees felt that it could be useful for improving teamwork, local crisis response processes and patient safety in other areas of the hospital, such as emergency departments, intensive care units, post-operative care suites and in other surgical specialties such as obstetrics and gynaecology.
“We’ve got specialties that are really keen to do the programme but there are no scenarios for them.” [Instructor, Anaesthetic Technician]
“I’d like to see NetworkZ go into the different areas of the hospital, that we shouldn’t be the only ones that benefit from the training.” [Instructor, Anaesthetic Technician]
Theme 2: compatibility with existing systems
Strong compatibility of a programme with existing systems and values can increase the likelihood that it will be adopted and sustained.21 While the focus of NetworkZ was described as compatible with personal beliefs about the importance of great teamwork, the delivery of the courses was at times seen as challenging with available local resources.
Interviewees described resourcing of instructor time and theatre time as a key challenge. Some interviewees described friction between the ongoing delivery of the NetworkZ courses, and “time and money”, arising from busy staff workloads, fully booked theatres and the desire to fill theatres in order to maximise funding and reduce hospital deficits.
“It was a significant challenge for the DHB to be able to deliver that and maintain production.” [Manager, Surgeon]
“It’s been difficult to get that time allocated to the technical team so they’ve got time to go set up the day before.” [Instructor, Anaesthetist]
“We made a decision in the end that the only way we were going to make this work was to fit it into the programme of educational activities that we have once a month for an afternoon.” [Instructor, Anaesthetist]
As described in the quote above, a number of DHBs utilised pre-existing half-days set aside for education to run courses so that NetworkZ training had a negligible impact on their theatre lists. The use of non-operating education days minimised disruption to theatres but created an opportunity cost for instructors who missed other important education sessions that were happening at the same time. The time, cost and theatre space requirements of NetworkZ were larger than for many other courses held at these hospitals.
Theme 3: complexity of course delivery
Programme complexity can be a deterrent to programme adoption and sustainability.21 Instructing on local courses was described as a complex, and as described above, a resource-intensive experience. Interviewees referred to the “extra” tasks involved in setting up the courses and making them run smoothly on the day as often unaccounted for within the local resourcing budget.
“Not just an educational thing but all this. It goes where the manikin’s stored and who can touch it and who looks after it, the nurses can be freed up and contacting the surgeons to agree to it and all that sort of stuff.” [Instructor, Anaesthetist]
Tasks included recruiting staff, setting up the simulation equipment and organising course materials. For anaesthetic technicians and nurses, standard rostering practices did not offer instructors time to complete these tasks. For clinicians there was an option to use non-clinical time for this type of work, but interviewees noted there were many competing demands on this time. Thus, complexity in part reflected limited time to do the job as an “extra-curricular” instructor. Complexity was also generated by the multiple roles involved in delivering the courses, and the desire to ensure the courses were well run and were realistic for participants.
“It took us a long time to work out the different roles. [Instructor, Nurse]
There’s that pressure to run them smoothly on a, usually on a compressed timeframe.” [Instructor, Anaesthetist]
A number of interviewees also said that the complexity of course delivery meant they did not feel confident to provide optimal instructor training to their own local instructors. As such, they indicated a desire for ongoing national ‘expert’ support to deliver the instructors courses so that future staff were able to learn the content as it was originally intended. Similarly, many felt that national expertise would be important for troubleshooting, designing new scenarios and maintaining quality in the future. While people varied in their confidence about future delivery, some noted that they were likely to also need technical support to be able to deliver the programme beyond the initial period of support.
Interviewees in large DHBs reported challenges in course coordination and maintaining momentum when they did not have staff dedicated to support NetworkZ; those with dedicated staffing said this was critical to ensuring smooth running of the programme.
Theme 4: observability of programme impact
Observable benefits encourage adoption and continued support for programmes.19 The following sections discuss the improvements observed in (1) teamwork and communication, and (2) reduction in latent safety issues. Interviewees observed improved teamwork and communication at their DHB following NetworkZ training. Some interviewees referred generically to a sense of improved teamwork and communication skills, or improved teamwork behaviour.
“Having everyone work together during these simulations, it’s just really improving the communication between the groups and everyone’s starting to work together. It’s helping people get a better idea of everybody’s roles and what they’re capable of doing.” [Instructor, Anaesthetic Technician]
“Communication certainly seems to have improved in the fact that folk aren’t working in the little individual teams like the anaesthetics working independently from the scrub side of things. So there seems to be more teamwork.” [Instructor, Nurse]
Other observations about teamwork changes included better sharing of information between team members, improved confidence to speak up, better knowledge about their team members and reduced hierarchy.
“The briefings in the morning, they’re more clear. They’re more organised. So, calling each other by name.” [Instructor, Surgeon]
“Concepts like closed loop communication or introducing each other at the start or needing to have pre-briefings or sort of pauses in the middle of crises and stuff like that, that’s all just becoming a lot more familiar and expected and asked for.” [Instructor, Anaesthetist]
The delivery of the NetworkZ courses in local theatres provided opportunities to identify latent safety threats. Interviewees reported identifying gaps in staff knowledge of local crisis systems and equipment, and problems with those systems and equipment. Further training sessions were then held to respond to these gaps.
“People didn’t know how to use the defibrillator well and so it kind of brought out where the weaknesses were as a team.” [Instructor, Nurse]
“There was an issue with the Belmont [rapid infusion device] but there was some misunderstanding about how to set it up and of course that then generated an education session so the techs were more familiar with it.” [Instructor, Anaesthetist]
Other local improvements generated by the courses included changes to the process of ordering bloods in an emergency, the purchase of a second defibrillator and improved medication storage. Managers were generally supportive and hopeful about the programme impact. However, some felt they had not yet received sufficient information on course attendance or programme impact.
Discussion
Interviewees described the relative advantages of NetworkZ as multidisciplinary involvement, in-situ delivery, scenario realism, relevance to teamwork and communication in the operating theatre, and potential for translation beyond theatre to other areas in the DHB. While the training was compatible with local instructor and DHB interest in improved teamwork and perceived safety, it presented challenges due to pressure on staff time and operating theatre access. The perceived complexity of delivery suggested that ongoing dedicated expert support would be required. While those involved as instructors knew about the reported observable impacts of NetworkZ, this information was not always conveyed to senior management, who were the ones responsible for providing the time and resources required for course delivery.
Relative advantages
Overall interviewees described the programme as having several “relative advantages” over other teamwork or communication programmes they had attended, many of which did not involve established surgical teams working together in their own environment using realistic surgical models. These relative advantages drove engagement, and motivation to expand the programme to other hospital departments. Yet these advantages also made the programme more complex, requiring substantial resourcing. Decisions to cut programme costs or complexity need to be weighed up against the possibility that these will undercut programme benefits or stakeholder buy in. While interviewees spoke about a number of ways that the NetworkZ programme improved upon other available training programmes, funders will also need to view this programme as offering greater benefit than other available programmes. Continuing medical education resources are commonly allocated to passive forms of education such as conferences, yet these activities have only small effects on physician performance and outcomes.26,27 Sustaining a programme such as NetworkZ into the future will require a rethink of the way practitioners and institutions use their budgets for continuing medical education. Recognition by regulatory bodies of the need to improve teamwork and communication through multidisciplinary activities would support the sustainability of such initiatives.
Compatibility with existing systems
Programme adaptation to align with local context is another core component of most sustainability approaches.20,29 Other simulation training programmes have also noted challenges to sustainability and scalability, including challenges with financing, resourcing, recruiting and upskilling trainers.28 Our findings revealed a range of local ‘solutions’ to resourcing constraints. For example, some DHBs have chosen to run courses for half a day instead of a full day, and most have integrated the courses with existing time scheduled for education. Support from managers of each professional group can also assist in overcoming resourcing constraints.29,19 Further, national budget streams specifically designed for health workforce development could reduce some of the resourcing costs for DHBs, yet these currently do not have obvious mechanisms to facilitate or incentivise the pooling of training funds from different professional groups to enable the delivery of multidisciplinary training programmes.
Complexity of course delivery
The physical resources, instructor skill-levels and staff time required to deliver NetworkZ pose challenges for sustainability, due to the complexity of scheduling multidisciplinary participation, multiple roles and skillsets needed to run the simulation smoothly. Interviewees expressed the view that ongoing support would be needed beyond the current period of ACC funding. Declines in skill and fidelity are common for health interventions,29 and interviewees requested ongoing input from key experts to maintain the quality of NetworkZ. Other team simulation training programmes (eg, PROMPT) also use experts, rather than peers to train new instructors.10
Recruiting participants from all the professional groups that make up a team creates its own complexities and can be one of the biggest barriers to the delivery of simulation-based team training.30 Education has traditionally been delivered in professional silos, and multidisciplinary training is a paradigm shift creating challenges in timetabling, motivation to attend and funding streams. Leadership support is important for overcoming recruitment challenges,30 and adequate resourcing may assist here too. For large DHBs, dedicated staff time for a head instructor, or to cover recruitment, course coordination and timetabling was reportedly key to ensuring smooth delivery of the programme.
The existing delivery of NetworkZ would not have been possible without substantial funding from ACC. To sustain NetworkZ, ongoing funding will be required, either from a national source or local sites, or a combination of the two. Diverse sources of funding are likely to be more sustainable than a single funder.31
Observability of programme impacts
For managers, evidence of programme impact was of key interest. Ward et al32 similarly noted that research, clinical and patient experience, and local evidence was important for enabling the implementation of the TeamSTEPPS curriculum in rural hospitals. Likewise, a systematic review of sustainability approaches identified that building evidence was a key strategy of most sustainability models and approaches,20 to ensure decision makers are informed of the theoretical and empirical data justifying expenditure on the programme. Ultimately, as noted by some of the managers interviewed, the most convincing evidence for ongoing delivery of the programme would be improved patient outcomes. We are collecting evidence on patient outcomes, as well as teamwork and communication at a national level,24 but results will not be available before 2022. Thus collating short-term evidence that can be fed back to senior managers is also important. Documenting the improvements to local theatre systems, equipment and training may offer concrete, easy to collect, immediate evidence of programme impact for senior managers.
Strengths and limitations
A key strength of this work is sampling from multiple DHBs of different sizes from very large to very small, incorporating a range of perspectives from these varying contexts.
Limitations of the study include potential bias in data collection and analysis due to the researchers’ vested interest in the success of NetworkZ. The extent to which our findings can be generalised to other countries, other stakeholders not involved in establishing or running NetworkZ in their local DHBs and other similar training programmes remains to be tested.
One of the themes of Diffusion of Innovations theory that did not arise was ‘trialability’ of the intervention. The intervention was extensively trialled prior to obtaining funding for the programme. When signing up to the intervention, each DHB committed to training instructors, providing time for the course, and accepting responsibility for maintenance and depreciation of the gifted simulator. A trial run of the programme was therefore not feasible. However, as noted earlier, each DHB worked out for themselves how to implement the programme.
Future research
In this study we identified programme complexity as a potential challenge. The extent to which programme gains could be achieved by delivering parts of the course in a less resource-intensive workshop format or delivering some courses in purpose-built simulation facilities is yet to be tested. The surgical models, however, seem to be a valued, key component of the experience, allowing surgical participants to actively participate in the simulation. While studies suggest that repeated training is one way to increase the sustainability of simulation training benefits, other opportunities to sustain health programme benefits have yet to be explored,20 such as the value of incorporating the discussion of communication and teamwork concepts into the debriefing of real-life trauma cases. Resourcing instructors’ time arose as an issue in sustainability of the programme. The sustainability of unpaid instructor involvement in programmes such as NetworkZ has not, to our knowledge, been explored and may be a particular challenge for complex train-the-trainer programmes, particularly where these roles are part of organisational service but are not explicitly reimbursed. Optimal approaches to maintaining engaged instructors in sufficient numbers is a potential area for workforce development research.
Conclusion
NetworkZ invokes positive attitudes in local stakeholders, but its complexity and resource requirements pose a risk without adequate resourcing. Existing workforce development programmes are predominantly uni-professional, and the multi-professional requirements of NetworkZ pose additional complexity as existing systems for staff development within institutions, and discipline-specific continuing professional development are not set up to support multidisciplinary training. Team training programmes such as NetworkZ have the potential to yield important benefits for healthcare and require adequate resourcing. This will require a commitment from funders, institutions and professional bodies to building effective healthcare teams and better outcomes for patients.
Appendix
Semi-structured interview guide
Indicative interview questions and prompts
1. First I’d like to talk about your experience of the NetworkZ programme so far:
a. What’s been working well? What have been your highlights? What makes the course special, or different from other courses you’ve been involved in? (What do you see as the key strengths? Can you think of a time that illustrates X?)
2. What got you involved in NetworkZ in the first place? What do you value about being involved in the programme?
3. What positive impacts has the NetworkZ programme had in your DHB? (Are there any impacts that have come from delivering the course in-situ?)
4. In regards to your experience of the implementation of NetworkZ:
a. What’s been working well? What things were done in your DHB that you would recommend to other DHBs starting out? How have you overcome any challenges along the way?
5. It is intended that NetworkZ could become business as usual in the future. What is already happening to facilitate long-term sustainability of the programme in your DHB?
6. I’d like you to think about your aspirations for the NetworkZ programme, what benefits would you like to see the NetworkZ programme achieve for staff and patients? (What ideas do you have for improve the programmes’ impact on teamwork and patient safety?)
7. What do you see as the optimal way of delivering NetworkZ in your own DHB over the longer term? Pprompt for frequency, integrated vs siloed from other training (development of future scenarios, who should deliver the training)
8. What needs to happen to make NetworkZ reflect your vision for an ideal programme? What resources would be required? What support would be needed? From whom?(this question was dropped in the second iteration as it was largely covered in other questions)
9. Thinking about the longer term, what else can be done to ensure the programme is sustainable? How would the courses be delivered and adapted over time? Who would be involved? What options might exist for funding?
10. Is there anything else you would like to tell me about in relation to the NetworkZ programme?
1Additions and deletions made over the course of the interviews are detailed in parentheses.
Summary
Abstract
Aim
Healthcare is delivered by teams, but the training of healthcare staff is commonly undertaken in professional silos. This study investigated local perspectives on the sustainability of NetworkZ, a New Zealand national simulation-based multi-disciplinary operating room team training programme.
Method
Local course instructors and managers were invited to participate in semi-structured interviews. Diffusion of innovations theory was utilised to frame deductive thematic analysis of interview data.
Results
Twenty-seven people participated. Interviewees described valuing NetworkZ for its multi-disciplinary orientation, in-situ delivery, scenario realism, relevance to teamwork and communication and potential for generalisability to other settings. Interviewees also identified NetworkZ as generating improvements in teamwork and crisis management. NetworkZ was described as complex, due to multidisciplinary participation and the multiple roles and skillsets of instructors needed to run simulations smoothly, making the programme resource intensive to deliver.
Conclusion
NetworkZ is appreciated as a valuable and unique programme for developing important teamwork and communication skills. Its sustainability is dependent on adequate resourcing and funding.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1. Lingard L. Rethinking competence in the context of teamwork. In: Hodges B, Lingard L, eds. The Question of Competence: Reconsidering Medical Education in the Twenty-first Century. New York: Cornell University Press 2012:42–69.
2. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004; 13:330–34.
3. Lingard L, Reznick R, Espin S, et al. Team communications in the operating room: talk patterns, sites of tension, and implications for novices. Acad Med. 2002; 77(3):232–37.
4. Varpio L, Hall P, Lingard L, Schryer, C F. Interprofessional communication and medical error: a reframing of research questions and approaches. Acad Med. 2008; 83(10):S76–S81.
5. Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med. 2004; 79(2):186–94.
6. Hodges B, Lingard L. The question of competence: reconsidering medical education in the twenty first century. Ithaca and London: Cornell University Press 2012.
7. Institute of Medicine CoQoHCiA. To err is human: building a safer health system. Washington (DC): National Academies Press 2000.
8. McPherson K, Headrick L, Moss F. Working and learning together: good quality care depends on it, but how can we achieve it? Qual Health Care. 2001; 10:ii46–ii53.
9. Harvey EM, Echols SR, Clark R, Lee, E. Comparison of Two TeamSTEPPS® Training Methods on Nurse Failure-to-Rescue Performance. Clin Simul Nurs.2014; 10(2):e57–e64.
10. PROMPT Practical Obstetric Multi-Professional Training PROMPT Maternity Foundation; [Available from: www.promptmaternity.org/training/ accessed 27 September 2019].
11. Shoushtarian M, Barnett M, McMahon F, Ferris, J. Impact of introducing Practical Obstetric Multi-Professional Training (PROMPT) into maternity units in Victoria, Australia. BJOG. 2014; 121(13):1710–18.
12. Agency for Healthcare Research and Quality. TeamSTEPPS [Available from: http://www.ahrq.gov/teamstepps/index.html accessed 27 September 2019].
13. Clancy CM, Tornberg DN. TeamSTEPPS: assuring optimal teamwork in clinical settings. Am J Med Qual. 2007; 22(3):214–17.
14. Arriaga AF, Gawande AA, Raemer DB, et al. Pilot testing of a model for insurer-driven, large-scale multicenter simulation training for operating room teams. Ann Surg. 2014; 259(3):403–10.
15. Weller JM, Cumin D, Torrie J, et al. Multidisciplinary operating room simulation-based team training to reduce treatment errors: a feasibility study in New Zealand hospitals. NZ Med J 2015; 128(1418):40–51.
16. Weller JM, Cumin D, Civil ID, et al. Improved scores for observed teamwork in the clinical environment following a multidisciplinary operating room simulation intervention. NZ Med J 2016; 129(1439):59–67.
17. Accident Compensation Corporation. ACC 2018 [Available from: www.acc.govt.nz, accessed 18 October 2019].
18. NetworkZ. NetworkZ - Better outcomes in acute care 2019 [Available from: http://www.networkz.ac.nz/, accessed 18 October 2019].
19. Jowsey T, Beaver P, Long J, et al. Towards a safer culture: implementing multidisciplinary simulation-based team training in New Zealand operating theatres, a framework analysis. BMJ Open. 2019; 9:e027122.
20. Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018; 13(1):27.
21. Rogers EM. Diffusion of Innovations. New York: Free Press 2003.
22. Nowell LS, Norris JM, White DE, et al. Thematic analysis: Striving to meet the trustworthiness criteria. Int J Qual Methods. 2017; 16:1–13.
23. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006; 3(2):77–101.
24. Weller JM, Long JA, Beaver P, et al. Evaluation of the effect of multidisciplinary simulation-based team training on patients, staff and organisations: protocol for a stepped-wedge cluster mixed methods study of a national, insurer-funded initiative for surgical teams in New Zealand public hospitals. BMJ Open. 2020; 10:e032997
25. NVivo qualitative data analysis Software [program]. Version 12 version. Southport, United Kingdom, 2018.
26. Forsetlund L, Bjørndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009(2).
27. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof 2007; 27(1):6–15.
28. Delisle M, Pradarelli JC, Panda N, et al. Methods for scaling simulation-based teamwork training. BMJ Qual Saf 2020; 29(2):98–102.
29. Stirman SW, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012; 7:17–17.
30. Cumin D, Boyd MJ, Webster CS, Weller JM. A systematic review of simulation for multidisciplinary team training in operating rooms. Simul Healthc. 2013; 8(3):171–9.
31. Whelan J, Love P, Pettman T, et al. Cochrane Update: Predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health. 2014; 36(2):347–51.
32. Ward MM, Baloh J, Zhu X, Stewart, GL. Promoting action on research implementation in health services framework applied to TeamSTEPPS implementation in small rural hospitals. Health Care Manage Rev. 2017; 42(1):2–13.
For the PDF of this article,
contact nzmj@nzma.org.nz
Sustaining multidisciplinary team training in New Zealand hospitals: a qualitative study of a national simulation-based initiative | OPEN ACCESS
View Article PDF
Healthcare today is predominantly delivered by teams.1 There is burgeoning literature on teamwork, team competence and interprofessional learning, with convincing evidence that failures in teamwork and communication can lead to bad outcomes for patients.2–5 This suggests that team training should be incorporated into everyday practice.6–8 In New Zealand, multidisciplinary team training is not yet ‘business as usual’.
Evidence is emerging that the use of simulation can generate greater improvements in teamwork skills than team-based training delivered didactically.9 Internationally, there are a number of simulation-based team training initiatives such as PROMPT,10,11 TeamSTEPPS12,13 and a Harvard insurer-funded simulation programme for operating theatre teams.14 We set out to implement simulation-based team training for operating theatre teams in New Zealand.
We set out to develop simulations of surgical cases that equally engaged all members of the multidisciplinary surgical team. While technologies to simulate anaesthetic tasks were readily available, we were unable to source surgical models that we could integrate with an anaesthesia simulator to enable surgeons to perform procedures such as incision, resection, suturing or haemorrhage control. We therefore created these models working with a medical special effects company (MedicFX). The course underwent an extensive pilot with 20 full surgical teams from two large hospitals. Each team participated in a full-day course. Participant evaluations were very positive and we demonstrated improved scores for teamwork in the workplace.15,16 Two further courses in an operating theatre suite tested the feasibility of running the simulations in situ.
With funding from New Zealand’s national no-fault accident insurer, the Accident Compensation Corporation (ACC),17 we developed NetworkZ,18 a simulation-based multi-disciplinary team training programme for operating theatre teams, and established it in New Zealand public hospitals (see Table 1). The programme is described on the website www.networkz.ac.nz. Implementation of NetworkZ began in 2017 and has sequentially rolled out across New Zealand public hospitals over four years. At the time of writing, 1,082 participants had attended a NetworkZ course, and 279 local DHB health practitioners had begun training to be a NetworkZ instructor. Participants reflect the full range of operating room roles; consultant surgeons (15%), consultant anaesthetists (15%), surgical trainees (7%), anaesthetic trainees (5%), nurses (38%), anaesthetic technicians (13%) and other staff such as healthcare assistants (7%).
Table 1: The NetworkZ programme.
We know of no other national programme for multidisciplinary operating theatre teams that integrates a high fidelity anaesthesia simulator such as the Laerdal 3G simulator with surgical models on which surgeons can perform operative procedures.
In a qualitative study in the first Cohort of five district health boards (DHBs) Jowsey and colleagues19 identified local factors associated with implementation challenges and successes, which informed our approach to establishing NetworkZ in later groups of DHBs. As the implementation phase will be completed by the end of 2020, our attention is now on the sustainability of the programme.
Theoretical framework
We defined sustainability of NetworkZ in terms of the maintenance of programme activities into the future.20 As our theoretical framework we used Rogers’ Diffusion of Innovation theory,21 which explains how new practices become embedded in organisations or populations. Rogers proposes that influential early adopters are important, as are the preceding conditions, such as decision-maker characteristics and communication behaviour, and features of the intervention itself. Rogers proposes that the key features of interventions that promote positive attitudes towards an intervention are; (1) the perceived relative advantage of the intervention, (2) compatibility of the intervention with existing structures, (3) (low) intervention complexity, (4) the ability to test out the intervention prior to full implementation and (5) observability of the interventions’ impact. These attributes in turn influence adoption and discontinuation decisions.21
In the present study we explored the perspectives on long-term sustainability of NetworkZ with DHB staff involved in its local delivery or establishment. Using the theoretical lens of Rogers’ Diffusion of Innovation theory21 to interpret the data, we aimed to identify the elements of the NetworkZ programme that promoted positive attitudes, and elements that posed challenges for sustainability.
Method
In this qualitative study we undertook semi-structured interviews of staff involved with delivering or establishing the NetworkZ programme and used deductive thematic analysis22,23 of the data based on Diffusion of Innovation theory.21 The study is part of a larger programme of evaluation of NetworkZ, registered with ANZCTR (ACTRN12617000017325) and approved by the NZ Health and Disability Ethics Committee (16/NTB/143).24
Interview
The semi-structured interview guide focused on programme strengths, impact, implementation experiences and needs for ongoing sustainability (Appendix). Minor changes to the interview guide were made iteratively to prompt participants to elaborate further on topics relevant to the project.
All interviews were conducted by one member of the research team (JL) who has a doctorate in psychology, and has been trained in interview techniques. She introduced herself to participants as a researcher on the NetworkZ evaluation team.
Sample population
At the start of the rollout, all 20 DHBs were divided into five groups (which we call cohorts), representing similar population catchments. Cohort 2, the focus of this study, included DHBs from each size grouping. The two largest DHBs in Cohort 2 were in large metropolitan centres each with multiple hospital sites and their own local simulation centre.
Potential interview participants were selected from existing lists of people involved in course delivery (instructors) and those who were involved in the establishment of NetworkZ through their role as managers (managers). Management roles included clinical director, chief medical officer, nurse manager, operating room manager, charge nurse, project manager or quality specialist. Instructor roles included course instructor, convenor and simulation technicians. In order to preserve confidentiality, we refer to these people in the findings as either managers or instructors. Potential interviewees were approached via email and invited to participate in a telephone or face-to-face interview. Interviews were conducted between December 2018 and March 2019.
Sampling
We used purposive sampling to recruit potential interviewees in proportion to the size of each DHB, NetworkZ role and professional group. Interviewees were recruited until coding saturation was achieved.
Data management and analysis
Interviews were transcribed verbatim and de-identified. The interview data were initially coded using semantic thematic analysis,23 focusing on what was said (rather than what was implied). Codes were then deductively developed into themes aligned to Diffusion of Innovation Theory’s five main factors. Coding was conducted in QSR Nvivo 12 Pro software25 by author 1 and reviewed by author 2.
Results
Interviews were conducted between December 2018 and April 2019. Forty-nine people were invited via email to participate and 27 agreed. Of those who declined, reasons given were leaving their role, not knowing enough about NetworkZ or lack of time to participate. Interview length ranged from 8 to 51 minutes. Interviewee characteristics are outlined in Table 2.
Table 2: Interviewee characteristics.
Thematic overview
Elements of the programme that influenced attitudes to NetworkZ, and its ongoing sustainability are presented as themes using the Diffusion of Innovations framework (relative advantage, compatibility with existing systems, complexity of course delivery and observability of programme impacts) and subthemes specific to the interview data (Figure 1). Of note, a theme around trialability, one of the elements of the Diffusions of Innovations framework, did not emerge.
Figure 1: Thematic map.
Theme 1: relative advantage
Interviewees perceived the key strengths of NetworkZ to be: multidisciplinary focus, in-situ delivery, realism of the courses and the relevance of the teamwork and communication focus, and generalisability of the course to other settings.
The concurrent training of all members of the multidisciplinary team was seen as an advantage of the courses. Interviewees valued the uniqueness of having a course for a full theatre team, the opportunity to act in their own role, and to listen and support each other in the debrief after the scenarios.
“I’ve done a lot of courses for anaesthetists with pretend surgeons. I can’t think of anything like this where it was so multidisciplinary.” [Instructor, Anaesthetist]
“Something simple that came up between the nurses and the surgeons in terms of when to do the count. And I can’t remember, the surgeons and the nurses each getting a shared insight into what each other’s priorities are at that time and how they can easily adjust or not to make it work for both of them.” [Instructor, Anaesthetist]
“The simulation process allows them to reflect on their degree of situational awareness they have, how they work safer as a team than as individuals in the team.” [Manager, Other]
The delivery of the courses in-situ using usual equipment and setup was described as facilitating engagement from participants by improving scenario realism (also discussed later), allowing for the troubleshooting of local systems and making access to simulation training possible for DHBs that did not have a local simulation centre.
“What happens in the simulation centre is everybody’s stressed. And everybody’s in an unfamiliar environment and so nobody performs to their best anyway. And besides, it goes back to the whole thing of, well actually if you’re stressing your own systems then you can improve your systems.” [Manager, Anaesthetist]
“It’s also checking the processes and the policies and things and how they work in real life situations. So, I think that’s what, one of the major benefits of doing in situ simulation is. Is that you’re actually trying it out and seeing what works and if it doesn’t work you can change things.” [Instructor, Anaesthetic Technician]
The realism of scenarios was described as a key strength that assisted with buy in to the programme, and active engagement during the course scenarios. Realism was described as particularly important for eliciting surgeon buy-in to the course.
“At the start of the session there’s varying degrees of engagement. And then once you actually run the scenarios which are quite realistic, and people actually do their role and they feel like it was kind of like a normal day in theatre.” [Instructor, Surgeon]
“I was involved with one of the scenarios and you’re actually sort of physically sweating… pretend scenarios where there’s no blood or anything is very hard to take seriously.” [Manager, Nurse]
“We practically had exactly the same scenarios happen with the guy with the amputated leg. We actually had pretty had much a very similar scenario happen about a year and a half ago.” [Instructor, Nurse]
The programme’s goals were described as highly relevant to the teamwork and communication priorities of instructors and managers alike.
“A lot of our issues are around communication breakdowns. So, you need any tools that we have that can help highlight and break down the hierarchical barriers.” [Instructor, Anaesthetic Technician]
“Our operating theatres should be a positive place for people to work, where there’s less of the hierarchical stuff of the past, where people can communicate and I think that’s going to lead to better patient outcomes.” [Instructor, Surgeon]
As described above, NetworkZ was viewed as uniquely valuable relative to other team training interventions currently available in New Zealand due to its multidisciplinary participation, in-situ delivery, realism and relevance to clinical practice. As such, interviewees felt that it could be useful for improving teamwork, local crisis response processes and patient safety in other areas of the hospital, such as emergency departments, intensive care units, post-operative care suites and in other surgical specialties such as obstetrics and gynaecology.
“We’ve got specialties that are really keen to do the programme but there are no scenarios for them.” [Instructor, Anaesthetic Technician]
“I’d like to see NetworkZ go into the different areas of the hospital, that we shouldn’t be the only ones that benefit from the training.” [Instructor, Anaesthetic Technician]
Theme 2: compatibility with existing systems
Strong compatibility of a programme with existing systems and values can increase the likelihood that it will be adopted and sustained.21 While the focus of NetworkZ was described as compatible with personal beliefs about the importance of great teamwork, the delivery of the courses was at times seen as challenging with available local resources.
Interviewees described resourcing of instructor time and theatre time as a key challenge. Some interviewees described friction between the ongoing delivery of the NetworkZ courses, and “time and money”, arising from busy staff workloads, fully booked theatres and the desire to fill theatres in order to maximise funding and reduce hospital deficits.
“It was a significant challenge for the DHB to be able to deliver that and maintain production.” [Manager, Surgeon]
“It’s been difficult to get that time allocated to the technical team so they’ve got time to go set up the day before.” [Instructor, Anaesthetist]
“We made a decision in the end that the only way we were going to make this work was to fit it into the programme of educational activities that we have once a month for an afternoon.” [Instructor, Anaesthetist]
As described in the quote above, a number of DHBs utilised pre-existing half-days set aside for education to run courses so that NetworkZ training had a negligible impact on their theatre lists. The use of non-operating education days minimised disruption to theatres but created an opportunity cost for instructors who missed other important education sessions that were happening at the same time. The time, cost and theatre space requirements of NetworkZ were larger than for many other courses held at these hospitals.
Theme 3: complexity of course delivery
Programme complexity can be a deterrent to programme adoption and sustainability.21 Instructing on local courses was described as a complex, and as described above, a resource-intensive experience. Interviewees referred to the “extra” tasks involved in setting up the courses and making them run smoothly on the day as often unaccounted for within the local resourcing budget.
“Not just an educational thing but all this. It goes where the manikin’s stored and who can touch it and who looks after it, the nurses can be freed up and contacting the surgeons to agree to it and all that sort of stuff.” [Instructor, Anaesthetist]
Tasks included recruiting staff, setting up the simulation equipment and organising course materials. For anaesthetic technicians and nurses, standard rostering practices did not offer instructors time to complete these tasks. For clinicians there was an option to use non-clinical time for this type of work, but interviewees noted there were many competing demands on this time. Thus, complexity in part reflected limited time to do the job as an “extra-curricular” instructor. Complexity was also generated by the multiple roles involved in delivering the courses, and the desire to ensure the courses were well run and were realistic for participants.
“It took us a long time to work out the different roles. [Instructor, Nurse]
There’s that pressure to run them smoothly on a, usually on a compressed timeframe.” [Instructor, Anaesthetist]
A number of interviewees also said that the complexity of course delivery meant they did not feel confident to provide optimal instructor training to their own local instructors. As such, they indicated a desire for ongoing national ‘expert’ support to deliver the instructors courses so that future staff were able to learn the content as it was originally intended. Similarly, many felt that national expertise would be important for troubleshooting, designing new scenarios and maintaining quality in the future. While people varied in their confidence about future delivery, some noted that they were likely to also need technical support to be able to deliver the programme beyond the initial period of support.
Interviewees in large DHBs reported challenges in course coordination and maintaining momentum when they did not have staff dedicated to support NetworkZ; those with dedicated staffing said this was critical to ensuring smooth running of the programme.
Theme 4: observability of programme impact
Observable benefits encourage adoption and continued support for programmes.19 The following sections discuss the improvements observed in (1) teamwork and communication, and (2) reduction in latent safety issues. Interviewees observed improved teamwork and communication at their DHB following NetworkZ training. Some interviewees referred generically to a sense of improved teamwork and communication skills, or improved teamwork behaviour.
“Having everyone work together during these simulations, it’s just really improving the communication between the groups and everyone’s starting to work together. It’s helping people get a better idea of everybody’s roles and what they’re capable of doing.” [Instructor, Anaesthetic Technician]
“Communication certainly seems to have improved in the fact that folk aren’t working in the little individual teams like the anaesthetics working independently from the scrub side of things. So there seems to be more teamwork.” [Instructor, Nurse]
Other observations about teamwork changes included better sharing of information between team members, improved confidence to speak up, better knowledge about their team members and reduced hierarchy.
“The briefings in the morning, they’re more clear. They’re more organised. So, calling each other by name.” [Instructor, Surgeon]
“Concepts like closed loop communication or introducing each other at the start or needing to have pre-briefings or sort of pauses in the middle of crises and stuff like that, that’s all just becoming a lot more familiar and expected and asked for.” [Instructor, Anaesthetist]
The delivery of the NetworkZ courses in local theatres provided opportunities to identify latent safety threats. Interviewees reported identifying gaps in staff knowledge of local crisis systems and equipment, and problems with those systems and equipment. Further training sessions were then held to respond to these gaps.
“People didn’t know how to use the defibrillator well and so it kind of brought out where the weaknesses were as a team.” [Instructor, Nurse]
“There was an issue with the Belmont [rapid infusion device] but there was some misunderstanding about how to set it up and of course that then generated an education session so the techs were more familiar with it.” [Instructor, Anaesthetist]
Other local improvements generated by the courses included changes to the process of ordering bloods in an emergency, the purchase of a second defibrillator and improved medication storage. Managers were generally supportive and hopeful about the programme impact. However, some felt they had not yet received sufficient information on course attendance or programme impact.
Discussion
Interviewees described the relative advantages of NetworkZ as multidisciplinary involvement, in-situ delivery, scenario realism, relevance to teamwork and communication in the operating theatre, and potential for translation beyond theatre to other areas in the DHB. While the training was compatible with local instructor and DHB interest in improved teamwork and perceived safety, it presented challenges due to pressure on staff time and operating theatre access. The perceived complexity of delivery suggested that ongoing dedicated expert support would be required. While those involved as instructors knew about the reported observable impacts of NetworkZ, this information was not always conveyed to senior management, who were the ones responsible for providing the time and resources required for course delivery.
Relative advantages
Overall interviewees described the programme as having several “relative advantages” over other teamwork or communication programmes they had attended, many of which did not involve established surgical teams working together in their own environment using realistic surgical models. These relative advantages drove engagement, and motivation to expand the programme to other hospital departments. Yet these advantages also made the programme more complex, requiring substantial resourcing. Decisions to cut programme costs or complexity need to be weighed up against the possibility that these will undercut programme benefits or stakeholder buy in. While interviewees spoke about a number of ways that the NetworkZ programme improved upon other available training programmes, funders will also need to view this programme as offering greater benefit than other available programmes. Continuing medical education resources are commonly allocated to passive forms of education such as conferences, yet these activities have only small effects on physician performance and outcomes.26,27 Sustaining a programme such as NetworkZ into the future will require a rethink of the way practitioners and institutions use their budgets for continuing medical education. Recognition by regulatory bodies of the need to improve teamwork and communication through multidisciplinary activities would support the sustainability of such initiatives.
Compatibility with existing systems
Programme adaptation to align with local context is another core component of most sustainability approaches.20,29 Other simulation training programmes have also noted challenges to sustainability and scalability, including challenges with financing, resourcing, recruiting and upskilling trainers.28 Our findings revealed a range of local ‘solutions’ to resourcing constraints. For example, some DHBs have chosen to run courses for half a day instead of a full day, and most have integrated the courses with existing time scheduled for education. Support from managers of each professional group can also assist in overcoming resourcing constraints.29,19 Further, national budget streams specifically designed for health workforce development could reduce some of the resourcing costs for DHBs, yet these currently do not have obvious mechanisms to facilitate or incentivise the pooling of training funds from different professional groups to enable the delivery of multidisciplinary training programmes.
Complexity of course delivery
The physical resources, instructor skill-levels and staff time required to deliver NetworkZ pose challenges for sustainability, due to the complexity of scheduling multidisciplinary participation, multiple roles and skillsets needed to run the simulation smoothly. Interviewees expressed the view that ongoing support would be needed beyond the current period of ACC funding. Declines in skill and fidelity are common for health interventions,29 and interviewees requested ongoing input from key experts to maintain the quality of NetworkZ. Other team simulation training programmes (eg, PROMPT) also use experts, rather than peers to train new instructors.10
Recruiting participants from all the professional groups that make up a team creates its own complexities and can be one of the biggest barriers to the delivery of simulation-based team training.30 Education has traditionally been delivered in professional silos, and multidisciplinary training is a paradigm shift creating challenges in timetabling, motivation to attend and funding streams. Leadership support is important for overcoming recruitment challenges,30 and adequate resourcing may assist here too. For large DHBs, dedicated staff time for a head instructor, or to cover recruitment, course coordination and timetabling was reportedly key to ensuring smooth delivery of the programme.
The existing delivery of NetworkZ would not have been possible without substantial funding from ACC. To sustain NetworkZ, ongoing funding will be required, either from a national source or local sites, or a combination of the two. Diverse sources of funding are likely to be more sustainable than a single funder.31
Observability of programme impacts
For managers, evidence of programme impact was of key interest. Ward et al32 similarly noted that research, clinical and patient experience, and local evidence was important for enabling the implementation of the TeamSTEPPS curriculum in rural hospitals. Likewise, a systematic review of sustainability approaches identified that building evidence was a key strategy of most sustainability models and approaches,20 to ensure decision makers are informed of the theoretical and empirical data justifying expenditure on the programme. Ultimately, as noted by some of the managers interviewed, the most convincing evidence for ongoing delivery of the programme would be improved patient outcomes. We are collecting evidence on patient outcomes, as well as teamwork and communication at a national level,24 but results will not be available before 2022. Thus collating short-term evidence that can be fed back to senior managers is also important. Documenting the improvements to local theatre systems, equipment and training may offer concrete, easy to collect, immediate evidence of programme impact for senior managers.
Strengths and limitations
A key strength of this work is sampling from multiple DHBs of different sizes from very large to very small, incorporating a range of perspectives from these varying contexts.
Limitations of the study include potential bias in data collection and analysis due to the researchers’ vested interest in the success of NetworkZ. The extent to which our findings can be generalised to other countries, other stakeholders not involved in establishing or running NetworkZ in their local DHBs and other similar training programmes remains to be tested.
One of the themes of Diffusion of Innovations theory that did not arise was ‘trialability’ of the intervention. The intervention was extensively trialled prior to obtaining funding for the programme. When signing up to the intervention, each DHB committed to training instructors, providing time for the course, and accepting responsibility for maintenance and depreciation of the gifted simulator. A trial run of the programme was therefore not feasible. However, as noted earlier, each DHB worked out for themselves how to implement the programme.
Future research
In this study we identified programme complexity as a potential challenge. The extent to which programme gains could be achieved by delivering parts of the course in a less resource-intensive workshop format or delivering some courses in purpose-built simulation facilities is yet to be tested. The surgical models, however, seem to be a valued, key component of the experience, allowing surgical participants to actively participate in the simulation. While studies suggest that repeated training is one way to increase the sustainability of simulation training benefits, other opportunities to sustain health programme benefits have yet to be explored,20 such as the value of incorporating the discussion of communication and teamwork concepts into the debriefing of real-life trauma cases. Resourcing instructors’ time arose as an issue in sustainability of the programme. The sustainability of unpaid instructor involvement in programmes such as NetworkZ has not, to our knowledge, been explored and may be a particular challenge for complex train-the-trainer programmes, particularly where these roles are part of organisational service but are not explicitly reimbursed. Optimal approaches to maintaining engaged instructors in sufficient numbers is a potential area for workforce development research.
Conclusion
NetworkZ invokes positive attitudes in local stakeholders, but its complexity and resource requirements pose a risk without adequate resourcing. Existing workforce development programmes are predominantly uni-professional, and the multi-professional requirements of NetworkZ pose additional complexity as existing systems for staff development within institutions, and discipline-specific continuing professional development are not set up to support multidisciplinary training. Team training programmes such as NetworkZ have the potential to yield important benefits for healthcare and require adequate resourcing. This will require a commitment from funders, institutions and professional bodies to building effective healthcare teams and better outcomes for patients.
Appendix
Semi-structured interview guide
Indicative interview questions and prompts
1. First I’d like to talk about your experience of the NetworkZ programme so far:
a. What’s been working well? What have been your highlights? What makes the course special, or different from other courses you’ve been involved in? (What do you see as the key strengths? Can you think of a time that illustrates X?)
2. What got you involved in NetworkZ in the first place? What do you value about being involved in the programme?
3. What positive impacts has the NetworkZ programme had in your DHB? (Are there any impacts that have come from delivering the course in-situ?)
4. In regards to your experience of the implementation of NetworkZ:
a. What’s been working well? What things were done in your DHB that you would recommend to other DHBs starting out? How have you overcome any challenges along the way?
5. It is intended that NetworkZ could become business as usual in the future. What is already happening to facilitate long-term sustainability of the programme in your DHB?
6. I’d like you to think about your aspirations for the NetworkZ programme, what benefits would you like to see the NetworkZ programme achieve for staff and patients? (What ideas do you have for improve the programmes’ impact on teamwork and patient safety?)
7. What do you see as the optimal way of delivering NetworkZ in your own DHB over the longer term? Pprompt for frequency, integrated vs siloed from other training (development of future scenarios, who should deliver the training)
8. What needs to happen to make NetworkZ reflect your vision for an ideal programme? What resources would be required? What support would be needed? From whom?(this question was dropped in the second iteration as it was largely covered in other questions)
9. Thinking about the longer term, what else can be done to ensure the programme is sustainable? How would the courses be delivered and adapted over time? Who would be involved? What options might exist for funding?
10. Is there anything else you would like to tell me about in relation to the NetworkZ programme?
1Additions and deletions made over the course of the interviews are detailed in parentheses.
Summary
Abstract
Aim
Healthcare is delivered by teams, but the training of healthcare staff is commonly undertaken in professional silos. This study investigated local perspectives on the sustainability of NetworkZ, a New Zealand national simulation-based multi-disciplinary operating room team training programme.
Method
Local course instructors and managers were invited to participate in semi-structured interviews. Diffusion of innovations theory was utilised to frame deductive thematic analysis of interview data.
Results
Twenty-seven people participated. Interviewees described valuing NetworkZ for its multi-disciplinary orientation, in-situ delivery, scenario realism, relevance to teamwork and communication and potential for generalisability to other settings. Interviewees also identified NetworkZ as generating improvements in teamwork and crisis management. NetworkZ was described as complex, due to multidisciplinary participation and the multiple roles and skillsets of instructors needed to run simulations smoothly, making the programme resource intensive to deliver.
Conclusion
NetworkZ is appreciated as a valuable and unique programme for developing important teamwork and communication skills. Its sustainability is dependent on adequate resourcing and funding.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1. Lingard L. Rethinking competence in the context of teamwork. In: Hodges B, Lingard L, eds. The Question of Competence: Reconsidering Medical Education in the Twenty-first Century. New York: Cornell University Press 2012:42–69.
2. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004; 13:330–34.
3. Lingard L, Reznick R, Espin S, et al. Team communications in the operating room: talk patterns, sites of tension, and implications for novices. Acad Med. 2002; 77(3):232–37.
4. Varpio L, Hall P, Lingard L, Schryer, C F. Interprofessional communication and medical error: a reframing of research questions and approaches. Acad Med. 2008; 83(10):S76–S81.
5. Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med. 2004; 79(2):186–94.
6. Hodges B, Lingard L. The question of competence: reconsidering medical education in the twenty first century. Ithaca and London: Cornell University Press 2012.
7. Institute of Medicine CoQoHCiA. To err is human: building a safer health system. Washington (DC): National Academies Press 2000.
8. McPherson K, Headrick L, Moss F. Working and learning together: good quality care depends on it, but how can we achieve it? Qual Health Care. 2001; 10:ii46–ii53.
9. Harvey EM, Echols SR, Clark R, Lee, E. Comparison of Two TeamSTEPPS® Training Methods on Nurse Failure-to-Rescue Performance. Clin Simul Nurs.2014; 10(2):e57–e64.
10. PROMPT Practical Obstetric Multi-Professional Training PROMPT Maternity Foundation; [Available from: www.promptmaternity.org/training/ accessed 27 September 2019].
11. Shoushtarian M, Barnett M, McMahon F, Ferris, J. Impact of introducing Practical Obstetric Multi-Professional Training (PROMPT) into maternity units in Victoria, Australia. BJOG. 2014; 121(13):1710–18.
12. Agency for Healthcare Research and Quality. TeamSTEPPS [Available from: http://www.ahrq.gov/teamstepps/index.html accessed 27 September 2019].
13. Clancy CM, Tornberg DN. TeamSTEPPS: assuring optimal teamwork in clinical settings. Am J Med Qual. 2007; 22(3):214–17.
14. Arriaga AF, Gawande AA, Raemer DB, et al. Pilot testing of a model for insurer-driven, large-scale multicenter simulation training for operating room teams. Ann Surg. 2014; 259(3):403–10.
15. Weller JM, Cumin D, Torrie J, et al. Multidisciplinary operating room simulation-based team training to reduce treatment errors: a feasibility study in New Zealand hospitals. NZ Med J 2015; 128(1418):40–51.
16. Weller JM, Cumin D, Civil ID, et al. Improved scores for observed teamwork in the clinical environment following a multidisciplinary operating room simulation intervention. NZ Med J 2016; 129(1439):59–67.
17. Accident Compensation Corporation. ACC 2018 [Available from: www.acc.govt.nz, accessed 18 October 2019].
18. NetworkZ. NetworkZ - Better outcomes in acute care 2019 [Available from: http://www.networkz.ac.nz/, accessed 18 October 2019].
19. Jowsey T, Beaver P, Long J, et al. Towards a safer culture: implementing multidisciplinary simulation-based team training in New Zealand operating theatres, a framework analysis. BMJ Open. 2019; 9:e027122.
20. Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018; 13(1):27.
21. Rogers EM. Diffusion of Innovations. New York: Free Press 2003.
22. Nowell LS, Norris JM, White DE, et al. Thematic analysis: Striving to meet the trustworthiness criteria. Int J Qual Methods. 2017; 16:1–13.
23. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006; 3(2):77–101.
24. Weller JM, Long JA, Beaver P, et al. Evaluation of the effect of multidisciplinary simulation-based team training on patients, staff and organisations: protocol for a stepped-wedge cluster mixed methods study of a national, insurer-funded initiative for surgical teams in New Zealand public hospitals. BMJ Open. 2020; 10:e032997
25. NVivo qualitative data analysis Software [program]. Version 12 version. Southport, United Kingdom, 2018.
26. Forsetlund L, Bjørndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009(2).
27. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof 2007; 27(1):6–15.
28. Delisle M, Pradarelli JC, Panda N, et al. Methods for scaling simulation-based teamwork training. BMJ Qual Saf 2020; 29(2):98–102.
29. Stirman SW, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012; 7:17–17.
30. Cumin D, Boyd MJ, Webster CS, Weller JM. A systematic review of simulation for multidisciplinary team training in operating rooms. Simul Healthc. 2013; 8(3):171–9.
31. Whelan J, Love P, Pettman T, et al. Cochrane Update: Predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health. 2014; 36(2):347–51.
32. Ward MM, Baloh J, Zhu X, Stewart, GL. Promoting action on research implementation in health services framework applied to TeamSTEPPS implementation in small rural hospitals. Health Care Manage Rev. 2017; 42(1):2–13.
Contact diana@nzma.org.nz
for the PDF of this article
Sustaining multidisciplinary team training in New Zealand hospitals: a qualitative study of a national simulation-based initiative | OPEN ACCESS
Healthcare today is predominantly delivered by teams. There is burgeoning literature on teamwork, team competence and interprofessional learning, with convincing evidence that failures in teamwork and communication can lead to bad outcomes for patients.
Subscriber Content
The full contents of this pages only available to subscribers.
Login, subscribe or email nzmj@nzma.org.nz to purchase this article.
