Journal of the New Zealand Medical Association, 06-August-2004, Vol 117 No 1199
Clinical practice guidelines’ development and use in New Zealand: an evolving process
Eileen McKinlay, Deborah McLeod, Antony Dowell, Catherine Marshall
New Zealand (NZ) guideline development began around 1992 with various organisations funding guidelines on specific topics. In 1996, the New Zealand Guidelines Group (NZGG) was formed. This initiative was funded by the National Health Committee and based on the guideline development processes introduced by the Seattle Group Health Co-operative.1 The first initiative involved establishing a number of fellowships (for opinion leaders from a broad spectrum of clinical and consumer interests) to study guideline development in Seattle. Later, US and other international guideline developers were invited to NZ to run intensive training courses. The NZGG became one of the founders of the Guidelines International Network, which shares ‘evidence’ internationally.
Guidelines were seen as a way of introducing and promoting evidence-based practice. New Zealand government directives support the use of evidence-based practice and clinical practice guidelines.2,3 The original aim of the NZGG guidelines programme included formally training medical practitioners to undertake guideline development, and for these clinicians to develop evidence-based guidelines using the prescribed development process. An anticipated outcome was a movement towards an explicit evidence-based approach to health care provision (Figure 1).
Figure 1. Continuum of clinical decision-making
Since then, clinicians from other disciplines (nursing, social work, community health workers, mental health workers, occupational therapists, and disability sector workers), consumers, and cultural representatives have undertaken guideline development training and been involved on guideline development groups.
General practitioners are considered an important target user group for the New Zealand guidelines that have been developed to date. Early NZ guidelines were often printed in full and disseminated by post to GPs throughout New Zealand with no other form of implementation. Although there is considerable international research literature regarding barriers to guideline uptake by doctors including general practitioners (GPs), little research has been undertaken in NZ.
Internationally, research literature identifies that there are barriers to the adoption and use of guidelines; including those guidelines that originate from the doctor4 or the structure they work in; or due to patient-related factors.5 Some doctors are hesitant to use guidelines; they are suspicious both of the philosophy6 and reasons behind them,7,8; and the content of the guidelines themselves.9 Whilst the literature strongly supports consumers’ using guidelines to inform themselves and their care,10 the impact and actual use of guidelines by consumers has not been evaluated.
Although many studies have been undertaken on implementation, and although universally it is agreed that implementation is pivotal in ensuring guidelines are used, there is no clarity about the most effective implementation methods.11–14
In NZ, Thornley et al15 examined the effectiveness of postal dissemination of evidence-based guidelines on heavy menstrual bleeding. Prior to updating the recommendations on heavy menstrual bleeding, Park and Farquhar surveyed NZ GPs and gynaecologists regarding their current practice in treating heavy menstrual bleeding as well as perceived barriers to the recommendations of the 1998 guidelines.16
Arroll et al determined the reported use and perceived usefulness of four national guidelines by New Zealand GPs.17 Factors included targeted clinician education following dissemination, availability of the recommendations, and accessible decision support through the Adis New Ethicals (drug information) Catalogue.
Wynn-Thomas et al18 in their study of the use of the Ottawa ankle rules, surveyed NZ GPs about their use of selected New Zealand and international guidelines. GPs reported they ‘hardly ever’ or ‘never’ used guidelines in clinical practice. The effectiveness of the dissemination of the New Zealand Guidelines for the Diagnosis and Treatment of Adult Asthma was examined by Martin and Reid.19 Although guidelines were being (routinely) sent to all GPs; 2 weeks after it was sent, almost one-third of GPs could not remember receiving it.
In 2004, 46 guidelines have been posted in the NZGG guideline library, available on http://www.nzgg.org.nz In addition to NZGG sponsored guidelines, other independent guideline-developers continue to develop or update guidelines; many of these guidelines are also available on the NZGG website.
Primary care clinician acceptance and use of guidelines has been limited, and anticipated changes in clinical practice have not yet occurred. This study aimed to explore the barriers to guideline use by NZ general practitioners and to develop strategies to overcome identified barriers.
To gather information about guidelines and the use of guidelines by New Zealand clinicians, a literature search was undertaken. A literature review reference document was developed after searching for guidelines literature from the main databases—including Medline, Embase, CINAHL, and psycINFO. The review summarised the key themes from the available literature.20 Few NZ published or unpublished works were located. Any (unpublished) projects undertaken to evaluate NZ guidelines were identified and audited.
A purposeful sample of currently practicing New Zealand general practitioners (GPs)—representative of age, gender, years in practice, and urban/rural status—were invited to take part in this project. A purposeful sample of guideline stakeholders were invited to nominate a representative to be interviewed. Stakeholders included the NZGG, the Ministry of Health (MOH), the then Health Funding Authority (HFA, a health service purchaser now devolved into the MOH), PHARMAC (the New Zealand pharmaceutical regulator), and an Independent Practitioner Association (IPA, an organisation of GP providers).
Interview schedules were developed in collaboration with stakeholders. Schedules included open-ended questions about knowledge and use of existing guidelines; the role and importance of guidelines; and additional questions for GPs concerning their use of guidelines, perceived barriers and facilitators to the use of guidelines, and perceived consumer use of guidelines or other evidence-based information. Interviews were generally undertaken by one researcher in 2000/2001 and were either face-to-face, or via email or telephone if a face-to-face interview was not possible. Interviews were audio-taped and transcribed. GP interviews were continued until data saturation was reached.
Data from interviews were analysed using inductive thematic analysis, identifying themes either held in common or disparate between those interviewed, and themes that coincided or were different from the literature. Emerging themes were discussed by the research team. The results were presented to the NZGG Board and strategies were developed to address a number of issues identified from the data.
In collaboration with the Department of General Practice at Wellington School of Medicine and Health Sciences, strategies were identified, and a number of initiatives have since been undertaken by the NZGG to enhance end-user acceptance of guidelines. In addition, the research team developed a guideline evaluation framework21 based on the AGREE model.22
Five stakeholder organisations were asked to nominate a representative for interview, and 13 currently practicing general practitioners (GPs) were approached to be interviewed. All those approached, agreed to participate. GPs ranged in experience from a newly started practitioner through to one nearly retiring. They were representative of urban and rural practice and from both the North and South Islands.
At the time the interviews were undertaken, NZGG was an evolving organisation. A new full-time Chief Executive had been appointed, and several guidelines were ‘in production’. The reason for wanting to foster development of guidelines included ‘having assurance of a robust process to determine the evidence around the relative efficacy of clinical treatments’ and to ‘ensure that the general public were receiving interventions or services that would be of maximum benefit’. It was also hoped that guidelines could be used to advocate for access to the most effective forms of treatments (including cost effectiveness), even if sometimes this could require changes in prescribing regulations and pharmaceutical schedule funding. (NZGG believed that guideline development and use would contribute to a cultural shift over time towards the use of research evidence in clinical practice.)
The HFA believed guidelines had the potential to improve overall care, including the provision of consumer information, thus giving patients an expectation of care delivery. The HFA viewed the NZGG as an independent body with expertise and networks.
The MOH believed that evidence-based guidelines were one of a range of tools which could be used to enhance the appropriate quality and standards of the health service. The MOH believed that they should be responsible for creating an environment where guidelines were seen as useful tools. However, they did not believe it was their role to develop structural processes or to sponsor implementation of a specific quality improvement method to enhance guideline usage.
PHARMAC perceived NZGG’s main role as the development of evidence-based guidelines, and supported the use of guidelines in general, particularly those which met their funding/affordability objectives. The development of evidence-based guidelines appeared to influence funding decisions that PHARMAC made. However, PHARMAC recognised there was an inevitable tension when guidelines were released containing pharmaceutical recommendations that PHARMAC could not fund, or when there was a lag time in funding a guideline pharmaceutical recommendation.
At the time of interview, the IPA had no formal interfaces with the NZGG. They believed that ideally they should have an interface with the NZGG and identify clinical practice issues for consideration as guideline topics. IPAs had developed their own guidelines and education/implementation strategies for members, which they felt were effective. They noted that general practitioners preferred to use locally developed IPA guidelines, possibly because of their involvement in development and recommendation of services available locally.
New Zealand GPs’ use of (and attitudes to) guidelines
The key themes (to emerge from interviews with GPs) related to GP recognition of guideline formats; stakeholder endorsement; prioritisation of guideline development; GP information overload; guideline implementation issues; the relevance of guideline recommendations to general practice; and GP participation in guideline development groups.
GP recognition of guideline formats—Visual recognition of guideline documents within NZ appeared to be low with confusion between evidence-based guidelines and quasi guidelines or other information produced and disseminated to GPs by drug companies or interest groups.
Stakeholder impact—Endorsement by professional colleges or other professional networks was perceived to have a positive influence on the recognition and possible uptake of guidelines. Conversely, influence by Government or other health regulatory organisations was viewed negatively. GPs expressed concern that guidelines could be linked to contracts and that failure to comply with guidelines may have medicolegal implications.
Guideline development prioritisation—GPs were uncertain whether current NZGG guidelines met their need for evidence-based information, and believed the process of prioritising topics for guideline development was unclear. Furthermore, they believed that they needed to be involved in this prioritisation process through GP organisations (such as IPAs).
Information overload—The GPs reported that they were overwhelmed by the written material sent via post and electronically—with some Wellington GPs in 2000 having to read as much as 52 pieces of postal mail per day (excluding email).23 Newly arrived guidelines have to compete with patient-related and other essential information. GPs also reported difficulty in establishing effective storage and management systems for hardcopy guidelines, which inhibited timely retrieval.
Implementation issues—The interviewed GPs felt that effective implementation was essential to enhance the uptake of guideline recommendations. They did not feel that postal dissemination of guidelines (on its own) to GPs had been effective. Implementation strategies such as working in conjunction with stakeholders to locally redevelop and implement national guidelines were suggested. Short education programmes within scheduled GP organisation or peer review meetings and one-to-one practice education (academic credentialling) were also suggested.
Relevance of guideline recommendations to practice—Many of the GPs interviewed saw it as unfortunate that there were several recommendations made within the NZ guidelines (which were not relevant to their practice or were inaccessible, or out of the scope of, their practice) thus reflecting a stakeholder objective in developing the guideline to change policy rather than practice. An example frequently cited by general practitioners was the recommendation in the Guidelines for the treatment and management of depression by primary health professionals24 to use cognitive behavioural therapy in the treatment of depression, when this therapy was neither publicly funded nor freely available. Another example was the recommendation (made in the Guidelines) to use tranexamic acid for the management of heavy menstrual bleeding25 when, at the time the Guideline was released (and for some time after), this medication could only be prescribed by a specialist gynaecologist or obstetrician.
GP participation in guideline development groups—GP participation in guideline groups was viewed as onerous because of the time commitment involved through undertaking guideline development training, meeting time, and work required in between meetings. For GPs who had never been involved in guideline development, there was a perceived lack of GP involvement in guideline development teams, and (when GPs were known to be involved) a lack of a defined process for GP selection. GPs felt they were not consulted whilst the guideline was being developed.
Implications: NZGG initiatives to address barriers to guideline use
Strategy to ensure recognisability of NZGG guidelines—A uniform and recognisable appearance/brand has been developed for guidelines sponsored by the NZGG. Documents now routinely include abbreviated formats and consumer information. The NZGG logo is prominently displayed along with the logos of supporting organisations and professional bodies. This allows practitioners to easily differentiate between NZGG guidelines and guidelines developed by specific interest groups, therefore providing quality assurance.
Strategy to identify stakeholders—NZGG has established strong links with appropriate stakeholders to ensure goals are congruent and acceptable to guidelines end-users. Stakeholders nominate members to the guideline development teams, and are invited to peer review draft guidelines and comment on the penultimate version of the guidelines as part of the formal endorsement process.
Strategy to address guideline development priorities—The NZGG attempts to closely monitor possible need for guideline development though contacts with their funding bodies. To a large degree, the priority topics have been driven by the NZ Health Strategy26 or those areas where the greatest gap between current practice and evidence-based practice has been perceived. NZGG is working with the Ministry of Health to set up a process for liaising with stakeholders to discuss priorities for the future, although there is no guaranteed funding for priority area guideline development.
Strategy to address the need for appropriate information—The NZGG are attempting to provide additional print and electronic formats that enhance different reading styles. Summary sheets are available in hard copy and electronic format. Online access is available, and some IPAs are providing GPs with the NZGG guidelines and other information on CD-ROM.27 NZGG are now producing several information resources for consumers and other provider groups.
Strategy to address guidelines implementation issues—NZGG strongly supports the implementation of guidelines, and now requires guideline-development groups to also develop an implementation strategy. However, they acknowledge their dependency on funders to also support implementation. (In the past there has been no mandate for funders to routinely implement new guidelines.) Currently, several Primary Health Organisations (PHOs) are using the guidelines to build their primary care preventative programmes. In addition, other organisations such as IPAs, the Best Practice Advocacy Centre (BPAC), and the Goodfellow Unit (Department of General Practice and Primary Health Care, University of Auckland) have used the guidelines as base resources to develop CME programmes.
Since 2001 implementation strategies have been increasingly tailored according to the guideline topic and the end-user population. At regional and national meetings of healthcare practitioners, NZGG actively promote the main messages from guidelines.
A typical implementation strategy targeting GPs might now include: dissemination by post (in conjunction with material known to be read by GPs such as the biweekly medical newspapers); wide national and medical media coverage of the recommendations; PHARMAC- and District Health Board-funding (aimed at raising awareness of evidence-based strategies); development of a brief laminated guideline summary; guideline promotion at the RNZCGP or similar conferences; working with Consumer magazine to produce an article on the guideline recommendations for consumers; commissioning a patient-information resource; and running Continuing Medical Education (CME) sessions and sponsoring online CME .
Strategy to ensure there is relevance of guideline recommendations to clinical practice—Guideline-development groups now liaise with the Ministry of Health and other regulatory bodies (including Medsafe and PHARMAC) throughout the guideline-development process. Guideline-development teams are also asked to provide practical guidance to readers where there may be treatment or care options that are not affordable or accessible in NZ, and to make recommendations that are suited to the current NZ setting. Guidelines are also ‘road-tested’ before publication and when any impractical recommendations are identified, these are reviewed, and action is taken (where possible) to see if systemic changes can be encouraged to bring effect to the recommendations.
Strategy to address the involvement of GPs in guideline development teams—An independently appointed NZGG project manager now supports each development group. As many aspects of the guideline development process as possible are transparent, with guideline members disclosing vested or competing interest. Critical appraisal of literature relating to a clinical topic is now generally being undertaken by expert researchers rather than the development group themselves (which happened in the past). The NZGG has formulated a new evidence-grading system to include high quality qualitative research. This approach is important to enable the inclusion of evidence from cultural groups
There is now greater flexibility in timelines for guideline development, recognising that some clinical topics are complex to address. Generally speaking, NZGG are trying to speed up production of guidelines by taking the main bulk of the work away from the guideline-development team members. For example, clinicians are now routinely nominally paid for work undertaken in guideline development groups.
Since 1998 and the formation of the NZGG, increasing numbers of guidelines have been developed and disseminated for clinicians to use. The NZGG has fostered a robust, internationally recognised, evidence-based guideline-development process; and has rigorously protected the process.
There have been few evaluations of the impact and outcomes of guidelines produced and disseminated to New Zealand GPs. Those evaluations which have been undertaken suggest that the impact of guidelines has not been as substantial as hoped for.
Whilst there appears to be adequate knowledge around, and processes to undertake development of, ‘evidence-based’ guidelines; there has been a considerable knowledge-gap regarding the attitudes of NZ clinicians towards and current use of guidelines—including which guideline formats and implementation methods work best for NZ clinicians and consumers.
This qualitative review of the New Zealand guidelines movement has identified a number of issues impacting on guideline use by GPs. However, there are limitations in this study. The size of the sample may limit generalisability, and qualitative methods identify the range of opinion rather than the proportions of stakeholders who hold any given attitude. Nevertheless, the themes reported here recurred independently across interviews, and are consistent with international writing on guidelines.
New Zealand guideline-development appears to have been predominantly funder- rather than clinician-driven and this may have influenced the acceptability of the guidelines to GPs. Funders have initiated guideline-development for various reasons, primarily based on the need to close a perceived gap between current practice and evidence-based practice but also as a mechanism to drive policy changes. It is unfortunate that currently there is no formal mechanism for GPs to signal the need for guideline development on a particular topic.
Use of guidelines in the past to drive policy changes has lead to the inclusion of recommendations for pharmaceuticals and treatments that cannot be accessed by GPs—this has caused frustration for GPs. Lack of early liaison with stakeholder/regulator groups (including PHARMAC and the MOH) has exacerbated these issues. In addition, there is ongoing dialogue about whether recommendations (which promote interventions that are not available in NZ) should be included in a NZ guideline, even if they are recommended as ‘best’ practice.
In New Zealand, previously there has not always been a formal estimation of current practice prior to guideline-development, and clinical topics have been identified as priorities for guideline development without having this baseline information. Therefore it has been unclear whether NZGG guidelines have been effective in closing the gap between current- and evidence-based practice, or whether in fact the gap exists as a result of lack of knowledge of best practice or as a result of resource constraints or other factors.
NZGG have recently worked more closely with stakeholders in planning an overall guidelines strategy. Involving organisations such as the Royal College of General Practitioners and GP organisations potentially increases guidelines relevance for GPs. It is unfortunate that some early NZ guidelines were not well received by clinicians because of their format and/or their recommendations, and clinicians may have become averse to considering recently released guidelines. NZGG now informs stakeholder groups about any guidelines work in progress, and notifies them when to expect the release of new guidelines. This means that CME events can be arranged in advance. Unfortunately, the volume of guidelines being sent to GPs is still substantial (5 guidelines were released in 2002; 7 guidelines, 16 summaries and 1 evidence report completed in 2003, and 7 guidelines and 7 summaries anticipated to be completed in 2004). Thus, some mechanism whereby GPs can prioritise new guideline information and can request selected evidence summaries is still required.
In the past, there has not been a systematic process to implement guidelines in New Zealand. Guideline development groups have not been routinely funded to develop an implementation strategy and no single body has held responsibility to ensure implementation was occurring. Although guidelines are formally launched and disseminated by post to practicing clinicians, and some may receive media and other attention at the time of release, generally there is no ongoing formal implementation.
There is a substantial body of international literature on implementation, including literature that is specific to primary care.28–30 In summary, studies of guideline implementation have demonstrated that, despite positive attitudes towards guidelines,9,31 there is variable to low usage.9,32,33 However, the potential perceived by stakeholders (for guidelines to provide an effective vehicle for disseminating new information) drives continuing examination of different implementation strategies.
Meta-analyses and reviews conclude that different implementation interventions are effective under some circumstances, but none is effective in all circumstances.34–37 Implementation strategy must be planned, tailored to specific barriers to change38 and targets set.39
Although NZGG undertake the Agree Collaboration process for evaluating the rigor of guideline development, this model does not incorporate a formal evaluation of the effectiveness of dissemination and implementation. A model developed by the research team (based on the Agree Collaboration model) also includes an evaluation of the guideline topic selection process and the effectiveness of dissemination and implementation.21
In the 2 years since this data was collected, and in response to the project recommendations (based on international guideline research), the NZGG has actively attempted to address clinician barriers to using guidelines by various measures described above. There is a further commitment to addressing clinician barriers through implementation and by working more closely with practicing clinicians to determine professional needs that can be met by guideline development.
Author information: Eileen McKinlay, Lecturer in Primary Health Care, Department of General Practice, Wellington School of Medicine and Health Sciences, Otago University, Wellington; Deborah McLeod, Research Director, Department of General Practice, Wellington School of Medicine and Health Sciences, Otago University, Wellington; Antony Dowell, Professor and Head of Department, Department of General Practice, Wellington School of Medicine and Health Sciences, Otago University, Wellington; Catherine Marshall, Chief Executive, New Zealand Guidelines Group Inc, Wellington
Acknowledgements: We acknowledge the funding and support of the New Zealand Guidelines Group; and we also thank the study participants.
Correspondence: Eileen McKinlay, Lecturer in Primary Health Care, Department of General Practice, Wellington School of Medicine and Health Sciences, Box 7343, Wellington South. Fax: (04) 385 5539; email: email@example.com
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals