NZMA Home

Table of contents
Current issue
Search journal
Archived issues
NZMJ Obituaries
Classifieds
Hotline (free ads)
How to subscribe
How to contribute
How to advertise
Contact Us
Copyright
Other journals
The New Zealand Medical Journal

 Journal of the New Zealand Medical Association, 13-December-2002, Vol 115 No 1167

Adverse events in New Zealand public hospitals I: occurrence and impact
Peter Davis, Roy Lay-Yee, Robin Briant, Wasan Ali, Alastair Scott and Stephan Schug
Abstract
Aim To assess the occurrence and impact of adverse events in New Zealand public hospitals.
Methods Two-stage retrospective review of 6579 medical records, selected by systematic list sample from admissions for 1998 in 13 generalist hospitals providing acute care. After initial screening, medical records were reviewed by trained medical practitioners using a standardised protocol.
Results Except for hospital stay, the sample appeared to be closely representative of New Zealand public hospital admissions for 1998 on key demographic and clinical criteria. The proportion of hospital admissions associated with an adverse event was 12.9% (incidence rate, 11.2%), of which nearly one fifth had occurred outside a public hospital (mainly doctor’s rooms, patient’s home, rest home, or private hospital). Most adverse events had minor patient impact, with less than 15% associated with permanent disability or death. Hospital workload was strongly affected, however, with adverse events adding an average of over nine days (median 4 days) to the expected hospital stay. There was limited evidence of patterning by diagnostic category. The elderly were disproportionately affected.
Conclusions The study provides representative base parameters that can contribute to the wider understanding, and potential improvement, of patient safety and the quality of care in New Zealand public hospitals.

The subject of patient safety, and the quality of healthcare, has gained increasing momentum internationally as a major focus of attention in professional and health policy circles. This has been highlighted recently by the report on patient safety from the Institute of Medicine in the United States,1 by an entire issue of the British Medical Journal devoted to medical error,2 and at least two high-profile reports on aspects of patient safety in the United Kingdom’s National Health Service.3,4 The matter has also gained attention in the UK because of the high level of public interest in the Bristol incident.5
In New Zealand, a number of reports have also highlighted quality issues,6,7 yet the question of patient safety has, to date, been the subject of relatively little systematic research. One of the first studies to use a standardised, epidemiological approach was a survey of adverse drug events among over 9000 admissions to Dunedin Hospital in the early 1970s.8 Although useful research since that time has been carried out on surgical audit9 and anaesthetic error,10 no generic, epidemiological data on adverse events have been published in this country. The absence of such data has been recognised as an obstacle to developing proposals for the regulation of safety in health and disability in New Zealand.11
A major scientific stimulus to rigorous epidemiological research on patient safety was the development of standardised procedures for the assessment of adverse events using medical records.12 This methodological approach was tested for its applicability in both the British13 and Australian contexts,14 and has further been tested for its feasibility in New Zealand.15,16
The objective of this study was to assess the occurrence and impact of adverse events in New Zealand public hospitals as revealed in an audit of a representative sample of medical records.

Methods

Sampling The survey population was defined as all patients admitted to 20 general hospitals with more than 100 beds for the calendar year 1998 (excluding day, psychiatric and rehabilitation-only cases). A random sample of 13 public hospitals was then selected following stratification by hospital type (ie, service facilities provided) and geographical area. The sampling frame for each hospital was a list of all eligible admissions in that hospital, ordered by admission date. To ensure distribution throughout the year, systematic list sampling from a random start point was used to identify 575 admissions from each hospital. Initially, 7475 records were selected for screening. However, for a variety of reasons – wrongly sampled record, missing record, current inpatient, and other – 744 records could not be screened, leaving 6579 records for assessment (after double admissions were also removed).
To be included in the study, an adverse event had to be related to, or have occurred during, the sampled admission. The feasibility of using this method was tested in three major hospitals in Auckland.16
Administrative data For each sampled admission, the New Zealand Health Information Service (NZHIS) provided admissions information (dates of admission and discharge, admission type (planned or acute), and admission source (routine or transfer)); socio-demographic data (age, gender, ethnicity, domicile code); and clinical data (diagnostic classification).17 NZDep96 quintiles were derived from patient domicile codes as a measure of residential area deprivation.18 Principal diagnosis or reason for admission was classified according to 25 Major Diagnostic Categories (MDCs) derived from Australian AN-DRG 3.1.17
Data collection The core data collection procedure of the study was a two-stage retrospective review of each selected medical record. This involved the use of two protocols. At the first stage, a screening protocol – Review Form 1 (RF1) – was administered by specially trained Registered Nurses (RNs) to determine if the sampled admission met any of 18 screening criteria selected as potentially indicative of an adverse event. The screening criteria included unplanned admission before the sampled admission, and unplanned readmission after discharge from the sampled admission, among others.19
Those records indicating positive on the initial screening stage were passed on for further consideration. The objective of the second stage was to determine whether the sampled admission was associated with an adverse event, and if so, to then characterise that adverse event according to key clinical criteria. Review Form 2 (RF2) guided these judgements – using structured implicit review – and was administered by specially trained and experienced Medical Officers (MOs). Both review forms were closely modelled on the comparable instruments in the American20,21 and Australian studies.14 Structured implicit review is the guided exercise of professional judgement to facilitate reliable detection and determination of adverse events. A series of seven evaluation questions were used to assist reviewers in arriving at this judgement. The degree of certainty accorded to this assessment was translated into a six-point confidence scale of evidence of causation by healthcare management. This ranged from 1 = virtually no evidence, to 6 = virtually certain evidence.19
An expert reviewer arbitrated on discrepant judgements (in which an RN and an MO disagreed), and carried out an independent review of a one in ten sub-sample of medical records. Measures of concurrent validity22 were used to determine the quality of screening and review.
Definitions An adverse event was operationally defined as: 1) an unintended injury; 2) resulting in disability; and 3) caused by healthcare management rather than the underlying disease process. Each of these criteria had to be fulfilled. Figure 1 provides two examples of unwanted outcomes of treatment; one is not classified as an adverse event and the other is an adverse event of low preventability.
Figure 1. Examples of event occurrences synthesised from real cases19

Example 1: Not an adverse event; outcome of disease

An 80-year-old man presented with a myocardial infarction, with three hours of chest pain. He was treated promptly with streptokinase, heparin and aspirin. On day three he had further chest pain, with new ECG changes, and he died 12 hours later of cardiogenic shock.
No adverse event = no medical causation, outcome of disease

Example 2: Adverse event, operative(fracture management); low preventability*

Young, right-handed man sustained a fracture of the radius within the wrist joint. It required operative reduction, K-wire fixation and bone grafting. At the 10-day check, the position had shifted, and re-operation was required. The end result was very good.
Adverse event = operative, low preventability, moderate disability
* Preventability defined as an error in healthcare management due to failure to follow accepted practice at an individual or system level

Disability was categorised into one of the following types: temporary; lasting up to one year; permanent impairment of function; or death. Attributable bed days refer to those extra days associated with an adverse event that were spent in the study hospital during one or more admissions.
Results
For over 85% of sampled records, available information was sufficient to complete all aspects of the RF1. Similarly, for approximately 95% of all medical records classed as adverse events, the available information was sufficient to complete all aspects of the MO review using the RF2.19
The representativeness of the sample was assessed by comparing the distribution of key patient characteristics with the pattern for all New Zealand publicly-funded hospital admissions in 1998 (Table 1). Sample figures for age, gender, ethnic group, discharge status and mortality were all closely comparable to national data, whereas length of stay in the sample appeared to be notably shorter.

Table 1. Patient characteristics, 1998; study sample vs all publicly-funded hospital patients

Patient characteristics
Sample
New Zealand*
Number of inpatient admissions
Mean age (years)
Males (%)
Maori (%)
Routine discharge (%)
Deaths (%)
Mean hospital stay (days)
6,579
42.6
45.1
15.4
91.6
1.8
5.1
699,095
40.3
43.5
14.3
92.1
1.7
6.9
*All publicly-funded hospitalisation in New Zealand (may include private hospital admissions) (Source: New Zealand Health Information Service)
Excludes day and psychiatric patients
Excludes specialist public hospitals, public hospitals with under 100 beds, and rehabilitation-only patients

The frequency of occurrence of adverse events is considered in Table 2. Such events were associated with 12.9% of admissions. This represents all incidents recorded by a healthcare professional over the period 1998 to 2000 (field work date) in a population of hospital patients admitted in 1998. Technically, this is a prevalence figure. Adjusting for the differential probability of selection of admissions (attributable to the stratified cluster sample design) increased this proportion slightly (13.1%). The incidence rate – of 11.2% – represents only cases recorded during the 1998 sampled admission. This again increased slightly on adjustment for sample design (11.3%).

Table 2. Adverse events: prevalence and incidence


Prevalence*
Incidence
Crude rate
850/6579 = 12.9%
735/6579 = 11.2%
Adjusted for sample design
13.1%
(95% CI; 12.2–14.1)
11.3%
(95% CI; 10.5–12.2)
*Prevalence defined as all adverse events found by the study review process as a proportion of sampled 1998 admissions
Incidence rate defined as incidents recorded by a healthcare professional during the 1998 sampled admission (and later assessed to be an adverse event by a study reviewer)

While all the events reported here were recorded and treated in public hospitals, not all had occurred in such institutions (Table 3). While the great majority of adverse events had occurred inside a public hospital (80.4%), nearly one fifth had taken place outside; most commonly in a doctor’s rooms, at the patient’s home, or in a rest home or private hospital.

Table 3. Distribution of adverse events (AEs), by location of occurrence

Location
All AEs (%)
Inside public hospital

Outside public hospital:
doctor’s rooms
ambulatory care unit
home
rest home
private hospital
other
80.4

19.6
6.4
1.3
5.3
3.8
2.0
0.9
Total (n = 850)
100.0

In Table 4, the impact of adverse events is assessed according to patient disability and hospital workload (number of extra days’ stay). For the great majority of patients, disability was either minimal or moderate. However, it should be noted that nearly 15% of patients either suffered permanent disability or died. On average, an extra 9.3 days was required for treatment (median of 4 days). There was a close association between disability status and length of stay, with the permanently disabled requiring between three and five weeks’ extra stay.
Table 4. Impact of adverse events (AEs): patient disability status and workload (extra hospital stay)

Disability
AEs
(%)
Attributable bed days per AE*
mean (median)
Minimal <1 month
Moderate 1–12 months
Permanent ≤50% of function
Permanent >50% of function
Death
Unable to determine from medical record
61.6
19.0
7.9
2.3
4.5
4.7
4.7 (3)
13.8 (8)
23.8 (13)
38.7 (35)
11.5 (4)
11.6 (7)
All AEs (n = 850)
100.0
9.3 (4)
*Extra bed days associated with an adverse event that were spent in the study hospital during one or more admissions
Period of disability

The impact of adverse events in relation to major diagnostic criteria is represented in Table 5. This shows the distribution of admissions and adverse events, together with two impact criteria – permanent disability and death, and extra stay in hospital. There were few striking discrepancies; adverse events seemed to be over-represented in injury-related and musculoskeletal MDCs, and less common in birth-related admissions. Adverse events associated with digestive, respiratory and nervous systems seemed to show greater patient impact.

Table 5. Distribution of admissions and adverse events (AEs) and impact of AEs, by Major Diagnostic Category (MDC)

MDC*
Admissions (%)
(n = 6579)
AEs
(%)
(n = 850)
Permanent disability or death (%)
Mean attributable bed days per AE
Circulatory system
Musculoskeletal system
Pregnancy, childbirth
Digestive system
Respiratory system
Newborns/neonates
Nervous system
Skin, tissue
Kidney and urinary tract
Injuries, poisoning and drugs
Other (remaining 15 MDCs)
13.4
11.3
11.2
10.1
8.6
7.4
6.5
3.9
3.7
3.3
20.7
13.5
17.7
6.6
11.7
6.0
4.0
4.9
3.9
4.9
6.5
20.4
17.4
16.7
3.6
19.2
23.5
2.9
28.6
6.1
19.1
3.6
12.1
9.0
11.2
2.5
14.6
8.9
3.5
8.7
6.8
11.5
7.9
9.0
Total
100.0
100.0
14.5
9.3
*Principal diagnosis was classified according to 25 Major Diagnostic Categories (MDCs) derived from Australian AN-DRG 3.1 and ordered according to percentage of admissions
Extra bed days associated with an adverse event that were spent in the study hospital during one or more admissions

The same set of data is presented in Table 6 for a range of socio-demographic factors; namely, age group, gender, ethnic group, and area deprivation. Overall, the pattern of adverse events mirrored that of admissions closely for all comparisons, except for patients over the age of 65. While this group accounted for one third of admissions, it was associated with 40% of adverse events. Patient impact and workload was also slightly higher for those over 65.

Table 6. Distribution of admissions, adverse events (AEs) and impact of AEs, by socio-demographic factors

Socio-demographic characteristics
Admissions (%)
(n = 6579)
AEs (%)
(n = 850)
Impact of AEs



Permanent disability or death (%)
Mean attributable bed days per AE*
Age group
0–14
15–29
30–44
45–64
65+

20.5
16.4
15.3
18.0
29.9

12.0
12.2
15.2
19.9
40.7

6.9
1.9
10.9
21.3
18.8

11.0
3.6
6.7
10.6
10.9
Gender
Male
Female

45.1
54.9

44.7
55.3

16.3
13.2

10.5
8.4
Ethnic group
European
Maori
Pacific
Other

71.7
15.4
3.7
9.2

73.7
15.9
3.8
6.7

16.0
10.4
9.4
12.3

9.6
9.2
6.2
8.6
Area deprivation score (quintiles)
1
2
3
4
5
(n = 6502)

12.7
14.0
20.8
24.4
28.2
(n = 843)§

11.4
15.2
22.0
24.6
26.9

8.3
14.1
16.2
15.0
16.3

8.8
8.1
10.0
9.7
9.4
*Extra bed days associated with an adverse event that were spent in the study hospital during one or more admissions
NZDep96 quintiles were derived from patient domicile codes as a measure of residential area deprivation; quintile 5 represents the highest level of deprivation
77 of the admissions could not be coded
§7 of the adverse events could not be coded

Discussion
A primary objective of this investigation was to establish the occurrence of adverse events in New Zealand public hospitals by assessing a representative sample of admission records according to a standardised audit protocol. Using this methodology, it was estimated that 12.9% of hospital admissions were associated with an adverse event. This rate stands almost midway between the levels recorded in two countries with shared medical traditions in training and practice: Australia (16.6%)14 and the UK (10.8%).13
A second major objective of the investigation was to assess the impact of adverse events, both on patients and on hospital workload. The noteworthy finding here is the quite mixed signals on the magnitude of impact (Tables 4 and 6). Less than 15% of adverse events were associated with permanent disability or death (and the great majority of events resulted in relatively minor impact on patients). This outcome is consistent with findings in other studies.13,14,20 However, when considering the impact on hospital workload, adverse events added an average of over nine days to expected hospital stay, an outcome that was similar to the Australian finding of an average of just over seven days.14 It should also be noted that there were few demographic or clinical patterns in the data, aside from the evident vulnerability of older patients.
Another area of interest that emerged from this study is the significant proportion – about one fifth – of adverse events that originated outside a public hospital (Table 3). This result has not been previously reported in the international literature and points to the potential importance of quality and safety issues in primary and community care and in other institutional settings.
The great strength of the study is its representativeness. On key criteria, the sample of records shows a close approximation to the pattern of admissions for all New Zealand hospitals in 1998. However, it should be noted that the sample draws only on generalist, acute hospitals with 100 beds or more, and that length of stay was on average lower (5.1 in the sample, 6.9 in all publicly-funded hospital admissions). The documentation in sampled medical records was sufficiently detailed and comprehensive to permit full completion of study instruments, and there was evidence of internal consistency in the data on key study variables (for example, the relationship between assessed patient disability and extra hospital workload).
Yet, there still remain questions about the quality of key study measures, such as adverse event status and preventability. Study instruments were directly applied, with little if any modification, from internationally established protocols. These rely on the guided judgement of screeners and reviewers – structured implicit review – and are thus potentially subject to observer variability. Hence, for example, the measure of agreement between MO reviewers and the expert reviewer on adverse event determination in this study – kappa 0.47 – was only of moderate strength,19 although within the range for comparable studies.14,23 Furthermore, while the adverse event rate reported in this study clusters with those for Australia14 and the UK,13 it differs significantly from those reported for the United States.20,21,23
An analysis of this overall discrepancy in adverse event rate between Australia and the United States suggests that slight differences in methodology were partly to account, but that the principal explanation lay in contrasting study purposes – medicolegal in one case (the United States), quality in the other (Australia).24 Nevertheless, there remains an irreducible element of subjectivity to the core study instrument, with the potential for considerable observer variability. This constrains the interpretation of any apparent variations in adverse event rates.
This investigation establishes broad clinical and managerial parameters for our understanding of patient safety and the quality of care in New Zealand public hospitals. The findings suggest that adverse events are as significant a problem in New Zealand as they are in Australia, the UK, and the United States. In essence, about one in eight admissions to a hospital are associated with adverse events (which may have occurred within or outside public hospitals). The majority of such incidents have a relatively minor impact on patients (though there is a significant proportion who suffer permanent disability or death), but their effects on hospital workload, and thus costs to the health system, are substantial.
There remain a number of issues unresolved from this investigation. First, there are still questions about the measurement properties of structured implicit review in identifying adverse events from medical records. Further methodological work is required in this area. Second, more detailed analysis of the data from this study – and others – is required in order to provide insight into the detailed patterns of adverse event occurrence and determination, particularly in relation to preventability. Preliminary work in this area has shown that only 6.3% of admissions to New Zealand public hospitals were associated with adverse events that were both preventable and occurred in hospital.25 From such work may come indications for quality improvement initiatives, together with testable propositions for strategies designed to reduce the level of preventable adverse events.
In summary, the first nationally representative audit of medical records in New Zealand public hospitals has identified a level of medical injury that is similar to that recorded in comparable countries. There is a considerable impact of adverse events on hospital workload, and a significant minority of patients suffers death or permanent disability.
Author information: Peter Davis, Professor, Department of Public Health and General Practice, Christchurch School of Medicine and Health Sciences, University of Otago; Roy Lay-Yee, Analyst; Robin Briant, Clinical Director, Division of Community Health, Faculty of Medical and Health Sciences, University of Auckland; Wasan Ali, Visiting Research Fellow, Department of Public Health and General Practice, Christchurch School of Medicine and Health Science, University of Otago; Alastair Scott, Professor, Department of Statistics, University of Auckland; Stephan Schug, Professor, Department of Anaesthesia, University of Western Australia, Perth, Australia
Acknowledgements: Work on this study was funded by the Health Research Council of New Zealand. We thank the 13 New Zealand hospitals that participated in the study and Dr David Richmond (Chair) and members of the study’s Advisory and Monitoring Committee. We also thank Sandra Johnson and Wendy Bingley, our medical review and data processing teams, and hospital records staff.
Correspondence: Professor Peter Davis, Department of Public Health and General Practice, Christchurch School of Medicine, University of Otago, P O Box 4345, Christchurch. Fax: (03) 364 0425; email: peter.davis@chmeds.ac.nz
References:
  1. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human. Building a safer health system. Washington, DC: National Academy Press, 1999. Available online. URL: http://www.nap.edu/books/0309068371/html/ Accessed December 2002.
  2. BMJ themed issue. Reducing error, improving safety. BMJ 2000; 320:725-814.
  3. National Health Service. An organisation with a memory: A report of an expert group on learning from adverse events in the NHS. London: Department of Health, 2000.
  4. National Health Service. Building a safer NHS for patients. London: Department of Health, 2001.
  5. The Bristol Royal Infirmary Inquiry. The Report of the Public Inquiry into Children’s Heart Surgery at the Bristol Royal Infirmary 1984–1995. London: The Stationary Office, 2001.
  6. Stent RK. Canterbury Health Ltd. A Report by the Health and Disability Commissioner. April 1998.
  7. Paterson R. Gisborne Hospital 1999–2000. A Report by the Health and Disability Commissioner. March 2001.
  8. Smidt NA, McQueen EG. Adverse reactions to drugs: a comprehensive hospital inpatient survey. NZ Med J 1972;76:397–401.
  9. Pettigrew RA, McDonald JR, van Rij AM. Developing a system for surgical audit. Aust NZ J Surg 1991;61:563–9.
  10. Merry AF, Peck DJ. Anaesthetists, errors in drug administration and the law. NZ Med J 1995;108:185–7.
  11. Ministry of Health. Purchasing for your health 1994/95. Wellington: Ministry of Health, 1995.
  12. Hiatt HH, Barnes BA, Brennan TA, et al. A study of medical injury and medical malpractice. N Engl J Med 1989;321:480–4.
  13. Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322:517–9.
  14. Wilson RM, Runciman WB, Gibberd RW, et al. The quality in Australian health care study. Med J Aust 1995;163:458–71.
  15. Davis P, Lay-Yee R, Schug S, et al. Adverse events regional feasibility study: indicative findings. NZ Med J 2001;114:203–5.
  16. Davis P, Lay-Yee R, Schug S, et al. Adverse events regional feasibility study: methodological results. NZ Med J 2001;114:200–2.
  17. New Zealand Health Information Service. Data Dictionary National Minimum Dataset. Wellington: Ministry of Health, 1998.
  18. Crampton P, Salmond C, Sutton F. NZDep96: Index of Deprivation. Report No 8. Wellington: Health Services Research Centre, 1997.
  19. Davis P, Lay-Yee R, Briant R, et al. Adverse events in New Zealand public hospitals: principal findings from a national survey. Occasional Paper No 3. Wellington: Ministry of Health, 2001. Available online. URL: http://www.moh.govt.nz/moh.nsf Accessed December 2002.
  20. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370–6.
  21. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324:377–84.
  22. Aday LA. Designing and conducting health surveys: a comprehensive guide. 2nd ed. San Francisco: Jossey-Bass Publishers, 1996.
  23. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38:261–71.
  24. Thomas EJ, Studdert DM, Runciman WB, et al. A comparison of iatrogenic injury studies in Australia and the USA I: context, methods, casemix, population, patient and hospital characteristics. International Journal for Quality in Health Care 2000;12:371–8.
  25. Davis P, Lay-Yee R, Briant R, et al. Adverse events in New Zealand public hospitals II: preventability and clinical context. NZ Med J (under review)


     
Current issue | Search journal | Archived issues | Classifieds | Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals