 
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Journal of the New Zealand Medical Association,
13-December-2002, Vol 115 No 1167
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Adverse events in New Zealand public hospitals I: occurrence
and impact
Peter Davis, Roy Lay-Yee, Robin Briant, Wasan Ali, Alastair
Scott and Stephan Schug
The subject of patient safety, and the quality of
healthcare, has gained increasing momentum internationally as a major focus of
attention in professional and health policy circles. This has been highlighted
recently by the report on patient safety from the Institute of Medicine in the
United States,1 by an entire issue of the
British Medical Journal devoted to
medical error,2 and at least two high-profile
reports on aspects of patient safety in the United Kingdom’s National
Health Service.3,4 The matter has also gained
attention in the UK because of the high level of public interest in the Bristol
incident.5
In New Zealand, a number of reports have also highlighted
quality issues,6,7 yet the question of patient
safety has, to date, been the subject of relatively little systematic research.
One of the first studies to use a standardised, epidemiological approach was a
survey of adverse drug events among over 9000 admissions to Dunedin Hospital in
the early 1970s.8 Although useful research
since that time has been carried out on surgical
audit9 and anaesthetic
error,10 no generic, epidemiological data on
adverse events have been published in this country. The absence of such data has
been recognised as an obstacle to developing proposals for the regulation of
safety in health and disability in New
Zealand.11
A major scientific stimulus to rigorous epidemiological
research on patient safety was the development of standardised procedures for
the assessment of adverse events using medical
records.12 This methodological approach was
tested for its applicability in both the
British13 and Australian
contexts,14 and has further been tested for its
feasibility in New Zealand.15,16
The objective of this study was to assess the occurrence and
impact of adverse events in New Zealand public hospitals as revealed in an audit
of a representative sample of medical records.
MethodsSampling
The survey population was defined as all patients admitted to 20
general hospitals with more than 100 beds
for the calendar year 1998 (excluding day, psychiatric and rehabilitation-only
cases). A random sample of 13 public hospitals was then selected following
stratification by hospital type (ie, service facilities provided) and
geographical area. The sampling frame for each hospital was a list of all
eligible admissions in that hospital, ordered by admission date. To ensure
distribution throughout the year, systematic list sampling from a random start
point was used to identify 575 admissions from each
hospital. Initially, 7475 records were
selected for screening. However, for a variety of reasons – wrongly
sampled record, missing record, current inpatient, and other – 744 records
could not be screened, leaving 6579 records for assessment (after double
admissions were also removed).
To be included in the study, an adverse event had to be related to, or have occurred during, the sampled admission. The feasibility of using this method was tested in three major hospitals in Auckland.16 Administrative data For each sampled admission, the New Zealand Health Information Service (NZHIS) provided admissions information (dates of admission and discharge, admission type (planned or acute), and admission source (routine or transfer)); socio-demographic data (age, gender, ethnicity, domicile code); and clinical data (diagnostic classification).17 NZDep96 quintiles were derived from patient domicile codes as a measure of residential area deprivation.18 Principal diagnosis or reason for admission was classified according to 25 Major Diagnostic Categories (MDCs) derived from Australian AN-DRG 3.1.17 Data collection The core data collection procedure of the study was a two-stage retrospective review of each selected medical record. This involved the use of two protocols. At the first stage, a screening protocol – Review Form 1 (RF1) – was administered by specially trained Registered Nurses (RNs) to determine if the sampled admission met any of 18 screening criteria selected as potentially indicative of an adverse event. The screening criteria included unplanned admission before the sampled admission, and unplanned readmission after discharge from the sampled admission, among others.19 Those records indicating positive on the initial screening stage were passed on for further consideration. The objective of the second stage was to determine whether the sampled admission was associated with an adverse event, and if so, to then characterise that adverse event according to key clinical criteria. Review Form 2 (RF2) guided these judgements – using structured implicit review – and was administered by specially trained and experienced Medical Officers (MOs). Both review forms were closely modelled on the comparable instruments in the American20,21 and Australian studies.14 Structured implicit review is the guided exercise of professional judgement to facilitate reliable detection and determination of adverse events. A series of seven evaluation questions were used to assist reviewers in arriving at this judgement. The degree of certainty accorded to this assessment was translated into a six-point confidence scale of evidence of causation by healthcare management. This ranged from 1 = virtually no evidence, to 6 = virtually certain evidence.19 An expert reviewer arbitrated on discrepant judgements (in which an RN and an MO disagreed), and carried out an independent review of a one in ten sub-sample of medical records. Measures of concurrent validity22 were used to determine the quality of screening and review. Definitions An adverse event was operationally defined as: 1) an unintended injury; 2) resulting in disability; and 3) caused by healthcare management rather than the underlying disease process. Each of these criteria had to be fulfilled. Figure 1 provides two examples of unwanted outcomes of treatment; one is not classified as an adverse event and the other is an adverse event of low preventability. Figure 1. Examples of event occurrences synthesised
from real cases19
* Preventability defined as an error in healthcare
management due to failure to follow accepted practice at an individual or system
level
Disability was categorised into one of the following
types: temporary; lasting up to one year; permanent impairment of function; or
death. Attributable bed days refer to those extra days associated with an
adverse event that were spent in the study hospital during one or more
admissions.
Results
For over 85% of sampled records, available information was
sufficient to complete all aspects of the RF1. Similarly, for approximately 95%
of all medical records classed as adverse events, the available information was
sufficient to complete all aspects of the MO review using the
RF2.19
The representativeness of the sample was assessed by
comparing the distribution of key patient characteristics with the pattern for
all New Zealand publicly-funded hospital admissions in 1998 (Table 1). Sample
figures for age, gender, ethnic group, discharge status and mortality were all
closely comparable to national data, whereas length of stay in the sample
appeared to be notably shorter.
Table 1. Patient characteristics, 1998; study sample vs
all publicly-funded hospital patients
*All
publicly-funded hospitalisation in New Zealand (may include private hospital
admissions) (Source: New Zealand Health Information
Service)
†Excludes day and psychiatric patients ‡Excludes specialist public hospitals, public hospitals with under 100 beds, and rehabilitation-only patients The frequency of occurrence of adverse events is considered
in Table 2. Such events were associated with 12.9% of admissions. This
represents all incidents recorded by a healthcare professional over the period
1998 to 2000 (field work date) in a population of hospital patients admitted in
1998. Technically, this is a prevalence figure. Adjusting for the differential
probability of selection of admissions (attributable to the stratified cluster
sample design) increased this proportion slightly (13.1%). The incidence rate
– of 11.2% – represents only cases recorded during the 1998 sampled
admission. This again increased slightly on adjustment for sample design
(11.3%).
Table 2. Adverse events: prevalence and
incidence
*Prevalence defined as all adverse events found by the
study review process as a proportion of sampled 1998
admissions
†Incidence rate defined as incidents recorded by a healthcare professional during the 1998 sampled admission (and later assessed to be an adverse event by a study reviewer) While all the events reported here were recorded and treated
in public hospitals, not all had occurred in such institutions (Table 3). While
the great majority of adverse events had occurred inside a public hospital
(80.4%), nearly one fifth had taken place outside; most commonly in a
doctor’s rooms, at the patient’s home, or in a rest home or private
hospital.
Table 3. Distribution of adverse events (AEs), by
location of occurrence
In Table 4, the impact of adverse events is assessed
according to patient disability and hospital workload (number of extra
days’ stay). For the great majority of patients, disability was either
minimal or moderate. However, it should be noted that nearly 15% of patients
either suffered permanent disability or died. On average, an extra 9.3 days was
required for treatment (median of 4 days). There was a close association between
disability status and length of stay, with the permanently disabled requiring
between three and five weeks’ extra stay.
Table 4. Impact of adverse events (AEs): patient
disability status and workload (extra hospital stay)
*Extra bed days associated with an adverse event that
were spent in the study hospital during one or more
admissions
†Period of disability The impact of adverse events in relation to major diagnostic
criteria is represented in Table 5. This shows the distribution of admissions
and adverse events, together with two impact criteria – permanent
disability and death, and extra stay in hospital. There were few striking
discrepancies; adverse events seemed to be over-represented in injury-related
and musculoskeletal MDCs, and less common in birth-related admissions. Adverse
events associated with digestive, respiratory and nervous systems seemed to show
greater patient impact.
Table 5. Distribution of admissions and adverse events
(AEs) and impact of AEs, by Major Diagnostic Category (MDC)
*Principal diagnosis was classified according to 25
Major Diagnostic Categories (MDCs) derived from Australian AN-DRG 3.1 and
ordered according to percentage of
admissions
†Extra bed days associated with an adverse event that were spent in the study hospital during one or more admissions The same set of data is presented in Table 6 for a range of
socio-demographic factors; namely, age group, gender, ethnic group, and area
deprivation. Overall, the pattern of adverse events mirrored that of admissions
closely for all comparisons, except for patients over the age of 65. While this
group accounted for one third of admissions, it was associated with 40% of
adverse events. Patient impact and workload was also slightly higher for those
over 65.
Table 6. Distribution of admissions, adverse events
(AEs) and impact of AEs, by socio-demographic factors
*Extra bed days associated with an adverse event that
were spent in the study hospital during one or more
admissions
†NZDep96 quintiles were derived from patient domicile codes as a measure of residential area deprivation; quintile 5 represents the highest level of deprivation ‡77 of the admissions could not be coded §7 of the adverse events could not be coded Discussion
A primary objective of this investigation was to establish
the occurrence of adverse events in New Zealand public hospitals by assessing a
representative sample of admission records according to a standardised audit
protocol. Using this methodology, it was estimated that 12.9% of hospital
admissions were associated with an adverse event. This rate stands almost midway
between the levels recorded in two countries with shared medical traditions in
training and practice: Australia (16.6%)14 and
the UK (10.8%).13
A second major objective of the investigation was to assess
the impact of adverse events, both on patients and on hospital workload. The
noteworthy finding here is the quite mixed signals on the magnitude of impact
(Tables 4 and 6). Less than 15% of adverse events were associated with permanent
disability or death (and the great majority of events resulted in relatively
minor impact on patients). This outcome is consistent with findings in other
studies.13,14,20 However, when considering the
impact on hospital workload, adverse events added an average of over nine days
to expected hospital stay, an outcome that was similar to the Australian finding
of an average of just over seven days.14 It
should also be noted that there were few demographic or clinical patterns in the
data, aside from the evident vulnerability of older patients.
Another area of interest that emerged from this study is the
significant proportion – about one fifth – of adverse events that
originated outside a public hospital (Table 3). This result has not been
previously reported in the international literature and points to the potential
importance of quality and safety issues in primary and community care and in
other institutional settings.
The great strength of the study is its representativeness.
On key criteria, the sample of records shows a close approximation to the
pattern of admissions for all New Zealand hospitals in 1998. However, it should
be noted that the sample draws only on generalist, acute hospitals with 100 beds
or more, and that length of stay was on average lower (5.1 in the sample, 6.9 in
all publicly-funded hospital admissions). The documentation in sampled medical
records was sufficiently detailed and comprehensive to permit full completion of
study instruments, and there was evidence of internal consistency in the data on
key study variables (for example, the relationship between assessed patient
disability and extra hospital workload).
Yet, there still remain questions about the quality of key
study measures, such as adverse event status and preventability. Study
instruments were directly applied, with little if any modification, from
internationally established protocols. These rely on the guided judgement of
screeners and reviewers – structured implicit review – and are thus
potentially subject to observer variability. Hence, for example, the measure of
agreement between MO reviewers and the expert reviewer on adverse event
determination in this study – kappa 0.47 – was only of moderate
strength,19 although within the range for
comparable studies.14,23 Furthermore, while the
adverse event rate reported in this study clusters with those for
Australia14 and the
UK,13 it differs significantly from those
reported for the United
States.20,21,23
An analysis of this overall discrepancy in adverse event
rate between Australia and the United States suggests that slight differences in
methodology were partly to account, but that the principal explanation lay in
contrasting study purposes – medicolegal in one case (the United States),
quality in the other (Australia).24
Nevertheless, there remains an irreducible element of subjectivity to the core
study instrument, with the potential for considerable observer variability. This
constrains the interpretation of any apparent variations in adverse event
rates.
This investigation establishes broad clinical and managerial
parameters for our understanding of patient safety and the quality of care in
New Zealand public hospitals. The findings suggest that adverse events are as
significant a problem in New Zealand as they are in Australia, the UK, and the
United States. In essence, about one in eight admissions to a hospital are
associated with adverse events (which may have occurred within or outside public
hospitals). The majority of such incidents have a relatively minor impact on
patients (though there is a significant proportion who suffer permanent
disability or death), but their effects on hospital workload, and thus costs to
the health system, are substantial.
There remain a number of issues unresolved from this
investigation. First, there are still questions about the measurement properties
of structured implicit review in identifying adverse events from medical
records. Further methodological work is required in this area. Second, more
detailed analysis of the data from this study – and others – is
required in order to provide insight into the detailed patterns of adverse event
occurrence and determination, particularly in relation to preventability.
Preliminary work in this area has shown that only 6.3% of admissions to New
Zealand public hospitals were associated with adverse events that were both
preventable and occurred in hospital.25 From
such work may come indications for quality improvement initiatives, together
with testable propositions for strategies designed to reduce the level of
preventable adverse events.
In summary, the first nationally representative audit of
medical records in New Zealand public hospitals has identified a level of
medical injury that is similar to that recorded in comparable countries. There
is a considerable impact of adverse events on hospital workload, and a
significant minority of patients suffers death or permanent
disability.
Author information:
Peter Davis, Professor, Department of Public Health and General Practice,
Christchurch School of Medicine and Health Sciences, University of Otago; Roy
Lay-Yee, Analyst; Robin Briant, Clinical Director, Division of Community Health,
Faculty of Medical and Health Sciences, University of Auckland; Wasan Ali,
Visiting Research Fellow, Department of Public Health and General Practice,
Christchurch School of Medicine and Health Science, University of Otago;
Alastair Scott, Professor, Department of Statistics, University of Auckland;
Stephan Schug, Professor, Department of Anaesthesia, University of Western
Australia, Perth, Australia
Acknowledgements:
Work on this study was funded by the Health Research Council of New
Zealand. We thank the 13 New Zealand hospitals that participated in the study
and Dr David Richmond (Chair) and members of the study’s Advisory and
Monitoring Committee. We also thank Sandra Johnson and Wendy Bingley, our
medical review and data processing teams, and hospital records staff.
Correspondence:
Professor Peter Davis, Department of Public Health and General Practice,
Christchurch School of Medicine, University of Otago, P O Box 4345,
Christchurch. Fax: (03) 364 0425; email: peter.davis@chmeds.ac.nz
References:
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| | Current
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads) Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals |
|