 
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Journal of the New Zealand Medical Association,
05-March-2010, Vol 123 No 1310
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
How well does routine hospitalisation data capture
information on comorbidity in New Zealand?
Comorbidities are diseases or disorders that coexist with a
disease of interest.1 The importance of
comorbidity has long been recognised in the clinical management of patients, but
there is now increasing recognition of its importance in health related research
and policy. Comorbidity can affect quality of life, increase mortality,
influence treatment decisions, prolong hospitalisation and confound
analysis.1-5 As the population ages, these
issues will become increasingly common and pressing.
To date there has been very little work published on
comorbidity in New Zealand. Davis et al published a study in
20022 investigating the burden of comorbid
disease in major Auckland hospitals. They found that over a third of patients
admitted had at least one comorbid condition, and that comorbidity was
associated with length of stay, mortality and the occurrence of adverse events.
Similarly Stevens et al found that comorbidity was very common among a cohort of
lung cancer patients, and that it was adversely associated with
survival.6
Currently it is unclear how common comorbidity is in New
Zealand more generally, or how well routine hospitalisation data captures
information on important comorbid conditions. This latter point is important as
the majority of health policy, service planning and research projects requiring
information on comorbidity will rely on secondary data. This paper aims to
assess how well data on comorbidity are captured in routine databases in New
Zealand by comparing detailed comorbidity data extracted by a physician from
hospital records of patients with routinely collected hospitalisation data from
these patients.
MethodsData for this study come from two sources, firstly from
a cohort study which investigated factors affecting colon cancer survival; and
secondly from routine hospitalisation data obtained from New Zealand Health
Information Service (NZHIS).
Cohort study—Details of this
study are available elsewhere.7 In brief, the
cohort was made up of patients with first primary colon cancer diagnosed between
1996 and 2003, and notified to the New Zealand Cancer Registry (ICD-10-AM site
codes C18-C19 excluding 18.1). Patients were ineligible if they were less than
25 years at diagnosis, or were diagnosed after death. All Māori patients
meeting the above criteria were included along with an approximately equal
number of randomly-sampled non-Māori patients. This was to allow an
assessment of survival disparities between Māori and non-Māori
patients with colon cancer.7
Clinical data were abstracted directly from
patients’ hospital medical notes during 2006-07. These were recorded on a
standardised form by a physician (SH) and double-entered into an electronic
database. Data were collected on all major comorbid conditions present at the
time of diagnosis and all conditions included in the Charlson comorbidity index.
The Charlson index was developed in 1987 using data
from a cohort of 607 medical patients, and validated with a population of breast
cancer patients. Nineteen conditions are allocated a weight of 1 to 6 depending
on the adjusted relative risk of 1-year mortality, and summed to give an overall
score. 8
In addition to the conditions included in the Charlson
Index, data were collected on the following conditions: angina, essential
hypertension, cardiac arrhythmias, previous pulmonary embolism, cardiac valvular
disease, inflammatory bowel disease, other neurological conditions (including
multiple sclerosis, Parkinson’s disease, other abnormal movement
disorders, epilepsy, spinocerebellar disease, anterior horn disease, other
disease of the spinal cord, other demyelinating diseases of the CNS, cerebral
palsy, myoneural disorders and muscular dystrophies) and major psychiatric
conditions (including schizophrenia, bipolar disease, and depressive
psychosis).
Comorbidities were classed in three different ways:
Administrative
data—Routine hospital discharge data coded to ICD-9-CM-A were
obtained from NZHIS in 2005 on the cohort specified above. These data are coded
routinely from patient discharge records by coders based at District Health
Boards and sent electronically in agreed format to NZHIS. We treated the
admission for surgical resection of colon cancer as the index admission. Where a
patient did not receive surgical resection, we treated the first hospital
admission with colon cancer as primary diagnosis as the index admission. Those
without such an admission were excluded from the study.
One of the problems with using administrative data to
assess comorbidity is deciding on an optimal comorbidity ascertainment lookback
period. Shorter periods may be more likely to identify currently active health
issues, while longer periods may be more likely to identify all important
comorbidity.9 In this study we assessed two
lookback periods; 1 and 8 years, 8 years being the longest available time for
the earliest cancer registrations.
We used both the principal and secondary diagnoses
fields to identify comorbid conditions from the administrative dataset. We used
the Deyo et al10 system which provides a method
of translating the Charlson index for use on administrative data using ICD
coding. The algorithm was modified to take account of the fact that we collected
data on additional conditions to those included in the Charlson Index. These are
listed in Table 1. Because it can be difficult to differentiate between
pre-existing conditions and complications of treatment, some conditions are only
included in the definition of comorbidity if they are listed prior to the index
admission.
We followed the approach used by Deyo et
al10, except that we included non-colorectal
malignancies in our definition of comorbidity if they were listed in index or
prior hospital discharges.11
Table 1. Diagnostic codes used for
mapping
* Included in definition of a comorbidity if they are
listed either in the index or prior hospital discharge; other codes only
included if they are recorded prior to index admission
‡ Not included
as part of Charlson Comorbidity Index
a Includes multiple
sclerosis, Parkinson’s disease, other abnormal movement disorders,
epilepsy, spinocerebellar disease, anterior horn disease, other diseases of
spinal cord, other demyelinating diseases of CNS, cerebral palsy, myoneural
disorders, muscular dystrophies.
b Includes
schizophrenia, bipolar disease and depressive psychosis
Analysis—To calculate the
maximum comorbidity we could identify from all data we had available, we first
calculated the total number and proportion of patients who were recorded with
each condition either in the medical notes review, or in the administrative data
combined (separately for 1 and 8 year lookback). We then compared the proportion
of these who had been identified in the notes, the administrative data or both,
and calculated p-values using McNemar’s test to test whether the number of
people with the condition differed significantly between the medical notes and
administrative data.
We calculated the distribution of Charlson score and
comorbidity count using medical notes, and administrative data with 1 and 8 year
lookback. We then measured cross-source agreement for each condition as well as
for the Charlson score and comorbidity count (uncategorised) using the weighted
kappa statistic with quadratic (Fleiss-Cohen)
weights.12
This statistic approximates the intraclass correlation
coefficient and provides a measure of reliability that adjusts for agreement
that occurs by chance. We considered scores of <0.40 to suggest poor
agreement, 0.40 to 0.74 to suggest moderate agreement and 0.75 or higher to
suggest very good agreement.
We assessed the association of comorbidity and
all-cause survival among this cohort with colon cancer using Cox proportional
hazards regression models. We fitted a baseline model that included sex, age,
and ethnicity, year of registration, stage, grade and site of disease. The fit
of the baseline model was compared to various models that included comorbidity
using the likelihood ratio test. For these models comorbidity was measured using
Charlson categories or individual conditions.
The conditions were selected on the basis that they had
been previously shown to be related to survival from colon cancer in this
cohort4, and that there were a minimum of 10
cases within the cohort (these conditions were previous myocardial infarction,
congestive heart failure, diabetes, chronic respiratory disease, renal disease,
cardiac arrhythmias, non-cerebrovascular neurological conditions and peripheral
vascular disease). We compared results from models that included comorbidity
measured using data from medial notes to those using administrative data.
Often comorbidity will not be an exposure of interest,
but a potential confounding factor in another putative association. Researchers
therefore have an interest in knowing how much of the ‘true’
confounding by comorbidity might be captured when adjusting for a misclassified
measure such as that from routine administrative data. We explored this for the
putative association of ethnicity with survival, and how much of the association
might be due to confounding/ mediation by comorbidity (we know that Māori
experience poorer survival from colon cancer than non-Māori, and that some
of this association is due to Māori carrying a higher burden of comorbidity
than non-Māori7).
We measured the hazard ratio for all-cause mortality of
Māori compared with non-Māori having adjusted for sex, age, year of
registration, stage, grade and site. We then added to the model comorbidity
measured using the individual conditions specified above identified either in
the notes, or in the administrative data to assess the extent to which each
changed the underlying hazard ratio.
Approval for this study was granted by the New Zealand
Multi-Region Ethics Committee.
ResultsA total of 685 patients met the eligibility criteria for the
cohort study, and full data were obtained for 92% of eligible patients to give
an initial study sample of 642 (308 Māori and 334 non-Māori). When
these cases were matched to the routine hospitalisation data, 73 were excluded
because they did not have an admission that met the criteria for the index
admission giving a final cohort for this study of 569 patients, 515 having an
admission for surgical resection of colon cancer.
Tables 2 and 3 show the comparison of medical notes data
with administrative data with 1- and 8-year lookback respectively. They show
that there were considerable differences in the comorbidity data obtained from
these two data sources. For most conditions, higher numbers of patients were
identified with notes review data than administrative data, and this effect was
more marked with 1-year than 8-year lookback.
This pattern was reversed for diabetes and renal disease for
both lookback periods, as well as non-colorectal malignancy, cardiac valve
disease and hemiplegia with the longer lookback period. There was very good
agreement (kappa=0.77 and 0.75 for 1- and 8-year lookback respectively) between
the sources of data for only one condition (mild to moderate diabetes).
For the 1-year lookback, 11 conditions showed moderate
agreement (kappa 0.40 to 0.74), and the remaining five showed poor agreement
(kappa<0.40). Agreement between the two data sources improved with the longer
lookback period with 14 conditions showing moderate and two showing poor
agreement.
View Table 2 and Table 3 here
As expected, both Charlson scores and comorbidity counts
tended to be higher when calculated from data extracted from medical notes than
from administrative data with 1 or 8 year lookback, and the highest scores were
obtained by combining both data sources (Figures 1 and 2). For the Charlson
index, agreement between the medical notes data and the administrative data was
somewhat better for the longer lookback period (kappa=0.66; 95% CI: 0.57-0.75)
than the shorter one (kappa =0.61; 95% CI: 0.51-0.70).
Figure 1. Comparison of Charlson comorbidity
scores calculated using medical notes or administrative data
 A similar pattern was seen for comorbidity count, although
because more conditions were included in this count, scores were generally
higher (Figure 2). The agreement between notes and administrative data was also
better with kappa coefficients of 0.66 (95% CI 0.60-0.73) and 0.77 (95% CI
0.72-0.81) for administrative data with 1 and 8 year lookback
respectively.
We found that comorbidity measures added significantly to
the ability of the base model to explain all-cause survival regardless of
whether comorbidity was measured using the Charlson score or individual
conditions, or whether data was collected from medical notes, administrative
data or both (in all cases likelihood ratio test p<0.0001 for model including
comorbidity measured compared with base model).
In this cohort, we found that after adjusting for sex, age,
year of registration, stage, grade and site, the baseline hazard ratio of
all-cause mortality for Māori compared with non-Māori was 1.34 (95% CI
1.03-1.74). When we adjusted for comorbidity using data from both sources
combined, the excess hazard ratio decreased to 1.17 (0.89-1.53). Adjusting for
comorbidity using either notes or administrative-based data alone resulted in
somewhat less reduction in the hazard ratio to 1.23 (95% CI 0.94-1.60), and 1.26
(95% CI 0.96-1.64) respectively.
Figure 2. Comparison of comorbidity counts
calculated using medical notes or administrative data
 DiscussionWe found that there were considerable differences in the
comorbidity data held in the routine administrative hospitalisation database in
New Zealand compared with that collected by a physician from medical records. In
general, more comorbidity was identified from medical records, however some
conditions were more frequently identified from administrative data notably
diabetes and renal failure. Agreement between the two data sources improved with
a longer lookback period for the administrative data. Despite these differences,
any of the measures of comorbidity that we used, regardless of the source of the
data, improved the ability of multivariable model to predict all-cause survival
in this cohort of colon cancer patients.
This is the first study in New Zealand to assess the quality
of routinely collected comorbidity data, which are being increasingly used for
health service funding and planning, and research. This is reasonable because
although medical notes review data is generally considered superior to
administrative data, it is not gold standard. While there may be concern about
the accuracy of diagnoses recorded in administrative data, medical notes are
also not entirely complete, standardised or error
free.13
Furthermore, the results here and elsewhere clearly show
that administrative comorbidity data are not a subset of medical notes data, and
it is likely that combining datasets provides less misclassification of
comorbidity than either source alone.13-15 This
is, of course, rarely possible.
Given that both sources result in misclassification of the
(immeasurable) underlying construct of ‘true’ comorbidity, it is
also possible, or even likely, that each of the sources of data correlates more
strongly with this third measure than they do with each other, assuming that the
misclassification errors in administrative and notes review data are independent
of each other. That is, the kappa comparing the administrative and note-based
comorbidity indices probably underestimate the correlation of each with a
‘true’ measure of comorbidity (unless errors in administrative and
notes-based measures are moderately to highly correlated).
Furthermore, routinely collected data are considerably more
accessible for large population groups than notes review, and a number of
approaches to dealing with administratively collected comorbidity data are
possible depending on the purpose of the data, and the outcome being
assessed.1, 16-20
Our finding that medical notes review results in higher
ascertainment of comorbidity is consistent with other studies.
13-15, 21, 22 The extent of this difference
depends on a number of factors including the measure or condition that is being
compared, the mapping algorithm used and the lookback period used for
administrative data. There is considerable variability between conditions in
terms of their ascertainment in administrative compared with medical notes data.
For the administrative data with 8-year lookback, this
varied from kappa coefficients of 0.32 to 0.75. This variation is likely to
depend in part on the seriousness of the condition, and coding practices
relating to administrative data. As a general rule in New Zealand, comorbidities
are only coded in administrative data if they co-exist or arise during a given
episode of care and that they affect patient management in a way which
might extend length of hospital stay. This approach is likely to result in an
emphasis on the most active and clinically important conditions, and will
explain some of the difference between notes and administrative comorbidity
data.
It is not entirely clear how one should map conditions from
clinical notes to ICD codes, and there has been dissent expressed on this in the
literature.10,11,21,23,24 We employed a
commonly used approach, but one that has also been criticised by some
authors.11,21 For example, we found that for
six of the nine mismatches for diabetes with end organ damage, had been coded as
diabetes without mention of complication.
Currently no gold standard mapping approach has been
developed. The length of the lookback period also makes a difference, but the
ideal lookback period seems to depend on the outcome for which the data is being
collected.9,25 For example Preen et al
(2006)9 found that a one-year lookback provided
better comorbidity data to predict mortality while five-year lookback was better
for readmission rates. In our study the longer lookback period seemed to give
more comparable data to the notes review.
Both the kappa coefficients for the individual conditions
and those for the Charlson and comorbidity count (0.66 and 0.77 respectively)
compare favourably with similar comparisons carried out
elsewhere.15,22 For example, Kieszak et
al22 compared comorbidity derived from medical
notes with administrative data in the United States and found that only three of
16 individual conditions had kappa coefficients greater than 0.4 (compared with
15/17 for our data with 8 year lookback), and that the correlation between the
notes Charlson index and the administrative index was only 0.47.
Regardless of the source of data used, we found that any
measure of comorbidity improved multivariable model fit compared with using
none. We also found in this study that using data from both sources combined
resulted in somewhat better risk adjustment than either source separately.
However, both the notes and administrative-based comorbidity measures
substantially reduced the excess mortality hazard for a model comparing
Māori with non-Māori for colon cancer survival, although more so for
the use of notes-based comorbidity index consistent with an a priori
expectation that it is a superior estimate of comorbidity.
Furthermore, given that including both the notes and
administrative-based measures of comorbidity resulted in greater reduction again
in the hazard ratio, it seems reasonable to conclude that neither measure alone
(notes or administrative-based) fully captures the confounding or mediating
effects of comorbidity.
Of note, is that this study focused solely on patients with
colon cancer. Patients with other primary conditions may have different patterns
of comorbidity, but it seems unlikely that this will affect the quality of the
recording of their comorbidity data. In that respect, it seems reasonable to be
able to generalise the findings of this study to hospital-based comorbidity data
in New Zealand.
In conclusion, measuring comorbidity is potentially
important for risk adjustment in health service policy, funding and planning,
and health-related research. Data from clinical notes review are often
considered superior but are rarely available. The correlation between clinical
notes and administrative data in New Zealand is moderate and varies considerably
between individual conditions. However, administrative data provides a source of
relatively accessible comorbidity data which we have found allows for reasonable
risk adjustment in the cohort presented here, although not quite as good as for
a notes-based or combined comorbidity measure.
Competing interests: None known.
Author information: Diana Sarfati, FAFPHM,
Senior Research Fellow, Department of Public Health, University of Otago,
Wellington; Sarah Hill, FAFPHM, Senior Research Fellow, Department of Public
Health, University of Otago, Wellington; Gordon Purdie, Biostatistician,
Department of Public Health, University of Otago, Wellington; Liz Dennett,
FRACS, Senior lecturer, Department of Surgery and Anaesthesia, University of
Otago, Wellington; Tony Blakely, Research Professor, Department of Public
Health, University of Otago, Wellington
Acknowledgements: We thank Bridget Robson,
Donna Cormack and Kevin Dew who were investigators in the colon cancer cohort
study from which data for this study was collected, and for their helpful
comments on earlier drafts of this paper.
The authors also acknowledge the Cancer Society of New
Zealand for providing funding for this study (grant 05/16). The Cancer Society
of New Zealand had no role in the study design; in the collection, analysis or
interpretation of data; in the writing of the manuscript or the decision to
submit the manuscript for publication.
Correspondence: Diana Sarfati, Department
of Public Health, University of Otago, Wellington, PO Box 7343, Wellington
South, New Zealand. Fax: +64 (0)4 3895319; email: diana.sarfati@otago.ac.nz
References:
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| | Current
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads) Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals |
|