Journal of the New Zealand Medical Association, 06-June-2003, Vol 116 No 1175
Unlocking the numerator-denominator bias III: adjustment ratios by ethnicity for 1981–1999 mortality data. The New Zealand Census-Mortality Study
Shilpi Ajwani, Tony Blakely, Bridget Robson, June Atkinson and Cindy Kiro
A focus on reducing health inequalities between ethnic groups is both a legislative and policy imperative for the health sector of Aotearoa/New Zealand.1 The Government has committed itself to reducing health inequalities, especially those experienced by Maori and Pacific peoples.2 In an evidence-based health policy environment, data quality underpins the ability of indigenous peoples to obtain collective wellbeing.3 The Crown has a Treaty responsibility (of good governance) to maintain Maori health data collections to at least the same quality as those of non-Maori.
Over the past quarter century, the definition of ethnicity used in the New Zealand Census has changed from a biological concept to one of cultural affiliation.4 However, this was not the case with mortality data, which rely on the death registration form. It is only since September 1995 that the ethnicity question on the death registration form has been consistent with that on the (1996) census form. Furthermore, there is a difference between the data collection methods used in the death registration and the census forms (eg, funeral-director-elicited versus self-identified). These discrepancies have created major problems for monitoring ethnic mortality trends over time.
Anonymous and probabilistic record linkage of census and mortality data in the New Zealand Census-Mortality Study (NZCMS) allows a direct comparison of the self-identified ethnicity at the previous census and the ethnicity recorded in the mortality data. Consequently, it is possible to quantify or ‘unlock’ the numerator-denominator bias that causes ethnic-specific mortality rates to be calculated incorrectly. We have previously determined that under-counting of Maori and Pacific deaths in 1991–1994 mortality data (numerator) relative to the 1991 Census (denominator) causes a severe underestimate of Maori and Pacific mortality rates.5
The objective of the current study is to additionally estimate the numerator-denominator bias for 1981–1984, 1986–1989, and 1996–1999, allowing a determination of trends in bias for the 1980s and 1990s. The findings of this study subsume those of the earlier study of bias in 1991–1994 census-mortality records5 and present analyses for all four (1981–1984, 1986–1989, 1991–1994, and 1996–1999) census-mortality cohorts. This study uses ‘age at death’ (instead of ‘age at census’ as used in the previous study) and an improved method for weighting.
The methodology has been described in detail in technical reports.6–8 Briefly, approximately three quarters of eligible mortality records (aged 0–74 years at census night) for three years subsequent to each of the 1981, 1986, 1991 and 1996 Censuses were anonymously and probabilistically linked to census records. For the purposes of determining numerator-denominator bias, the linked records were further restricted to highly probably links (HPL), where ethnicity had no effect on linkage probability. In the previously reported 1991–1994 numerator-denominator analyses, the HPL were restricted to records with agreement on meshblock (ie, small areas of residence containing approximately 100 people).5 In this study, we also included links with agreement on area unit (approximately 2000 people) in our HPL data sets to increase linkages for rural decedents that seldom had a meshblock assigned. The number of mortality records in the HPL data set (and as a percentage of eligible mortality records) for 1981–1984, 1986–1989, 1991–1994, and 1996–1999, respectively, were 28 470 (64%), 30 891 (69%), 30 789 (75%), and 29 637 (75%).
Ethnicity definitions Until September 1995, the death register classified each decedent as one of three groups: NZ Maori (½ or more Maori ‘blood’); Pacific Island (more than ½ Pacific Island ‘blood’); or Other (non-Maori non-Pacific). After September 1995, the death register collected data on ‘ethnic group’ and coded up to three ethnic groups per person. An alternative (although not commonly used) source of ethnicity information for deaths is the National Health Index (NHI), from which records from 1988 onwards can be linked to the mortality data by way of the unique NHI number.
The censuses collected data on ‘ethnic origin’ in 1981 and 1986 and ‘ethnic group’ in 1991 and 1996. Each census (and post-1995 mortality data) allowed for multiple groups to be self-identified, thereby allowing an assignation of ‘sole’ and ‘prioritised’ ethnic groups:
The ethnicity question on the 1996 Census was modified to include extra categories and encouraged multi-ethnic responses. Consequently, the 1996 Census ‘sole’ groups became smaller and the ‘prioritised’ groups became larger compared with the 1991 Census. We also categorised an Asian group for 1996–1999 data, with the ‘prioritised Asian’ group being assigned if one of the self-identified ethnic groups was Asian and they had no Maori or Pacific self-identified ethnic groups.
Determining the numerator-denominator bias A major reason for determining the numerator-denominator bias is to correct mortality rates by ethnicity. From 1981 to 1985, official Maori and Pacific mortality rates were calculated using the ½ or more Maori (or Pacific) ‘blood’ categorisation in both the deaths and census data. Between 1986 and 1995, sole ethnic group from the census was used as the denominator, with the ½ or more Maori (or Pacific) ‘blood’ category in the mortality data. From 1996 onwards the Ministry of Health has used prioritised ethnicity for both census and mortality data to calculate rates. Therefore, we present numerator-denominator bias results in this paper for:
For completeness, we also present adjustment ratios comparing the sole populations in 1996–1999.
The numerator-denominator bias was determined by cross-classification of the weighted HPL data set by census and death registration ethnicity. (The weights make the HPL data set representative of all eligible mortality records; their calculation is described in detail elsewhere.7) This cross-classification was also conducted by strata of age at death, regional health association (RHA), and rurality, in order to determine heterogeneity of the numerator-denominator bias. The bias, calculated as an ‘adjustment ratio’ that could be applied to historic mortality data, equals the estimated number of census respondents of the given ethnic group divided by the number of mortality records of the given ethnic group.
It is possible that our calculations may be biased due to the HPL data set excluding 25% to 30% of eligible mortality records. Extensive sensitivity analyses for Maori presented elsewhere suggest that any such bias was minimal.6,7 Briefly, the key assumption for the validity of the adjustment ratio calculations is that within each stratum of mortality record ([ethnicity] by [sex] by [age at death] by [RHA] by [NZDep] by [rurality]) the distribution of census self-identified ethnicity was the same for mortality records included in the HPL data set as it was for all eligible mortality records in the given stratum. As most Maori identified as such according to mortality data also self-identified as Maori on census data, it is only among non-Maori according to mortality data that any residual bias may arise. We tested this possibility for 1991–1994 by re-calculating unlock ratios under the (somewhat extreme) scenario that non-Maori non-Pacific decedents excluded from the HPL data set (meshblock-only links) were 15% more likely to self-identify as sole Maori on the census than those included in the final data set within all strata used for weighting. Our (previous5) overall estimate of the Maori adjustment ratio of 1.29 became 1.33, or 1.31 if a 7.5% rather than a 15% estimate was used.6 The analyses in this current paper included census-mortality links formed at the census-area-unit level as well as the meshblock level, resulting in an overall Maori unlock ratio of 1.32. Thus, based on the more inclusive HPL data set used in this current paper, and further sensitivity analyses published elsewhere,6,7 we conclude that the adjustment ratios presented in this paper are reasonably accurate.
The overall adjustment ratios for the 1981–1984, 1986–1989, 1991–1994 and 1996–1999 census-mortality cohorts are shown in Table 1. In the 1981–1984 cohort, for example, 3615 of a total of 44 703 deaths were estimated as having identified as sole Maori on the 1981 Census; 16% more than the 3108 recorded as Maori on the death registration form. Therefore, the Maori mortality data for 1981–1984 would have to be multiplied by the adjustment ratio of 1.16 (ie, 3615/3108) to make them comparable with the sole Maori population from the 1981 Census.
Table 1. Census ethnicity and death-registration-form ethnicity totals for 0–74 year old decedents during 1981–1984, 1986–1989, 1991–1994 and 1996–1999
All the numbers are weighted, and then random rounded to a multiple of three as per Statistics New Zealand protocol. Minimum cell size is 6. *The census to mortality ratio is the census total divided by the death registration form total eg, for Maori sole ethnicity 1.32 is the correction factor to apply to 1986–1989 ethnic specific mortality rates calculated using sole ethnicity as the denominator.
The Maori adjustment ratio in both 1986–1989 and 1991–1994 was 1.32 using the census ‘sole’ categorisation. With the change in ethnicity data collection on death registrations in 1995, the ratio for sole Maori deaths reversed to 0.86 for the 1996–1999 cohort. However, using the ‘prioritised’ classification in 1996–1999, the study found 7% more decedents identified as Maori on census than on death register (a ratio of 1.07). (Currently published Maori mortality rates for the late 1990s use the ‘prioritised’ classification,9 and are therefore still modestly underestimating Maori mortality rates.) For Pacific peoples, the trend over time was similar for the first three cohorts, although the bias was greater than for Maori, with ratios of 1.55, 1.76, and 1.68 respectively. Like Maori, Pacific deaths were over-reported according to sole ethnicity in the 1996–1999 cohort, but little bias was observed for the prioritised Pacific ethnic group with a ratio of 0.99.
*ratios for Asian are calculated in the 1996–1999 cohort only; †ratio is for non-Maori non-Pacific for the first three cohorts, and is for non-Maori non-Pacific non-Asian in the 1996–1999 cohort only
There was little variation in the adjustment ratios by sex, except for Pacific people in the 1981–1984 cohort (1.48 for men vs 1.68 for women). The Pacific ratios by sex were, however, unstable due to the smaller number of linked records and must be treated with caution.
By age, the Maori adjustment ratios were substantially greater at younger age groups, at least in the 1981–1984, 1986–1989, and 1991–1994 cohorts (Table 2). For example, the underestimation of deaths among the 0–14 year age group in the respective three cohorts was 34% ([0.52/1.52] x 100), 36% and 44%, compared with 11%, 15% and 19% among the 65–77 year age group. There was no clear pattern by age for Maori in the 1996–1999 cohort, but young adults (prioritised ethnicity) had the highest under-reporting with a ratio of 1.13. No pattern by age was observed for Pacific people. The Pacific ratios were higher for all age categories in the first three cohorts and close to one in the final cohort.
Table 3. Census ethnicity to death-registration-form ethnicity ratios by Regional Health Authority (RHA)
*ratios for Asian are calculated in the 1996–1999 cohort only; †ratio is for non-Maori non-Pacific for the first three cohorts, and is for non-Maori non-Pacific non-Asian in the 1996–1999 cohort only
There was variation in the numerator-denominator bias for Maori by region (Table 3), being greatest for the Central and Southern RHA with adjustment ratios of 1.34 for the 1981–1984, 1.90 for the 1991–1994, and 1.13 for the 1996–1999 cohort. However, an aberration was noted for the 1986–1989 cohort, when the greatest bias for Maori was in the Northern RHA. Further investigation of the northern region for 1986–1989 disclosed a major, localised problem. At the Onehunga Registration Office, 908 deaths were reported during 1981–1984, of which 280 were Maori (approximately 30%). Of the total 1408 deaths reported in that office in 1986–1989, therefore, approximately 420 should have been recorded as Maori, but only 18 Maori deaths were reported.
There were very few Pacific (or Asian) deaths reported in rural areas, and the urban adjustment ratios for Pacific and Asian deaths were very similar to the overall ratios reported in Table 1. Hence, we report urban and rural ratios for Maori and non-Maori only. As observed in Table 4, the underestimation of Maori deaths was higher in urban areas than in rural, and the pattern was similar across all four cohorts.
The adjustment ratios for Asian deaths were available only for the 1996–1999 period, and were close to one without any substantial variation by other demographic factors.
A comparison using ethnicity from the National Health Index (NHI) file, instead of the death-registration-form ethnicity, found that the NHI ethnicity data were more accurate than the mortality data for the 1991–1994 cohort. For the 1996–1999 cohort, the NHI ethnicity data were reasonably accurate for the sole-Maori group (with a ratio of 1.05), and somewhat less accurate for the sole-Pacific group (ratio of 1.14). However, for the prioritised-Maori and -Pacific ethnic groups, the NHI data were considerably less accurate than the mortality data for 1996–1999. The census to NHI ratios for 1996–1999 using ‘prioritised’ categorisations were 1.38 for the Maori ethnic group and 1.30 for Pacific peoples (compared with 1.07 and 0.99 respectively for the death registrations).
More detailed results, including the results for numerator-denominator bias at two levels of stratification and ratios using other ethnicity combinations (census sole by mortality prioritised, and census prioritised by mortality sole for the 1996–1999 cohort, and NHI file ethnicity) are published in a technical report.7
Maori and Pacific deaths were substantially underestimated during the late 1980s and early 1990s. Approximately one third more decedents self-identified as sole Maori on census data than were identified as Maori on mortality data for this period, and approximately two thirds more decedents self-identified as sole Pacific. The change to compulsory recording of ethnicity, through incorporation of the (1996) census question on mortality data following September 1995, caused a dramatic reduction in numerator-denominator bias. During the late 1990s, only 7% more decedents self-identified as prioritised Maori on census data than were identified as prioritised Maori in mortality data. There was negligible remaining bias for prioritised Pacific deaths. However, sole Maori and sole Pacific counts on mortality data are overestimated in the late 1990s relative to census data.
We believe our quantification of numerator-denominator bias is reasonably accurate. First, our results are based on attempted record linkage for all mortality records in the three-year period after each census. Second, whilst only about three quarters of mortality records were linked to a census record, we were able to use weighted analyses of these linked records to estimate the numerator-denominator bias for all eligible mortality records. Extensive sensitivity analyses about our methods and results are published elsewhere.6,7
It is clear from our findings that there was large numerator-denominator bias, especially for Pacific people, younger Maori and Maori living in the Central and Southern RHA at least prior to 1996, even when using sole ethnicity populations as a denominator. As a result, both Maori and Pacific mortality rates and ethnic mortality disparities have been severely underestimated for these periods. For example, an unadjusted relative risk of 1.0 for Maori children aged 0–14 years compared with non-Maori non-Pacific in the early 1980s corresponds to an adjusted relative risk of 1.0 x 1.52/0.89 = 1.71 (where 1.52 is the adjustment ratio for Maori 0–14 year olds and 0.89 is the adjustment ratio for non-Maori non-Pacific 0–14 years olds (Table 2)). This large underestimation of ethnic inequalities among the young during the 1980s and 1990s needs further consideration. Social and economic policies implemented during that period have resulted in increasing inequalities between Maori and non-Maori in education, income and employment. The impact of such policies on the health of Maori and Pacific children and young people may not have been given due consideration given that the extent of those inequalities was not accurately represented in official statistics. The current trend of increasing ethnic disparities in young people means that improving the living circumstances of Maori and Pacific families is an urgent priority.
The failure to accurately record deaths has policy and research implications beyond the practices of data collection and processing. Maori researchers have argued that such under-counting is tantamount to a form of institutional racism (even if unintentional by individual agents), effectively rendering the problem of ethnic disparities invisible.10
More accurate reporting of Maori deaths in rural areas than in urban areas was observed in this study. Previous reports that life expectancy was higher for urban Maori than for rural Maori, while the opposite trend was reported for non-Maori, should be reconsidered in light of this new finding.9
A north-south variation in bias was observed for Maori mortality – a finding consistent with previous studies.11,12 This regional variation in ethnicity data quality should be particularly noted by those involved in researching, monitoring or reporting time trends in ethnic differences by region or in developing policies based on regionally specific data (eg, DHB funding formulae, calculation of population estimates, priority setting, health-service planning and monitoring of inequalities). The possibility of a north-south gradient in ethnic misclassification in other data collections, such as hospital admissions, should also be considered. The reasons why such a trend exists need investigation, but potential factors might include:
Beyond these potential reasons why such a regional trend exists, the matter of whether or not these variations could be indicative of other regional issues that impact on the health of Maori (and, if found to be true, in what way) requires investigation.
Our findings show that it is possible to rapidly improve ethnicity data quality through the implementation of a consistent and compulsory ethnicity question (ie, the change in September 1995 to include the census ethnicity question on mortality data). Ethnicity data collections from hospitals, primary care, cancer registrations, public health data and other registers, such as cervical screening and immunisation, may also benefit from ensuring the ethnicity question is standardized and compulsory for all administrative interactions. Given that the 2001 Census ethnicity question will remain in the 2006 Census and will therefore form the basis of denominator data until 2010, health services should now invest in implementing the standard question in their data collection systems.
Conversely, the finding that sole Maori are now overestimated in mortality data indicates that there are still improvements to be made in death registrations in the identification and recording of multiple ethnicity. Some funeral directors report that their data processing systems allow only a single ethnic group to be recorded.14 This would result in under-recording of those with multiple ethnic groups, and prevents accurate monitoring of mortality trends for the sole-Maori population, the group that currently experiences the highest mortality disparities. Furthermore, the mortality rate for the Maori ethnic group (prioritised) was still underestimated by 7% on a national level during the late 1990s, with a higher under-count in the central/southern region (13%) and in young adults (13% for 15–24 years, and 10% for 25–44 years). This may seem minor when compared with the severity of the previous problem but needs to be addressed nevertheless.
The current legislative and policy imperatives to reduce inequalities by improving the health of Maori and Pacific peoples1,2,15,16 necessitate accurate recording of ethnicity data across the entire health sector. This allows benchmarks to be set and the impact of policy changes to be monitored, along with the contributions of various parts of the sector to reducing inequalities. Given the current commitment by the Crown to reducing inequalities in health it is important that improvements to ethnicity data quality be made quickly and comprehensively. This would enable the monitoring of actual outcomes rather than artefactual effects of changes in data quality.
Principles of good practice in ethnicity data collection need further promotion among funeral directors as well as throughout the health sector and the general public. These include information about what is ‘ethnicity’, that it is self-identified (not based on ‘blood’ quantum), that it is possible for a person to identify with more than one ethnic group, and that ethnic identification may change throughout one’s life. Families could also be alerted to their right to fill in the death registration form themselves. Areas that require extra attention include training of data collectors, informing the public about ethnicity and their rights to accurate information being recorded, data input and output systems, and ongoing audit to ensure standards of collection are maintained. Our study found that ethnicity recording on mortality data is now more accurate than that on the National Health Index. Hospital data collections need improvement and primary care organisations should be cautious of obtaining ethnicity data for their patients from the NHI, which still under-reports Maori (and Pacific) patients.
While this research does not address specific policy interventions, the underestimation of Maori and Pacific mortality identified by this research provides increased evidence to support strong policy imperatives for addressing health inequalities and reinforces the need for proactive strategies to address premature mortality among Maori and Pacific peoples. Through such actions, the Treaty risk from ‘inaction in the face of need’ will be minimised and Treaty rights to collective wellbeing maximised.
Using the quantification of numerator-denominator bias presented in this paper, we are currently re-calculating mortality rates and trends by ethnicity over the last twenty years. Only then can we accurately determine if ethnic mortality ‘gaps’ are closing or widening.
Author information: Shilpi Ajwani, Research Fellow; Tony Blakely, Senior Research Fellow; Bridget Robson, Research Fellow (Te Ropu Rangahau Hauora a Eru Pomare); June Atkinson, Data Manager/Biostatistician, Department of Public Health, Wellington School of Medicine and Health Sciences, University of Otago; Cindy Kiro, Waiora: Centre for Public Health Research, Massey University, Auckland
Acknowledgements: The New Zealand Census Mortality Study (NZCMS) is conducted in collaboration with Statistics New Zealand and within the confines of the Statistics Act 1975. The NZCMS is funded by the Health Research Council of New Zealand, with cofunding from the Ministry of Health.
Summary Statistics New Zealand security statement: The New Zealand Census Mortality Study (NZCMS) is a study of the relationship between socioeconomic factors and mortality in New Zealand, based on the integration of anonymised population census data from Statistics New Zealand and mortality data from the New Zealand Health Information Service. The project was approved by Statistics New Zealand as a Data Laboratory project under the Microdata Access Protocols in 1997. The data sets created by the integration process are covered by the Statistics Act and can be used for statistical purposes only. Only approved researchers who have signed Statistics New Zealand’s declaration of secrecy can access the integrated data in the Data Laboratory. A full security statement is published in a technical report at the NZCMS web site http://www.wnmeds.ac.nz/nzcms-info.htm. For further information about confidentiality matters in regard to this study please contact Statistics New Zealand.
Correspondence: Dr Tony Blakely, Department of Public Health, Wellington School of Medicine and Health Sciences, University of Otago, P O Box 7343, Wellington. Fax: (04) 389 5319; email: firstname.lastname@example.org
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals