Journal of the New Zealand Medical Association, 04-August-2006, Vol 119 No 1239
Investigating the accuracy of ethnicity data in New Zealand hospital records: still room for improvement
Judith Swan, Steven Lillis, David Simmons
The accuracy of ethnicity information in New Zealand hospital records has been investigated several times1–3 over the last 24 years. Each of these investigations has demonstrated inadequacies in the accuracy of ethnicity information contained in hospital records, particularly in relation to Māori. Such miscoding can lead to under- or over-estimates of differences in health outcomes or health resource utilisation in differing ethnic groups in New Zealand. The accuracy of hospital records is particularly important, as data from the hospital records constitute one of the few data sets available to describe morbidity in the New Zealand context.4
The Barriers to Diabetes Care in the Waikato Study (Barriers) commenced in 2003. This study was established to identify perceived obstacles to the provision and receipt of quality diabetes care in the Waikato region. This study is unique in New Zealand diabetes research in asking the same questions of both providers and consumers of diabetes care. The structure of the Barriers Study enables the comparison of perceived obstacles, and possible solutions, between providers and consumers, between primary and secondary providers, and between differing geographic and demographic groups.
Once obstacles to care have been identified, strategies can be planned to improve the delivery of diabetes care in an environment characterised by rapidly escalating disease burden. This paper reports on one demographic variable—ethnicity, and extends the knowledge of the accuracy or otherwise of this information in the hospital records.
The Waikato District Health Board (WDHB) catchment area comprises approximately 8.3% of New Zealand’s land area in the central North Island. The population count of the WDHB in the 2001 census was 317,751 people, 8.5% of the national population. Two urban areas: Hamilton, a main urban area, and Tokoroa, a secondary urban area; accommodate 56.8% of the district’s population.5
Ethnically, the WDHB population has more people who identify as Māori than the national population (20.2% and 14.1% respectively), fewer self-identified Pacific people (2.1%, 5.4%), fewer self-identified Asian people (3.3%, 6.1%), and the same European / 'Other' people (74.4%).5 This source combines European and 'Other' ethnic groups together, while this paper reports on these groups separately.
The exact numbers of people with diabetes in the WDHB region are unknown. The WDHB estimated in 2003 that there were 12,487 people who live with diabetes in the region. Of these, 5734 (45.9) were Māori, 6253 (50.1%) were European, and 500 (4.0%) were Pacific people.5 Other similar estimates have been published.6,7
A validated four-item open questionnaire8 was posted to people who have diabetes; all general practitioners and practice nurses; all diabetes staff; and all relevant hospital-based medical, senior nursing, and allied health staff. The questionnaire asked the participant to identify what prevents diabetes care, how to improve diabetes care, what worries them about diabetes care, and to make any other comments. Additional closed questions elicited demographic and health history information.
Planned follow-up of non-participants was based on methods described by Dillman.9 The follow-up consisted of posting a second questionnaire, ‘in person’ follow-up to selected groups, and phone follow-up for all others. Consent was requested to access hospital records.
As part of the structured questionnaire, a single question asked “Which ethnic group(s) are you?” Participants could select, from six options (European, Māori, Pacific Island, South Asian, Asian and ‘Other’), and could list up to three ethnic groups. The order of ethnicities indicated was not analysed separately.
The hospital record data was drawn from the Regional Diabetes Database, a standalone National Health Index-based database developed and managed by the Regional Diabetes Service. This database permits recording of a single, patient-selected ethnicity.
Ethnicity data from completed questionnaires was compared with ethnicity information in the hospital-based medical record, for those who had consented. The self-identified ethnicity, from the Barriers Study, was taken as the standard, against which the hospital record was compared.
This paper reports on Māori–non-Māori differences as well as disaggregating the non-Māori group. In the Māori–non-Māori comparisons, if Māori was one of the reported ethnic identities, for those who reported multiple ethnicities, these participants were counted as Māori. All the other ethnic identities were grouped together and counted as non-Māori. In the analysis across all ethnic groups no prioritisation was applied.
Using percentage and 95% confidence intervals,10 the results were analysed using self-identified ethnicity and hospital record ethnicity as the denominator. The accuracy of the hospital record was assessed using the first analysis, while the latter provides a prediction of an individual’s ethnicity based on the hospital record material. The study was approved by the Waikato Ethics Committee.
The Barriers questionnaire was sent to 6881 people who live with diabetes. Of these, 4499 (65.4%) have been received back and 3500 provided consent to accessing of their hospital record. Ethnicity data from both the hospital record and the Barriers Study were available for all of these 3500 individual records. Data from these two sources are shown in Table 1.
Table 1. Variation in ethnic identity between self-identification (Barriers) and hospital records; Māori and non-Māori (n=3500)
These data show that 71.2 (95% CI: 67.2–75.0)% of persons who self-identify as Māori were correctly recorded as such in their hospital record. For those who self-identify in a non-Māori ethnic group, 99.3 (98.8–99.7)% were correctly recorded in their hospital record.
Considering the predictive value of a hospital record of ethnicity, the Barriers data showed that 94.9 (92.2–96.7)% of persons described in their hospital record as Māori identified themselves as Māori. Similarly, 95.2 (94.4–95.9)% of those described in their hospital records as non-Māori identified themselves as belonging to a non-Māori ethnic group.
Nearly one in three participants who self-identified as Māori were recorded as non-Māori in their hospital record. However, examining these 149 participants in more detail reveals a situation that is more complex than just incorrectly recorded ethnicity. Table 2 describes the diversity of these participants.
Table 2. Detail of those who self-identify as Māori and whose hospital record shows non-Māori (n=149)
These data show that two-thirds of this group self-identify with multiple ethnic groups (11 participants identified with three ethnic groups). Of those who identified with multiple ethnic groups the majority (88%) were correctly recorded in the hospital record for one of their ethnic identities.
When combined with the data from Table 1, this shows that 88.2 (85.2–90.8)% of participants who self-identify Māori as one of their ethnic groups had their ethnicity correctly recorded for one of their ethnic identities. However, this leaves 11.8 (9.3–14.9)% of participants who self-identify as Māori with their ethnicity incorrectly recorded in their hospital record. Most of these are participants who self-identified solely as Māori.
Closer examination of the much smaller group of participants who self-identify in a non-Māori ethnic group but are recorded as Māori in their hospital record shows 17 of the 20 self-identify as solely European. Two participants self-identified as solely Pacific Islander. The one remaining participant identified in two ethnic groups, these being European and Pacific Islander.
Combining the data from Tables 1 and 2 allows consideration of ethnic identity across all participants and all of their (sometimes multiple) ethnic identities. At this level, the data show 97.7 (96.8–98.3)% of participants had an identical ethnic group recorded for one of the ethnic identities.
Table 3 shows the data across all the ethnic groups in the Barriers dataset. While in all groups (except the ‘Other’ group) the majority of respondents (64.1–89.1%) had the same ethnicity data self-identified and recorded in the hospital notes, several variations emerged.
The self-identified ethnic group was correctly recorded in the hospital record for 89.1 (87.3–90.6)% of Europeans; 67.2 (54.7–77.8)% of Pacific Island ethnic groups; 70.2 (57.3–80.6)% of South Asian ethnic groups; and 64.1 (48.4–77.4)% of Asian ethnic groups—but only 41.0 (27.1–56.7)% of ‘Other’ ethnic groups. Most participants who had their ethnicity recorded differently to their self-identified status in the 'Other' group had European as their ethnic group in the hospital record.
Table 3. Variation in ethnic identity between self-identification (Barriers) and hospital records; all ethnic groups (n=3500)
The hospital record predicted the self-identified ethnic group for 94.5 (93.1–95.6)% of Europeans; 94.6 of Māori; and 93.2 (81.2–98.4)% of Pacific Island ethnic groups. However, for Asian ethnic group participants, the predictive value of the hospital record fell to 86.2 (68.9–95.2); and to 74.1 (61.0–84.1)% for South Asian ethnic group participants. The hospital record predicted only 4.5 (2.7–7.2)% of those who identified in the ‘Other’ ethnic group. Most [78.8 (74.2–82.7)%] respondents recorded in the ‘Other’ ethnic group self-identified as European.
These findings have significant policy implications. With such major discrepancies in ethnicity reporting among non-European groups in the major national outcome dataset (besides death certificates), it is difficult for the impact of policies focussing on inter-ethnic health disparities to be interpreted.
The accuracy of self-identified ethnic group hospital records has been reported on several times by different researchers.1–3 Table 4 shows how self-identity has compared with hospital records for Māori and non-Māori over the last 24 years. Our data are consistent with the two earlier studies, but not Priest and Jackson.
Indeed, one of the earlier reports was from the Waikato.2 Whether our results reflect national coding validity is unknown, but the Waikato itself is a significant proportion of the national population.
Table 4. Self-identity and correctly recorded ethnic group in hospital data 1980–2004 in New Zealand; various researchers
Limited information appears to have been published specifically addressing the accuracy of hospital data for ethnicity variables elsewhere. Boehmer et al11 reported on a review of a US dental department’s outpatient clinic files compared with self-reported race/ethnicity. These authors show that 77.1% of clinic files were correct for those who self-reported as white. The accuracy fell to 4.6% for those who identified as American Indian— with accuracy at 68.9% for Black or African American; 61.0% for Spanish, Hispanic, or Latino; and 54.0% for Asian people. The accuracy of the clinic records was lower in all ethnic groups when respondents identified with multiple ethnicities.
Western Australian data from an interview study12 showed 85.8% of indigenous people were correctly recorded in hospital records. Non-indigenous people had the correct ethnicity recorded in 99.5% of hospital records.
When comparing these studies it must be noted that each used a slightly different method for data collection, which may explain some of the variation in the results shown in Table 4. It is known that ethnic responses may alter depending on a range of factors, including broad social structures, perceived fiscal or political benefit, psychological, and familial.13 Technical issues such as question design, collection methods, timing and coding can also have an impact.13,14
Notwithstanding these methodological differences, it appears that ethnic identity has been recorded in hospital records less accurately for those who self-identify as Māori than for those self-identify in a non-Māori ethnic group. Moreover, this situation seems to be a long-standing tradition which does not seem ready to change. While the high and rising levels of accuracy in the non-Māori ethnic group appears reassuring, this may be misleading. The Barriers Study data (Table 3) clearly shows that when the non-Māori ethnic group is disaggregated into its component parts, all ethnic groups experience mis-recording in their hospital records.
The mis-recording of ethnic identity is particularly problematic in the ‘Other’ ethnic group, where the predictive value of the hospital record is also highly suspect. The ‘Other’ group may require specific attention from data entry personnel to increase the accuracy of ethnic self identity when people present for hospitalisation.
At more defined levels of ethnic identity there may be further concerns, not able to be reported on with the Barriers Study data. This study requested participant information at aggregated levels, thus the study cannot report on the accuracy of hospital records, for those who identify, for example, as Tongan or Sri Lankan.
More positively, it is important to note that nearly 98% of all participants had an ethnic group correctly recorded in the hospital record for one of their ethnic groups. While multiple ethnic identities do increase the complexity of demographic analysis, in any field, it is important that effort is expended to reflect the everyday realities of participants.
Lastly the predictive value of ethnicity information in hospital records show high levels of accuracy for Māori, Non-Māori, and Pacific Island ethnic groups (the latter two at aggregated levels). However there is room for improvement in the Asian and South Asian ethnic groups. The accuracy of the hospital record must be questioned when it states an individual belong to the ‘Other’ ethnic group.
As has been shown by past researchers, hospital records continue to mis-record ethnicity when compared to self-identified ethnic group. This occurs for all ethnic groups. The hospital record can predict ethnicity accurately but only at higher levels of aggregation: more detailed levels of ethnic group extracted from the hospital record must be treated with caution.
The discrepancies between the self-identified ethnicity and the ethnic group recorded in hospital record demonstrate several concerns which researchers, clinicians, and policy makers must be cognisant of. This continuing limitation of data in the contemporary New Zealand health environment must act as an impetus to improve the collection and accuracy of all datasets, but especially hospital records. Hospital data comparing ethnic groups needs to be viewed cautiously and the implications of misclassification should be reported quantitatively.
Author information: Judith Swan, Diabetes Research Fellow; Steven Lillis, Senior Lecturer in General Practice; David Simmons, Professor of Medicine; Waikato Clinical School, University of Auckland, Hamilton
Acknowledgements: The Barriers to Diabetes Care in the Waikato Study was supported with funding from the Waikato Medical Research Foundation and the Waikato Local Diabetes Team. The authors also acknowledge the assistance of Dr Jarrod Harr in the coding process.
Correspondence: Judith Swan, c/- Barriers to Diabetes Care in the Waikato, First Floor, Hockin Building, Waikato Hospital, PO Box 934, Hamilton. Fax: (07) 834 3615; email: email@example.com
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals