Ethnicity data audit in a secondary care gastroenterology service
View Article PDF
Accurate ethnicity data are important because they are used to quantify health inequities, including monitoring trends in disease rates and healthcare delivery. In relation to Māori health equity, high-quality ethnicity data in health are also an obligation under Te Tiriti.[[1]] In addition, accurate ethnicity data are essential if targeted interventions to address inequity are to be successful, such as the soon to be implemented targeted screening for bowel cancer recommended by a number of Māori health experts.[[2]] Ethnicity data collection in New Zealand health settings is guided by the Ministry of Health 2017 “HISO 10001:2017 Ethnicity Data Protocols”, which identify the gold standard for collecting ethnicity data in New Zealand as self-reported ethnicity collected using the Stats NZ 2018 Census ethnicity question. These protocols also state that “ethnicity data must be collected at least every three years”.[[3]] However, ethnicity data in hospital databases are variable in the extent to which they meet these standards.[[4]]
In New Zealand, ethnicity data have been shown to be less accurate for those identifying as Māori, other minority ethnic groups and for those identifying with multiple ethnicities.[[5–8]] There are also variations in the quality of ethnicity data across datasets.[[4]] There have been several evaluations of New Zealand ethnicity data in a range of contexts over the last two decades.[[5–10]] The vast majority have consistently found an undercount of Māori in routinely collected health datasets when they are compared to known high-quality ethnicity data, such as newly collected and standardised self-reported ethnicity. In a 2006 study, a sample of 3,500 patients with diabetes in Waikato found that non-Māori had 99.3% concordance in self-reported ethnicity data (collected for the study) with the hospital database compared to just 71.2% for Māori participants.[[5]] Eleven years later, a 2017 audit in Waikato District Health Board (WDHB) found Māori undercounts of 17% in the hospital National Health Index (NHI) database and 21% in the trauma registry record.[[6]] These issues are not only found in hospital data, with a 2007 study of Waitematā Primary Health Organisation (PHO) data showing an undercount for Māori twice that of NZ Europeans when compared to a fresh self-identified collection (on the National Immunisation Collection),[[7]] and a 2008 study in a large Auckland PHO finding a concordance of 64.9% of Māori, compared with 90.9% of NZ Europeans.[[8]] Ongoing data quality issues, particularly systematic undercounting of Māori, suggest a level of non-adherence to ethnicity data collection protocols for the health sector.[[3]]
Undercounts of Māori at the health data aggregate level, and the accuracy of the proportion of Māori (percentage of Māori in the dataset) have important implications. Aggregated data can be used for planning services, measuring rates of disease and in the measurement and monitoring of health inequities. They are also used for understanding the size of different ethnic groups, and to estimate how many people from different ethnic groups have been reached, such as with vaccination rollouts for Māori.[[11]] Equally important is the accuracy of ethnicity data at the individual level, where this information may be used to target health services in New Zealand to individuals including the use of ethnicity as part of cardiovascular risk assessment,[[12]] diabetes screening[[13]] and sore throat swabbing.[[14]] Information on the accuracy of ethnicity data at the individual level can also help inform us about the quality of aggregate level ethnicity data, as it provides important information on ethnicity misclassification and whether this is differential or not.
This audit aims to comprehensively evaluate the accuracy of ethnicity data at both the aggregate and individual levels by comparing a fresh collection of self-identified ethnicity data with the ethnicity data stored under (NHI) in the Hutt Hospital database for a sample of 200 consecutive patients presenting to Hutt DHB gastroenterology services. Hutt Hospital is an urban tertiary teaching centre, where the gastroenterology department cared for more than 13,000 patients who attended outpatient clinics and endoscopies between 1 April 2020 and 31 March 2021. It provides care to those living in the Hutt Valley District Health Board (HVDHB) region, serving a population of 156,790 people—17.9% of whom are estimated to be Māori based on 2020/2021 population projections by the Ministry of Health.[[15]]
Methods
This study was a prospective ethnicity audit of 200 consecutive patients aged 16 years and over presenting to the HVDHB gastroenterology service as outpatients to either a gastroenterology clinic or an endoscopy between 2–11 November 2021. The aim of this study was to audit the quality of ethnicity data by comparing current self-identified ethnicity (directly collected from patients for this audit), hereby referred to as “self-identified ethnicity”, to that recorded under the NHI in the Hutt Hospital database. We identified the level of match or mismatch present between the collections. We note that while ethnicity in all health data is also supposed to be self-identified according to protocols for the health sector, that is not always the case.
Data collection and analysis
This audit followed the methods described in the Ministry of Health’s “Primary Care Ethnicity Data Toolkit” published in 2013. Ethnicity data were collected from consecutive patients on each day of data collection until a total sample size of 200 patients was achieved. The protocol methods align with the Ministry of Health 2017 “HISO 10001:2017 Ethnicity Data Protocols” and thus self-identified ethnicity collected in this study is considered the “gold standard” in comparison to the NHI ethnicity recorded on the hospital NHI database.
The audit was powered to detect whether a mismatch is present between self-identified ethnicity data and the NHI hospital database ethnicity data. With 200 patients total and 90% accuracy in ethnicity data overall, the overall margin of error is roughly +/-5% (proportion/accuracy=90, lower/upper bound of 95% CI, 85.0/93.8).
All self-reported ethnicity data were collected by JR using the 2001 New Zealand Census ethnicity question via a patient-completed form after verbal consent was obtained. From the time of invitation to take part in the audit until the patient had completed their self-identified ethnicity, the data-collector was blind to the hospital database ethnicity data. Patients’ full names and dates of birth were also collected for the purposes of correctly identifying the patients in the hospital database. NHI-linked ethnicity data in the HVDHB Hospital database for each patient were then accessed and recorded by JR. All hospital database ethnicity data were collected on the same day as the patient presentation. Ethnicity data from both sources were coded to Level 2 of the 2004 Ministry of Health ethnicity data protocols for analysis.[[16]]
We used three measures to evaluate the accuracy of the hospital NHI database: ethnic proportions, individual mismatch and undercounts. Ethnic proportions were calculated separately for the self-identified collection and the hospital database (NHI) ethnicity, and were calculated as the number of individuals who reported being a member of the ethnic group divided by the total population size in the health dataset. For each individual, the level of match of ethnicity data between self-identified and NHI hospital database was categorised as match, partial match and total mismatch (Table 1).[[16]] The measurement of undercount assumes that self-identified ethnicity is the gold standard collection. It gives the proportion of individuals who self-identified an ethnic group that were not recorded with that ethnic group in the hospital database.
Statistical analysis
Data were analysed in Microsoft Excel version 2110 and R Studio (Version 1.4.1106) to generate descriptive statistics and confidence intervals.
Results
Of the 202 patients invited to take part in the study, 200 (99%) consented to participate. The median age of participants was 56.5 years (IQR 42–69.25) with an age range of participants from 16 to 93 years. Slightly more participants were female (n=107, 53.5%). Participants were a mix of those presenting for endoscopy (n=114, 57%) and gastroenterology clinics (n=86, 43%).
In the sample of 200 patients, 77.5% (n=155) self-identified as New Zealand European (either alone or in combination with other ethnicities) and 9% (n=18) identified themselves as Māori (Table 2). Half of self-identified Māori reported belonging to more than one ethnic group (9/18) compared to just 3% (5/182) of non-Māori (Table 3). Overall, those self-identifying with multiple ethnicities had a 50% (95% CI, 23.0–77.0) match (with all ethnicities also recorded on the hospital NHI database) compared to 91.9% (95% CI, 87.0–95.4) for patients who self-identified with a single ethnicity (Table 3). All mismatches in the multiple ethnicities group were partial mismatches, where some but not all of the self-identified ethnicities were recorded.
View Tables 1–3.
The overall level of match between the self-identified and the hospital NHI database ethnicity was 89% (95% CI, 83.8–93.0). Partial mismatches were found for 5% (10/200) of patients and total mismatch for 6% (12/200). Among non-Māori patients only 3.8% (95% CI, 1.6–7.8) of patients had partial mismatch of ethnicity data and 6.6% (95% CI, 3.5–11.2) had total mismatch of data. In comparison the match for Māori was 83.3% (95% CI, 58.6–96.4), 16.7% (95% CI, 3.6–41.4) of Māori patients had partial mismatch of ethnicity data and there were no total mismatches. Importantly, all the partial mismatches for Māori resulted in the patient being unable to be identified as Māori on the hospital NHI database. This means that the undercount for Māori on the hospital NHI database when compared to the self-identified fresh collection was 16.7%. The undercount in the NZ European group was 5.8%.
In both the self-reported ethnicity data and the hospital NHI database, the ethnic proportion of Māori was 9% (18/200). There were three patients who were identified as Māori in the hospital NHI database that did not self-report Māori ethnicity—two of whom identified themselves as NZ European only, and one who self-identified as NZ European and Niuean. Additionally, there were three patients who were identified as NZ European only in the hospital NHI database but self-identified as both Māori and NZ European.
There was some variation in the level of match for ethnicity when stratified by other demographics. Males had a lower match rate (81.7%) than females (95.3%), with a higher rate of total mismatch in males (11.8%) compared to females (0.9%). Most participants were aged between 40 and 79, and mismatch was relatively evenly distributed by age of around 10–15% mismatch, with the exception of being slightly higher in the 70- to 79-year-old category at 17% (Figure 1). The proportion of (mis)match did not differ between patients presenting to clinic or endoscopy.
Figure 1: Any mismatch by age for total sample population.
Discussion
This study audited the accuracy of NHI ethnicity data in a hospital NHI database against self-reported ethnicity, using the methods described in the Ministry of Health’s “Primary Care Ethnicity Data Toolkit”. We found an overall match rate of 89% in our sample and a match rate of 83.3% for Māori.
All the mismatches found for people self-reporting Māori ethnicity in our sample resulted in them being unidentifiable in the hospital NHI database, with an undercount of Māori on the hospital NHI database measuring 16.7%. In comparison, the undercount for the NZ European group was 5.8%. Undercounts of Māori have been found in several studies over the last two decades in a variety of areas of the health system. Swan et al. audited the accuracy of ethnicity data in a group of 3,500 diabetes patients in the Waikato in 2006 using a non-standard ethnicity question,[[5]] and in 2007 Bramley et al. looked at 57,612 vaccination records of school-aged children with data collected using the census ethnicity question.[[7]] These studies found Māori undercounts of 28.8% and 37.1% respectively. Undercounts found in more recent studies have been more variable. A 2017 study by Scott et al. found an undercount of Māori by 17% in the hospital NHI database and 21% in the Trauma Registry.[[6]] In contrast, another 2017 study of 227 patients with kidney disease found a Māori undercount of 7.3%, but this was still twice the undercount of 3.6% found in NZ Europeans.[[9]] The results of our audit would support evidence that there remains inequity between the size of the undercounts for Māori when compared to the NZ European population.
In our study, the ethnic proportions of Māori in the self-reported ethnicity data and the hospital NHI database were both 9%. This finding differs from other New Zealand-based hospital ethnicity audits that have found both an undercount of Māori participants in the hospital NHI database and a lower ethnic proportion of Māori.[[5,6]] This difference may reflect the smaller sample of Māori in our audit due to the lower ethnic proportion of Māori in the HVDHB catchment and potential selection bias in terms of who is able to navigate the healthcare pathway to access gastroenterology services (discussed further below).
At the individual level, we found differences in the individuals identified/recorded as Māori in the datasets, with three participants identified as Māori in the hospital NHI database who did not self-report Māori ethnicity, and three participants not identified as Māori in the hospital NHI database who did self-identify as Māori. This has important implications for our ability to identify the correct individuals to invite to existing and future health programmes that use NHI ethnicity data to target individuals. Both the cardiovascular disease risk assessment and[[12]] and diabetes screening[[13]] in primary care start for Māori, Pasifika and South Asian patients at an earlier age. Being of Māori or Pacific ethnicity is also used in guidance for sore throat swabbing for group A streptococcal pharyngitis to prevent rheumatic fever.[[14]] More recently, COVID-19 vaccination efforts for Māori relied on the accurate identification of individuals (as Māori) from existing data collections, as well as the use of aggregate-level denominators sourced from health service data, for reporting of vaccine coverage.[[11]]
Similar to other studies, we found that there was lower accuracy for those self-identifying with multiple ethnicities.[[6,7]] A complete match was found for only 50% of the 14 participants that had multiple self-reported ethnicities, compared to 91.9% in those who only self-reported one ethnicity. Importantly, half of the Māori participants (compared to just 3% of non-Māori) in our sample self-reported multiple ethnicities and so the impact of this inaccuracy may be greater for Māori. Consistent with the Primary Care Ethnicity Data Audit Toolkit our audit was powered to detect overall mismatch and was therefore underpowered to detect differences for multiple ethnicities. In addition, due to a smaller Māori sample than anticipated, some of our confidence intervals around our best estimates are overlapping. However, our findings are consistent with other research and indicate an ongoing issue with ethnicity data quality.
Māori were under-represented in our sample, making up just 9% of patients, compared to 17.9% in the HVDHB population (a difference that remained when we limited the sample to those aged 30–69 years at 10% versus 16.7% for HVDHB).[[15,17]] While there is the possibility of sampling error (in terms of Māori just not being booked on the days the audit was run), it is likely that the under-representation of Māori in our sample also reflects that Māori have inequitable access to this service. In general, Māori carry a higher rate of comorbidities and have higher rates of a large number of gastroenterological illnesses including stomach cancer,[[18]] hepatocellular carcinoma,[[19]] gastric ulcers [[20]] and upper gastrointestinal bleeding,[[20]] and similar rates of bowel cancer (but worse survival).[[21]] There are lower rates of diagnosed inflammatory bowel disease in Māori [[22]]. On balance, we expect Māori have greater need for gastroenterological services than non-Māori, and so we were surprised to find that in this study Māori were underrepresented in the gastroenterology service.
The inequitable rates of gastroenterology service use identified in this study may be an indication of barriers along the healthcare pathway that are experienced by Māori.[[23]] There are well-documented inequities in access to primary care,[[24]] and a higher level of unmet healthcare need for Māori,[[24]] which then impacts on access to secondary services. Those able to navigate the healthcare system into secondary care may have had more opportunities to update their ethnicity data. For example, all patients presenting to a gastroenterology clinic or endoscopy at Hutt Hospital are asked to review a paper copy of their hospital details at each presentation, including their ethnicity information. Our sample of patients, particularly those who saw gastroenterology services regularly, would have had multiple opportunities to review and change their ethnicity information, potentially (although not necessarily) improving accuracy. Therefore, our estimate of the level of undercount of Māori ethnicity may be smaller than the undercount for those less able to navigate the healthcare system, in settings where patients can present to services directly (e.g., emergency department), or in other district health boards with different processes around ethnicity data collection, and recording.
Ethnicity data collection needs to be an ongoing activity as ethnicity can change over time, and this has been shown to be a greater issue for Māori, Pasifika and Asian populations more than other ethnicities.[[25]] A 2009 New Zealand study found that 8% of individuals changed their self-reported ethnicity over a 3-year period, and the strongest predictors of this change were reporting Māori, Pasifika or Asian identity at the beginning of the study, as well as reporting multiple ethnic groups.[[25]] This also means that the ability to regularly update self-reported ethnicity will have an effect on accuracy, and that this will impact Māori and other minoritised groups more than NZ Europeans.
All areas of health in New Zealand need to collect, and maintain access to, high-quality ethnicity data. The New Zealand Ministry of Health ethnicity data protocols published in 2017 already provide guidance on ways to improve ethnicity data quality.[[3]] This guidance includes ensuring ethnicity is self-identified, using only the census ethnicity question for collection and a maximum of 3 years between each collection of ethnicity data. However, this guidance requires supported implementation to be followed appropriately with ongoing quality assessment. There are also other opportunities to collect updated ethnicity data that could improve quality on a national level that could form part of a wider strategy, such as collection of updated data as part of vaccination programmes. A key finding in this study is the importance of measuring the accuracy of ethnicity data in multiple ways to reflect their multiple uses.
Conclusion
Ethnicity data are fundamental to the monitoring and provision of equitable health and healthcare, with a range of applications in the health sector. Data can accurately reflect ethnic proportions at a group level but be inaccurate at the individual level, which can be falsely reassuring about overall data quality. Our findings of poor-quality ethnicity data in the hospital NHI record are consistent with previous studies. Māori have been shown to be particularly affected by inaccurate ethnicity data in a range of settings that are important for measuring and acting to correct health inequities.
Summary
Abstract
Aim
To audit the quality of ethnicity data stored under National Health Index (NHI) in the Hutt Hospital database against a fresh collection of self-identified ethnicity to identify the level of (mis)match present between the datasets.
Method
Self-identified ethnicity data was collected from 200 consecutive patients presenting to outpatient gastroenterology services and compared to National Health Index (NHI) in the Hutt Hospital database, using the process outlined in the Primary Care Ethnicity Data Audit Toolkit.
Results
The overall level of match between the individual’s self-identified ethnicity and that recorded in the hospital database was 89% (95% CI [83.8–93.0]). Eighteen patients (9%) self-identified as Māori, 16.7% (95% CI [3.6–41.4]) of whom were not recorded as Māori in the hospital database. Three patients were recorded as Māori in the hospital database but did not self-identify as Māori.
Conclusion
Ethnicity data are fundamental to the monitoring and provision of equitable health and healthcare, with a range of applications in the health sector. Our findings of poor-quality ethnicity data for Māori in a hospital NHI database are consistent with previous studies. The assessment of ethnicity data quality must be done in multiple ways to reflect its multiple uses.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) Te Rōpū Rangahau Hauora a Eru Pōmare. Counting for nothing: understanding the issues in monitoring disparities in health. Soc Policy J N Z. 2000;(14):1-6.
2) McLeod M, Harris R, Paine S-J, Crengle S et al. Bowel cancer screening age range for Māori: what is all the fuss about? N Z Med J. 2021;134(1535):71-7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/34012141.
3) NZ Ministry of Health. 2017 Ethnicity Data Protocols [Internet]. Wellington; 2017 [cited 2022 Feb 4]. Available from: https://www.health.govt.nz/publication/hiso-100012017-ethnicity-data-protocols.
4) Cormack D, McLeod M. Improving and maintaining quality in ethnicity data collections in the health and disability sector. Wellington: Te Rōpū Rangahau Hauora a Eru Pōmare; 2010.
5) Swan J, Lillis S, Simmons D. Investigating the accuracy of ethnicity data in New Zealand hospital records: still room for improvement. N Z Med J. 2006 Aug 4;119(1239):U2103. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16912721.
6) Scott N, Clark H, Kool B, Ameratunga S et al. Audit of ethnicity data in the Waikato hospital patient management system and trauma registry: Pilot of the hospital ethnicity data audit toolkit. N Z Med J. 2018;131(1483):21-9.
7) Bramley D, Latimer S. The accuracy of ethnicity data in primary care. N Z Med J. 2007 Oct 26;120(1264):U2779. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17972986.
8) Riddell T, Lindsay G, Kenealy T, Jackson R et al. The accuracy of ethnicity data in primary care and its impact on cardiovascular risk assessment and management - PREDICT CVD-8. N Z Med J. 2008;121(1281):40-8.
9) Page MJ, Wyeth EH, Samaranayaka A, McNoe B et al. Accuracy of ethnicity data recorded in hospital-based patient clinical records and the Australia and New Zealand dialysis and transplant registry. N Z Med J. 2017;130(1454):65-71.
10) Rumball-Smith J, Sarfati D. Improvement in the accuracy of hospital ethnicity data. N Z Med J. 2011 Aug 1;124:96-7.
11) NZ Ministry of Health. COVID-19 Māori Vaccine and Immunisation Plan [Internet]. Wellington; 2021 Mar [cited 2022 Jan 24]. Available from: https://www.health.govt.nz/publication/covid-19-maori-vaccine-and-immunisation-plan-supplementary-updated-covid-19-maori-health-response.
12) NZ Ministry of Health. Cardiovascular Disease Risk Assessment and Management for Primary Care 2018. Available at: https://www.health.govt.nz/system/files/documents/publications/cardiovascular-disease-risk-assessment-management-primary-care-feb18-v4_0.pdf.
13) NZ Ministry of Health, NZ Society for the Study of Diabetes. Screening for diabetes in asymptomatic adults - Type 2 Diabetes Management Guidelines [Internet]. Type 2 Diabetes Management Guidance Website. 2022 [cited 2022 Jan 24]. Available from: https://t2dm.nzssd.org.nz/Section-112-Screening-for-diabetes-in-asymptomatic-adults.
14) National Heart Foundation of New Zealand. Evidence-based, best practice New Zealand Guidelines for Rheumatic Fever. Auckland; 2019.
15) NZ Ministry of Health. My DHB: Hutt Valley DHB [Internet]. Ministry of Health Website. 2021 [cited 2021 Dec 29]. Available from: https://www.health.govt.nz/new-zealand-health-system/my-dhb/hutt-valley-dhb.
16) Ministry of Health. Primary Care Ethnicity Data Audit Toolkit [Internet]. 2013. Available from: http://www.health.govt.nz/publication/primary-care-ethnicity-data-audit-toolkit.
17) Stats NZ. 2018 Census. 2020.
18) Ellison-Loschmann L, Sporle A, Corbin M, Cheng S et al. Risk of stomach cancer in Aotearoa/New Zealand: A Māori population based case-control study. PLoS One. 2017 Jul 1 [cited 2022 Feb 15];12(7). Available from: /pmc/articles/PMC5521812/.
19) Chamberlain J, Sarfati D, Cunningham R, Koea J et al. Incidence and management of hepatocellular carcinoma among Māori and non-Māori New Zealanders. Aust N Z J Public Health. 2013 Dec;37(6):520-6. Available from: https://onlinelibrary.wiley.com/doi/10.1111/1753-6405.12108.
20) Irwin J, Ferguson R, Weilert F, Smith A. Incidence of upper gastrointestinal haemorrhage in Maori and New Zealand European ethnic groups, 2001-2010. Intern Med J [Internet]. 2014 [cited 2022 Feb 15];44(8):735-41. Available from: https://pubmed.ncbi.nlm.nih.gov/24893826/.
21) Blackmore T, Lao C, Chepulis L, Page B et al. The characteristics and outcomes of patients with colorectal cancer in New Zealand, analysed by Cancer Network. N Z Med J. 2020 [cited 2022 Feb 15];133:1513. Available from: www.nzma.org.nz/journal.
22) Coppell KJ, Galts CP-C, Huizing FY, Norton JK et al. Annual Incidence and Phenotypic Presentation of IBD in Southern New Zealand: An 18-Year Epidemiological Analysis. Inflamm Intest Dis [Internet]. 2018 [cited 2022 Feb 15];3(1):329. Available from: https://www.karger.com/Article/FullText/492615.
23) Espiner E, Paine SJ, Weston M, Curtis E. Barriers and facilitators for Māori in accessing hospital services in Aotearoa New Zealand. N Z Med J. 2021;134(1546):47-58.
24) NZ Ministry of Health. Annual Data Explorer 2020/2021: New Zealand Health survey [Data File] [Internet]. 2021 [cited 2022 Feb 28]. Available from: https://minhealthnz.shinyapps.io/nz-health-survey-2020-21-annual-data-explorer/.
25) Carter K, Hayward M, Blakely T, Shaw C. How much and for whom does self-identified ethnicity change over time in New Zealand? Results from a longitudinal study. Soc Policy J N Z. 2009 Aug 1;36.
For the PDF of this article,
contact nzmj@nzma.org.nz
View Article PDF
Accurate ethnicity data are important because they are used to quantify health inequities, including monitoring trends in disease rates and healthcare delivery. In relation to Māori health equity, high-quality ethnicity data in health are also an obligation under Te Tiriti.[[1]] In addition, accurate ethnicity data are essential if targeted interventions to address inequity are to be successful, such as the soon to be implemented targeted screening for bowel cancer recommended by a number of Māori health experts.[[2]] Ethnicity data collection in New Zealand health settings is guided by the Ministry of Health 2017 “HISO 10001:2017 Ethnicity Data Protocols”, which identify the gold standard for collecting ethnicity data in New Zealand as self-reported ethnicity collected using the Stats NZ 2018 Census ethnicity question. These protocols also state that “ethnicity data must be collected at least every three years”.[[3]] However, ethnicity data in hospital databases are variable in the extent to which they meet these standards.[[4]]
In New Zealand, ethnicity data have been shown to be less accurate for those identifying as Māori, other minority ethnic groups and for those identifying with multiple ethnicities.[[5–8]] There are also variations in the quality of ethnicity data across datasets.[[4]] There have been several evaluations of New Zealand ethnicity data in a range of contexts over the last two decades.[[5–10]] The vast majority have consistently found an undercount of Māori in routinely collected health datasets when they are compared to known high-quality ethnicity data, such as newly collected and standardised self-reported ethnicity. In a 2006 study, a sample of 3,500 patients with diabetes in Waikato found that non-Māori had 99.3% concordance in self-reported ethnicity data (collected for the study) with the hospital database compared to just 71.2% for Māori participants.[[5]] Eleven years later, a 2017 audit in Waikato District Health Board (WDHB) found Māori undercounts of 17% in the hospital National Health Index (NHI) database and 21% in the trauma registry record.[[6]] These issues are not only found in hospital data, with a 2007 study of Waitematā Primary Health Organisation (PHO) data showing an undercount for Māori twice that of NZ Europeans when compared to a fresh self-identified collection (on the National Immunisation Collection),[[7]] and a 2008 study in a large Auckland PHO finding a concordance of 64.9% of Māori, compared with 90.9% of NZ Europeans.[[8]] Ongoing data quality issues, particularly systematic undercounting of Māori, suggest a level of non-adherence to ethnicity data collection protocols for the health sector.[[3]]
Undercounts of Māori at the health data aggregate level, and the accuracy of the proportion of Māori (percentage of Māori in the dataset) have important implications. Aggregated data can be used for planning services, measuring rates of disease and in the measurement and monitoring of health inequities. They are also used for understanding the size of different ethnic groups, and to estimate how many people from different ethnic groups have been reached, such as with vaccination rollouts for Māori.[[11]] Equally important is the accuracy of ethnicity data at the individual level, where this information may be used to target health services in New Zealand to individuals including the use of ethnicity as part of cardiovascular risk assessment,[[12]] diabetes screening[[13]] and sore throat swabbing.[[14]] Information on the accuracy of ethnicity data at the individual level can also help inform us about the quality of aggregate level ethnicity data, as it provides important information on ethnicity misclassification and whether this is differential or not.
This audit aims to comprehensively evaluate the accuracy of ethnicity data at both the aggregate and individual levels by comparing a fresh collection of self-identified ethnicity data with the ethnicity data stored under (NHI) in the Hutt Hospital database for a sample of 200 consecutive patients presenting to Hutt DHB gastroenterology services. Hutt Hospital is an urban tertiary teaching centre, where the gastroenterology department cared for more than 13,000 patients who attended outpatient clinics and endoscopies between 1 April 2020 and 31 March 2021. It provides care to those living in the Hutt Valley District Health Board (HVDHB) region, serving a population of 156,790 people—17.9% of whom are estimated to be Māori based on 2020/2021 population projections by the Ministry of Health.[[15]]
Methods
This study was a prospective ethnicity audit of 200 consecutive patients aged 16 years and over presenting to the HVDHB gastroenterology service as outpatients to either a gastroenterology clinic or an endoscopy between 2–11 November 2021. The aim of this study was to audit the quality of ethnicity data by comparing current self-identified ethnicity (directly collected from patients for this audit), hereby referred to as “self-identified ethnicity”, to that recorded under the NHI in the Hutt Hospital database. We identified the level of match or mismatch present between the collections. We note that while ethnicity in all health data is also supposed to be self-identified according to protocols for the health sector, that is not always the case.
Data collection and analysis
This audit followed the methods described in the Ministry of Health’s “Primary Care Ethnicity Data Toolkit” published in 2013. Ethnicity data were collected from consecutive patients on each day of data collection until a total sample size of 200 patients was achieved. The protocol methods align with the Ministry of Health 2017 “HISO 10001:2017 Ethnicity Data Protocols” and thus self-identified ethnicity collected in this study is considered the “gold standard” in comparison to the NHI ethnicity recorded on the hospital NHI database.
The audit was powered to detect whether a mismatch is present between self-identified ethnicity data and the NHI hospital database ethnicity data. With 200 patients total and 90% accuracy in ethnicity data overall, the overall margin of error is roughly +/-5% (proportion/accuracy=90, lower/upper bound of 95% CI, 85.0/93.8).
All self-reported ethnicity data were collected by JR using the 2001 New Zealand Census ethnicity question via a patient-completed form after verbal consent was obtained. From the time of invitation to take part in the audit until the patient had completed their self-identified ethnicity, the data-collector was blind to the hospital database ethnicity data. Patients’ full names and dates of birth were also collected for the purposes of correctly identifying the patients in the hospital database. NHI-linked ethnicity data in the HVDHB Hospital database for each patient were then accessed and recorded by JR. All hospital database ethnicity data were collected on the same day as the patient presentation. Ethnicity data from both sources were coded to Level 2 of the 2004 Ministry of Health ethnicity data protocols for analysis.[[16]]
We used three measures to evaluate the accuracy of the hospital NHI database: ethnic proportions, individual mismatch and undercounts. Ethnic proportions were calculated separately for the self-identified collection and the hospital database (NHI) ethnicity, and were calculated as the number of individuals who reported being a member of the ethnic group divided by the total population size in the health dataset. For each individual, the level of match of ethnicity data between self-identified and NHI hospital database was categorised as match, partial match and total mismatch (Table 1).[[16]] The measurement of undercount assumes that self-identified ethnicity is the gold standard collection. It gives the proportion of individuals who self-identified an ethnic group that were not recorded with that ethnic group in the hospital database.
Statistical analysis
Data were analysed in Microsoft Excel version 2110 and R Studio (Version 1.4.1106) to generate descriptive statistics and confidence intervals.
Results
Of the 202 patients invited to take part in the study, 200 (99%) consented to participate. The median age of participants was 56.5 years (IQR 42–69.25) with an age range of participants from 16 to 93 years. Slightly more participants were female (n=107, 53.5%). Participants were a mix of those presenting for endoscopy (n=114, 57%) and gastroenterology clinics (n=86, 43%).
In the sample of 200 patients, 77.5% (n=155) self-identified as New Zealand European (either alone or in combination with other ethnicities) and 9% (n=18) identified themselves as Māori (Table 2). Half of self-identified Māori reported belonging to more than one ethnic group (9/18) compared to just 3% (5/182) of non-Māori (Table 3). Overall, those self-identifying with multiple ethnicities had a 50% (95% CI, 23.0–77.0) match (with all ethnicities also recorded on the hospital NHI database) compared to 91.9% (95% CI, 87.0–95.4) for patients who self-identified with a single ethnicity (Table 3). All mismatches in the multiple ethnicities group were partial mismatches, where some but not all of the self-identified ethnicities were recorded.
View Tables 1–3.
The overall level of match between the self-identified and the hospital NHI database ethnicity was 89% (95% CI, 83.8–93.0). Partial mismatches were found for 5% (10/200) of patients and total mismatch for 6% (12/200). Among non-Māori patients only 3.8% (95% CI, 1.6–7.8) of patients had partial mismatch of ethnicity data and 6.6% (95% CI, 3.5–11.2) had total mismatch of data. In comparison the match for Māori was 83.3% (95% CI, 58.6–96.4), 16.7% (95% CI, 3.6–41.4) of Māori patients had partial mismatch of ethnicity data and there were no total mismatches. Importantly, all the partial mismatches for Māori resulted in the patient being unable to be identified as Māori on the hospital NHI database. This means that the undercount for Māori on the hospital NHI database when compared to the self-identified fresh collection was 16.7%. The undercount in the NZ European group was 5.8%.
In both the self-reported ethnicity data and the hospital NHI database, the ethnic proportion of Māori was 9% (18/200). There were three patients who were identified as Māori in the hospital NHI database that did not self-report Māori ethnicity—two of whom identified themselves as NZ European only, and one who self-identified as NZ European and Niuean. Additionally, there were three patients who were identified as NZ European only in the hospital NHI database but self-identified as both Māori and NZ European.
There was some variation in the level of match for ethnicity when stratified by other demographics. Males had a lower match rate (81.7%) than females (95.3%), with a higher rate of total mismatch in males (11.8%) compared to females (0.9%). Most participants were aged between 40 and 79, and mismatch was relatively evenly distributed by age of around 10–15% mismatch, with the exception of being slightly higher in the 70- to 79-year-old category at 17% (Figure 1). The proportion of (mis)match did not differ between patients presenting to clinic or endoscopy.
Figure 1: Any mismatch by age for total sample population.
Discussion
This study audited the accuracy of NHI ethnicity data in a hospital NHI database against self-reported ethnicity, using the methods described in the Ministry of Health’s “Primary Care Ethnicity Data Toolkit”. We found an overall match rate of 89% in our sample and a match rate of 83.3% for Māori.
All the mismatches found for people self-reporting Māori ethnicity in our sample resulted in them being unidentifiable in the hospital NHI database, with an undercount of Māori on the hospital NHI database measuring 16.7%. In comparison, the undercount for the NZ European group was 5.8%. Undercounts of Māori have been found in several studies over the last two decades in a variety of areas of the health system. Swan et al. audited the accuracy of ethnicity data in a group of 3,500 diabetes patients in the Waikato in 2006 using a non-standard ethnicity question,[[5]] and in 2007 Bramley et al. looked at 57,612 vaccination records of school-aged children with data collected using the census ethnicity question.[[7]] These studies found Māori undercounts of 28.8% and 37.1% respectively. Undercounts found in more recent studies have been more variable. A 2017 study by Scott et al. found an undercount of Māori by 17% in the hospital NHI database and 21% in the Trauma Registry.[[6]] In contrast, another 2017 study of 227 patients with kidney disease found a Māori undercount of 7.3%, but this was still twice the undercount of 3.6% found in NZ Europeans.[[9]] The results of our audit would support evidence that there remains inequity between the size of the undercounts for Māori when compared to the NZ European population.
In our study, the ethnic proportions of Māori in the self-reported ethnicity data and the hospital NHI database were both 9%. This finding differs from other New Zealand-based hospital ethnicity audits that have found both an undercount of Māori participants in the hospital NHI database and a lower ethnic proportion of Māori.[[5,6]] This difference may reflect the smaller sample of Māori in our audit due to the lower ethnic proportion of Māori in the HVDHB catchment and potential selection bias in terms of who is able to navigate the healthcare pathway to access gastroenterology services (discussed further below).
At the individual level, we found differences in the individuals identified/recorded as Māori in the datasets, with three participants identified as Māori in the hospital NHI database who did not self-report Māori ethnicity, and three participants not identified as Māori in the hospital NHI database who did self-identify as Māori. This has important implications for our ability to identify the correct individuals to invite to existing and future health programmes that use NHI ethnicity data to target individuals. Both the cardiovascular disease risk assessment and[[12]] and diabetes screening[[13]] in primary care start for Māori, Pasifika and South Asian patients at an earlier age. Being of Māori or Pacific ethnicity is also used in guidance for sore throat swabbing for group A streptococcal pharyngitis to prevent rheumatic fever.[[14]] More recently, COVID-19 vaccination efforts for Māori relied on the accurate identification of individuals (as Māori) from existing data collections, as well as the use of aggregate-level denominators sourced from health service data, for reporting of vaccine coverage.[[11]]
Similar to other studies, we found that there was lower accuracy for those self-identifying with multiple ethnicities.[[6,7]] A complete match was found for only 50% of the 14 participants that had multiple self-reported ethnicities, compared to 91.9% in those who only self-reported one ethnicity. Importantly, half of the Māori participants (compared to just 3% of non-Māori) in our sample self-reported multiple ethnicities and so the impact of this inaccuracy may be greater for Māori. Consistent with the Primary Care Ethnicity Data Audit Toolkit our audit was powered to detect overall mismatch and was therefore underpowered to detect differences for multiple ethnicities. In addition, due to a smaller Māori sample than anticipated, some of our confidence intervals around our best estimates are overlapping. However, our findings are consistent with other research and indicate an ongoing issue with ethnicity data quality.
Māori were under-represented in our sample, making up just 9% of patients, compared to 17.9% in the HVDHB population (a difference that remained when we limited the sample to those aged 30–69 years at 10% versus 16.7% for HVDHB).[[15,17]] While there is the possibility of sampling error (in terms of Māori just not being booked on the days the audit was run), it is likely that the under-representation of Māori in our sample also reflects that Māori have inequitable access to this service. In general, Māori carry a higher rate of comorbidities and have higher rates of a large number of gastroenterological illnesses including stomach cancer,[[18]] hepatocellular carcinoma,[[19]] gastric ulcers [[20]] and upper gastrointestinal bleeding,[[20]] and similar rates of bowel cancer (but worse survival).[[21]] There are lower rates of diagnosed inflammatory bowel disease in Māori [[22]]. On balance, we expect Māori have greater need for gastroenterological services than non-Māori, and so we were surprised to find that in this study Māori were underrepresented in the gastroenterology service.
The inequitable rates of gastroenterology service use identified in this study may be an indication of barriers along the healthcare pathway that are experienced by Māori.[[23]] There are well-documented inequities in access to primary care,[[24]] and a higher level of unmet healthcare need for Māori,[[24]] which then impacts on access to secondary services. Those able to navigate the healthcare system into secondary care may have had more opportunities to update their ethnicity data. For example, all patients presenting to a gastroenterology clinic or endoscopy at Hutt Hospital are asked to review a paper copy of their hospital details at each presentation, including their ethnicity information. Our sample of patients, particularly those who saw gastroenterology services regularly, would have had multiple opportunities to review and change their ethnicity information, potentially (although not necessarily) improving accuracy. Therefore, our estimate of the level of undercount of Māori ethnicity may be smaller than the undercount for those less able to navigate the healthcare system, in settings where patients can present to services directly (e.g., emergency department), or in other district health boards with different processes around ethnicity data collection, and recording.
Ethnicity data collection needs to be an ongoing activity as ethnicity can change over time, and this has been shown to be a greater issue for Māori, Pasifika and Asian populations more than other ethnicities.[[25]] A 2009 New Zealand study found that 8% of individuals changed their self-reported ethnicity over a 3-year period, and the strongest predictors of this change were reporting Māori, Pasifika or Asian identity at the beginning of the study, as well as reporting multiple ethnic groups.[[25]] This also means that the ability to regularly update self-reported ethnicity will have an effect on accuracy, and that this will impact Māori and other minoritised groups more than NZ Europeans.
All areas of health in New Zealand need to collect, and maintain access to, high-quality ethnicity data. The New Zealand Ministry of Health ethnicity data protocols published in 2017 already provide guidance on ways to improve ethnicity data quality.[[3]] This guidance includes ensuring ethnicity is self-identified, using only the census ethnicity question for collection and a maximum of 3 years between each collection of ethnicity data. However, this guidance requires supported implementation to be followed appropriately with ongoing quality assessment. There are also other opportunities to collect updated ethnicity data that could improve quality on a national level that could form part of a wider strategy, such as collection of updated data as part of vaccination programmes. A key finding in this study is the importance of measuring the accuracy of ethnicity data in multiple ways to reflect their multiple uses.
Conclusion
Ethnicity data are fundamental to the monitoring and provision of equitable health and healthcare, with a range of applications in the health sector. Data can accurately reflect ethnic proportions at a group level but be inaccurate at the individual level, which can be falsely reassuring about overall data quality. Our findings of poor-quality ethnicity data in the hospital NHI record are consistent with previous studies. Māori have been shown to be particularly affected by inaccurate ethnicity data in a range of settings that are important for measuring and acting to correct health inequities.
Summary
Abstract
Aim
To audit the quality of ethnicity data stored under National Health Index (NHI) in the Hutt Hospital database against a fresh collection of self-identified ethnicity to identify the level of (mis)match present between the datasets.
Method
Self-identified ethnicity data was collected from 200 consecutive patients presenting to outpatient gastroenterology services and compared to National Health Index (NHI) in the Hutt Hospital database, using the process outlined in the Primary Care Ethnicity Data Audit Toolkit.
Results
The overall level of match between the individual’s self-identified ethnicity and that recorded in the hospital database was 89% (95% CI [83.8–93.0]). Eighteen patients (9%) self-identified as Māori, 16.7% (95% CI [3.6–41.4]) of whom were not recorded as Māori in the hospital database. Three patients were recorded as Māori in the hospital database but did not self-identify as Māori.
Conclusion
Ethnicity data are fundamental to the monitoring and provision of equitable health and healthcare, with a range of applications in the health sector. Our findings of poor-quality ethnicity data for Māori in a hospital NHI database are consistent with previous studies. The assessment of ethnicity data quality must be done in multiple ways to reflect its multiple uses.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) Te Rōpū Rangahau Hauora a Eru Pōmare. Counting for nothing: understanding the issues in monitoring disparities in health. Soc Policy J N Z. 2000;(14):1-6.
2) McLeod M, Harris R, Paine S-J, Crengle S et al. Bowel cancer screening age range for Māori: what is all the fuss about? N Z Med J. 2021;134(1535):71-7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/34012141.
3) NZ Ministry of Health. 2017 Ethnicity Data Protocols [Internet]. Wellington; 2017 [cited 2022 Feb 4]. Available from: https://www.health.govt.nz/publication/hiso-100012017-ethnicity-data-protocols.
4) Cormack D, McLeod M. Improving and maintaining quality in ethnicity data collections in the health and disability sector. Wellington: Te Rōpū Rangahau Hauora a Eru Pōmare; 2010.
5) Swan J, Lillis S, Simmons D. Investigating the accuracy of ethnicity data in New Zealand hospital records: still room for improvement. N Z Med J. 2006 Aug 4;119(1239):U2103. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16912721.
6) Scott N, Clark H, Kool B, Ameratunga S et al. Audit of ethnicity data in the Waikato hospital patient management system and trauma registry: Pilot of the hospital ethnicity data audit toolkit. N Z Med J. 2018;131(1483):21-9.
7) Bramley D, Latimer S. The accuracy of ethnicity data in primary care. N Z Med J. 2007 Oct 26;120(1264):U2779. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17972986.
8) Riddell T, Lindsay G, Kenealy T, Jackson R et al. The accuracy of ethnicity data in primary care and its impact on cardiovascular risk assessment and management - PREDICT CVD-8. N Z Med J. 2008;121(1281):40-8.
9) Page MJ, Wyeth EH, Samaranayaka A, McNoe B et al. Accuracy of ethnicity data recorded in hospital-based patient clinical records and the Australia and New Zealand dialysis and transplant registry. N Z Med J. 2017;130(1454):65-71.
10) Rumball-Smith J, Sarfati D. Improvement in the accuracy of hospital ethnicity data. N Z Med J. 2011 Aug 1;124:96-7.
11) NZ Ministry of Health. COVID-19 Māori Vaccine and Immunisation Plan [Internet]. Wellington; 2021 Mar [cited 2022 Jan 24]. Available from: https://www.health.govt.nz/publication/covid-19-maori-vaccine-and-immunisation-plan-supplementary-updated-covid-19-maori-health-response.
12) NZ Ministry of Health. Cardiovascular Disease Risk Assessment and Management for Primary Care 2018. Available at: https://www.health.govt.nz/system/files/documents/publications/cardiovascular-disease-risk-assessment-management-primary-care-feb18-v4_0.pdf.
13) NZ Ministry of Health, NZ Society for the Study of Diabetes. Screening for diabetes in asymptomatic adults - Type 2 Diabetes Management Guidelines [Internet]. Type 2 Diabetes Management Guidance Website. 2022 [cited 2022 Jan 24]. Available from: https://t2dm.nzssd.org.nz/Section-112-Screening-for-diabetes-in-asymptomatic-adults.
14) National Heart Foundation of New Zealand. Evidence-based, best practice New Zealand Guidelines for Rheumatic Fever. Auckland; 2019.
15) NZ Ministry of Health. My DHB: Hutt Valley DHB [Internet]. Ministry of Health Website. 2021 [cited 2021 Dec 29]. Available from: https://www.health.govt.nz/new-zealand-health-system/my-dhb/hutt-valley-dhb.
16) Ministry of Health. Primary Care Ethnicity Data Audit Toolkit [Internet]. 2013. Available from: http://www.health.govt.nz/publication/primary-care-ethnicity-data-audit-toolkit.
17) Stats NZ. 2018 Census. 2020.
18) Ellison-Loschmann L, Sporle A, Corbin M, Cheng S et al. Risk of stomach cancer in Aotearoa/New Zealand: A Māori population based case-control study. PLoS One. 2017 Jul 1 [cited 2022 Feb 15];12(7). Available from: /pmc/articles/PMC5521812/.
19) Chamberlain J, Sarfati D, Cunningham R, Koea J et al. Incidence and management of hepatocellular carcinoma among Māori and non-Māori New Zealanders. Aust N Z J Public Health. 2013 Dec;37(6):520-6. Available from: https://onlinelibrary.wiley.com/doi/10.1111/1753-6405.12108.
20) Irwin J, Ferguson R, Weilert F, Smith A. Incidence of upper gastrointestinal haemorrhage in Maori and New Zealand European ethnic groups, 2001-2010. Intern Med J [Internet]. 2014 [cited 2022 Feb 15];44(8):735-41. Available from: https://pubmed.ncbi.nlm.nih.gov/24893826/.
21) Blackmore T, Lao C, Chepulis L, Page B et al. The characteristics and outcomes of patients with colorectal cancer in New Zealand, analysed by Cancer Network. N Z Med J. 2020 [cited 2022 Feb 15];133:1513. Available from: www.nzma.org.nz/journal.
22) Coppell KJ, Galts CP-C, Huizing FY, Norton JK et al. Annual Incidence and Phenotypic Presentation of IBD in Southern New Zealand: An 18-Year Epidemiological Analysis. Inflamm Intest Dis [Internet]. 2018 [cited 2022 Feb 15];3(1):329. Available from: https://www.karger.com/Article/FullText/492615.
23) Espiner E, Paine SJ, Weston M, Curtis E. Barriers and facilitators for Māori in accessing hospital services in Aotearoa New Zealand. N Z Med J. 2021;134(1546):47-58.
24) NZ Ministry of Health. Annual Data Explorer 2020/2021: New Zealand Health survey [Data File] [Internet]. 2021 [cited 2022 Feb 28]. Available from: https://minhealthnz.shinyapps.io/nz-health-survey-2020-21-annual-data-explorer/.
25) Carter K, Hayward M, Blakely T, Shaw C. How much and for whom does self-identified ethnicity change over time in New Zealand? Results from a longitudinal study. Soc Policy J N Z. 2009 Aug 1;36.
For the PDF of this article,
contact nzmj@nzma.org.nz
View Article PDF
Accurate ethnicity data are important because they are used to quantify health inequities, including monitoring trends in disease rates and healthcare delivery. In relation to Māori health equity, high-quality ethnicity data in health are also an obligation under Te Tiriti.[[1]] In addition, accurate ethnicity data are essential if targeted interventions to address inequity are to be successful, such as the soon to be implemented targeted screening for bowel cancer recommended by a number of Māori health experts.[[2]] Ethnicity data collection in New Zealand health settings is guided by the Ministry of Health 2017 “HISO 10001:2017 Ethnicity Data Protocols”, which identify the gold standard for collecting ethnicity data in New Zealand as self-reported ethnicity collected using the Stats NZ 2018 Census ethnicity question. These protocols also state that “ethnicity data must be collected at least every three years”.[[3]] However, ethnicity data in hospital databases are variable in the extent to which they meet these standards.[[4]]
In New Zealand, ethnicity data have been shown to be less accurate for those identifying as Māori, other minority ethnic groups and for those identifying with multiple ethnicities.[[5–8]] There are also variations in the quality of ethnicity data across datasets.[[4]] There have been several evaluations of New Zealand ethnicity data in a range of contexts over the last two decades.[[5–10]] The vast majority have consistently found an undercount of Māori in routinely collected health datasets when they are compared to known high-quality ethnicity data, such as newly collected and standardised self-reported ethnicity. In a 2006 study, a sample of 3,500 patients with diabetes in Waikato found that non-Māori had 99.3% concordance in self-reported ethnicity data (collected for the study) with the hospital database compared to just 71.2% for Māori participants.[[5]] Eleven years later, a 2017 audit in Waikato District Health Board (WDHB) found Māori undercounts of 17% in the hospital National Health Index (NHI) database and 21% in the trauma registry record.[[6]] These issues are not only found in hospital data, with a 2007 study of Waitematā Primary Health Organisation (PHO) data showing an undercount for Māori twice that of NZ Europeans when compared to a fresh self-identified collection (on the National Immunisation Collection),[[7]] and a 2008 study in a large Auckland PHO finding a concordance of 64.9% of Māori, compared with 90.9% of NZ Europeans.[[8]] Ongoing data quality issues, particularly systematic undercounting of Māori, suggest a level of non-adherence to ethnicity data collection protocols for the health sector.[[3]]
Undercounts of Māori at the health data aggregate level, and the accuracy of the proportion of Māori (percentage of Māori in the dataset) have important implications. Aggregated data can be used for planning services, measuring rates of disease and in the measurement and monitoring of health inequities. They are also used for understanding the size of different ethnic groups, and to estimate how many people from different ethnic groups have been reached, such as with vaccination rollouts for Māori.[[11]] Equally important is the accuracy of ethnicity data at the individual level, where this information may be used to target health services in New Zealand to individuals including the use of ethnicity as part of cardiovascular risk assessment,[[12]] diabetes screening[[13]] and sore throat swabbing.[[14]] Information on the accuracy of ethnicity data at the individual level can also help inform us about the quality of aggregate level ethnicity data, as it provides important information on ethnicity misclassification and whether this is differential or not.
This audit aims to comprehensively evaluate the accuracy of ethnicity data at both the aggregate and individual levels by comparing a fresh collection of self-identified ethnicity data with the ethnicity data stored under (NHI) in the Hutt Hospital database for a sample of 200 consecutive patients presenting to Hutt DHB gastroenterology services. Hutt Hospital is an urban tertiary teaching centre, where the gastroenterology department cared for more than 13,000 patients who attended outpatient clinics and endoscopies between 1 April 2020 and 31 March 2021. It provides care to those living in the Hutt Valley District Health Board (HVDHB) region, serving a population of 156,790 people—17.9% of whom are estimated to be Māori based on 2020/2021 population projections by the Ministry of Health.[[15]]
Methods
This study was a prospective ethnicity audit of 200 consecutive patients aged 16 years and over presenting to the HVDHB gastroenterology service as outpatients to either a gastroenterology clinic or an endoscopy between 2–11 November 2021. The aim of this study was to audit the quality of ethnicity data by comparing current self-identified ethnicity (directly collected from patients for this audit), hereby referred to as “self-identified ethnicity”, to that recorded under the NHI in the Hutt Hospital database. We identified the level of match or mismatch present between the collections. We note that while ethnicity in all health data is also supposed to be self-identified according to protocols for the health sector, that is not always the case.
Data collection and analysis
This audit followed the methods described in the Ministry of Health’s “Primary Care Ethnicity Data Toolkit” published in 2013. Ethnicity data were collected from consecutive patients on each day of data collection until a total sample size of 200 patients was achieved. The protocol methods align with the Ministry of Health 2017 “HISO 10001:2017 Ethnicity Data Protocols” and thus self-identified ethnicity collected in this study is considered the “gold standard” in comparison to the NHI ethnicity recorded on the hospital NHI database.
The audit was powered to detect whether a mismatch is present between self-identified ethnicity data and the NHI hospital database ethnicity data. With 200 patients total and 90% accuracy in ethnicity data overall, the overall margin of error is roughly +/-5% (proportion/accuracy=90, lower/upper bound of 95% CI, 85.0/93.8).
All self-reported ethnicity data were collected by JR using the 2001 New Zealand Census ethnicity question via a patient-completed form after verbal consent was obtained. From the time of invitation to take part in the audit until the patient had completed their self-identified ethnicity, the data-collector was blind to the hospital database ethnicity data. Patients’ full names and dates of birth were also collected for the purposes of correctly identifying the patients in the hospital database. NHI-linked ethnicity data in the HVDHB Hospital database for each patient were then accessed and recorded by JR. All hospital database ethnicity data were collected on the same day as the patient presentation. Ethnicity data from both sources were coded to Level 2 of the 2004 Ministry of Health ethnicity data protocols for analysis.[[16]]
We used three measures to evaluate the accuracy of the hospital NHI database: ethnic proportions, individual mismatch and undercounts. Ethnic proportions were calculated separately for the self-identified collection and the hospital database (NHI) ethnicity, and were calculated as the number of individuals who reported being a member of the ethnic group divided by the total population size in the health dataset. For each individual, the level of match of ethnicity data between self-identified and NHI hospital database was categorised as match, partial match and total mismatch (Table 1).[[16]] The measurement of undercount assumes that self-identified ethnicity is the gold standard collection. It gives the proportion of individuals who self-identified an ethnic group that were not recorded with that ethnic group in the hospital database.
Statistical analysis
Data were analysed in Microsoft Excel version 2110 and R Studio (Version 1.4.1106) to generate descriptive statistics and confidence intervals.
Results
Of the 202 patients invited to take part in the study, 200 (99%) consented to participate. The median age of participants was 56.5 years (IQR 42–69.25) with an age range of participants from 16 to 93 years. Slightly more participants were female (n=107, 53.5%). Participants were a mix of those presenting for endoscopy (n=114, 57%) and gastroenterology clinics (n=86, 43%).
In the sample of 200 patients, 77.5% (n=155) self-identified as New Zealand European (either alone or in combination with other ethnicities) and 9% (n=18) identified themselves as Māori (Table 2). Half of self-identified Māori reported belonging to more than one ethnic group (9/18) compared to just 3% (5/182) of non-Māori (Table 3). Overall, those self-identifying with multiple ethnicities had a 50% (95% CI, 23.0–77.0) match (with all ethnicities also recorded on the hospital NHI database) compared to 91.9% (95% CI, 87.0–95.4) for patients who self-identified with a single ethnicity (Table 3). All mismatches in the multiple ethnicities group were partial mismatches, where some but not all of the self-identified ethnicities were recorded.
View Tables 1–3.
The overall level of match between the self-identified and the hospital NHI database ethnicity was 89% (95% CI, 83.8–93.0). Partial mismatches were found for 5% (10/200) of patients and total mismatch for 6% (12/200). Among non-Māori patients only 3.8% (95% CI, 1.6–7.8) of patients had partial mismatch of ethnicity data and 6.6% (95% CI, 3.5–11.2) had total mismatch of data. In comparison the match for Māori was 83.3% (95% CI, 58.6–96.4), 16.7% (95% CI, 3.6–41.4) of Māori patients had partial mismatch of ethnicity data and there were no total mismatches. Importantly, all the partial mismatches for Māori resulted in the patient being unable to be identified as Māori on the hospital NHI database. This means that the undercount for Māori on the hospital NHI database when compared to the self-identified fresh collection was 16.7%. The undercount in the NZ European group was 5.8%.
In both the self-reported ethnicity data and the hospital NHI database, the ethnic proportion of Māori was 9% (18/200). There were three patients who were identified as Māori in the hospital NHI database that did not self-report Māori ethnicity—two of whom identified themselves as NZ European only, and one who self-identified as NZ European and Niuean. Additionally, there were three patients who were identified as NZ European only in the hospital NHI database but self-identified as both Māori and NZ European.
There was some variation in the level of match for ethnicity when stratified by other demographics. Males had a lower match rate (81.7%) than females (95.3%), with a higher rate of total mismatch in males (11.8%) compared to females (0.9%). Most participants were aged between 40 and 79, and mismatch was relatively evenly distributed by age of around 10–15% mismatch, with the exception of being slightly higher in the 70- to 79-year-old category at 17% (Figure 1). The proportion of (mis)match did not differ between patients presenting to clinic or endoscopy.
Figure 1: Any mismatch by age for total sample population.
Discussion
This study audited the accuracy of NHI ethnicity data in a hospital NHI database against self-reported ethnicity, using the methods described in the Ministry of Health’s “Primary Care Ethnicity Data Toolkit”. We found an overall match rate of 89% in our sample and a match rate of 83.3% for Māori.
All the mismatches found for people self-reporting Māori ethnicity in our sample resulted in them being unidentifiable in the hospital NHI database, with an undercount of Māori on the hospital NHI database measuring 16.7%. In comparison, the undercount for the NZ European group was 5.8%. Undercounts of Māori have been found in several studies over the last two decades in a variety of areas of the health system. Swan et al. audited the accuracy of ethnicity data in a group of 3,500 diabetes patients in the Waikato in 2006 using a non-standard ethnicity question,[[5]] and in 2007 Bramley et al. looked at 57,612 vaccination records of school-aged children with data collected using the census ethnicity question.[[7]] These studies found Māori undercounts of 28.8% and 37.1% respectively. Undercounts found in more recent studies have been more variable. A 2017 study by Scott et al. found an undercount of Māori by 17% in the hospital NHI database and 21% in the Trauma Registry.[[6]] In contrast, another 2017 study of 227 patients with kidney disease found a Māori undercount of 7.3%, but this was still twice the undercount of 3.6% found in NZ Europeans.[[9]] The results of our audit would support evidence that there remains inequity between the size of the undercounts for Māori when compared to the NZ European population.
In our study, the ethnic proportions of Māori in the self-reported ethnicity data and the hospital NHI database were both 9%. This finding differs from other New Zealand-based hospital ethnicity audits that have found both an undercount of Māori participants in the hospital NHI database and a lower ethnic proportion of Māori.[[5,6]] This difference may reflect the smaller sample of Māori in our audit due to the lower ethnic proportion of Māori in the HVDHB catchment and potential selection bias in terms of who is able to navigate the healthcare pathway to access gastroenterology services (discussed further below).
At the individual level, we found differences in the individuals identified/recorded as Māori in the datasets, with three participants identified as Māori in the hospital NHI database who did not self-report Māori ethnicity, and three participants not identified as Māori in the hospital NHI database who did self-identify as Māori. This has important implications for our ability to identify the correct individuals to invite to existing and future health programmes that use NHI ethnicity data to target individuals. Both the cardiovascular disease risk assessment and[[12]] and diabetes screening[[13]] in primary care start for Māori, Pasifika and South Asian patients at an earlier age. Being of Māori or Pacific ethnicity is also used in guidance for sore throat swabbing for group A streptococcal pharyngitis to prevent rheumatic fever.[[14]] More recently, COVID-19 vaccination efforts for Māori relied on the accurate identification of individuals (as Māori) from existing data collections, as well as the use of aggregate-level denominators sourced from health service data, for reporting of vaccine coverage.[[11]]
Similar to other studies, we found that there was lower accuracy for those self-identifying with multiple ethnicities.[[6,7]] A complete match was found for only 50% of the 14 participants that had multiple self-reported ethnicities, compared to 91.9% in those who only self-reported one ethnicity. Importantly, half of the Māori participants (compared to just 3% of non-Māori) in our sample self-reported multiple ethnicities and so the impact of this inaccuracy may be greater for Māori. Consistent with the Primary Care Ethnicity Data Audit Toolkit our audit was powered to detect overall mismatch and was therefore underpowered to detect differences for multiple ethnicities. In addition, due to a smaller Māori sample than anticipated, some of our confidence intervals around our best estimates are overlapping. However, our findings are consistent with other research and indicate an ongoing issue with ethnicity data quality.
Māori were under-represented in our sample, making up just 9% of patients, compared to 17.9% in the HVDHB population (a difference that remained when we limited the sample to those aged 30–69 years at 10% versus 16.7% for HVDHB).[[15,17]] While there is the possibility of sampling error (in terms of Māori just not being booked on the days the audit was run), it is likely that the under-representation of Māori in our sample also reflects that Māori have inequitable access to this service. In general, Māori carry a higher rate of comorbidities and have higher rates of a large number of gastroenterological illnesses including stomach cancer,[[18]] hepatocellular carcinoma,[[19]] gastric ulcers [[20]] and upper gastrointestinal bleeding,[[20]] and similar rates of bowel cancer (but worse survival).[[21]] There are lower rates of diagnosed inflammatory bowel disease in Māori [[22]]. On balance, we expect Māori have greater need for gastroenterological services than non-Māori, and so we were surprised to find that in this study Māori were underrepresented in the gastroenterology service.
The inequitable rates of gastroenterology service use identified in this study may be an indication of barriers along the healthcare pathway that are experienced by Māori.[[23]] There are well-documented inequities in access to primary care,[[24]] and a higher level of unmet healthcare need for Māori,[[24]] which then impacts on access to secondary services. Those able to navigate the healthcare system into secondary care may have had more opportunities to update their ethnicity data. For example, all patients presenting to a gastroenterology clinic or endoscopy at Hutt Hospital are asked to review a paper copy of their hospital details at each presentation, including their ethnicity information. Our sample of patients, particularly those who saw gastroenterology services regularly, would have had multiple opportunities to review and change their ethnicity information, potentially (although not necessarily) improving accuracy. Therefore, our estimate of the level of undercount of Māori ethnicity may be smaller than the undercount for those less able to navigate the healthcare system, in settings where patients can present to services directly (e.g., emergency department), or in other district health boards with different processes around ethnicity data collection, and recording.
Ethnicity data collection needs to be an ongoing activity as ethnicity can change over time, and this has been shown to be a greater issue for Māori, Pasifika and Asian populations more than other ethnicities.[[25]] A 2009 New Zealand study found that 8% of individuals changed their self-reported ethnicity over a 3-year period, and the strongest predictors of this change were reporting Māori, Pasifika or Asian identity at the beginning of the study, as well as reporting multiple ethnic groups.[[25]] This also means that the ability to regularly update self-reported ethnicity will have an effect on accuracy, and that this will impact Māori and other minoritised groups more than NZ Europeans.
All areas of health in New Zealand need to collect, and maintain access to, high-quality ethnicity data. The New Zealand Ministry of Health ethnicity data protocols published in 2017 already provide guidance on ways to improve ethnicity data quality.[[3]] This guidance includes ensuring ethnicity is self-identified, using only the census ethnicity question for collection and a maximum of 3 years between each collection of ethnicity data. However, this guidance requires supported implementation to be followed appropriately with ongoing quality assessment. There are also other opportunities to collect updated ethnicity data that could improve quality on a national level that could form part of a wider strategy, such as collection of updated data as part of vaccination programmes. A key finding in this study is the importance of measuring the accuracy of ethnicity data in multiple ways to reflect their multiple uses.
Conclusion
Ethnicity data are fundamental to the monitoring and provision of equitable health and healthcare, with a range of applications in the health sector. Data can accurately reflect ethnic proportions at a group level but be inaccurate at the individual level, which can be falsely reassuring about overall data quality. Our findings of poor-quality ethnicity data in the hospital NHI record are consistent with previous studies. Māori have been shown to be particularly affected by inaccurate ethnicity data in a range of settings that are important for measuring and acting to correct health inequities.
Summary
Abstract
Aim
To audit the quality of ethnicity data stored under National Health Index (NHI) in the Hutt Hospital database against a fresh collection of self-identified ethnicity to identify the level of (mis)match present between the datasets.
Method
Self-identified ethnicity data was collected from 200 consecutive patients presenting to outpatient gastroenterology services and compared to National Health Index (NHI) in the Hutt Hospital database, using the process outlined in the Primary Care Ethnicity Data Audit Toolkit.
Results
The overall level of match between the individual’s self-identified ethnicity and that recorded in the hospital database was 89% (95% CI [83.8–93.0]). Eighteen patients (9%) self-identified as Māori, 16.7% (95% CI [3.6–41.4]) of whom were not recorded as Māori in the hospital database. Three patients were recorded as Māori in the hospital database but did not self-identify as Māori.
Conclusion
Ethnicity data are fundamental to the monitoring and provision of equitable health and healthcare, with a range of applications in the health sector. Our findings of poor-quality ethnicity data for Māori in a hospital NHI database are consistent with previous studies. The assessment of ethnicity data quality must be done in multiple ways to reflect its multiple uses.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) Te Rōpū Rangahau Hauora a Eru Pōmare. Counting for nothing: understanding the issues in monitoring disparities in health. Soc Policy J N Z. 2000;(14):1-6.
2) McLeod M, Harris R, Paine S-J, Crengle S et al. Bowel cancer screening age range for Māori: what is all the fuss about? N Z Med J. 2021;134(1535):71-7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/34012141.
3) NZ Ministry of Health. 2017 Ethnicity Data Protocols [Internet]. Wellington; 2017 [cited 2022 Feb 4]. Available from: https://www.health.govt.nz/publication/hiso-100012017-ethnicity-data-protocols.
4) Cormack D, McLeod M. Improving and maintaining quality in ethnicity data collections in the health and disability sector. Wellington: Te Rōpū Rangahau Hauora a Eru Pōmare; 2010.
5) Swan J, Lillis S, Simmons D. Investigating the accuracy of ethnicity data in New Zealand hospital records: still room for improvement. N Z Med J. 2006 Aug 4;119(1239):U2103. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16912721.
6) Scott N, Clark H, Kool B, Ameratunga S et al. Audit of ethnicity data in the Waikato hospital patient management system and trauma registry: Pilot of the hospital ethnicity data audit toolkit. N Z Med J. 2018;131(1483):21-9.
7) Bramley D, Latimer S. The accuracy of ethnicity data in primary care. N Z Med J. 2007 Oct 26;120(1264):U2779. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17972986.
8) Riddell T, Lindsay G, Kenealy T, Jackson R et al. The accuracy of ethnicity data in primary care and its impact on cardiovascular risk assessment and management - PREDICT CVD-8. N Z Med J. 2008;121(1281):40-8.
9) Page MJ, Wyeth EH, Samaranayaka A, McNoe B et al. Accuracy of ethnicity data recorded in hospital-based patient clinical records and the Australia and New Zealand dialysis and transplant registry. N Z Med J. 2017;130(1454):65-71.
10) Rumball-Smith J, Sarfati D. Improvement in the accuracy of hospital ethnicity data. N Z Med J. 2011 Aug 1;124:96-7.
11) NZ Ministry of Health. COVID-19 Māori Vaccine and Immunisation Plan [Internet]. Wellington; 2021 Mar [cited 2022 Jan 24]. Available from: https://www.health.govt.nz/publication/covid-19-maori-vaccine-and-immunisation-plan-supplementary-updated-covid-19-maori-health-response.
12) NZ Ministry of Health. Cardiovascular Disease Risk Assessment and Management for Primary Care 2018. Available at: https://www.health.govt.nz/system/files/documents/publications/cardiovascular-disease-risk-assessment-management-primary-care-feb18-v4_0.pdf.
13) NZ Ministry of Health, NZ Society for the Study of Diabetes. Screening for diabetes in asymptomatic adults - Type 2 Diabetes Management Guidelines [Internet]. Type 2 Diabetes Management Guidance Website. 2022 [cited 2022 Jan 24]. Available from: https://t2dm.nzssd.org.nz/Section-112-Screening-for-diabetes-in-asymptomatic-adults.
14) National Heart Foundation of New Zealand. Evidence-based, best practice New Zealand Guidelines for Rheumatic Fever. Auckland; 2019.
15) NZ Ministry of Health. My DHB: Hutt Valley DHB [Internet]. Ministry of Health Website. 2021 [cited 2021 Dec 29]. Available from: https://www.health.govt.nz/new-zealand-health-system/my-dhb/hutt-valley-dhb.
16) Ministry of Health. Primary Care Ethnicity Data Audit Toolkit [Internet]. 2013. Available from: http://www.health.govt.nz/publication/primary-care-ethnicity-data-audit-toolkit.
17) Stats NZ. 2018 Census. 2020.
18) Ellison-Loschmann L, Sporle A, Corbin M, Cheng S et al. Risk of stomach cancer in Aotearoa/New Zealand: A Māori population based case-control study. PLoS One. 2017 Jul 1 [cited 2022 Feb 15];12(7). Available from: /pmc/articles/PMC5521812/.
19) Chamberlain J, Sarfati D, Cunningham R, Koea J et al. Incidence and management of hepatocellular carcinoma among Māori and non-Māori New Zealanders. Aust N Z J Public Health. 2013 Dec;37(6):520-6. Available from: https://onlinelibrary.wiley.com/doi/10.1111/1753-6405.12108.
20) Irwin J, Ferguson R, Weilert F, Smith A. Incidence of upper gastrointestinal haemorrhage in Maori and New Zealand European ethnic groups, 2001-2010. Intern Med J [Internet]. 2014 [cited 2022 Feb 15];44(8):735-41. Available from: https://pubmed.ncbi.nlm.nih.gov/24893826/.
21) Blackmore T, Lao C, Chepulis L, Page B et al. The characteristics and outcomes of patients with colorectal cancer in New Zealand, analysed by Cancer Network. N Z Med J. 2020 [cited 2022 Feb 15];133:1513. Available from: www.nzma.org.nz/journal.
22) Coppell KJ, Galts CP-C, Huizing FY, Norton JK et al. Annual Incidence and Phenotypic Presentation of IBD in Southern New Zealand: An 18-Year Epidemiological Analysis. Inflamm Intest Dis [Internet]. 2018 [cited 2022 Feb 15];3(1):329. Available from: https://www.karger.com/Article/FullText/492615.
23) Espiner E, Paine SJ, Weston M, Curtis E. Barriers and facilitators for Māori in accessing hospital services in Aotearoa New Zealand. N Z Med J. 2021;134(1546):47-58.
24) NZ Ministry of Health. Annual Data Explorer 2020/2021: New Zealand Health survey [Data File] [Internet]. 2021 [cited 2022 Feb 28]. Available from: https://minhealthnz.shinyapps.io/nz-health-survey-2020-21-annual-data-explorer/.
25) Carter K, Hayward M, Blakely T, Shaw C. How much and for whom does self-identified ethnicity change over time in New Zealand? Results from a longitudinal study. Soc Policy J N Z. 2009 Aug 1;36.
Contact diana@nzma.org.nz
for the PDF of this article
Ethnicity data audit in a secondary care gastroenterology service
Accurate ethnicity data are important because they are used to quantify health inequities, including monitoring trends in disease rates and healthcare delivery. In relation to Māori health equity, high-quality ethnicity data in health are also an obligation under Te Tiriti.
Subscriber Content
The full contents of this pages only available to subscribers.
Login, subscribe or email nzmj@nzma.org.nz to purchase this article.
