The New Zealand Ministry of Health (MoH) maintains a number of centralised databases, known as National Collections, which are used for the purposes of policy formation, performance monitoring, research, and review.1In theory, these collections offer an opportunity for health service providers and health researchers alike to monitor and investigate disease burden, determinants and other such factors at a national level. However, the MoH estimates that these data collections probably underestimate receipt of cancer treatment, particularly with respect to outpatient care.2There are two likely reasons for this. First, collections such as the National Non-Admitted Patient Collection (NNPAC) serve as administrative databases, with the primary purpose of assisting the reimbursement of health care providers such as District Health Boards (DHBs) for provision of services.Therefore, the completeness of treatment data will depend on the completeness of claims made by a given service provider. Second, private healthcare facilities are currently not mandated to report data on the provision of privately-funded treatment to the MoH.3Most private facilities do report, but there are some notable exceptions. Since a substantial minority of cancer patients will privately fund their treatment particularly those with highly-prevalent breast and colon cancers the absence of data for these patients could have a measurable effect on population estimates of treatment provision. In the case of colon cancer, at least 18% of non-Mori and 5% of Mori patients will receive their definitive treatment in private facilities.4These factors combine to potentially undermine the validity of the MoH collections in achieving their stated purpose. This is important, because data from the National Collections are used to inform policy and assess system performance. For example, the Price of Cancer report published in 2011 2 attempted to estimate the cost of cancer to the New Zealand government, and the likely costs into the future, using data from the National Collections.However, as acknowledged in the report itself, underreporting of cancer treatment to the National Collections is likely to have resulted in an underestimate of treatment provision, with flow-on effects to estimates of treatment cost. Also, since incompleteness of treatment data would result in erroneous estimates of receipt of cancer treatment, any measure of the effectiveness of Government policy around this issue would also be prone to error.In this short report, we demonstrate the extent to which the centralised National Collections underestimate cancer treatment provision, using breast, colon and renal cancer patients from the Auckland region as exemplars. We hypothesised that the greatest undercount would be observed for breast and colon cancers due to a privately-funded treatment effect, while any undercount for renal cancer would be more modest since care is less likely to be privately-funded for this disease.As part of a larger study investigating cancer care and outcomes (the Cancer, Comorbidity and Care [C3] study; ethics reference MEC/10/042/EXP), we collected treatment data from the MoH collections and augmented this with data from six Cancer Centres, breast cancer registers, private hospitals and personal clinician databases.All patients diagnosed with breast, colon or renal cancers between July 2006 and June 2008 were identified from the New Zealand Cancer Registry (NZCR), with staging and demographic information also taken from this collection. The National Minimum Dataset (NMDS) and NNPAC were used to determine baseline treatment data.For the purposes of this report, we focus on the Auckland region as an example because (1) we were able to secure the greatest depth of data augmentation for this region (for example, Auckland Breast Cancer Register data were complete and available for our study period); and (2) because we hypothesised that Auckland is likely to have the highest proportion of privately-funded healthcare nationally, and therefore the effect of missing private hospital data could potentially have the greatest effect on data completeness for this region.The NMDS and NNPAC collections were augmented with data from the: Auckland Cancer Centre (chemo- and radiotherapy data); Auckland Breast Cancer Register (surgery, chemo- and radiotherapy data); and A large non-reporting private facility in Auckland (surgery data). Figure 1. Crude proportion (%) of a) breast, colon and renal cancer patients with local or regional extent of disease who received surgery, by dataset and b) breast cancer patients with regional extent of disease who received chemo- and/or radiotherapy, by dataset; c) difference between MoH collections and augmented dataset in crude proportion (%) treated, by ethnicityWe used clinical codes (e.g. ICD-10-AM) and/or procedure descriptions to determine relevant treatments from the MoH Collections and for the non-reporting private facility, with other data sources providing dates of definitive treatment only.We determined the proportion of breast (n=1,328; 119 Mori; 1209 non-Mori), colon (n=724; 14 Mori; 710 non-Mori) and renal (n=143; 11 Mori; 132 non-Mori) cancer patients with local or regional extent of disease (NZCR/SEER extent B, C or D)5 who received definitive surgical treatment within 90 days of diagnosis. We also determined the proportion of breast cancer patients (n=517; 47 Mori; 470 non-Mori) with regional extent of disease (NZCR/SEER extent C or D) who received chemo- and/or radiotherapy within 1 year of diagnosis, since these adjuvant therapies may (or may not) be offered at this stage of disease.We observed that the MoH National Collections substantially undercounted the proportion of breast (crude proportion treated: National Collections [NC] 76%; Augmented Dataset [AD] 95%), colon (NC 81%; AD 96%] and renal (NC 83%; AD 95%) cancer patients who received definitive surgical treatment. The greatest undercounts were observed for breast and colon cancer (Figure 1a).Similarly, the MoH data collections substantially undercounted the proportion of breast cancer patients with regional stage of disease who received chemo- (NC 40%; AD 59%) and/or radiotherapy (NC 52%; AD 67%; Figure 1b).We also observed that receipt of treatment was undercounted to a greater degree by the National Collections for non-Mori patients than for Mori patients (Figure 1c). This undercount was pronounced for surgery (absolute difference between proportion treated on National Collection vs augmented dataset: breast non-Mori 20%, Mori 7%; colon non-Mori 14%, Mori 7%; renal non-Mori 13%, Mori 9%), and similarly large for breast cancer chemo- (non-Mori 19%, Mori 15%) and radiotherapy (non-Mori 16%, Mori 11%).Two key findings from our analysis point toward a potential non-reporting private hospital effect on completeness of surgical treatment data on the MoH collections: (1) we observed that breast and colon cancers showed the greatest change following data augmentation, and anecdotally we know that these two cancers have a relatively high proportion of patients who receive treatment in the private sector compared to other cancers; (2) we observed a greater increase in data completeness for non-Mori following data augmentation (Figure 1c), and we know that non-Mori are more likely to access privately-funded cancer treatment than Mori.4In terms of adjuvant therapy, the substantial difference observed in chemo- and radiotherapy receipt between the National Collections and our augmented dataset may be a reflection of underreporting of adjuvant therapy provision by DHBs in the first instance,2 and (to a lesser degree) an increasing tendency for these adjuvant services to be privately funded.Despite reasonably comprehensive augmentation, we still suspect that our augmented dataset undercounts receipt of treatment (particularly for adjuvant chemotherapy). Assuming this is the case, the true difference between actual treatment receipt and that which was recorded in the National Collections will be larger than that which we could ascertain with our dataset. It should be noted that post-2008 (i.e. after our study period) the reporting of pharmaceutical cancer treatment data by hospitals to the National Collections became mandatory, and it is hoped that this will result in an improvement in the completeness of available chemotherapy data. Further investigation is required to determine whether such improvements occur.As previously mentioned, we hypothesised that proportionally more patients were treated privately in Auckland than in any other region. For this reason, it should be noted that any private hospital effect on data completeness particularly for receipt of surgery may not be as pronounced for the rest of the country. For example, we observed that the crude proportion observed on the National Collections to have received surgery was higher in regions outside Auckland compared to within Auckland (breast cancer: 88% outside Auckland vs. 76% within Auckland; colon cancer: 92% outside Auckland vs. 81% within Auckland).There were similarly higher proportions of patients with regional breast cancer observed to have received chemotherapy outside Auckland compared to within Auckland (chemotherapy: 52% outside Auckland vs. 40% within Auckland; radiotherapy: 62% outside Auckland vs. 52% within Auckland).In summary, our findings show that the MoH National Collections substantially undercount receipt of treatment for patients with breast and colon cancers, and to a lesser extent for renal cancer, in the Auckland region. Since the magnitude of this undercount may be the result of missing data from privately-funded events, this discrepancy is unlikely to improve until reporting of all cancer treatment irrespective of funding source is mandated and facilitated, as is the case in other contexts such as Australia6 and Denmark.7

The New Zealand Ministry of Health (MoH) maintains a number of centralised databases, known as National Collections, which are used for the purposes of policy formation, performance monitoring, research, and review.1In theory, these collections offer an opportunity for health service providers and health researchers alike to monitor and investigate disease burden, determinants and other such factors at a national level. However, the MoH estimates that these data collections probably underestimate receipt of cancer treatment, particularly with respect to outpatient care.2There are two likely reasons for this. First, collections such as the National Non-Admitted Patient Collection (NNPAC) serve as administrative databases, with the primary purpose of assisting the reimbursement of health care providers such as District Health Boards (DHBs) for provision of services.Therefore, the completeness of treatment data will depend on the completeness of claims made by a given service provider. Second, private healthcare facilities are currently not mandated to report data on the provision of privately-funded treatment to the MoH.3Most private facilities do report, but there are some notable exceptions. Since a substantial minority of cancer patients will privately fund their treatment particularly those with highly-prevalent breast and colon cancers the absence of data for these patients could have a measurable effect on population estimates of treatment provision. In the case of colon cancer, at least 18% of non-Mori and 5% of Mori patients will receive their definitive treatment in private facilities.4These factors combine to potentially undermine the validity of the MoH collections in achieving their stated purpose. This is important, because data from the National Collections are used to inform policy and assess system performance. For example, the Price of Cancer report published in 2011 2 attempted to estimate the cost of cancer to the New Zealand government, and the likely costs into the future, using data from the National Collections.However, as acknowledged in the report itself, underreporting of cancer treatment to the National Collections is likely to have resulted in an underestimate of treatment provision, with flow-on effects to estimates of treatment cost. Also, since incompleteness of treatment data would result in erroneous estimates of receipt of cancer treatment, any measure of the effectiveness of Government policy around this issue would also be prone to error.In this short report, we demonstrate the extent to which the centralised National Collections underestimate cancer treatment provision, using breast, colon and renal cancer patients from the Auckland region as exemplars. We hypothesised that the greatest undercount would be observed for breast and colon cancers due to a privately-funded treatment effect, while any undercount for renal cancer would be more modest since care is less likely to be privately-funded for this disease.As part of a larger study investigating cancer care and outcomes (the Cancer, Comorbidity and Care [C3] study; ethics reference MEC/10/042/EXP), we collected treatment data from the MoH collections and augmented this with data from six Cancer Centres, breast cancer registers, private hospitals and personal clinician databases.All patients diagnosed with breast, colon or renal cancers between July 2006 and June 2008 were identified from the New Zealand Cancer Registry (NZCR), with staging and demographic information also taken from this collection. The National Minimum Dataset (NMDS) and NNPAC were used to determine baseline treatment data.For the purposes of this report, we focus on the Auckland region as an example because (1) we were able to secure the greatest depth of data augmentation for this region (for example, Auckland Breast Cancer Register data were complete and available for our study period); and (2) because we hypothesised that Auckland is likely to have the highest proportion of privately-funded healthcare nationally, and therefore the effect of missing private hospital data could potentially have the greatest effect on data completeness for this region.The NMDS and NNPAC collections were augmented with data from the: Auckland Cancer Centre (chemo- and radiotherapy data); Auckland Breast Cancer Register (surgery, chemo- and radiotherapy data); and A large non-reporting private facility in Auckland (surgery data). Figure 1. Crude proportion (%) of a) breast, colon and renal cancer patients with local or regional extent of disease who received surgery, by dataset and b) breast cancer patients with regional extent of disease who received chemo- and/or radiotherapy, by dataset; c) difference between MoH collections and augmented dataset in crude proportion (%) treated, by ethnicityWe used clinical codes (e.g. ICD-10-AM) and/or procedure descriptions to determine relevant treatments from the MoH Collections and for the non-reporting private facility, with other data sources providing dates of definitive treatment only.We determined the proportion of breast (n=1,328; 119 Mori; 1209 non-Mori), colon (n=724; 14 Mori; 710 non-Mori) and renal (n=143; 11 Mori; 132 non-Mori) cancer patients with local or regional extent of disease (NZCR/SEER extent B, C or D)5 who received definitive surgical treatment within 90 days of diagnosis. We also determined the proportion of breast cancer patients (n=517; 47 Mori; 470 non-Mori) with regional extent of disease (NZCR/SEER extent C or D) who received chemo- and/or radiotherapy within 1 year of diagnosis, since these adjuvant therapies may (or may not) be offered at this stage of disease.We observed that the MoH National Collections substantially undercounted the proportion of breast (crude proportion treated: National Collections [NC] 76%; Augmented Dataset [AD] 95%), colon (NC 81%; AD 96%] and renal (NC 83%; AD 95%) cancer patients who received definitive surgical treatment. The greatest undercounts were observed for breast and colon cancer (Figure 1a).Similarly, the MoH data collections substantially undercounted the proportion of breast cancer patients with regional stage of disease who received chemo- (NC 40%; AD 59%) and/or radiotherapy (NC 52%; AD 67%; Figure 1b).We also observed that receipt of treatment was undercounted to a greater degree by the National Collections for non-Mori patients than for Mori patients (Figure 1c). This undercount was pronounced for surgery (absolute difference between proportion treated on National Collection vs augmented dataset: breast non-Mori 20%, Mori 7%; colon non-Mori 14%, Mori 7%; renal non-Mori 13%, Mori 9%), and similarly large for breast cancer chemo- (non-Mori 19%, Mori 15%) and radiotherapy (non-Mori 16%, Mori 11%).Two key findings from our analysis point toward a potential non-reporting private hospital effect on completeness of surgical treatment data on the MoH collections: (1) we observed that breast and colon cancers showed the greatest change following data augmentation, and anecdotally we know that these two cancers have a relatively high proportion of patients who receive treatment in the private sector compared to other cancers; (2) we observed a greater increase in data completeness for non-Mori following data augmentation (Figure 1c), and we know that non-Mori are more likely to access privately-funded cancer treatment than Mori.4In terms of adjuvant therapy, the substantial difference observed in chemo- and radiotherapy receipt between the National Collections and our augmented dataset may be a reflection of underreporting of adjuvant therapy provision by DHBs in the first instance,2 and (to a lesser degree) an increasing tendency for these adjuvant services to be privately funded.Despite reasonably comprehensive augmentation, we still suspect that our augmented dataset undercounts receipt of treatment (particularly for adjuvant chemotherapy). Assuming this is the case, the true difference between actual treatment receipt and that which was recorded in the National Collections will be larger than that which we could ascertain with our dataset. It should be noted that post-2008 (i.e. after our study period) the reporting of pharmaceutical cancer treatment data by hospitals to the National Collections became mandatory, and it is hoped that this will result in an improvement in the completeness of available chemotherapy data. Further investigation is required to determine whether such improvements occur.As previously mentioned, we hypothesised that proportionally more patients were treated privately in Auckland than in any other region. For this reason, it should be noted that any private hospital effect on data completeness particularly for receipt of surgery may not be as pronounced for the rest of the country. For example, we observed that the crude proportion observed on the National Collections to have received surgery was higher in regions outside Auckland compared to within Auckland (breast cancer: 88% outside Auckland vs. 76% within Auckland; colon cancer: 92% outside Auckland vs. 81% within Auckland).There were similarly higher proportions of patients with regional breast cancer observed to have received chemotherapy outside Auckland compared to within Auckland (chemotherapy: 52% outside Auckland vs. 40% within Auckland; radiotherapy: 62% outside Auckland vs. 52% within Auckland).In summary, our findings show that the MoH National Collections substantially undercount receipt of treatment for patients with breast and colon cancers, and to a lesser extent for renal cancer, in the Auckland region. Since the magnitude of this undercount may be the result of missing data from privately-funded events, this discrepancy is unlikely to improve until reporting of all cancer treatment irrespective of funding source is mandated and facilitated, as is the case in other contexts such as Australia6 and Denmark.7