View Article PDF

The problemIn 1938 the Social Security Act for health care was passed in New Zealand with the aim of creating universal access to a comprehensive national health service in which barriers to accessing required healthcare, including financial, would be removed. Ten years later the National Health Service (NHS) with similar aims was enabled in the United Kingdom.1Major restructuring of the health sector has occurred, sometimes at frequent intervals, in these and many other developed countries, yet documentation of the overall success (or otherwise) of these changes has often been ignored even though they can produce unintended consequences. For example, 4 years after initiation of radical healthcare restructuring in New Zealand in 1993 there had been no formal, comprehensive review of achievements and outcomes.2 However, predicted outcomes of the market oriented alternative to state control in New Zealand, such as hospital profits and provider competition, failed to appear. Meanwhile, positives, such as better organisation of general practice, improvements in Māori health organisation, and creation of the national pharmaceutical purchasing agency (PHARMAC), which did eventuate were unforeseen.3The 1990s "big bang" approach to structural change, utilised under New Zealand's unicameral political system, seems more likely to produce results that are unpredictable than an incremental approach to change which is more typically undertaken in countries with a bicameral political system.3 In England, reorganisation (or breakup) of the NHS is underway with fears that, despite its bicameral political system, privatisation will play a prominent role.4In the absence of broad-based objective data collected routinely on an ongoing basis, ratings of the outcome of such policy changes or a health system per se can be readily reported as anywhere from superlative (usually by political leaders and public officials) to a dismal failure—often via anecdotal patient or health professional experience.In this brief article, we contend that in predominantly tax-funded systems with goals such as those of New Zealand and England, unmet healthcare need should be defined and included in any performance assessment.This should apply across restructuring of the health sector, following the implementation of new models of care whatever their objective, and even in the absence of modifications in the health system when changes in population demographics can alter health requirements substantially. We note the potential for unbalanced reporting in its absence, and describe some ways in which unmet need could be measured.A recent example of the problemInadequacies in assessing the performance of a healthcare system were brought home to us recently. On the one hand, a King's Fund investigation reported in 2013 that over some 5 years the District Health Board in Canterbury, New Zealand "..has moved towards (a more integrated system of healthcare) that manages demand more effectively in primary care and allows the hospitals to run more efficiently, thus concentrating more of their care on those who actually need to be in hospital".5 This report formed the basis for an editorial article in the British Medical Journal (BMJ) which suggested that the Canterbury experience offers several insights for the National Health Service (NHS) in England.6On the other hand a perceived serious and possibly increasing level of unmet healthcare need in the same Canterbury region has seen the establishment of the Canterbury Charity Hospital. A recent three year (2010–2012) review of this places on record the services, mostly surgical, provided to patients unable to access treatment in the public system and who could not afford private care.7 This review noted that, beyond the hundreds of patients treated, many appointment requests to the Charity Hospital were rejected because volunteer services in some specialties were not available. The experience of the Charity Hospital led to the conclusion that there are "… substantial, undocumented unmet healthcare needs in the region."7The former report by the King's Fund5 and the BMJ editorial6 suggest that substantial improvements in healthcare in Canterbury have occurred whilst the latter article on the Charity Hospital suggests that, across the same time period, unmet healthcare needs in the region are "alarmingly high".7 Neither report is particularly helpful in assessing the overall effectiveness or otherwise of changes to the organisation of healthcare services in Canterbury.We agree with the key comment in the King's Fund report that "Canterbury is far from alone in facing the challenge of measuring the impact of more integrated care".5 In our view the moral (and fiscal) imperative dictates that no major reform of a healthcare system should be considered, let alone implemented, in the absence of measuring its impact on a broad canvas of healthcare needs, including unmet need, in the target population.We also assert that unmet healthcare needs should be a core component of any health system assessment, even in the absence of major restructuring or the implementation of new models. This is for the simple reason that failure to meet such needs, particularly for medical and surgical services, is in breach of a basic human right to good health and healthcare.The way forwardWe concur with the growing body of research indicating that the performance of a healthcare system should be measured regularly, objectively and comprehensively by a body at arm's length from, and preferably independent of, the healthcare system.8,9 This, however, is easily stated but less easily implemented with a wide range of possibilities and different combinations when it comes to approach and indicators that could be used.10,11Generally speaking, there are three different sources of information: routine data relating to service coverage, scope and costs, and the incidence of disease; reports from healthcare professionals; and reports from patients. Often these are considered together, as in measurement of hospital performance, but none is ideal and frequently there is an absence of one of the three aforementioned perspectives on performance.For example, some indices built from routine data (longevity, perinatal mortality rate, the number of emergency medical admissions, readmission rates and childhood inoculation rates, for example) are important and readily quantified—but provide a narrow perspective of overall health delivery in a community.When it comes to health professional reported data, estimating patient access to services is problematic,12 can be readily manipulated, and cannot alone be relied upon to provide robust data.13 It may be possible for medical professionals to provide accurate information regarding the status of their patients across periods of reform or collectively raise concerns when standards of care are unacceptable. However, the inadequate performance of health professionals in the Mid Staffordshire events highlights the problems associated with such reporting.14,15As is now well known, conditions of appalling healthcare flourished in the main hospital of the Mid Staffordshire NHS Foundation Trust between 2005 and 2008, amid cost cutting and a drive to meet government targets, yet the response of health professionals was inadequate. It would be naïve to consider that the situation in Mid Staffordshire was and is unique. As working relationships (including employment links) between medical staff in primary and secondary care in particular change over time, so perceptions by medical staff of patient needs will inevitably alter with time.Accordingly, data, provided by medical and other healthcare staff, can be insightful but, if not standardised and collected in accordance with carefully defined criteria, need to be viewed with a degree of caution.From a patient's perspective there are possibly three core elements to a health care system which should be measured; accessibility, timeliness and quality of care. The first and second incorporate unmet need whilst the third, quality of care, is readily quantified by, for example, readmission rates and other quantitative indicators, and is also amenable to process and experience measurement.In our view, the centre point to assessing the performance of a healthcare system should utilise documentation of unmet healthcare needs as perceived by representative segments of the population at formal interview. At present, various agencies and research groups have attempted this, some in a robust and objective manner16,17 and some with sometimes questionable methods, for example, a tele-marketing approach of phoning individuals until a quota is reached.18 While providing a reasonable snap-shot of unmet need in different countries, such studies can easily be disregarded by policy-makers.A representative study of unmet need should involve selection of a random sample identified using robust statistical sampling procedures rather than accessible and potentially biased samples of convenience which characterise many of the ‘patient exit' and other surveys conducted by healthcare providers.Representative interviews, we suggest, should be repeated regularly and cover all aspects of unmet healthcare needs including dental, psychiatric, birth control and disability, as well as unmet general medical and surgical needs. The interview protocol should, ideally, be identical across many countries to enable time-matched comparisons between different healthcare systems and longitudinal assessments of the effects of organisational changes within any one country, as advocated by agencies such as the OECD but not yet achieved.19With this approach healthcare sector performance with regard to unmet need could be assessed, first, by documenting changes within the country over time and across any restructuring period and, second, by comparing changes in unmet need between countries some of which did not undergo restructuring.As a starting point we suggest that population based sampling methods such as those used by The New Zealand Health Survey and others could be the basis for such surveys.16 17 20 Currently the New Zealand example of a national Health Survey captures data on, amongst other things, health utilisation and management of diagnosed medical conditions, but does not attempt to measure the core elements of unmet need: undiagnosed conditions or conditions for which health care has been sought but has not been provided.Questions about avoiding health care due to financial barriers were asked in a related New Zealand government-sponsored Survey of Family, Income and Employment but did not assess the specific conditions for which healthcare needs were not met and was, in any case, discontinued in 2010.21 Surveys of this kind, coupled with information from healthcare providers, especially general practitioners, could provide robust estimates of unmet need and potentially identify specific problematic conditions, as well as identify demographic features of those most likely to be affected. In this way, there would be capacity to bring together a range of measures from the three different information vantage-points cited above in developing new methods for assessing unmet need, which also meet with suggested benchmarks for this.22It is important that the range of measures used be standardised to avoid the possibility of a positive, but limited, finding as indicating general good health of a healthcare system. Of course, it is possible that variables from Statistics New Zealand's Integrated Data Infrastructure could also be drawn upon.While for a different population and component of its healthcare system, the United States Centers for Medicare and Medicaid Services (CMS) has demonstrated that it is possible to implement a nation-wide standard survey of a random sample of public and private hospital patients (the Hospital Consumer Assessment of Healthcare Providers and Systems—or HCAHPS).23 The government of Thailand has similarly commissioned a random household survey focused specifically on unmet need for healthcare services.17It is time now for policymakers in New Zealand, England and elsewhere to follow suit in further developing such techniques and extending them beyond only patients who have been able to access hospital care.22 In this way, they may be able to respond to the need to better understand and comprehensively measure the important and unexplored issue of unmet need for healthcare in our communities. Without this, goals of delivering on fundamental human rights to good health, as well as robust health system performance measurement, cannot be delivered.The result will be the continued delivery of reports such as the King's Fund's5,6 that provide what is perhaps an important story, but present only one facet of the reality.\r\n

Summary

Abstract

Major restructuring of the health sector has been undertaken in many countries, including New Zealand and England, yet objective assessment of the outcomes has rarely been recorded. In the absence of comprehensive objective data, the success or otherwise of health reforms has been inferred from narrowly-focussed data or anecdotal accounts. A recent example relates to a buoyant Kings Fund report on the quest for integrated health and social care in Canterbury, New Zealand which prompted an equally supportive editorial article in the British Medical Journal (BMJ) suggesting it may contain lessons for Englands National Health Service. At the same time, a report published in the New Zealand Medical Journal expressed concerns at the level of unmet healthcare needs in Canterbury. Neither report provided objective information about changes over time in the level of unmet healthcare needs in Canterbury. -We propose that the performance of healthcare systems should be measured regularly, objectively and comprehensively through documentation of unmet healthcare needs as perceived by representative segments of the population at formal interview. Thereby the success or otherwise of organisational changes to a health system and its adequacy as demographics of the population evolve, even in the absence of major restructuring of the health sector, can be better documented.

Aim

Method

Results

Conclusion

Author Information

Robin Gauld, Professor of Health Policy, Department of Preventive and Social Medicine, University of Otago - Dunedin, Dunedin; Antony Raymont, Senior Research Officer, Waitemata District Health Board, Auckland; Philip F Bagshaw, Chair, Canterbury Charity Hospital Trust and Clinical Associate Professor, University of Otago - Christchurch, Christchurch; M Gary Nicholls, Emeritus Professor, Department of Medicine, University of Otago - Christchurch, Christchurch Hospital, Christchurch; Christopher M Frampton, Associate Professor, University of Otago - Christchurch, Christchurch

Acknowledgements

Correspondence

Robin Gauld, Department of Preventive and Social Medicine, University of Otago, PO Box 913, Dunedin 9054, New Zealand.

Correspondence Email

robin.gauld@otago.ac.nz

Competing Interests

Nil.

1. Gauld R. Questions about New Zealand's health system in 2013, its 75th anniversary year. New Zealand Medical Journal 2013;126(1380):1-7. 2. Hornblow A. New Zealand's health reforms: a clash of cultures. British Medical Journal 1997;314(1892-1894). 3. Gauld R. Big Bang and the Policy Prescription: Health Care Meets the Market in New Zealand. Journal of Health Politics, Policy and Law 2000;25(5):815-44. 4. Godlee F. Why markets don't work in healthcare. British Medical Journal 2012;344:33300. 5. Timmins N, Ham C. The Quest for Integrated Health and Social Care: A Case Study in Canterbury, New Zealand. London: The King's Fund, 2013. 6. Mays N, Smith J. What can England's NHS learn from Canterbury New Zealand? British Medical Journal 2013;347:f6513. 7. Bagshaw P, Maimbo-M'siska M, Nicholls M, Shaw C, Allardyce R, Bagshaw S, et al. The Canterbury Charity Hospital: an update (2010-2012) and effects of the earthquakes. New Zealand Medical Journal 2013;126(1386):31-42. 8. World Health Organization. The World Health Report 2000: Health Systems: Improving Performance. Geneva: World Health Organization, 2000. 9. van den Berg M, Kringos D, Marks L, Klazinga N. The Dutch health care performance report: seven years of health care performance assessment in the Netherlands. Health Research Policy and Systems 2014;12(1):1. 10. Gauld R, Al-Wahaibi S, Chisholm J, Crabbe R, Kwon B, Oh T, et al. Scorecards for health system performance assessment: the New Zealand example. Health Policy 2011;103:200-08. 11. Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Cambridge: Cambridge University Press, 2009. 12. Derrett S, Bevin T, Herbison P, Paul C. Access to elective surgery in New Zealand: considering equity and the public and private mix. International Journal of Health Planning and Management 2009;24:147-60. 13. Derrett S, Cousins K, Gauld R. A messy reality: an analysis of New Zealand's elective surgery scoring system via media sources, 2000-2006. International Journal of Health Planning and Management 2013;28:48-62. 14. Spence D. Mid-Staffs shames us all. British Medical Journal 2013;346:1927. 15. Delamothe T. Culture change: Robert Francis' prescription for the NHS. British Medical Journal 2013;346:f979. 16. Groen R, Samai M, Steward K, Cassidy L, Kamara T, Tambasu S, et al. Untreated surgical conditions in Sierra Leone: a cluster randomised, cross-sectional, countrywide survey. The Lancet 2012;380:1082-87. 17. Thammatacharee N, Tisayaticom K, Suphanchaimat R, Limwattananon S, Putthasri W, Netsaengtip R, et al. Prevalence and profiles of unmet healthcare need in Thailand. BMC Public Health 2012;12(923):1-8. 18. Schoen C, Osborn R, Squires D, Doty M. Access, affordability, and insurance complexity are often worse in the United States compared to ten other countries. Health Affairs 2013;32(12):1-11. 19. Arah OA, Westert GP, Hurst J, Klazinga NS. A Conceptual Framework for the OECD Health Care Quality Indicators Project. International Journal for Quality in Health Care 2006;18(1):5-13. 20. Ministry of Health. A Portrait of Health: Key Results of the 2006/07 New Zealand Health Survey. Wellington: Ministry of Health, 2008. 21. Jatrana S, Crampton P. Primary health care in New Zealand: who has access? Health Policy 2009;93:1-10. 22. Harrison J, Young J, Butow P, Solomon M. Needs in health care: what beast is that? International Journal of Health Services 2013;43(3):567-85. 23. Giordano L, Elliot M, Goldstein E, Lehrman W, Spencer P. Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review 2010;67(1):27-37.

For the PDF of this article,
contact nzmj@nzma.org.nz

View Article PDF

The problemIn 1938 the Social Security Act for health care was passed in New Zealand with the aim of creating universal access to a comprehensive national health service in which barriers to accessing required healthcare, including financial, would be removed. Ten years later the National Health Service (NHS) with similar aims was enabled in the United Kingdom.1Major restructuring of the health sector has occurred, sometimes at frequent intervals, in these and many other developed countries, yet documentation of the overall success (or otherwise) of these changes has often been ignored even though they can produce unintended consequences. For example, 4 years after initiation of radical healthcare restructuring in New Zealand in 1993 there had been no formal, comprehensive review of achievements and outcomes.2 However, predicted outcomes of the market oriented alternative to state control in New Zealand, such as hospital profits and provider competition, failed to appear. Meanwhile, positives, such as better organisation of general practice, improvements in Māori health organisation, and creation of the national pharmaceutical purchasing agency (PHARMAC), which did eventuate were unforeseen.3The 1990s "big bang" approach to structural change, utilised under New Zealand's unicameral political system, seems more likely to produce results that are unpredictable than an incremental approach to change which is more typically undertaken in countries with a bicameral political system.3 In England, reorganisation (or breakup) of the NHS is underway with fears that, despite its bicameral political system, privatisation will play a prominent role.4In the absence of broad-based objective data collected routinely on an ongoing basis, ratings of the outcome of such policy changes or a health system per se can be readily reported as anywhere from superlative (usually by political leaders and public officials) to a dismal failure—often via anecdotal patient or health professional experience.In this brief article, we contend that in predominantly tax-funded systems with goals such as those of New Zealand and England, unmet healthcare need should be defined and included in any performance assessment.This should apply across restructuring of the health sector, following the implementation of new models of care whatever their objective, and even in the absence of modifications in the health system when changes in population demographics can alter health requirements substantially. We note the potential for unbalanced reporting in its absence, and describe some ways in which unmet need could be measured.A recent example of the problemInadequacies in assessing the performance of a healthcare system were brought home to us recently. On the one hand, a King's Fund investigation reported in 2013 that over some 5 years the District Health Board in Canterbury, New Zealand "..has moved towards (a more integrated system of healthcare) that manages demand more effectively in primary care and allows the hospitals to run more efficiently, thus concentrating more of their care on those who actually need to be in hospital".5 This report formed the basis for an editorial article in the British Medical Journal (BMJ) which suggested that the Canterbury experience offers several insights for the National Health Service (NHS) in England.6On the other hand a perceived serious and possibly increasing level of unmet healthcare need in the same Canterbury region has seen the establishment of the Canterbury Charity Hospital. A recent three year (2010–2012) review of this places on record the services, mostly surgical, provided to patients unable to access treatment in the public system and who could not afford private care.7 This review noted that, beyond the hundreds of patients treated, many appointment requests to the Charity Hospital were rejected because volunteer services in some specialties were not available. The experience of the Charity Hospital led to the conclusion that there are "… substantial, undocumented unmet healthcare needs in the region."7The former report by the King's Fund5 and the BMJ editorial6 suggest that substantial improvements in healthcare in Canterbury have occurred whilst the latter article on the Charity Hospital suggests that, across the same time period, unmet healthcare needs in the region are "alarmingly high".7 Neither report is particularly helpful in assessing the overall effectiveness or otherwise of changes to the organisation of healthcare services in Canterbury.We agree with the key comment in the King's Fund report that "Canterbury is far from alone in facing the challenge of measuring the impact of more integrated care".5 In our view the moral (and fiscal) imperative dictates that no major reform of a healthcare system should be considered, let alone implemented, in the absence of measuring its impact on a broad canvas of healthcare needs, including unmet need, in the target population.We also assert that unmet healthcare needs should be a core component of any health system assessment, even in the absence of major restructuring or the implementation of new models. This is for the simple reason that failure to meet such needs, particularly for medical and surgical services, is in breach of a basic human right to good health and healthcare.The way forwardWe concur with the growing body of research indicating that the performance of a healthcare system should be measured regularly, objectively and comprehensively by a body at arm's length from, and preferably independent of, the healthcare system.8,9 This, however, is easily stated but less easily implemented with a wide range of possibilities and different combinations when it comes to approach and indicators that could be used.10,11Generally speaking, there are three different sources of information: routine data relating to service coverage, scope and costs, and the incidence of disease; reports from healthcare professionals; and reports from patients. Often these are considered together, as in measurement of hospital performance, but none is ideal and frequently there is an absence of one of the three aforementioned perspectives on performance.For example, some indices built from routine data (longevity, perinatal mortality rate, the number of emergency medical admissions, readmission rates and childhood inoculation rates, for example) are important and readily quantified—but provide a narrow perspective of overall health delivery in a community.When it comes to health professional reported data, estimating patient access to services is problematic,12 can be readily manipulated, and cannot alone be relied upon to provide robust data.13 It may be possible for medical professionals to provide accurate information regarding the status of their patients across periods of reform or collectively raise concerns when standards of care are unacceptable. However, the inadequate performance of health professionals in the Mid Staffordshire events highlights the problems associated with such reporting.14,15As is now well known, conditions of appalling healthcare flourished in the main hospital of the Mid Staffordshire NHS Foundation Trust between 2005 and 2008, amid cost cutting and a drive to meet government targets, yet the response of health professionals was inadequate. It would be naïve to consider that the situation in Mid Staffordshire was and is unique. As working relationships (including employment links) between medical staff in primary and secondary care in particular change over time, so perceptions by medical staff of patient needs will inevitably alter with time.Accordingly, data, provided by medical and other healthcare staff, can be insightful but, if not standardised and collected in accordance with carefully defined criteria, need to be viewed with a degree of caution.From a patient's perspective there are possibly three core elements to a health care system which should be measured; accessibility, timeliness and quality of care. The first and second incorporate unmet need whilst the third, quality of care, is readily quantified by, for example, readmission rates and other quantitative indicators, and is also amenable to process and experience measurement.In our view, the centre point to assessing the performance of a healthcare system should utilise documentation of unmet healthcare needs as perceived by representative segments of the population at formal interview. At present, various agencies and research groups have attempted this, some in a robust and objective manner16,17 and some with sometimes questionable methods, for example, a tele-marketing approach of phoning individuals until a quota is reached.18 While providing a reasonable snap-shot of unmet need in different countries, such studies can easily be disregarded by policy-makers.A representative study of unmet need should involve selection of a random sample identified using robust statistical sampling procedures rather than accessible and potentially biased samples of convenience which characterise many of the ‘patient exit' and other surveys conducted by healthcare providers.Representative interviews, we suggest, should be repeated regularly and cover all aspects of unmet healthcare needs including dental, psychiatric, birth control and disability, as well as unmet general medical and surgical needs. The interview protocol should, ideally, be identical across many countries to enable time-matched comparisons between different healthcare systems and longitudinal assessments of the effects of organisational changes within any one country, as advocated by agencies such as the OECD but not yet achieved.19With this approach healthcare sector performance with regard to unmet need could be assessed, first, by documenting changes within the country over time and across any restructuring period and, second, by comparing changes in unmet need between countries some of which did not undergo restructuring.As a starting point we suggest that population based sampling methods such as those used by The New Zealand Health Survey and others could be the basis for such surveys.16 17 20 Currently the New Zealand example of a national Health Survey captures data on, amongst other things, health utilisation and management of diagnosed medical conditions, but does not attempt to measure the core elements of unmet need: undiagnosed conditions or conditions for which health care has been sought but has not been provided.Questions about avoiding health care due to financial barriers were asked in a related New Zealand government-sponsored Survey of Family, Income and Employment but did not assess the specific conditions for which healthcare needs were not met and was, in any case, discontinued in 2010.21 Surveys of this kind, coupled with information from healthcare providers, especially general practitioners, could provide robust estimates of unmet need and potentially identify specific problematic conditions, as well as identify demographic features of those most likely to be affected. In this way, there would be capacity to bring together a range of measures from the three different information vantage-points cited above in developing new methods for assessing unmet need, which also meet with suggested benchmarks for this.22It is important that the range of measures used be standardised to avoid the possibility of a positive, but limited, finding as indicating general good health of a healthcare system. Of course, it is possible that variables from Statistics New Zealand's Integrated Data Infrastructure could also be drawn upon.While for a different population and component of its healthcare system, the United States Centers for Medicare and Medicaid Services (CMS) has demonstrated that it is possible to implement a nation-wide standard survey of a random sample of public and private hospital patients (the Hospital Consumer Assessment of Healthcare Providers and Systems—or HCAHPS).23 The government of Thailand has similarly commissioned a random household survey focused specifically on unmet need for healthcare services.17It is time now for policymakers in New Zealand, England and elsewhere to follow suit in further developing such techniques and extending them beyond only patients who have been able to access hospital care.22 In this way, they may be able to respond to the need to better understand and comprehensively measure the important and unexplored issue of unmet need for healthcare in our communities. Without this, goals of delivering on fundamental human rights to good health, as well as robust health system performance measurement, cannot be delivered.The result will be the continued delivery of reports such as the King's Fund's5,6 that provide what is perhaps an important story, but present only one facet of the reality.\r\n

Summary

Abstract

Major restructuring of the health sector has been undertaken in many countries, including New Zealand and England, yet objective assessment of the outcomes has rarely been recorded. In the absence of comprehensive objective data, the success or otherwise of health reforms has been inferred from narrowly-focussed data or anecdotal accounts. A recent example relates to a buoyant Kings Fund report on the quest for integrated health and social care in Canterbury, New Zealand which prompted an equally supportive editorial article in the British Medical Journal (BMJ) suggesting it may contain lessons for Englands National Health Service. At the same time, a report published in the New Zealand Medical Journal expressed concerns at the level of unmet healthcare needs in Canterbury. Neither report provided objective information about changes over time in the level of unmet healthcare needs in Canterbury. -We propose that the performance of healthcare systems should be measured regularly, objectively and comprehensively through documentation of unmet healthcare needs as perceived by representative segments of the population at formal interview. Thereby the success or otherwise of organisational changes to a health system and its adequacy as demographics of the population evolve, even in the absence of major restructuring of the health sector, can be better documented.

Aim

Method

Results

Conclusion

Author Information

Robin Gauld, Professor of Health Policy, Department of Preventive and Social Medicine, University of Otago - Dunedin, Dunedin; Antony Raymont, Senior Research Officer, Waitemata District Health Board, Auckland; Philip F Bagshaw, Chair, Canterbury Charity Hospital Trust and Clinical Associate Professor, University of Otago - Christchurch, Christchurch; M Gary Nicholls, Emeritus Professor, Department of Medicine, University of Otago - Christchurch, Christchurch Hospital, Christchurch; Christopher M Frampton, Associate Professor, University of Otago - Christchurch, Christchurch

Acknowledgements

Correspondence

Robin Gauld, Department of Preventive and Social Medicine, University of Otago, PO Box 913, Dunedin 9054, New Zealand.

Correspondence Email

robin.gauld@otago.ac.nz

Competing Interests

Nil.

1. Gauld R. Questions about New Zealand's health system in 2013, its 75th anniversary year. New Zealand Medical Journal 2013;126(1380):1-7. 2. Hornblow A. New Zealand's health reforms: a clash of cultures. British Medical Journal 1997;314(1892-1894). 3. Gauld R. Big Bang and the Policy Prescription: Health Care Meets the Market in New Zealand. Journal of Health Politics, Policy and Law 2000;25(5):815-44. 4. Godlee F. Why markets don't work in healthcare. British Medical Journal 2012;344:33300. 5. Timmins N, Ham C. The Quest for Integrated Health and Social Care: A Case Study in Canterbury, New Zealand. London: The King's Fund, 2013. 6. Mays N, Smith J. What can England's NHS learn from Canterbury New Zealand? British Medical Journal 2013;347:f6513. 7. Bagshaw P, Maimbo-M'siska M, Nicholls M, Shaw C, Allardyce R, Bagshaw S, et al. The Canterbury Charity Hospital: an update (2010-2012) and effects of the earthquakes. New Zealand Medical Journal 2013;126(1386):31-42. 8. World Health Organization. The World Health Report 2000: Health Systems: Improving Performance. Geneva: World Health Organization, 2000. 9. van den Berg M, Kringos D, Marks L, Klazinga N. The Dutch health care performance report: seven years of health care performance assessment in the Netherlands. Health Research Policy and Systems 2014;12(1):1. 10. Gauld R, Al-Wahaibi S, Chisholm J, Crabbe R, Kwon B, Oh T, et al. Scorecards for health system performance assessment: the New Zealand example. Health Policy 2011;103:200-08. 11. Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Cambridge: Cambridge University Press, 2009. 12. Derrett S, Bevin T, Herbison P, Paul C. Access to elective surgery in New Zealand: considering equity and the public and private mix. International Journal of Health Planning and Management 2009;24:147-60. 13. Derrett S, Cousins K, Gauld R. A messy reality: an analysis of New Zealand's elective surgery scoring system via media sources, 2000-2006. International Journal of Health Planning and Management 2013;28:48-62. 14. Spence D. Mid-Staffs shames us all. British Medical Journal 2013;346:1927. 15. Delamothe T. Culture change: Robert Francis' prescription for the NHS. British Medical Journal 2013;346:f979. 16. Groen R, Samai M, Steward K, Cassidy L, Kamara T, Tambasu S, et al. Untreated surgical conditions in Sierra Leone: a cluster randomised, cross-sectional, countrywide survey. The Lancet 2012;380:1082-87. 17. Thammatacharee N, Tisayaticom K, Suphanchaimat R, Limwattananon S, Putthasri W, Netsaengtip R, et al. Prevalence and profiles of unmet healthcare need in Thailand. BMC Public Health 2012;12(923):1-8. 18. Schoen C, Osborn R, Squires D, Doty M. Access, affordability, and insurance complexity are often worse in the United States compared to ten other countries. Health Affairs 2013;32(12):1-11. 19. Arah OA, Westert GP, Hurst J, Klazinga NS. A Conceptual Framework for the OECD Health Care Quality Indicators Project. International Journal for Quality in Health Care 2006;18(1):5-13. 20. Ministry of Health. A Portrait of Health: Key Results of the 2006/07 New Zealand Health Survey. Wellington: Ministry of Health, 2008. 21. Jatrana S, Crampton P. Primary health care in New Zealand: who has access? Health Policy 2009;93:1-10. 22. Harrison J, Young J, Butow P, Solomon M. Needs in health care: what beast is that? International Journal of Health Services 2013;43(3):567-85. 23. Giordano L, Elliot M, Goldstein E, Lehrman W, Spencer P. Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review 2010;67(1):27-37.

For the PDF of this article,
contact nzmj@nzma.org.nz

View Article PDF

The problemIn 1938 the Social Security Act for health care was passed in New Zealand with the aim of creating universal access to a comprehensive national health service in which barriers to accessing required healthcare, including financial, would be removed. Ten years later the National Health Service (NHS) with similar aims was enabled in the United Kingdom.1Major restructuring of the health sector has occurred, sometimes at frequent intervals, in these and many other developed countries, yet documentation of the overall success (or otherwise) of these changes has often been ignored even though they can produce unintended consequences. For example, 4 years after initiation of radical healthcare restructuring in New Zealand in 1993 there had been no formal, comprehensive review of achievements and outcomes.2 However, predicted outcomes of the market oriented alternative to state control in New Zealand, such as hospital profits and provider competition, failed to appear. Meanwhile, positives, such as better organisation of general practice, improvements in Māori health organisation, and creation of the national pharmaceutical purchasing agency (PHARMAC), which did eventuate were unforeseen.3The 1990s "big bang" approach to structural change, utilised under New Zealand's unicameral political system, seems more likely to produce results that are unpredictable than an incremental approach to change which is more typically undertaken in countries with a bicameral political system.3 In England, reorganisation (or breakup) of the NHS is underway with fears that, despite its bicameral political system, privatisation will play a prominent role.4In the absence of broad-based objective data collected routinely on an ongoing basis, ratings of the outcome of such policy changes or a health system per se can be readily reported as anywhere from superlative (usually by political leaders and public officials) to a dismal failure—often via anecdotal patient or health professional experience.In this brief article, we contend that in predominantly tax-funded systems with goals such as those of New Zealand and England, unmet healthcare need should be defined and included in any performance assessment.This should apply across restructuring of the health sector, following the implementation of new models of care whatever their objective, and even in the absence of modifications in the health system when changes in population demographics can alter health requirements substantially. We note the potential for unbalanced reporting in its absence, and describe some ways in which unmet need could be measured.A recent example of the problemInadequacies in assessing the performance of a healthcare system were brought home to us recently. On the one hand, a King's Fund investigation reported in 2013 that over some 5 years the District Health Board in Canterbury, New Zealand "..has moved towards (a more integrated system of healthcare) that manages demand more effectively in primary care and allows the hospitals to run more efficiently, thus concentrating more of their care on those who actually need to be in hospital".5 This report formed the basis for an editorial article in the British Medical Journal (BMJ) which suggested that the Canterbury experience offers several insights for the National Health Service (NHS) in England.6On the other hand a perceived serious and possibly increasing level of unmet healthcare need in the same Canterbury region has seen the establishment of the Canterbury Charity Hospital. A recent three year (2010–2012) review of this places on record the services, mostly surgical, provided to patients unable to access treatment in the public system and who could not afford private care.7 This review noted that, beyond the hundreds of patients treated, many appointment requests to the Charity Hospital were rejected because volunteer services in some specialties were not available. The experience of the Charity Hospital led to the conclusion that there are "… substantial, undocumented unmet healthcare needs in the region."7The former report by the King's Fund5 and the BMJ editorial6 suggest that substantial improvements in healthcare in Canterbury have occurred whilst the latter article on the Charity Hospital suggests that, across the same time period, unmet healthcare needs in the region are "alarmingly high".7 Neither report is particularly helpful in assessing the overall effectiveness or otherwise of changes to the organisation of healthcare services in Canterbury.We agree with the key comment in the King's Fund report that "Canterbury is far from alone in facing the challenge of measuring the impact of more integrated care".5 In our view the moral (and fiscal) imperative dictates that no major reform of a healthcare system should be considered, let alone implemented, in the absence of measuring its impact on a broad canvas of healthcare needs, including unmet need, in the target population.We also assert that unmet healthcare needs should be a core component of any health system assessment, even in the absence of major restructuring or the implementation of new models. This is for the simple reason that failure to meet such needs, particularly for medical and surgical services, is in breach of a basic human right to good health and healthcare.The way forwardWe concur with the growing body of research indicating that the performance of a healthcare system should be measured regularly, objectively and comprehensively by a body at arm's length from, and preferably independent of, the healthcare system.8,9 This, however, is easily stated but less easily implemented with a wide range of possibilities and different combinations when it comes to approach and indicators that could be used.10,11Generally speaking, there are three different sources of information: routine data relating to service coverage, scope and costs, and the incidence of disease; reports from healthcare professionals; and reports from patients. Often these are considered together, as in measurement of hospital performance, but none is ideal and frequently there is an absence of one of the three aforementioned perspectives on performance.For example, some indices built from routine data (longevity, perinatal mortality rate, the number of emergency medical admissions, readmission rates and childhood inoculation rates, for example) are important and readily quantified—but provide a narrow perspective of overall health delivery in a community.When it comes to health professional reported data, estimating patient access to services is problematic,12 can be readily manipulated, and cannot alone be relied upon to provide robust data.13 It may be possible for medical professionals to provide accurate information regarding the status of their patients across periods of reform or collectively raise concerns when standards of care are unacceptable. However, the inadequate performance of health professionals in the Mid Staffordshire events highlights the problems associated with such reporting.14,15As is now well known, conditions of appalling healthcare flourished in the main hospital of the Mid Staffordshire NHS Foundation Trust between 2005 and 2008, amid cost cutting and a drive to meet government targets, yet the response of health professionals was inadequate. It would be naïve to consider that the situation in Mid Staffordshire was and is unique. As working relationships (including employment links) between medical staff in primary and secondary care in particular change over time, so perceptions by medical staff of patient needs will inevitably alter with time.Accordingly, data, provided by medical and other healthcare staff, can be insightful but, if not standardised and collected in accordance with carefully defined criteria, need to be viewed with a degree of caution.From a patient's perspective there are possibly three core elements to a health care system which should be measured; accessibility, timeliness and quality of care. The first and second incorporate unmet need whilst the third, quality of care, is readily quantified by, for example, readmission rates and other quantitative indicators, and is also amenable to process and experience measurement.In our view, the centre point to assessing the performance of a healthcare system should utilise documentation of unmet healthcare needs as perceived by representative segments of the population at formal interview. At present, various agencies and research groups have attempted this, some in a robust and objective manner16,17 and some with sometimes questionable methods, for example, a tele-marketing approach of phoning individuals until a quota is reached.18 While providing a reasonable snap-shot of unmet need in different countries, such studies can easily be disregarded by policy-makers.A representative study of unmet need should involve selection of a random sample identified using robust statistical sampling procedures rather than accessible and potentially biased samples of convenience which characterise many of the ‘patient exit' and other surveys conducted by healthcare providers.Representative interviews, we suggest, should be repeated regularly and cover all aspects of unmet healthcare needs including dental, psychiatric, birth control and disability, as well as unmet general medical and surgical needs. The interview protocol should, ideally, be identical across many countries to enable time-matched comparisons between different healthcare systems and longitudinal assessments of the effects of organisational changes within any one country, as advocated by agencies such as the OECD but not yet achieved.19With this approach healthcare sector performance with regard to unmet need could be assessed, first, by documenting changes within the country over time and across any restructuring period and, second, by comparing changes in unmet need between countries some of which did not undergo restructuring.As a starting point we suggest that population based sampling methods such as those used by The New Zealand Health Survey and others could be the basis for such surveys.16 17 20 Currently the New Zealand example of a national Health Survey captures data on, amongst other things, health utilisation and management of diagnosed medical conditions, but does not attempt to measure the core elements of unmet need: undiagnosed conditions or conditions for which health care has been sought but has not been provided.Questions about avoiding health care due to financial barriers were asked in a related New Zealand government-sponsored Survey of Family, Income and Employment but did not assess the specific conditions for which healthcare needs were not met and was, in any case, discontinued in 2010.21 Surveys of this kind, coupled with information from healthcare providers, especially general practitioners, could provide robust estimates of unmet need and potentially identify specific problematic conditions, as well as identify demographic features of those most likely to be affected. In this way, there would be capacity to bring together a range of measures from the three different information vantage-points cited above in developing new methods for assessing unmet need, which also meet with suggested benchmarks for this.22It is important that the range of measures used be standardised to avoid the possibility of a positive, but limited, finding as indicating general good health of a healthcare system. Of course, it is possible that variables from Statistics New Zealand's Integrated Data Infrastructure could also be drawn upon.While for a different population and component of its healthcare system, the United States Centers for Medicare and Medicaid Services (CMS) has demonstrated that it is possible to implement a nation-wide standard survey of a random sample of public and private hospital patients (the Hospital Consumer Assessment of Healthcare Providers and Systems—or HCAHPS).23 The government of Thailand has similarly commissioned a random household survey focused specifically on unmet need for healthcare services.17It is time now for policymakers in New Zealand, England and elsewhere to follow suit in further developing such techniques and extending them beyond only patients who have been able to access hospital care.22 In this way, they may be able to respond to the need to better understand and comprehensively measure the important and unexplored issue of unmet need for healthcare in our communities. Without this, goals of delivering on fundamental human rights to good health, as well as robust health system performance measurement, cannot be delivered.The result will be the continued delivery of reports such as the King's Fund's5,6 that provide what is perhaps an important story, but present only one facet of the reality.\r\n

Summary

Abstract

Major restructuring of the health sector has been undertaken in many countries, including New Zealand and England, yet objective assessment of the outcomes has rarely been recorded. In the absence of comprehensive objective data, the success or otherwise of health reforms has been inferred from narrowly-focussed data or anecdotal accounts. A recent example relates to a buoyant Kings Fund report on the quest for integrated health and social care in Canterbury, New Zealand which prompted an equally supportive editorial article in the British Medical Journal (BMJ) suggesting it may contain lessons for Englands National Health Service. At the same time, a report published in the New Zealand Medical Journal expressed concerns at the level of unmet healthcare needs in Canterbury. Neither report provided objective information about changes over time in the level of unmet healthcare needs in Canterbury. -We propose that the performance of healthcare systems should be measured regularly, objectively and comprehensively through documentation of unmet healthcare needs as perceived by representative segments of the population at formal interview. Thereby the success or otherwise of organisational changes to a health system and its adequacy as demographics of the population evolve, even in the absence of major restructuring of the health sector, can be better documented.

Aim

Method

Results

Conclusion

Author Information

Robin Gauld, Professor of Health Policy, Department of Preventive and Social Medicine, University of Otago - Dunedin, Dunedin; Antony Raymont, Senior Research Officer, Waitemata District Health Board, Auckland; Philip F Bagshaw, Chair, Canterbury Charity Hospital Trust and Clinical Associate Professor, University of Otago - Christchurch, Christchurch; M Gary Nicholls, Emeritus Professor, Department of Medicine, University of Otago - Christchurch, Christchurch Hospital, Christchurch; Christopher M Frampton, Associate Professor, University of Otago - Christchurch, Christchurch

Acknowledgements

Correspondence

Robin Gauld, Department of Preventive and Social Medicine, University of Otago, PO Box 913, Dunedin 9054, New Zealand.

Correspondence Email

robin.gauld@otago.ac.nz

Competing Interests

Nil.

1. Gauld R. Questions about New Zealand's health system in 2013, its 75th anniversary year. New Zealand Medical Journal 2013;126(1380):1-7. 2. Hornblow A. New Zealand's health reforms: a clash of cultures. British Medical Journal 1997;314(1892-1894). 3. Gauld R. Big Bang and the Policy Prescription: Health Care Meets the Market in New Zealand. Journal of Health Politics, Policy and Law 2000;25(5):815-44. 4. Godlee F. Why markets don't work in healthcare. British Medical Journal 2012;344:33300. 5. Timmins N, Ham C. The Quest for Integrated Health and Social Care: A Case Study in Canterbury, New Zealand. London: The King's Fund, 2013. 6. Mays N, Smith J. What can England's NHS learn from Canterbury New Zealand? British Medical Journal 2013;347:f6513. 7. Bagshaw P, Maimbo-M'siska M, Nicholls M, Shaw C, Allardyce R, Bagshaw S, et al. The Canterbury Charity Hospital: an update (2010-2012) and effects of the earthquakes. New Zealand Medical Journal 2013;126(1386):31-42. 8. World Health Organization. The World Health Report 2000: Health Systems: Improving Performance. Geneva: World Health Organization, 2000. 9. van den Berg M, Kringos D, Marks L, Klazinga N. The Dutch health care performance report: seven years of health care performance assessment in the Netherlands. Health Research Policy and Systems 2014;12(1):1. 10. Gauld R, Al-Wahaibi S, Chisholm J, Crabbe R, Kwon B, Oh T, et al. Scorecards for health system performance assessment: the New Zealand example. Health Policy 2011;103:200-08. 11. Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Cambridge: Cambridge University Press, 2009. 12. Derrett S, Bevin T, Herbison P, Paul C. Access to elective surgery in New Zealand: considering equity and the public and private mix. International Journal of Health Planning and Management 2009;24:147-60. 13. Derrett S, Cousins K, Gauld R. A messy reality: an analysis of New Zealand's elective surgery scoring system via media sources, 2000-2006. International Journal of Health Planning and Management 2013;28:48-62. 14. Spence D. Mid-Staffs shames us all. British Medical Journal 2013;346:1927. 15. Delamothe T. Culture change: Robert Francis' prescription for the NHS. British Medical Journal 2013;346:f979. 16. Groen R, Samai M, Steward K, Cassidy L, Kamara T, Tambasu S, et al. Untreated surgical conditions in Sierra Leone: a cluster randomised, cross-sectional, countrywide survey. The Lancet 2012;380:1082-87. 17. Thammatacharee N, Tisayaticom K, Suphanchaimat R, Limwattananon S, Putthasri W, Netsaengtip R, et al. Prevalence and profiles of unmet healthcare need in Thailand. BMC Public Health 2012;12(923):1-8. 18. Schoen C, Osborn R, Squires D, Doty M. Access, affordability, and insurance complexity are often worse in the United States compared to ten other countries. Health Affairs 2013;32(12):1-11. 19. Arah OA, Westert GP, Hurst J, Klazinga NS. A Conceptual Framework for the OECD Health Care Quality Indicators Project. International Journal for Quality in Health Care 2006;18(1):5-13. 20. Ministry of Health. A Portrait of Health: Key Results of the 2006/07 New Zealand Health Survey. Wellington: Ministry of Health, 2008. 21. Jatrana S, Crampton P. Primary health care in New Zealand: who has access? Health Policy 2009;93:1-10. 22. Harrison J, Young J, Butow P, Solomon M. Needs in health care: what beast is that? International Journal of Health Services 2013;43(3):567-85. 23. Giordano L, Elliot M, Goldstein E, Lehrman W, Spencer P. Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review 2010;67(1):27-37.

Contact diana@nzma.org.nz
for the PDF of this article

Subscriber Content

The full contents of this pages only available to subscribers.

LOGINSUBSCRIBE