Journal of the New Zealand Medical Association, 05-November-2010, Vol 123 No 1325
The Voice of Experience: Results from Cancer Control New Zealand’s first national cancer care survey
Inga O’Brien, Emma Britton, Diana Sarfati, Wayne Naylor, Barry Borman, Lis Ellison-Loschmann, Andrew Simpson, Craig Tamblyn, Chris Atkinson
Quality cancer care has both technical and service components. Technical skills are critical to the effective diagnosis and treatment of disease. Service skills are required to holistically meet the needs of patients’ and address their expectations. A combination of technical and service skills is necessary to address cancer patients’ medical and non-medical needs and wants over the duration of a cancer journey. Patients’ experiences of care can be sought and reviewed with the goal of incorporating these voices of experience into quality improvement efforts.
The importance of including the patient’s perspective in evaluations of care is reflected in key New Zealand government documents such as the New Zealand Health Strategy (2000), which has an underlying fundamental principle that there should be “active involvement of consumers and communities at all levels”.1 Additionally, two of the guiding principles of the New Zealand Cancer Control Strategy (2003) are that activities should “ reflect a person-centred approach” and “actively involve consumers and communities”.2
Furthermore, the recent Ministerial review of the Health System has described the “...Government’s vision of a public health and disability service that is more patient- than provider-centric, giving patients more control”.3 Analysing patients’ experiences of care, to inform improvements in health services, adheres to these principles and ideas.
With the general public, healthcare professionals and the Government all looking for improvements in the quality and delivery of cancer care, surveying cancer patients’ experiences is a promising method of exploring if the services consumers receive match their needs and expectations. This type of survey puts the consumer at the centre of the evaluation of care and the results provide stakeholders with a unique perspective on New Zealand’s performance in providing quality cancer care.
In 2009, Cancer Control New Zealand (CCNZ), formerly known as the Cancer Control Council, initiated the Cancer Care Survey, which was the first national survey of cancer patient experiences.
The overall aim of this study was to collect data on how well outpatient cancer treatment services were meeting the needs and expectations of cancer patients. CCNZ’s specific 2009 Cancer Care Survey objectives were to:
Three reports on the Cancer Care Survey have been written and are available on CCNZ’s website (www.cancercontrolnz.govt.nz). The Voice of Experience Part One report provides preliminary national results from the survey.4 The Voice of Experience companion report provides the results for the eight participating cancer treatment services.5 The Voice of Experience Part Two report6 includes a select review of the patient experience literature, in-depth analysis of the survey data, and recommendations.
This article focuses on summarising the overall national findings from the survey.
Participants and setting—The target population for the survey was patients 18 years of age and older with a confirmed diagnosis of cancer who had undergone, or were undergoing, publicly funded cancer outpatient treatment (specifically chemotherapy or radiotherapy) in New Zealand. Patients were excluded from the survey if they had no fixed address, had moved out of New Zealand, were not residents of New Zealand, had received only inpatient services, or were deceased.
The participating facilities included the six regional cancer centres providing the majority of cancer services in New Zealand (Canterbury Oncology Service, Northern Region Cancer Centre, Palmerston North Regional Cancer Treatment Service, Southern Blood and Cancer Service, Waikato Regional Cancer Centre, and Wellington Blood and Cancer Centre) and two satellite chemotherapy treatment facilities with vocationally registered oncologists on staff (Nelson Oncology Service and Tauranga & Whakatane Cancer Centres). These services provided lists of all their outpatients who met the criteria for the study during the 6-month period from 1 October 2008 to 31 March 2009.
A sample was randomly drawn from these lists with an aim to select 500 people from each cancer treatment service. If less than 500 people were submitted by a cancer treatment service then all individuals were selected.
All those who had their ethnicity recorded as Māori but had not been randomly selected, were also included in the sample. This oversampling process aimed to increase the precision of the estimates for Māori. Prior to proceeding with the survey, ethics approval was obtained from the Multi-Region Ethics Committee (MEC/09/22/EXP).
Survey instrument—The 2009 Cancer Care Survey, a tool developed and validated7 by NRC+Picker (USA), was mailed to the selected participants. The questionnaire included 96 questions, with multiple response options, covering a variety of patient experiences related to their cancer treatment, including their diagnosis, treatment (surgery, chemotherapy and radiotherapy), symptom management, healthcare team, care environment and overall impressions of care. The survey also contained demographic questions including the respondent’s age, gender, household annual income after tax and ethnicity.
Surveys were posted to 3525 selected participants and were also available online. A covering letter, signed by Dame Catherine Tizard (CCNZ chair 2005–2009), explaining the importance of the survey was attached to each survey form. To increase the response rate, posters about the survey were placed in cancer treatment service waiting rooms and patient support service buildings. A telephone help line was set up to answer any queries from survey recipients. Reminder post cards and a second survey were sent out to those who had not responded to the initial survey after 6 weeks.
Analysis—The data analysis was carried out using Stata (StataCorp LP) software, versions 9.2 and 10. The distribution of the target, surveyed and eligible respondent populations were compared across key demographic characteristics, and the response rates were also compared across the different ethnic, age, gender, income and cancer treatment service groups.
The sample data were weighted to take into account the probability of selection. Post stratification weights were also calculated for each respondent using the age and ethnicity distribution of the target population in each cancer treatment service.
Sixty-five of the 96 survey questions had ordinal categorical response options which could be grouped into dichotomised positive and negative response categories. In this ‘top box’ scoring approach, only the ‘ideal’ response, such as “always” or “definitely”, was equated with a positive response. This categorisation process allowed the questions to be rank ordered from the highest percentage of positive responses to those with the lowest percentage of positive responses, so the questions in the upper and lower quartiles could be examined both nationally and at the individual cancer treatment service levels.
In total 3525 people were mailed surveys. A total of 410 people were excluded (136 during pre-mailing checks, 254 post mailing and 20 post data cleaning). Replacements were randomly selected from the corresponding patient lists for the 136 people excluded pre-mailing.
Reasons for the 274 exclusions post-mailing included: the person was deceased, the person stated they did not receive treatment for cancer, the person was too unwell or otherwise unable to complete the survey, the survey being returned to sender as undeliverable, or the returned questionnaire being illegible or damaged. The final total survey population was 3251. Of the 3251 eligible respondents, surveys were returned by 2221 people resulting in a 68% final response rate.
According to the ethnicity information obtained from the cancer treatment services, New Zealand Europeans made up 69% of the respondent population, while 13% were Māori, 3% were Asian, and 2% were Pacific Islanders (Table 1). The highest proportion of eligible respondents was in the 50–69 years age group and over half of the survey population was female.
The proportion of those within the different ethnic, age, and gender groups in the targeted, surveyed and respondent populations were similar (Table 1). For example, 50% of the targeted population, 49% of the surveyed population and 51% of the respondent population were in the 50–69 years age group.
Although the overall response rate for Māori was lower (51%) than for New Zealand Europeans (74%), the proportion of Māori in the surveyed population was higher than the proportion of Māori in the targeted population due to deliberate over-sampling.
* The original target population includes everyone submitted from the cancer service patient lists and includes people who were subsequently excluded on the basis of being deceased, not having received treatment for cancer, being too unwell or unable to complete the survey, the survey being returned to sender as undeliverable, or the returned questionnaire being illegible or damaged.
** Includes Maori selected by the oversampling process.
† Cancer treatment services are required to collect ethnicity data from people accessing their services based on the Ministry of Health Ethnicity Data Protocols for the Health and Disability Sector. There were minor variations between the ethnicity data collected by the cancer treatment services and the self-reported ethnicity data collected by the survey.
‡ For the sample and eligible respondent populations, gender was imputed from the person’s title or name, as gender information was not included in the cancer treatment services’ lists. Target population proportions for gender were estimated using sample proportions.
Figure 1 presents the range of national and individual treatment service percent positive score results for a subset of survey questions. The national score is shown alongside the participating cancer treatment service scores for comparison purposes. Overall, the survey questions tended to have a high percentage of positive responses.
In Figure 1, the questions with the highest percentage of positive responses were related to specialist care coordination (91% positive response; 95% CI: 90–93) and the level of dignity and respect provided by healthcare professionals (86% positive response; 95% CI: 85–88). Whereas, the questions on information clarity (54% positive response; 95% CI: 52–57) and provision of referrals for emotional support (36% positive response; 95% CI: 33–39) received a lower percentage of positive responses.
Figure 1. National responses to a selection of key survey questions
For most survey questions, the individual treatment service results were tightly clustered around the national-level result (Figure 1). However, as shown in Table 2, 12 survey questions demonstrated at least a 20% difference between the lowest and highest treatment service scores, indicating some significant differences in patient experiences between treatment services.
The key national strengths and opportunities for improvement are shown in Tables 3 and 4. These were identified from examining the upper and lower quartiles of the ranked survey questions. The aspects of care with a high percentage of positive responses were identified as strengths (Table 3).
Table 2. Aspects of care with substantial differences in positive scores between cancer treatment services
CI=confidence interval; * For these confidence intervals the sample sizes in the strata were very small, so strata were collapsed together to calculate the confidence intervals. The confidence intervals here are very wide indicating that the point estimate is not a reliable estimate of the ‘true’ value.
Table 3. National strengths of outpatient cancer care, as indicated by aspects of care receiving the highest percentage of positive responses in the survey
The aspects of care with a low percentage of positive responses were identified as opportunities for improvement (Table 4).
Table 4. National opportunities for improvement in outpatient cancer care, as indicated by aspects of care receiving the lowest percentage of positive responses in the survey
Most of the people who participated in the 2009 Cancer Care Survey responded very positively to questions on care coordination and the level of privacy, dignity and respect provided. The majority also felt that care providers were doing everything they could to treat their cancer. Even though overall care was rated highly, focusing on patients’ experiences exposed areas where action is needed to improve the quality of care. These areas include providing more information on potential changes in aspects of day-to-day living, facilitating opportunities for emotional support, providing better explanations for waiting times, and treating patients more within the context of their own lives.
The results of this first Cancer Care Survey in New Zealand provide a baseline for future monitoring of patients’ experiences of cancer care. This survey found that each cancer treatment service had different areas where it performed relatively well and where there were opportunities for improvement.
While cancer treatment service results tended to follow the same general pattern as the national-level results, responses to 12 questions demonstrated variation with at least a 20% difference in scores between the cancer treatment service with the highest positive score and the cancer treatment service with the lowest positive score.
Although this analysis found that there is no single ‘best’ patient experience of care model, cancer treatment services can collaborate to allow regional champions for particular aspects of care to share approaches for improving the patient experience.
Both technical and service aspects of care comprise quality service delivery just as both clinical outcome and personal experiences are important aspects of patient expectations.8
Results from this first Cancer Care Survey suggest that technical aspects of care tend to meet most patients’ expectations while service aspects of care often do not meet patients’ highest expectations. These indicate important opportunities for improvement given that service responsiveness has been posited to:
Because quality care is the “...cumulative result of the interactions of people, individuals, teams, organizations and systems”,11 multiple strategies and voices will be required to advocate for the need to optimise both the technical and service aspects of providing effective cancer care.
This was a relatively large survey, of a randomly selected population, which used a questionnaire that, although it contained minor adaptations for a New Zealand context, was based on material that had been well tested and administered internationally.7,12,13 A high response rate was achieved.
The study did, however, have several limitations. The strengths and opportunities for improvement sections presented in the results reflect the experiences of the majority of respondents, whose experiences may be different from those of minority groups within the sample, and those who chose not to respond to the survey. Furthermore, there is no single definition of ‘best patient experience’ due to differing expectations and perceptions of what is effective and appropriate cancer care delivery across individuals and population groups. Additionally, as this was a quantitative study, responses were restricted to categories provided in the survey, limiting the amount of information that could be gathered on the context of the patients’ experiences.
Whilst the questionnaire has been validated in the USA by NRC+Picker, and utilised in Australia and Canada, a few of the questions may not be as applicable to the New Zealand health system. However, the authors believe that the survey utilised provides valuable insights into the perceptions of cancer management in New Zealand and that no survey instrument can perfectly measure the complexities of patient experiences of healthcare services.
Further research could assist in addressing some of these limitations. There is a need for more in-depth exploration of Pacific and Asian cancer patient experiences as these population groups were under-represented in this survey. Further, large cancer centres, particularly Auckland, have been under-represented in this project and would benefit from follow-up research to validate the findings from this survey. In regard to the data analysis, a sensitivity analysis of the different positive response thresholds would provide useful information on the utility of the ‘top box’ scoring approach for reporting results.
It is recommended that the findings from this survey progress the debate around the role and meaning of patient-centred care in New Zealand. Results can also inform the development of actionable and standardised patient-reported outcomes. Further, survey findings can be linked to findings from related projects, such as the National Cancer Psychosocial Services Stocktake 2005–2006,14 to inform development of the Implementation Plan for the Supportive Care Guidance.15
Finally, a robust cancer patient needs assessment, one that is able to identify changing needs along a cancer journey, can provide a broader patient-focused perspective on quality improvement priorities.
The 2009 Cancer Care Survey data provides a consumer-perspective on the performance of cancer outpatient services. CCNZ has found that cancer patient experience surveys can achieve high response rates and generate useful consumer-reported views on cancer care delivery that could be integrated with evidence-based practice to inform quality improvement efforts.
The results of this first Cancer Care Survey in New Zealand will act as a benchmark against which the results of future cancer patient experience surveys can be compared. This will be crucial for monitoring the impact of new initiatives to improve the quality of cancer care.
Notes: This is a summary of information presented in the Voice of Experience report series available from Cancer Control New Zealand (www.cancercontrolnz.govt.nz).
Competing interests: None.
Author information: Inga O’Brien, Senior Analyst, Cancer Control New Zealand, Wellington; Emma Britton, Public Health Registrar, Cancer Control New Zealand, Wellington; Diana Sarfati, Senior Lecturer and Public Health Physician, Department of Public Health, University of Otago, Wellington; Wayne Naylor, Senior Analyst, Palliative Care Council of New Zealand, Wellington; Barry Borman, Associate Professor, Centre for Public Health Research, Massey University, Wellington; Lis Ellison-Loschmann, Research Fellow, Centre for Public Health Research, Massey University, Wellington; Andrew Simpson, Clinical Director, Wellington Blood & Cancer Centre, Capital & Coast District Health Board, Wellington; Craig Tamblyn, General Manager, Cancer Control New Zealand, Wellington; Chris Atkinson, Chair, Cancer Control New Zealand, Wellington
Acknowledgements: We thank Cancer Control New Zealand and The New Zealand Population Health Charitable Trust for project funding; the 2009 Cancer Care Survey Advisory Group (Dr Diana Sarfati, Associate Professor Barry Borman, Professor Neil Pearce, Professor Tony Blakely, Dr Andrew Simpson, Ms Astrid Koorneef and Dr Lis Ellison-Loschmann) for survey advice; and Robert Templeton (Principal Technical Specialist) and Roy Costilla (Advisor-Statistics) at the Ministry of Health for statistical advice. Lastly, we gratefully acknowledge the willingness of people affected by cancer to share their views and experiences of health services they received.
Correspondence: Craig Tamblyn, Cancer Control New Zealand, PO Box 5013, Wellington, New Zealand. Fax: +64 (0)4 8159249; email: email@example.com
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals