Journal of the New Zealand Medical Association, 20-April-2012, Vol 125 No 1353
Great expectations: use of molecular tests and computerised prognostic tools in New Zealand cancer care
Deborah M Wright, Rob McNeill, Arend E H Merrie, Cristin G Print
There is an international drive to improve outcomes for patients with cancer by individualising cancer treatment using technologies including molecular tests (MT) and computerised prognostic tools (CPT).1,2 MT utilise molecular information, for example variations in DNA sequence or RNA expression levels, to diagnose disease or to predict susceptibility or treatment outcome. CPT use computerised statistical models to combine large datasets with individuals’ clinical details to infer individualised prognoses.
MT and CPT designed to aid clinical decision making for patients with a range of malignancies have been described.3 Molecular tests available in New Zealand (NZ) include: MammaPrint4 and Oncotype DX,5 which use gene expression analysis to derive a recurrence risk score for patients with early breast cancer; FLT3, NPM1 and CEBPA mutation analysis which provide prognostic information for patients with cytogenetically normal acute myeloid leukaemia (CN-AML) and are recommended in World Health Organization (WHO) guidelines;6,7 KRAS mutation analysis, which predicts response to cetuximab, an unfunded treatment for metastatic colorectal cancer;8 UGT1A1 mutation analysis to predict irinotecan toxicity;9 EGFR mutation analysis to predict response to gefinitib and erlotinib for patients with non-small cell lung cancer.10
In NZ we also have free online access to a number of CPT including Adjuvant!, which estimates recurrence risk and treatment benefit for patients with breast, colon or lung cancer.11 Further details of these examples of MT and CPT are given in Table 1.
Table 1 Examples of molecular tests and computerised prognostic tools currently available in New Zealand for the care of patients with cancer
CN-AML=cytogenetically normal acute myeloid leukaemia; CRC=colorectal cancer; ER=(o)estrogen receptor; qRT-PCR=quantitative reverse transcription polymerase chain reaction.
The utilisation of MT and CPT during the management of patients with solid organ and haematological malignancy is likely to have a significant impact on clinical practice and health economics in NZ, however it has not been evaluated to date. The intent of this study is to determine the awareness and specific utilisation of MT and CPT amongst NZ cancer clinicians treating solid organ and haematological malignancy, and to ascertain their predictions for the impact of these technologies over the next 10 years.
An anonymous online questionnaire was used to survey clinicians who treat patients with cancer in NZ.
The questionnaire was implemented using LimeSurvey software (Carsten Schmitz, Germany), a free open source survey application. It comprised 185 questions in three sections. Most questions in sections one and two had fixed ‘click button’ answer options and a free text ‘other’ option; where a numeric answer was required a free text box or slide rule was provided.
In section 3, participants were shown clinical scenarios relating to their area of specialty. The scenarios presented situations in which molecular tests are purported to assist with clinical decision-making: stage II breast cancer, stage II colon cancer and CN-AML in remission after chemotherapy. Participants were invited to leave free text comments at the end of each section of the survey.
The questions presented to each participant were determined by their previous responses such that each participant saw only those questions relevant to their clinical practice. The questionnaire took approximately 15 minutes to complete.
Please visit http://www.bioinformatics.auckland.ac.nz/doc/project_data/Supplemetary_figure_and_tables_FINAL.docx to view the questionnaire in full and all supplementary figures and tables. The University of Auckland Human Participants Ethics Committee granted ethical approval for this study.
Medical and radiation oncologists, haematologists, pathologists and general surgeons practicing in NZ at specialist and trainee level were invited to participate by email via their professional societies and colleges. All trainees were enrolled in college-approved training programmes. Reminder emails were sent out 2 and 4 weeks after the initial invitation. Participation was incentivised with an iPad (Apple Inc., California, USA), won by a participant selected using a random number generator.
The survey was conducted over 11 weeks, from 17th May to 1st August 2010. Responses from clinicians practicing outside NZ were excluded from analysis, as were incomplete responses that did not include details of the participant’s specialty and seniority. Data analysis was carried out using PASW Statistics 18.0 (IBM Corp., NY, USA), Excel 2008 version 12.2.9 (Microsoft Corp., Washington, USA) and VassarStats (faculty.vassar.edu/lowry/VassarStats.html). Relationships between independent categorical variables were analysed using the chi-square test for independence of association, relationships between non-independent variables were analysed using McNemar’s test. Where multiple tests were performed the Bonferroni correction was used. A P value of <0.05 was held to be significant and P<0.01 as highly significant.
Survey participants - 739 clinicians were invited to participate in the survey. 186 clinicians accessed the online questionnaire; 137 completed it (Figure 1). Participants represented all invited specialties and included both specialists and trainees (Table 2). Specialists were significantly under represented relative to trainees (P<0.01); pathologists were significantly under represented relative to other specialties (P<0.05 for both pathology specialists and trainees). Participants worked in secondary, tertiary, academic and private practice settings.
Figure 1. Participation in a survey investigating utilisation of molecular tests and computerised prognostic tools
Table 2. Seniority and specialty of survey participants
Current practice—A greater proportion of participants were aware of MT than CPT (92% vs. 69%, P<0.01) (Table 3). Awareness of MT by specialists vs. registrars showed no statistically significant difference (6% and 10%, respectively), however specialists were significantly less likely to be aware of CPT than registrars (60% vs. 81%, P <0.05). Fewer participants had ever used MT than CPT (43 vs. 78, P<0.01). Of participants aware of MT, 59/126 (47%) reported that they had never used MT relevant to their clinical practice. Of participants aware of CPT, 12/94 (13%) reported that they had never used CPT relevant to their clinical practice.
Table 3. Awareness and utilisation of molecular tests and computerised prognostic tools amongst New Zealand cancer clinicians
Table 4 presents data on factors that limited the use of those MT and CPT most commonly used in NZ. Supplementary Tables 1 and 2 present this data for all of the MT and CPT included in the survey.
Factors reported to limit the use of MT and CPT varied. For example, awareness of both the CPT Adjuvant! and the MT Oncotype DX was high (78% and 86%) amongst participants who managed breast cancer (n=94) but while the use of Oncotype DX was most commonly limited by cost, use of Adjuvant! was most commonly limited by lack of clinical time (Table 4).
For participants who prescribed chemotherapy, both the cost of mutation analysis and, in some instances, the cost of unfunded medications (e.g. cetuximab) limited their uptake of MT.
Table 4. Factors that limited the use† of molecular tests and computerised prognostic models for the management of patients with cancer in New Zealand
At the time of the survey 80% of participants managing breast cancer (75/94) were aware of the prognostic MT Oncotype DX and MammaPrint; Oncotype DX was currently being used by six, MammaPrint by two. Of the 26 participants managing CN-AML, 92% had heard of FLT3, NPM1 or CEBPA mutation analysis; FLT3, NPM1 and CEBPA mutation analysis were currently being used by 22 (85%), 15 (41%) and two (8%) of these clinicians, respectively. Thirty-four participants prescribed chemotherapeutic agents of whom 29 (85%) had heard of KRAS, UGT1A1, or EGFR testing. Twelve (35%), one (3%) and four (12%) of these clinicians currently used KRAS, UGT1A1 and EGFR mutation analysis, respectively.
Table 5. Impact of molecular tests and computerised prognostic tools on the management of patients with cancer in New Zealand
Participants were asked to comment on the influence these tools had on their practice (Table 5). For 19 participants FLT3 mutation analysis affected their clinical decisions; 16 were more likely to offer allogeneic stem cell transplantation, two were more likely to suggest deferring treatment and one was more likely to offer chemotherapy. For 10 participants KRAS mutation analysis, which predicts response to cetuximab, affected their clinical decisions; six reported that it resulted in offering fewer patients treatment with this drug and two that they offered more patients treatment with cetuximab.
Twenty-eight participants reported that Adjuvant! for breast cancer affected their clinical decisions; 18 considered adjuvant therapy for fewer patients and 10 for more.
Overall, MT were more likely than CPT (P<0.01) to affect the clinical decisions of participants currently using them. However, because fewer participants used MT than CPT, the global effect of MT and CPT on clinical decisions was similar; 33/137 participants reported that MT affected their clinical decisions, 35/137 participants reported that CPT did so.
Estimated value of molecular tests—The median estimated value of hypothetical tests that provided reliable patient-specific recurrence and response data was $1000 (Table 6). The majority of participants concluded that such a test could save the health service money, with no significant difference between the scenarios offered.
Table 6. Participants estimated the value of hypothetical molecular tests in response to clinical scenarios, and whether such tests could reduce health costs
Estimates of test worth are median value in NZ$ (interquartile range). All other data are number of participants (percentage). AML=acute myeloid anaemia.
Predictions for the future – All participants (n=137) were asked to predict the change in impact of MT and CPT on the care of patients with cancer over the next decade. Over 85% of participants, whether or not they currently used these tests and tools, predicted that they would have a greater influence and a stronger evidence base within the next 10 years (Table 7).
Table 7. Predicted change in the influence and impact of molecular tests and computerised prognostic tools on the management of patients with cancer over the next 10 years
This study has elucidated the use of molecular tests (MT) and computerised prognostic tools (CPT) by 137 clinicians treating patients with solid organ and haematological malignancy in NZ, the factors that limit their uptake and their predicted impact over the coming decade. For each point below we will first draw conclusions from our data and then discuss the potential role of MT and CPT in NZ cancer care.
Survey response rate—The ‘click through’ response rate to our survey was 25% (186/739); most clinicians who visited the survey completed it (137/186, 72%). However the figure of 186 responders to 739 invitations may significantly underestimate response due to difficulties in accurately determining the number of eligible participants. Some members of the relevant colleges and professional societies are members of more than one organisation (e.g. haematologists may be members of both RACP and RCPA), others are currently practicing overseas and are likely to have determined that they were ineligible to participate prior to accessing the survey’s website. Participation was unevenly distributed amongst the invited specialities; a significantly smaller proportion of invited pathologists participated than clinicians invited other specialties.
In order to maximise participation we utilised strategies that have been found effective including reminder notices and incentivisation;15 participation was modest nonetheless. Studies have found that clinicians have the lowest survey response rate of all health care providers,16 with Australasian physicians less likely to participate than their international colleagues.17 It has also been shown that response rates to electronic surveys vary widely, from 0.1%18 to 83%,19,16 but tend to be lower than to postal surveys.18 Reviews of survey-based research have commented that surveys with low response rates can provide useful and representative data.16 We are therefore confident that our data is a helpful contribution to this field.
Current use—We found that MT and CPT currently influence the treatment offered to a significant number of patients with cancer in NZ; our data suggests that the care of up to 80% of patients with CN-AML is impacted by the use of FLT3 mutation analysis and that the care of up to 40% of women with early breast cancer is impacted by the use of the CPT Adjuvant!. 67-73% of participants who used these technologies believed that they positively impact patient outcomes. Overall a greater number of participants were aware of MT than were aware of CPT, but CPT were more commonly used.
It is interesting to speculate on the factors that may explain this difference. We propose that awareness of MT may be enhanced by the larger number of publications about them than about CPT (8,600 versus 159 PubMed-referenced publications in 2010) and by the effort of manufacturers to raise the profile of some expensive MT within Australasia.
The MT discussed in this paper range in cost from around $300 per patient for FLT3 mutation analysis testing (Canterbury Health Laboratories, http://www.labnet.co.nz/testmanager/) to around $4500 per patient for MammaPrint (personal communication with Ronald van Klaveren, Agendia, March 2011). In contrast we suggest that the greater uptake of CPT may be because they are often available free of charge and can be accessed using computer hardware and software commonly available in clinical settings.
Use of MT and CPT may also be influenced by their inclusion in current clinical guidelines. For example FLT3 testing for patients with CN-AML is recommended in the current WHO guidelines6 and was used by 85% of participants who treat this malignancy. In contrast, MammaPrint, which was used by only 2% of participants who manage breast cancer, is not mentioned in NZ’s Early Breast Cancer Guidelines.20
64 to 78% of participants estimated that the use of hypothetical MT might reduce healthcare costs even at prices that would significantly increase the cost of pathological assessment.21 In the USA, industry-associated studies have previously calculated that use of MammaPrint22 and Oncotype DX23 may indeed reduce healthcare costs. However, some may argue that assessing the economic value of MT in NZ may be premature before more robustly establishing their ability to improve patient outcomes.24,25
Future use—Nearly all clinicians forecast that MT and CPT will be used more frequently and will have a greater influence on clinical decisions within the next decade. Participants predicted that this increased impact and influence would be supported by a stronger evidence base and greater ability to improve patient outcomes. Less than 1% of respondents believed that these tools would become less important over the next 10 years.
Discussion of the role of MT and CPT in NZ cancer care—This survey showed that clinicians are currently using MT and CPT to make clinical decisions about patients with cancer in NZ and have great expectations for their increasing contribution over the next 10 years. It also suggested that a subset of clinicians saw the relative lack of research into the effect of MT or CPT on patient outcomes as limiting MT or CPT uptake. MammaPrint, Oncotype DX and other MT that have not yet completed prospective trials are currently influencing patient care in this country.26-28 The NZ Cancer Control Strategy supports an evidence-based approach to the management of patients with malignancy.29 Therefore, we would like to suggest that high quality research evaluating the effects of MT and CPT on patient outcome should be a priority. This view is backed by overseas studies, which have found that some MT have worrying variations in their technical use,30 that others are marketed before a convincing evidence base has been assembled31 and that the clinical evaluation of some MT and CPT has lagged behind the technological leaps that have allowed these tests to be used.32,33
Defining the role of MT and CPT in NZ cancer care requires input from a wide range of clinical specialists and scientists. Pathologists were under-represented amongst survey respondents, yet their involvement in a multidisciplinary effort to integrate traditional histopathology with developments in the molecular understanding of cancer can not be overestimated.34 For example, Cummings et al stress that new MT will only produce maximal clinical benefit for patients with breast cancer if they are used by pathologists as an adjunct to their existing armamentarium.35
In conclusion, our survey suggested that MT and CPT already influence treatment provided to NZ cancer patients and that NZ cancer clinicians overwhelmingly expect their use and influence to increase. This has important clinical and health economic implications for NZ. Although these technologies may represent exciting opportunities to improve cancer care and patient outcomes it seems important that their use is supported by high quality evidence and that research is undertaken into their effects on both patient outcome and future health resource utilisation.
As with any health care intervention, MT and CPT cannot be considered in isolation, but rather should be considered as elements of a co-ordinated strategy that includes primary prevention, early referral, screening, and optimal specialist management to improve the quality of cancer care in NZ.
Competing interests: None.
Author information: Deborah M Wright, PhD Student;1,2 Rob McNeill, Senior Lecturer;3 Arend EH Merrie*, Colorectal Surgeon, Honorary Clinical Senior Lecturer;2,4 Cristin Print*, Associate Professor of Pathology.1,5
1Department of Molecular Medicine and Pathology, University of Auckland, Auckland.
2Department of Surgery, University of Auckland, Auckland.
3School of Population Health, University of Auckland, Auckland.
4Colorectal Unit, Department of Surgery, Auckland City Hospital, Auckland.
5Bioinformatics Institute, University of Auckland, Auckland.
*Joint Senior Authors.
Acknowledgements: This work is supported by the Newmarket Rotary Charitable Foundation Oncology Award, the Foundation for Surgery NZ Research Fellowship, and the Foundation for Research, Science and Technology.
The authors also thank Professor Peter Browett for his technical advice; Dr Ben Lawrence for his assistance with survey development; Gail Le Claire and Helena Cox of the RANZCR, Sue Jansen of the RCPANZ, Heather Rosser of the RACP, and Bronwen Evans of the NZAGS for distributing invitations to participate to their members; and all those who completed our questionnaire.
Correspondence: Dr Deborah Wright, Department of Molecular Medicine and Pathology, Faculty of Medical and Health Sciences, University of Auckland, Private Bag 92019, Auckland, 1142, New Zealand. Fax: +64 (0)9 3737492; email: email@example.com
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals