Journal of the New Zealand Medical Association, 21-November-2003, Vol 116 No 1186
Randomised trials in general practice – a New Zealand experience in recruitment
Ann Pearl, Susan Wright, Greg Gamble, Robert Doughty and Norman Sharpe
There is a well-recognised need to carry out trials in the primary care setting.1–4 Results of randomised trials from secondary and tertiary healthcare environments may not have relevance in the primary sector where the majority of patient contacts occur.3,4 Increasingly, trials that examine tests and interventions may be conducted in primary care in order to develop evidence-based healthcare in this setting. However, it has been well acknowledged that conducting trials in primary care is difficult,2–6 and not all trials are successful.1,3
Gaining access to primary care settings,7 and achieving good levels of patient recruitment,2,3,5,6 are key components in an effective randomised trial and are invariably dependent on general practitioner (GP) participation. It has been shown, however, that GPs may not have a great interest in research.8 Strategies that facilitate GP and patient involvement in clinical trials are therefore essential. Overseas studies have suggested that the following factors are important in determining GP participation: clinically relevant study,2,6 personal approach by study investigators,2,4,7 perceived benefit for GPs and patients,4,6 no interference in the doctor–patient relationship,6 simple and quick study documentation,2,6 effective communication and support from study personnel,2,4,6 and appropriate financial reimbursement.4,6,9
There have been few reports of the experience of conducting randomised clinical trials in primary care in New Zealand. This paper examines GP and patient recruitment issues in a clinical trial based in Auckland.
A randomised controlled trial, the Natriuretic Peptides in the Community Study,10 was developed to investigate the usefulness of brain natriuretic peptide (BNP) measurement in the diagnosis of heart failure in the community setting. Briefly, patients aged 45 years or more who presented to their GP with symptoms of shortness of breath and/or oedema were eligible for the study. At this initial consultation the GP made a clinical diagnosis of whether or not heart failure was present. The GP initiated treatment as necessary. Patients were then referred by GPs to the hospital study centre where they had a full cardiological evaluation including clinical assessment, electrocardiogram (ECG), chest X-ray, echocardiography and measurement of BNP. Patients were randomly allocated as to whether or not their GP received the BNP result. Each patient then returned to his/her GP for a final review where the GP revised the diagnosis with or without the BNP result. For study purposes the decision as to whether or not heart failure was present was made by a panel of three cardiologists and one general physician who were blinded to the BNP result. The primary study end point was to compare the accuracy of the final diagnosis between the two groups of GPs. The overall study result was that diagnostic accuracy improved 21% in the BNP group compared with only 8% in the control group,10 demonstrating that BNP measurement can significantly improve the accuracy of the diagnosis of heart failure by GPs.
The trial required patients presenting with shortness of breath and/or oedema to be recruited from primary care. A multi-faceted approach was used to recruit patients. First, a small number of local GPs were consulted to estimate the number of patients with these symptoms presenting to a GP and to discuss aspects of the proposed study. A comprehensive list of Auckland GPs (excluding those based in accident and medical clinics) was compiled using a commercial database supplemented by the local phone directory and phone calls to practices. GPs were then approached on a suburb-by-suburb basis. Each GP was initially sent a letter. The letter outlined the study aims and requirements for participating GPs and patients, described patients who might be eligible for the study, and discussed benefits of participation for GPs and patients. One of the study investigators phoned each GP approximately one week after the letter had been sent to determine if the GP was eligible to participate (working more than four tenths) and interested in taking part. Each eligible, interested GP was visited by one of the study investigators. At this visit the study and the involvement of the GP and patients were explained in more detail. The GP was given a study pack that included study documentation, patient information sheets, the Heart Foundation heart failure guidelines, and information regarding BNP.
Each participating GP was asked to identify patients suitable for the study at normal consultations. Suitable patients were those aged 45 years or more who presented with shortness of breath and/or oedema, and who had not been admitted to hospital for heart failure or had a cardiological assessment in the previous year. The GP gave each eligible patient a brief explanation of the study and provided a patient information sheet. The GP completed a simple study documentation sheet that was faxed to the study centre. Each eligible patient was then seen at Auckland Hospital for independent cardiological evaluation. The GP had no further involvement in the study until approximately two weeks later when the GP reviewed the patient with or without a BNP result. At this time, final study documentation was completed.
Once the final review for each patient was completed the GP received a full cardiological report including echocardiography, ECG, chest X-ray, and clinical assessment. GPs received a payment (NZ$150.00) for each patient they enrolled in the study. This payment was reimbursement for time spent on study matters and the cost of the final consultation as this was to be free to the patient. Participating GPs (with their consent) were acknowledged as co-investigators. The study was also approved by the Royal New Zealand College of General Practitioners so that GPs could receive MOPS (Maintenance of Professional Standards) points for their involvement in the study. Participating GPs also received bimonthly, one-page newsletters to update them on study progress and inform them of new, relevant developments in heart failure management.
Following completion of the study all GPs who had agreed to participate were sent an evaluation questionnaire to complete. There were different questionnaires for those GPs who had referred and those who had not referred patients. Both questionnaires consisted of rating scales to determine GP attitudes to aspects of the study and research in general, with some items common to both questionnaires.
The study took place in Auckland and the Auckland Ethics Committee approved the study. GP recruitment began in November 1999 and continued until April 2001. Patient recruitment began in November 1999 and final patient data collection was completed in November 2001.
Statistical analysis Comparison of the characteristics of recruiters and non-recruiters was made using Wilcoxon unpaired test for continuous variables and Fisher’s exact test for categorical variables. The questionnaire was designed to facilitate multivariate analysis with a trunk statement – ‘Overall this has been a good study to be involved in’ – and branches addressing study communication, study organisation, patient involvement, and GP participation. Within each branch a number of specific points were tested by presenting a statement and then asking the respondent to answer with a rating from 1 to 5 (1 = strongly disagree, 5 = strongly agree). A second questionnaire was developed for GPs who agreed to participate but who were unable to refer any patients. Both questionnaires were analysed using a variety of iterative (stepwise, forward and backward selection), multivariate regression procedures; consistency and parsimony were sought from amongst the models. A probability of 0.15 was considered sufficient for inclusion in the model; however, the threshold for statistical significance was set at 5%. All analyses were performed using SAS (SAS Institute Inc).
The introductory letter was sent to 327 GPs from 135 practices. All these GPs were contacted by phone; 294 were eligible to take part in the study and 186 GPs (63% of eligible GPs) from 92 practices agreed to participate. From these participating GPs, 92 (31% of eligible GPs) from 62 practices referred a total of 307 patients to the study (Figure 1).
Figure 1. General practitioner participation in the Natriuretic Peptides in the Community Study
The GPs who agreed to participate and those who actually referred were very similar to the total group of GPs approached for the variables measured. Of the GPs approached, 59% (193/327) were male and 9% (29/327) were in solo practice. In the group of GPs that agreed to participate 61% (113/186) were male and 9% (17/186) were in solo practice. Similarly, in the group of GPs that actually referred, 58% (53/92) were male and 7% (6/92) were in solo practice (Table 1). There was no significant difference between the two groups – GPs approached and referring GPs – with respect to median years since graduation (p = 0.77) (Table 1).
Table 1. Sociodemographic characteristics of GPs involved in Natriuretic Peptides in the Community Study
IQR = inter-quartile range
Of the GPs who agreed to participate, 51% (94/186) did not refer any patients to the study (Table 2). The remaining 92 GPs referred 307 patients with a median of one patient per GP (range 1–14). A maximum of 10 patients per GP had been set but five GPs exceeded this number before they could be advised that the limit had been reached. Two female practitioners referred the largest number of participants, 14 each, but overall male GPs referred more patients to the study. Participating GPs began their involvement in the study at different times as a result of the rolling recruitment process. The participating GPs were involved in the study for a median of 332 days (inter-quartile range 161,452).
Table 2. Number of patients referred by GPs who agreed to participate
Evaluation questionnaire The response rate for the questionnaire sent to the referring GPs was 64% (59/92) and 27% (25/94) for the questionnaire sent to GPs who agreed to participate but did not refer. Of the referring GPs, 97% (57/59) agreed or strongly agreed that GPs should participate in research and 93% (55/59) agreed or strongly agreed that the Department of General Practice (in this study at University of Auckland) should be involved in research based in general practice (Table 3). Eighty five per cent (50/59) of referring GPs agreed or strongly agreed that GPs should be reimbursed for involvement in trials and 46% (27/59) agreed or strongly agreed that they could not participate without reimbursement (Table 3).
Table 3. Items from process evaluation questionnaire for referring GPs (n = 59)
The provision of MOPS points was not an important factor for referring GPs when deciding to participate in the study, while the bimonthly newsletter was helpful for 80% (47/59) of referring GPs (Table 3). Of the referring GPs, 97% (57/59) found it a good study to be involved in (Table 3), with multivariate analysis showing that overall satisfaction was independently related to the involvement of the Department of General Practice (partial r2 = 25%) and patient benefit (partial r2 = 17%).
Although the response rate for non-referring GPs was low (27%) similar responses were seen as those for the referring group: 92% (23/25) agreed that it was important that GPs participated in research; 76% (19/25) agreed that GPs should be paid for involvement in trials; and 36% (9/25) stated that they could not participate without reimbursement. The main reason for not referring was having no patients who met the study criteria.
This study, a randomised controlled effectiveness study demonstrating the positive effect of BNP measurement on the accurate diagnosis of heart failure in primary care, showed that randomised trials can be successfully conducted in the primary care setting by working collaboratively with GPs and paying consideration to their needs. It was essential that this study was executed in the primary care sector where heart failure can be very difficult to diagnose and access to investigations can be limited. Ninety two GPs from 62 practices successfully recruited 307 patients for the study. This, however, represents the involvement of approximately one third of all eligible GPs approached and half of those GPs who agreed to participate. These figures support findings from overseas studies that have shown that many GPs who agree to participate in trials do not subsequently recruit patients. Peto et al, in their trial involving patients with menorrhagia, found that 41% of GPs who agreed to participate actually referred patients.5 Other researchers have reported recruitment rates ranging from 8% of GPs who agreed to participate to 66%.1,6
Our finding that a common reason for GPs not referring was that they had no patients who met the study criteria is also echoed in other studies.2,5 Other reported reasons for non-recruitment include forgetfulness5,6 and time pressures.4–6 These factors did not appear to be so important for GPs involved in this study. One weakness of our study is that although patients were referred prospectively we do not know how many eligible patients actually presented to their GPs but were not referred. However, Peto et al found in a notes review of some of the practices involved in their study that only 20% of eligible patients had actually been referred.5
Our decision to reimburse GPs financially was initially questioned by the external study reviewers. However, this strategy clearly had significant support from the GPs, with 85% of referring GPs agreeing with the principle that GPs should be reimbursed for participation in trials. Silagy and Carson reported in 1989 that Australian GPs gave financial reward a low rating as a reason for participating in research.8 However, they acknowledged that financial reward may be important in the context of a busy GP’s life. The need for appropriate financial reward is increasingly acknowledged and accepted.2,6,9,11 Arguably, the acceptance of financial reimbursement in the New Zealand setting should be no different.
This trial attempted to maximise patient recruitment by using techniques and strategies that had been recommended by other researchers, and provided an opportunity to evaluate the effectiveness of these strategies with a post-study questionnaire. The use of these strategies, which included early consultation with GPs at the time of study design, a relevant study with good benefits for participating GPs and patients, and effective communication, resulted in successful recruitment. However, the study also demonstrated that many GPs who agree to participate will not recruit patients. Researchers carrying out trials in primary care may need to identify and work more closely with this group early in the course of the trial.
Primary care in New Zealand will in all likelihood continue to develop as an important research environment. The success of trials in this sector will depend on careful and sensitive planning with cooperation and collaboration between primary healthcare professionals and research teams.
Author information: Ann Pearl, Research Fellow, Department of General Practice and Primary Health Care; Susan Wright, HRC Trainee Fellow; Greg Gamble, Biostatistician; Norman Sharpe, Emeritus Professor of Medicine; Robert Doughty, Senior Lecturer in Cardiology, Department of Medicine, University of Auckland.
Acknowledgements: This trial was supported by grants from the Health Research Council of New Zealand and the New Zealand National Heart Foundation.
We acknowledge the participation of Auckland general practitioners who were co-investigators in this study (the NPC Study GP Collaborative Group):
Dr Abeysekera, Dr TR Akroyd, Dr SP Barclay, Dr RMD Batt, Dr JB Buckley, Dr HA Budelmann, Dr CM Built, Dr S Calthrop-Owen, Dr JG Carter, Dr R Chan, Dr SA Chand, Dr S Chunilal, Dr JD Clark, Dr PJ Clemo, Dr G Collinson, Dr PA Cotton-Barker, Dr SC Crerar, Dr SD Cutmore, Dr D Dalziel, Dr DM De Lacey, Dr GJ Desborough, Dr MI Eade, Dr TN Farquharson, Dr JC Fetherston, Dr J Frater, Dr G Ganesh, Dr J Gardner, Dr HM Gardyne, Dr DI Gibson, Dr DJ Gillanders, Dr DM Good, Dr RK Haydon, Dr NJH Hefford, Dr C Hong, Dr G Houng-Lee, Dr HI Jenkins, Dr L Johnson, Dr N Kerse, Dr W Khoo, Dr H King, Dr K Large, Dr RW Leitch, Dr A Leong, Dr R Levenberg, Dr C Lodder, Dr MV Long, Dr DBH Loos, Dr M Loten, Dr SA Newman, Dr PF Nola, Dr HM MacDonald, Dr BE McKinney, Dr IA Maclean, Dr RAP Mok, Dr S Moller, Dr JJ O'Sullivan, Dr N Patel, Dr R Potts, Dr N Rasalingam, Dr GW Robertson, Dr TA Robinson, Dr JJ Rosby, Dr BW Roy, Dr C Sanders, Dr A Sharma, Dr V Sharma, Dr KL Settle, Dr R Sood, Dr JRG Stewart, Dr RN Stirling, Dr MK Stone, Dr AG Svensen, Dr JCO Tseung, Dr SJ Turner, Dr AN Twhigg, Dr B Tye, Dr MD Wah, Dr RH Wallace, Dr G Wardrope, Dr HPV Waterfall, Dr RK Watt, Dr S Weeramuni, Dr AP Williams, Dr P Woolford, Dr L Young, Dr L Zhuang
Correspondence: Dr Ann Pearl, Department of General Practice and Primary Health Care, University of Auckland, Private Bag 92019, Auckland. Fax: (09) 373 7006; email: email@example.com
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals