The priorities for future clinical trials and large cohort studies addressing health and healthcare for mothers and babies in Aotearoa New Zealand
View Article PDF
Multicentre clinical trials and large cohort studies provide the most effective platforms to answer clinically important research questions that lead to improved health. The efficiency and cost-effectiveness of these types of research are significantly enhanced by well organised and co-ordinated clinical trials networks.[[1]] The ON TRACK Network: Better Health for Mothers and Babies, Te Awhi Rito was established in 2016 (http://ontrack.perinatalsociety.org.nz/) to be an Aotearoa New Zealand specific multidisciplinary maternal and perinatal clinical trials network. It aims to engage researchers, clinicians, consumers and other stakeholders in a systematic and constructive way across the whole pipeline of clinical research, with an ultimate goal to achieve greater equity, better health outcomes and improved quality of life for women and babies nationwide.
To enable future high quality multicentre clinical trials and large cohort studies, while ensuring the most efficient use of resources and maximum benefit, one of the first steps is to prioritise research questions.[[2]] Over recent decades, systematic approaches to research prioritisation have been developed.[[3–9]] However, no single gold standard research prioritisation method exists, as each method has advantages and disadvantages depending on the goals of the prioritisation project and the setting in which it takes place.
The aims of this project were to establish a Research Prioritisation Framework that was robust, transparent, systematic, equity-focused, and able to consider the unique social and geographic context of maternal and perinatal health and healthcare in Aotearoa New Zealand. The aim was then then to use this framework to establish the future priorities for clinical trials and large cohort studies in addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Methods
Development of a Research Prioritisation Framework
Established health research priority-setting methods were identified and reviewed, including: the Council of Health Research for Development;[[3]] Delphi studies;[[4]] Essential National Health Research;[[5]] 3D Combined Approach Matrix;[[6]] the Child Health Nutrition Research Initiative;[[7]] the James Lind Alliance;[[8]] and, within New Zealand, the Health Research Council’s ‘The New Zealand Health Research Prioritisation Framework’.[[9]] Each was assessed for quality using a recognised good practice tool,[[10]] the resources required per method, ability to engage with multiple disciplines and groups, applicability to Aotearoa New Zealand, and overall strengths and weaknesses. Considerations for the Aotearoa New Zealand context included the ability to be responsive to the needs of the Indigenous peoples, the ability to address equity, the opportunity for consumer participation, and the suitability for the New Zealand maternity care system in which the majority of pregnancy care is provided by independent practitioners. Using this information, we developed an Aotearoa New Zealand specific Framework that included 11 steps in four phases, involving four groups of participants: the Core Group, Audience Group, Advisory Group and Ranking Group (View Figure 1).
Information gathering
Maternal and perinatal health and healthcare was considered by the Core Group to include: before pregnancy (where there is impact on maternal and perinatal health outcomes); during pregnancy and antenatal care; at the time of birth; maternal postnatal care; and newborn health. Infertility was not included because a recent international consensus development study identifying top priorities for future infertility research, led by New Zealand investigators, has reported on issues relevant to Aotearoa New Zealand.[[11]] Knowledge gaps were collected from the Audience Group, and research priorities already identified were collected by a systematic review of published research prioritisation projects.
To enable a large and diverse group of individuals to participate in the Audience Group, we planned to use online semi-structured questionnaires and face-to-face (kanohi ki te kanohi) hui and focus groups. However, due to COVID-19-related restrictions, in-person group meetings were not feasible. We therefore used video and telephone semi-structured interviews for those who wished to interact in person. Some one-on-one in-person interviews were possible towards the end of the information gathering phase.
Invitations to contribute to the Audience Group were distributed via email and newsletters to: all ON TRACK Site Network leaders at each district health board participating in the Network; professional bodies of maternity and perinatal healthcare providers; consumer groups; Māori and Pacific maternity health groups; funders of research; academic institutions undertaking research; relevant Ministry appointed committees; and, directly to identified Māori and Pacific health researchers. Each group was asked to distribute the invitation to participate via membership circulation lists. Invitations, poster displays and social media posts included introductory information, contact details and links and QR codes to the online questionnaire.
Participants eligible to contribute to the Audience Group were 16 years or older, living in Aotearoa New Zealand, and belonging to at least one of three groups:
1. Healthcare professionals involved in the provision of care for pregnant women and their offspring in Aotearoa New Zealand, ie midwives, obstetricians, neonatologists, paediatricians, neonatal nurses, obstetric physicians and other allied health professionals.
2. Consumers involved or recently involved with pregnancy, defined as: woman with current or recent experience (within the last five years) of maternity healthcare (including preconception, pregnancy, pregnancy loss, birth, postnatal and lactation care); partner, whānau and support persons of a woman with current or recent experience in maternity healthcare; caregiver of a newborn/infant/child born within the last five years; or, consumer representative of a group related to maternal and perinatal health.
3. Other stakeholders involved in maternal and perinatal health and healthcare research such as scientists, researchers, funders, policy-makers and members of Ministry appointed committees.
Qualtrics® software was used to host the online questionnaire. Demographic data was requested from each participant, together with open-ended questions to invite free text responses relating to knowledge gaps before pregnancy, during pregnancy and antenatal care, at time of birth, postnatal care and newborn health. Responses were not linked to individuals. Two members of the project management team undertook all interviews using the same approach, and these were then recorded and transcribed.
The Collection of Knowledge Gap Study nested within the Research Prioritisation Project was approved by The University of Auckland Human Participants Ethics Committee (024469), April 2020. Participants gave written, informed consent for interviews. Submission of the questionnaire provided a record of consent for the online questionnaire.
Knowledge gaps were also identified by a systematic review of all prioritisation studies that included women and/or their offspring published since 2009. Only those specific to an Aotearoa New Zealand setting were included.
Content analysis
Knowledge gaps generated by all three methods (survey, interview and systematic review) were reviewed by the project management team. The depth and breadth of research ideas were considered as research avenues, options and questions.[[12]] Using clinical and research knowledge, specific research questions were assigned as already answered or unanswered. Unanswered questions that still require new knowledge were formulated into practical research questions, assigned to one of four domains (delivery, development, discovery, description)[[12]] and assessed for their suitability to be answered in a randomised clinical trial or a large cohort study.
Research questions suitable for assessment by these two methods were further analysed to consider: to whom the “intervention” would be applied (mother, offspring, mother and offspring, family/whānau, healthcare professional and healthcare system); the timing at which the “intervention” would be applied (preconception, in pregnancy/foetal, during birth, neonatal—preterm baby, neonatal—unspecified or term baby, postpartum—maternal or, unspecified/crossover); the intended health beneficiary (mother, offspring, mother and offspring); and, the most suitable Advisory Group member disciplines for review (up to four disciplines per research question including anaesthetics, epidemiology/public health, Māori health, maternal mental health, midwifery, primary care/general practice, neonatology, neonatal nursing, obstetric medicine, obstetrics, Pacific health, science, reproductive health, other allied health). Each question was specifically considered with regards to relevance to Māori and Pacific health (if the research question only included Māori and Pacific people, or if the health condition included was one where Māori and Pacific people are over-represented).
Once the analysis of all research questions was complete, the results were reviewed by two principal study investigators, any discrepancies in judgement were resolved by discussion. Closely related research questions were combined where feasible without impacting on the original meaning of the knowledge gap. Research questions generated from the Audience Group by survey and interview were then grouped by discipline, and circulated to the appropriate Advisory Group members via email.
Advisory Group members were asked to review their allocated research questions and consider whether this remained a genuine knowledge gap, to provide a reference to the evidence if this was no longer a knowledge gap, and to review the wording to ensure clinical relevance and understanding. Responses were collated and reviewed by two principal study investigators. Where discrepancies were identified, additional literature searches were undertaken, and research questions were retained unless there was clear evidence that the knowledge gap no longer existed.
Ranking
Members of the Ranking Group received written information including project background and ranking methodology before an online meeting, in which an overview of maternity and perinatal healthcare in Aotearoa New Zealand, and some of the challenges faced, were presented alongside an overview of the prioritisation framework. Time was allowed for discussion, including smaller discipline-based breakout groups.
The ranking of research questions was completed in two steps by Multi-Criteria Decision Analysis (MCDA) using an established online decision-making tool, developed by 1000minds Ltd. through the University of Otago (https://www.1000minds.com). To first determine the weighting of the different criteria, five rating criteria were selected from the Child Health and Nutrition Research Initiative (CHNRI) Guidelines for Implementation of CHNRI Method for Setting Priorities in Global Child Health Research Investments,[[7]] and adapted to suit maternal and perinatal health research in Aotearoa New Zealand (View Figure 2). The Ranking Group completed the weighting survey which used the PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) method to determine the relative importance of each criterion.[[13]]
Ranking Group members were then asked to review each research question against the five rating criteria as high, medium-high, medium, low-medium or low, in importance. They were supplied with detailed descriptors of each criterion to support their assessments (Appendix 1). To limit bias and the effect of reviewer fatigue, the order in which questions were presented to each reviewer was allocated in a random manner. Rating of research questions was undertaken independently by each reviewer following the online meeting. The content of individual research questions was not discussed by the Ranking Group.
The 1000minds software used reviewer responses, and applied the relative weighting for each criterion assigned by the weighting survey to score and generate a ranked list of research questions. The ranked list was reviewed by the Core Group and the top 49 were grouped by themes (models of care/service provision, workforce, and specific interventions/conditions).
Results
The Research Prioritisation Framework (Figure 1) developed includes: key features of a transparent, structured approach; Māori participation at all levels and all steps;[[14]] an equity lens applied at each step;[[15]] wide engagement with diverse stakeholders; women and whānau (consumer) participation at all levels and all steps;[[16]] methods to accommodate cultural needs and location; and, use of a validated metric-based decision-making tool. The Framework includes 11 steps, in 4 phases, including 4 groups of participants.
Invitations to join the Audience Group were distributed to: 17 professional colleges and societies; 7 funding agencies; 5 Māori and Pacific specific health groups and agencies; 19 consumer groups, 6 academic units/departments; 16 individual Māori health contacts; and, via 3 national newsletters. The online questionnaire was accessible from 1 May to 1 August 2020; 305 questionnaires were submitted. The majority of responders were New Zealand European/European (235, 77%), 29 (9.5%) were Māori and 4 (1.3%) were Pacific. Responses were received from all geographical regions and included a variety of stakeholder groups including 147 (48.1%) healthcare professionals, 122 (40.0%) consumers and 36 (11.8%) other stakeholders (Table 1). Interviews were offered from 1 May to 14 August 2020; 62 interviews were completed. The majority of interviewees were New Zealand European/European (39, 62.9%) but a higher proportion were Māori (12, 19.4%) and Pacific (8, 12.9%), in comparison to the online questionnaire. Interviewees covered a variety of stakeholder groups including 34 (54.8%) healthcare professionals, 21 (33.9%) consumers and 7 (11.3%) other stakeholders (Table 1). More than half were from the Auckland Region (34, 54.8%).
A total of 3347 knowledge gaps were identified: 1610 from online questionnaires, 892 from interviews, and 845 from the systematic review of 62 eligible research prioritisation projects. After content analysis, 358 unanswered research questions were considered potentially able to be answered in a randomised clinical trial or large cohort study and entered the Ranking phase (Table 2). Of these 358 research questions 175 were identified by on-line questionnaires, 115 by interviews and 68 by the systematic review.
Twenty-nine individuals contributed to the Advisory Group and a further 23 agreed to join the Ranking Group (52 in total), of whom 47 (90.0%) completed the weighting survey and 46 (88.5%) completed the rating survey. Six Ranking Group members (11.5%) identified as Māori, but only two (4.3%) completed both surveys. All three Pacific Ranking Group members completed the weighting (5.8%) and rating surveys (6.5%) (Table 3). Representation by regions was focussed to areas including academic units and involved all the major professional health disciplines that provide maternal and perinatal care (Table 3).
View Tables 1, 2 & 3.
The mean weights generated for the five rating criteria were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. Ranking Group members reviewed a mean of 239 (range 2–358) of the 358 questions against the five rating criteria. The total score generated, and the ranking of each research question, are shown in Appendix 2. The top 49 ranked research questions grouped by themes within each ranked score group are shown in Table 4. Of the 20 research questions ranked highest, 13 related to models of care and service provision, five related to the health workforce and only two to specific interventions (smoking cessation programmes and maternal mental health support); 14 were directly specific to Māori, two to Pacific peoples and four to both Māori and Pacific peoples, covering interventions for healthcare systems (16), the mother (3) or, healthcare professionals (1); and all having both the mother and offspring as intended beneficiaries of the intervention.
View Table 4.
Discussion
We developed a new Aotearoa New Zealand specific Research Prioritisation Framework and used this to identify and answer the most important research questions through randomised clinical trials or large cohort studies in maternal and perinatal health and healthcare. This is the first systematic health-area-wide research prioritisation project undertaken for Aotearoa New Zealand. The developed Framework includes all the steps and recommended questions in a checklist for good practice[[10]] and enabling an equity-oriented approach,[[15]] as well as all the critical steps, which have since been identified in the recently published Australian Clinical Trials Alliance (ACTA) guideline for clinical trial networks interested in setting priorities for the conduct of clinical trials.[[2]]
We were successful in addressing the majority of key features planned in our framework. Key stakeholder groups contributed to the Core, Advisory and Ranking groups and a diverse group of stakeholders were included in the Audience Group. Consumer participation was included in all steps of the process with the exception of the Advisory Group consultation, as this was deemed to be an area where clinical practice and evidenced-based knowledge was required.
The use of two different methods for the collection of knowledge gaps was intended to accommodate cultural needs and different locations of potential participants. On-line questionnaires provided an efficient and cost-effective method of data collection. This allowed participation by a large number and variety of stakeholders, including those who may prefer to participate anonymously or are geographically distant, using their own choice of timing and location. The inclusion of face-to-face data collection has been demonstrated to be effective in enabling participation by healthcare consumers in research[[17,18]] and kanohi ki te kanohi hui were also expected to support and encourage participation by Māori.[[14]] Although this could not be undertaken due to COVID-19 restrictions, the higher rates of Māori and Pacific peoples’ participation in the collection of knowledge gaps via interview suggests that the use of video and telephone calls still provided some level of personal connection, thereby supporting their willingness to participate. However, it is also possible that a more targeted approach to include Māori and Pacific people was possible by using interview rather than survey methodology.
In designing the Research Prioritisation Framework, equity was considered a key feature, and essential to supporting the vision of the ON TRACK Network “to achieve greater equity, health outcomes, and quality of life for women and babies nationwide”. Consideration of an equity lens, as developed to ensure an equity-oriented approach to agenda and priority setting of Cochrane Reviews,[[15]] was utilised at each step of the project. The importance of equity was further supported by the inclusion of the rating criteria “effect on equity” as part of the ranking process, which was found to be the criterion most heavily weighted by the Ranking Group (26.1%), and hence made the most significant contribution to overall research question scores. This approach appears to have been effective in prioritising equity focussed research questions, since all of the top 20 prioritised research questions related specifically to Māori and/or Pacific peoples’ health and healthcare, and predominantly focussed on models of care and service provision. Within the top 49 prioritised research questions, a further 12 were specific to Māori and/or Pacific people. The remainder were mainly focussed on specific health conditions where Māori and/or Pacific people are disproportionately over-represented (mental health, obesity and diabetes in pregnancy) or inequities existing due to social disadvantage, poverty, access to care, rural location or high needs communities.
Through the ranking process it was highlighted that equity issues, and meeting the obligations of Te Tiriti o Waitangi, must be considered separately. We were able to achieve Māori participation at all levels and all steps of the process but we were not able to achieve equal partnership with Māori (50% of all participants), or indeed, reach levels that are representative of Māori within the Aotearoa New Zealand population (16.7%, NZ Stats 2020)[[19]] with only 11.2% of the Audience Group and 11.5% of the Ranking Group identifying as Māori, and only two of the six Māori participants in the Ranking Group completing both the weighting and rating surveys. This partly related to the lack of capacity for overstretched Māori healthcare workers and health researchers, but also to reservations relating to the framework, which by the nature of its design does not follow Kaupapa Māori research theory and practice. It was suggested that a Māori-led and -focused future research project should be considered. However, this may not provide opportunities to directly compare, measure and prioritise research for Māori against research for non-Māori or the general population, and would currently be significantly limited by capacity of those Māori working in this area. The prioritisation project itself identified the need for further research on how to develop and support our Māori healthcare and health research workforce, with two research questions within the top 12 relating to the Māori maternity and newborn health workforce.
The list of prioritised research questions will be promoted by the ON TRACK Network through peer-reviewed publication, presentation at scientific and public meetings, through the media and social media including the ON TRACK “Forum for Women and Whānau” Facebook page for consumers, and through the ON TRACK Network’s monthly national newsletter which is distributed across district health board members of the Network. The ON TRACK Network will support development of the top prioritised research questions in future activities including their annual trial development workshop. We expect that the list of prioritised research questions will be used by clinicians and researchers to develop clinical trials and large cohort studies in the future. It is also anticipated the list will attract more clinicians and researchers to this area and be of value to hospitals, consumer and advocacy groups, colleges and societies, funders and policy-makers when considering what research to support and where resources should be applied.
Further exploration of data will identify any differences in research priorities by subgroups, such as healthcare provider groups and consumers, and will provide opportunity to explore the methodology used including the impact of each criteria and their relative weightings. Furthermore, where knowledge gaps were suggested but evidence exists, there is scope for a number of research translation and implementation projects as well as national practice guidelines, with work focussed both for clinicians and consumers. A number of broader research avenues and options were also commonly identified, for example, preterm birth and gestational diabetes, and these findings support larger programmes of research, many of which may need randomised trials nested within them. Research questions identified under the domains of discovery (fundamental and inventive research) and description (epidemiological analysis) should be of interest to scientists and epidemiologists as well to clinicians and clinical researchers, funders, hospital managers and policy-makers. The use of a robust, systematic and transparent approach to prioritisation will allow later evaluation and updates.
This first systematic health-area-wide research prioritisation project, undertaken specifically for Aotearoa New Zealand, has identified the most important questions for future clinical trials and large cohort studies in maternal and perinatal health and healthcare. This Framework and methodology could be adapted for use in other areas of health and healthcare in Aotearoa New Zealand.
View Appendices 1 & 2.
Summary
Abstract
Aim
To identify priorities for clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Method
An Aotearoa New Zealand specific Research Prioritisation Framework was developed. Knowledge gaps were collected from an Audience Group via online questionnaires, video call interviews, and by systematic review. These were formulated into research questions. An Advisory Group reviewed questions suited to a clinical trial or large cohort study. A Ranking Group weighted the ranking criteria and ranked the research questions.
Results
A total of 305 online questionnaires, 62 interviews and 62 published prioritisation projects generated 3,347 knowledge gaps. After content analysis, 358 unanswered research questions were ranked. Rating criteria weightings were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. All of the top 20 prioritised research questions directly related to Māori and/or Pacific health and predominantly involved research into healthcare systems and workforce rather than disease conditions.
Conclusion
This project has identified the most important questions for future clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand. The Framework and methodology can be adapted for use across all areas of health.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) The Joint ACTA/ACSQHC Working Group: Tate R, Britton E, Webb S, et al. The value proposition of investigator-initiated clinical trials conducted by networks. Med J Aust. 2021; doi:10.5694/mja2.50935.
2) Australian Clinical Trials Alliance. Research Prioritisation Framework. January 2020. https://clinicaltrialsalliance.org.au/wp-content/uploads/2020/02/Research-Prioritisation-Framework-FINAL.pdf Accessed 7th April 2021.
3) Council on Health Research for Development (COHRED). National health research and priority setting: lessons learned; 1997. Available: http://www.cohred.org/downloads/586.pdf. Accessed 7th April 2021.
4) Hsu C, Sandford BA. The Delphi Technique: Making Sense of Consensus Practical Assessment. Research and Evaluation 2007: DOI: https://doi.org/10.7275/pdz9-th90 Accessed 7th April 2021.
5) Okello D, Chongtrakul P, COHRED Working Group on Priority Setting: A Manual for Research Priority Setting using the ENHR Strategy. Council on Health Research for Development, 2000. https://www.cohred.org/publications/library-and-archive/a_manual_for_researc_1_0/ Accessed 7th April 2021.
6) Ghaffar A, Collins T, Matlin SA, Olifson S: The 3D Combined Approach Matrix: An Improved Tool for Setting Priorities in Research for Health. Global Forum for Health Research. http://www.bvs.hn/Honduras/PIS/MEC3DEnglish.pdf Accessed 7th April 2021.
7) Rudan I, Gibson JL, Ameratunga S, et al. Setting Priorities in Global Child Health Research Investments: Guidelines for Implementation of CHNRI Method. Croat Med J. 2008; 49:720-33.
8) The James Lind Alliance Guidebook Version 10, March 2021. https://www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-Version-10-March-2021.pdf Accessed 7th April 2021.
9) The Health Research Council of New Zealand, the Ministry of Business, Innovation and Employment and the Ministry of Health. The New Zealand Health Research Prioritisation Framework. December 2019. https://hrc.govt.nz/sites/default/files/2020-01/NZ%20Prioritisation-Framework-FA-web_0.pdf Accessed 7th April 2021.
10) Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010; 8, 36. Doi:10.1186/1478-4505-8-36.
11) Duffy JMN, Adamson GD, Benson E, et al. Top 10 priorities for future infertility research; an international consensus development study. Hum Reprod 2020: doi:10.1093/humrep/deaa242.
12) Rudan I, Yoshida S, Chan K, et al. Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method. J Glob Health 2017:7 (1), 011004. doi: 10.7189/jogh.07.011004.
13) Hansen P, Ombler F. A new method for scoring additive multi‐attribute value models using pairwise rankings of alternatives. J Multi‐Crit Decis Anal 2008; 15(3‐4),87-107.
14) Reid P, Paine SJ, Curtis E, et al. Achieving health equity in Aotearoa: strengthening responsiveness to Māori in health research. NZ Med J. 2017:130(1465):96-103.
15) Nasser M, Ueffing E, Welch V, Tugwell P. An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. J Clin Epidemiol. 2013;66(5):511-521.
16) Coney S for the New Zealand Guidelines Group. Effective Consumer Voice and Participation for New Zealand. A systematic review of the evidence 2004. https://www.health.govt.nz/system/files/documents/publications/050511_consumer_report.pdf Accessed 7th April 2021.
17) Pollock A, George BS, Fenton M, et al. Development of a new model to engage patients and clinicians in setting research priorities. Journal Health Serv Res Policy. 2013:19(1):12-18.
18) MacFarlane A, Galvin R, O'Sullivan M, et al. Participatory methods for research prioritization in primary care: an analysis of the World Cafe approach in Ireland and the USA. Fam Pract. 2017: 34(3):278-284.
19) Stats NZ. Maori Population Data 2020. www.stats.govt.nz Accessed 7th April 2021.
For the PDF of this article,
contact nzmj@nzma.org.nz
The priorities for future clinical trials and large cohort studies addressing health and healthcare for mothers and babies in Aotearoa New Zealand
View Article PDF
Multicentre clinical trials and large cohort studies provide the most effective platforms to answer clinically important research questions that lead to improved health. The efficiency and cost-effectiveness of these types of research are significantly enhanced by well organised and co-ordinated clinical trials networks.[[1]] The ON TRACK Network: Better Health for Mothers and Babies, Te Awhi Rito was established in 2016 (http://ontrack.perinatalsociety.org.nz/) to be an Aotearoa New Zealand specific multidisciplinary maternal and perinatal clinical trials network. It aims to engage researchers, clinicians, consumers and other stakeholders in a systematic and constructive way across the whole pipeline of clinical research, with an ultimate goal to achieve greater equity, better health outcomes and improved quality of life for women and babies nationwide.
To enable future high quality multicentre clinical trials and large cohort studies, while ensuring the most efficient use of resources and maximum benefit, one of the first steps is to prioritise research questions.[[2]] Over recent decades, systematic approaches to research prioritisation have been developed.[[3–9]] However, no single gold standard research prioritisation method exists, as each method has advantages and disadvantages depending on the goals of the prioritisation project and the setting in which it takes place.
The aims of this project were to establish a Research Prioritisation Framework that was robust, transparent, systematic, equity-focused, and able to consider the unique social and geographic context of maternal and perinatal health and healthcare in Aotearoa New Zealand. The aim was then then to use this framework to establish the future priorities for clinical trials and large cohort studies in addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Methods
Development of a Research Prioritisation Framework
Established health research priority-setting methods were identified and reviewed, including: the Council of Health Research for Development;[[3]] Delphi studies;[[4]] Essential National Health Research;[[5]] 3D Combined Approach Matrix;[[6]] the Child Health Nutrition Research Initiative;[[7]] the James Lind Alliance;[[8]] and, within New Zealand, the Health Research Council’s ‘The New Zealand Health Research Prioritisation Framework’.[[9]] Each was assessed for quality using a recognised good practice tool,[[10]] the resources required per method, ability to engage with multiple disciplines and groups, applicability to Aotearoa New Zealand, and overall strengths and weaknesses. Considerations for the Aotearoa New Zealand context included the ability to be responsive to the needs of the Indigenous peoples, the ability to address equity, the opportunity for consumer participation, and the suitability for the New Zealand maternity care system in which the majority of pregnancy care is provided by independent practitioners. Using this information, we developed an Aotearoa New Zealand specific Framework that included 11 steps in four phases, involving four groups of participants: the Core Group, Audience Group, Advisory Group and Ranking Group (View Figure 1).
Information gathering
Maternal and perinatal health and healthcare was considered by the Core Group to include: before pregnancy (where there is impact on maternal and perinatal health outcomes); during pregnancy and antenatal care; at the time of birth; maternal postnatal care; and newborn health. Infertility was not included because a recent international consensus development study identifying top priorities for future infertility research, led by New Zealand investigators, has reported on issues relevant to Aotearoa New Zealand.[[11]] Knowledge gaps were collected from the Audience Group, and research priorities already identified were collected by a systematic review of published research prioritisation projects.
To enable a large and diverse group of individuals to participate in the Audience Group, we planned to use online semi-structured questionnaires and face-to-face (kanohi ki te kanohi) hui and focus groups. However, due to COVID-19-related restrictions, in-person group meetings were not feasible. We therefore used video and telephone semi-structured interviews for those who wished to interact in person. Some one-on-one in-person interviews were possible towards the end of the information gathering phase.
Invitations to contribute to the Audience Group were distributed via email and newsletters to: all ON TRACK Site Network leaders at each district health board participating in the Network; professional bodies of maternity and perinatal healthcare providers; consumer groups; Māori and Pacific maternity health groups; funders of research; academic institutions undertaking research; relevant Ministry appointed committees; and, directly to identified Māori and Pacific health researchers. Each group was asked to distribute the invitation to participate via membership circulation lists. Invitations, poster displays and social media posts included introductory information, contact details and links and QR codes to the online questionnaire.
Participants eligible to contribute to the Audience Group were 16 years or older, living in Aotearoa New Zealand, and belonging to at least one of three groups:
1. Healthcare professionals involved in the provision of care for pregnant women and their offspring in Aotearoa New Zealand, ie midwives, obstetricians, neonatologists, paediatricians, neonatal nurses, obstetric physicians and other allied health professionals.
2. Consumers involved or recently involved with pregnancy, defined as: woman with current or recent experience (within the last five years) of maternity healthcare (including preconception, pregnancy, pregnancy loss, birth, postnatal and lactation care); partner, whānau and support persons of a woman with current or recent experience in maternity healthcare; caregiver of a newborn/infant/child born within the last five years; or, consumer representative of a group related to maternal and perinatal health.
3. Other stakeholders involved in maternal and perinatal health and healthcare research such as scientists, researchers, funders, policy-makers and members of Ministry appointed committees.
Qualtrics® software was used to host the online questionnaire. Demographic data was requested from each participant, together with open-ended questions to invite free text responses relating to knowledge gaps before pregnancy, during pregnancy and antenatal care, at time of birth, postnatal care and newborn health. Responses were not linked to individuals. Two members of the project management team undertook all interviews using the same approach, and these were then recorded and transcribed.
The Collection of Knowledge Gap Study nested within the Research Prioritisation Project was approved by The University of Auckland Human Participants Ethics Committee (024469), April 2020. Participants gave written, informed consent for interviews. Submission of the questionnaire provided a record of consent for the online questionnaire.
Knowledge gaps were also identified by a systematic review of all prioritisation studies that included women and/or their offspring published since 2009. Only those specific to an Aotearoa New Zealand setting were included.
Content analysis
Knowledge gaps generated by all three methods (survey, interview and systematic review) were reviewed by the project management team. The depth and breadth of research ideas were considered as research avenues, options and questions.[[12]] Using clinical and research knowledge, specific research questions were assigned as already answered or unanswered. Unanswered questions that still require new knowledge were formulated into practical research questions, assigned to one of four domains (delivery, development, discovery, description)[[12]] and assessed for their suitability to be answered in a randomised clinical trial or a large cohort study.
Research questions suitable for assessment by these two methods were further analysed to consider: to whom the “intervention” would be applied (mother, offspring, mother and offspring, family/whānau, healthcare professional and healthcare system); the timing at which the “intervention” would be applied (preconception, in pregnancy/foetal, during birth, neonatal—preterm baby, neonatal—unspecified or term baby, postpartum—maternal or, unspecified/crossover); the intended health beneficiary (mother, offspring, mother and offspring); and, the most suitable Advisory Group member disciplines for review (up to four disciplines per research question including anaesthetics, epidemiology/public health, Māori health, maternal mental health, midwifery, primary care/general practice, neonatology, neonatal nursing, obstetric medicine, obstetrics, Pacific health, science, reproductive health, other allied health). Each question was specifically considered with regards to relevance to Māori and Pacific health (if the research question only included Māori and Pacific people, or if the health condition included was one where Māori and Pacific people are over-represented).
Once the analysis of all research questions was complete, the results were reviewed by two principal study investigators, any discrepancies in judgement were resolved by discussion. Closely related research questions were combined where feasible without impacting on the original meaning of the knowledge gap. Research questions generated from the Audience Group by survey and interview were then grouped by discipline, and circulated to the appropriate Advisory Group members via email.
Advisory Group members were asked to review their allocated research questions and consider whether this remained a genuine knowledge gap, to provide a reference to the evidence if this was no longer a knowledge gap, and to review the wording to ensure clinical relevance and understanding. Responses were collated and reviewed by two principal study investigators. Where discrepancies were identified, additional literature searches were undertaken, and research questions were retained unless there was clear evidence that the knowledge gap no longer existed.
Ranking
Members of the Ranking Group received written information including project background and ranking methodology before an online meeting, in which an overview of maternity and perinatal healthcare in Aotearoa New Zealand, and some of the challenges faced, were presented alongside an overview of the prioritisation framework. Time was allowed for discussion, including smaller discipline-based breakout groups.
The ranking of research questions was completed in two steps by Multi-Criteria Decision Analysis (MCDA) using an established online decision-making tool, developed by 1000minds Ltd. through the University of Otago (https://www.1000minds.com). To first determine the weighting of the different criteria, five rating criteria were selected from the Child Health and Nutrition Research Initiative (CHNRI) Guidelines for Implementation of CHNRI Method for Setting Priorities in Global Child Health Research Investments,[[7]] and adapted to suit maternal and perinatal health research in Aotearoa New Zealand (View Figure 2). The Ranking Group completed the weighting survey which used the PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) method to determine the relative importance of each criterion.[[13]]
Ranking Group members were then asked to review each research question against the five rating criteria as high, medium-high, medium, low-medium or low, in importance. They were supplied with detailed descriptors of each criterion to support their assessments (Appendix 1). To limit bias and the effect of reviewer fatigue, the order in which questions were presented to each reviewer was allocated in a random manner. Rating of research questions was undertaken independently by each reviewer following the online meeting. The content of individual research questions was not discussed by the Ranking Group.
The 1000minds software used reviewer responses, and applied the relative weighting for each criterion assigned by the weighting survey to score and generate a ranked list of research questions. The ranked list was reviewed by the Core Group and the top 49 were grouped by themes (models of care/service provision, workforce, and specific interventions/conditions).
Results
The Research Prioritisation Framework (Figure 1) developed includes: key features of a transparent, structured approach; Māori participation at all levels and all steps;[[14]] an equity lens applied at each step;[[15]] wide engagement with diverse stakeholders; women and whānau (consumer) participation at all levels and all steps;[[16]] methods to accommodate cultural needs and location; and, use of a validated metric-based decision-making tool. The Framework includes 11 steps, in 4 phases, including 4 groups of participants.
Invitations to join the Audience Group were distributed to: 17 professional colleges and societies; 7 funding agencies; 5 Māori and Pacific specific health groups and agencies; 19 consumer groups, 6 academic units/departments; 16 individual Māori health contacts; and, via 3 national newsletters. The online questionnaire was accessible from 1 May to 1 August 2020; 305 questionnaires were submitted. The majority of responders were New Zealand European/European (235, 77%), 29 (9.5%) were Māori and 4 (1.3%) were Pacific. Responses were received from all geographical regions and included a variety of stakeholder groups including 147 (48.1%) healthcare professionals, 122 (40.0%) consumers and 36 (11.8%) other stakeholders (Table 1). Interviews were offered from 1 May to 14 August 2020; 62 interviews were completed. The majority of interviewees were New Zealand European/European (39, 62.9%) but a higher proportion were Māori (12, 19.4%) and Pacific (8, 12.9%), in comparison to the online questionnaire. Interviewees covered a variety of stakeholder groups including 34 (54.8%) healthcare professionals, 21 (33.9%) consumers and 7 (11.3%) other stakeholders (Table 1). More than half were from the Auckland Region (34, 54.8%).
A total of 3347 knowledge gaps were identified: 1610 from online questionnaires, 892 from interviews, and 845 from the systematic review of 62 eligible research prioritisation projects. After content analysis, 358 unanswered research questions were considered potentially able to be answered in a randomised clinical trial or large cohort study and entered the Ranking phase (Table 2). Of these 358 research questions 175 were identified by on-line questionnaires, 115 by interviews and 68 by the systematic review.
Twenty-nine individuals contributed to the Advisory Group and a further 23 agreed to join the Ranking Group (52 in total), of whom 47 (90.0%) completed the weighting survey and 46 (88.5%) completed the rating survey. Six Ranking Group members (11.5%) identified as Māori, but only two (4.3%) completed both surveys. All three Pacific Ranking Group members completed the weighting (5.8%) and rating surveys (6.5%) (Table 3). Representation by regions was focussed to areas including academic units and involved all the major professional health disciplines that provide maternal and perinatal care (Table 3).
View Tables 1, 2 & 3.
The mean weights generated for the five rating criteria were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. Ranking Group members reviewed a mean of 239 (range 2–358) of the 358 questions against the five rating criteria. The total score generated, and the ranking of each research question, are shown in Appendix 2. The top 49 ranked research questions grouped by themes within each ranked score group are shown in Table 4. Of the 20 research questions ranked highest, 13 related to models of care and service provision, five related to the health workforce and only two to specific interventions (smoking cessation programmes and maternal mental health support); 14 were directly specific to Māori, two to Pacific peoples and four to both Māori and Pacific peoples, covering interventions for healthcare systems (16), the mother (3) or, healthcare professionals (1); and all having both the mother and offspring as intended beneficiaries of the intervention.
View Table 4.
Discussion
We developed a new Aotearoa New Zealand specific Research Prioritisation Framework and used this to identify and answer the most important research questions through randomised clinical trials or large cohort studies in maternal and perinatal health and healthcare. This is the first systematic health-area-wide research prioritisation project undertaken for Aotearoa New Zealand. The developed Framework includes all the steps and recommended questions in a checklist for good practice[[10]] and enabling an equity-oriented approach,[[15]] as well as all the critical steps, which have since been identified in the recently published Australian Clinical Trials Alliance (ACTA) guideline for clinical trial networks interested in setting priorities for the conduct of clinical trials.[[2]]
We were successful in addressing the majority of key features planned in our framework. Key stakeholder groups contributed to the Core, Advisory and Ranking groups and a diverse group of stakeholders were included in the Audience Group. Consumer participation was included in all steps of the process with the exception of the Advisory Group consultation, as this was deemed to be an area where clinical practice and evidenced-based knowledge was required.
The use of two different methods for the collection of knowledge gaps was intended to accommodate cultural needs and different locations of potential participants. On-line questionnaires provided an efficient and cost-effective method of data collection. This allowed participation by a large number and variety of stakeholders, including those who may prefer to participate anonymously or are geographically distant, using their own choice of timing and location. The inclusion of face-to-face data collection has been demonstrated to be effective in enabling participation by healthcare consumers in research[[17,18]] and kanohi ki te kanohi hui were also expected to support and encourage participation by Māori.[[14]] Although this could not be undertaken due to COVID-19 restrictions, the higher rates of Māori and Pacific peoples’ participation in the collection of knowledge gaps via interview suggests that the use of video and telephone calls still provided some level of personal connection, thereby supporting their willingness to participate. However, it is also possible that a more targeted approach to include Māori and Pacific people was possible by using interview rather than survey methodology.
In designing the Research Prioritisation Framework, equity was considered a key feature, and essential to supporting the vision of the ON TRACK Network “to achieve greater equity, health outcomes, and quality of life for women and babies nationwide”. Consideration of an equity lens, as developed to ensure an equity-oriented approach to agenda and priority setting of Cochrane Reviews,[[15]] was utilised at each step of the project. The importance of equity was further supported by the inclusion of the rating criteria “effect on equity” as part of the ranking process, which was found to be the criterion most heavily weighted by the Ranking Group (26.1%), and hence made the most significant contribution to overall research question scores. This approach appears to have been effective in prioritising equity focussed research questions, since all of the top 20 prioritised research questions related specifically to Māori and/or Pacific peoples’ health and healthcare, and predominantly focussed on models of care and service provision. Within the top 49 prioritised research questions, a further 12 were specific to Māori and/or Pacific people. The remainder were mainly focussed on specific health conditions where Māori and/or Pacific people are disproportionately over-represented (mental health, obesity and diabetes in pregnancy) or inequities existing due to social disadvantage, poverty, access to care, rural location or high needs communities.
Through the ranking process it was highlighted that equity issues, and meeting the obligations of Te Tiriti o Waitangi, must be considered separately. We were able to achieve Māori participation at all levels and all steps of the process but we were not able to achieve equal partnership with Māori (50% of all participants), or indeed, reach levels that are representative of Māori within the Aotearoa New Zealand population (16.7%, NZ Stats 2020)[[19]] with only 11.2% of the Audience Group and 11.5% of the Ranking Group identifying as Māori, and only two of the six Māori participants in the Ranking Group completing both the weighting and rating surveys. This partly related to the lack of capacity for overstretched Māori healthcare workers and health researchers, but also to reservations relating to the framework, which by the nature of its design does not follow Kaupapa Māori research theory and practice. It was suggested that a Māori-led and -focused future research project should be considered. However, this may not provide opportunities to directly compare, measure and prioritise research for Māori against research for non-Māori or the general population, and would currently be significantly limited by capacity of those Māori working in this area. The prioritisation project itself identified the need for further research on how to develop and support our Māori healthcare and health research workforce, with two research questions within the top 12 relating to the Māori maternity and newborn health workforce.
The list of prioritised research questions will be promoted by the ON TRACK Network through peer-reviewed publication, presentation at scientific and public meetings, through the media and social media including the ON TRACK “Forum for Women and Whānau” Facebook page for consumers, and through the ON TRACK Network’s monthly national newsletter which is distributed across district health board members of the Network. The ON TRACK Network will support development of the top prioritised research questions in future activities including their annual trial development workshop. We expect that the list of prioritised research questions will be used by clinicians and researchers to develop clinical trials and large cohort studies in the future. It is also anticipated the list will attract more clinicians and researchers to this area and be of value to hospitals, consumer and advocacy groups, colleges and societies, funders and policy-makers when considering what research to support and where resources should be applied.
Further exploration of data will identify any differences in research priorities by subgroups, such as healthcare provider groups and consumers, and will provide opportunity to explore the methodology used including the impact of each criteria and their relative weightings. Furthermore, where knowledge gaps were suggested but evidence exists, there is scope for a number of research translation and implementation projects as well as national practice guidelines, with work focussed both for clinicians and consumers. A number of broader research avenues and options were also commonly identified, for example, preterm birth and gestational diabetes, and these findings support larger programmes of research, many of which may need randomised trials nested within them. Research questions identified under the domains of discovery (fundamental and inventive research) and description (epidemiological analysis) should be of interest to scientists and epidemiologists as well to clinicians and clinical researchers, funders, hospital managers and policy-makers. The use of a robust, systematic and transparent approach to prioritisation will allow later evaluation and updates.
This first systematic health-area-wide research prioritisation project, undertaken specifically for Aotearoa New Zealand, has identified the most important questions for future clinical trials and large cohort studies in maternal and perinatal health and healthcare. This Framework and methodology could be adapted for use in other areas of health and healthcare in Aotearoa New Zealand.
View Appendices 1 & 2.
Summary
Abstract
Aim
To identify priorities for clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Method
An Aotearoa New Zealand specific Research Prioritisation Framework was developed. Knowledge gaps were collected from an Audience Group via online questionnaires, video call interviews, and by systematic review. These were formulated into research questions. An Advisory Group reviewed questions suited to a clinical trial or large cohort study. A Ranking Group weighted the ranking criteria and ranked the research questions.
Results
A total of 305 online questionnaires, 62 interviews and 62 published prioritisation projects generated 3,347 knowledge gaps. After content analysis, 358 unanswered research questions were ranked. Rating criteria weightings were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. All of the top 20 prioritised research questions directly related to Māori and/or Pacific health and predominantly involved research into healthcare systems and workforce rather than disease conditions.
Conclusion
This project has identified the most important questions for future clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand. The Framework and methodology can be adapted for use across all areas of health.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) The Joint ACTA/ACSQHC Working Group: Tate R, Britton E, Webb S, et al. The value proposition of investigator-initiated clinical trials conducted by networks. Med J Aust. 2021; doi:10.5694/mja2.50935.
2) Australian Clinical Trials Alliance. Research Prioritisation Framework. January 2020. https://clinicaltrialsalliance.org.au/wp-content/uploads/2020/02/Research-Prioritisation-Framework-FINAL.pdf Accessed 7th April 2021.
3) Council on Health Research for Development (COHRED). National health research and priority setting: lessons learned; 1997. Available: http://www.cohred.org/downloads/586.pdf. Accessed 7th April 2021.
4) Hsu C, Sandford BA. The Delphi Technique: Making Sense of Consensus Practical Assessment. Research and Evaluation 2007: DOI: https://doi.org/10.7275/pdz9-th90 Accessed 7th April 2021.
5) Okello D, Chongtrakul P, COHRED Working Group on Priority Setting: A Manual for Research Priority Setting using the ENHR Strategy. Council on Health Research for Development, 2000. https://www.cohred.org/publications/library-and-archive/a_manual_for_researc_1_0/ Accessed 7th April 2021.
6) Ghaffar A, Collins T, Matlin SA, Olifson S: The 3D Combined Approach Matrix: An Improved Tool for Setting Priorities in Research for Health. Global Forum for Health Research. http://www.bvs.hn/Honduras/PIS/MEC3DEnglish.pdf Accessed 7th April 2021.
7) Rudan I, Gibson JL, Ameratunga S, et al. Setting Priorities in Global Child Health Research Investments: Guidelines for Implementation of CHNRI Method. Croat Med J. 2008; 49:720-33.
8) The James Lind Alliance Guidebook Version 10, March 2021. https://www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-Version-10-March-2021.pdf Accessed 7th April 2021.
9) The Health Research Council of New Zealand, the Ministry of Business, Innovation and Employment and the Ministry of Health. The New Zealand Health Research Prioritisation Framework. December 2019. https://hrc.govt.nz/sites/default/files/2020-01/NZ%20Prioritisation-Framework-FA-web_0.pdf Accessed 7th April 2021.
10) Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010; 8, 36. Doi:10.1186/1478-4505-8-36.
11) Duffy JMN, Adamson GD, Benson E, et al. Top 10 priorities for future infertility research; an international consensus development study. Hum Reprod 2020: doi:10.1093/humrep/deaa242.
12) Rudan I, Yoshida S, Chan K, et al. Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method. J Glob Health 2017:7 (1), 011004. doi: 10.7189/jogh.07.011004.
13) Hansen P, Ombler F. A new method for scoring additive multi‐attribute value models using pairwise rankings of alternatives. J Multi‐Crit Decis Anal 2008; 15(3‐4),87-107.
14) Reid P, Paine SJ, Curtis E, et al. Achieving health equity in Aotearoa: strengthening responsiveness to Māori in health research. NZ Med J. 2017:130(1465):96-103.
15) Nasser M, Ueffing E, Welch V, Tugwell P. An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. J Clin Epidemiol. 2013;66(5):511-521.
16) Coney S for the New Zealand Guidelines Group. Effective Consumer Voice and Participation for New Zealand. A systematic review of the evidence 2004. https://www.health.govt.nz/system/files/documents/publications/050511_consumer_report.pdf Accessed 7th April 2021.
17) Pollock A, George BS, Fenton M, et al. Development of a new model to engage patients and clinicians in setting research priorities. Journal Health Serv Res Policy. 2013:19(1):12-18.
18) MacFarlane A, Galvin R, O'Sullivan M, et al. Participatory methods for research prioritization in primary care: an analysis of the World Cafe approach in Ireland and the USA. Fam Pract. 2017: 34(3):278-284.
19) Stats NZ. Maori Population Data 2020. www.stats.govt.nz Accessed 7th April 2021.
For the PDF of this article,
contact nzmj@nzma.org.nz
The priorities for future clinical trials and large cohort studies addressing health and healthcare for mothers and babies in Aotearoa New Zealand
View Article PDF
Multicentre clinical trials and large cohort studies provide the most effective platforms to answer clinically important research questions that lead to improved health. The efficiency and cost-effectiveness of these types of research are significantly enhanced by well organised and co-ordinated clinical trials networks.[[1]] The ON TRACK Network: Better Health for Mothers and Babies, Te Awhi Rito was established in 2016 (http://ontrack.perinatalsociety.org.nz/) to be an Aotearoa New Zealand specific multidisciplinary maternal and perinatal clinical trials network. It aims to engage researchers, clinicians, consumers and other stakeholders in a systematic and constructive way across the whole pipeline of clinical research, with an ultimate goal to achieve greater equity, better health outcomes and improved quality of life for women and babies nationwide.
To enable future high quality multicentre clinical trials and large cohort studies, while ensuring the most efficient use of resources and maximum benefit, one of the first steps is to prioritise research questions.[[2]] Over recent decades, systematic approaches to research prioritisation have been developed.[[3–9]] However, no single gold standard research prioritisation method exists, as each method has advantages and disadvantages depending on the goals of the prioritisation project and the setting in which it takes place.
The aims of this project were to establish a Research Prioritisation Framework that was robust, transparent, systematic, equity-focused, and able to consider the unique social and geographic context of maternal and perinatal health and healthcare in Aotearoa New Zealand. The aim was then then to use this framework to establish the future priorities for clinical trials and large cohort studies in addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Methods
Development of a Research Prioritisation Framework
Established health research priority-setting methods were identified and reviewed, including: the Council of Health Research for Development;[[3]] Delphi studies;[[4]] Essential National Health Research;[[5]] 3D Combined Approach Matrix;[[6]] the Child Health Nutrition Research Initiative;[[7]] the James Lind Alliance;[[8]] and, within New Zealand, the Health Research Council’s ‘The New Zealand Health Research Prioritisation Framework’.[[9]] Each was assessed for quality using a recognised good practice tool,[[10]] the resources required per method, ability to engage with multiple disciplines and groups, applicability to Aotearoa New Zealand, and overall strengths and weaknesses. Considerations for the Aotearoa New Zealand context included the ability to be responsive to the needs of the Indigenous peoples, the ability to address equity, the opportunity for consumer participation, and the suitability for the New Zealand maternity care system in which the majority of pregnancy care is provided by independent practitioners. Using this information, we developed an Aotearoa New Zealand specific Framework that included 11 steps in four phases, involving four groups of participants: the Core Group, Audience Group, Advisory Group and Ranking Group (View Figure 1).
Information gathering
Maternal and perinatal health and healthcare was considered by the Core Group to include: before pregnancy (where there is impact on maternal and perinatal health outcomes); during pregnancy and antenatal care; at the time of birth; maternal postnatal care; and newborn health. Infertility was not included because a recent international consensus development study identifying top priorities for future infertility research, led by New Zealand investigators, has reported on issues relevant to Aotearoa New Zealand.[[11]] Knowledge gaps were collected from the Audience Group, and research priorities already identified were collected by a systematic review of published research prioritisation projects.
To enable a large and diverse group of individuals to participate in the Audience Group, we planned to use online semi-structured questionnaires and face-to-face (kanohi ki te kanohi) hui and focus groups. However, due to COVID-19-related restrictions, in-person group meetings were not feasible. We therefore used video and telephone semi-structured interviews for those who wished to interact in person. Some one-on-one in-person interviews were possible towards the end of the information gathering phase.
Invitations to contribute to the Audience Group were distributed via email and newsletters to: all ON TRACK Site Network leaders at each district health board participating in the Network; professional bodies of maternity and perinatal healthcare providers; consumer groups; Māori and Pacific maternity health groups; funders of research; academic institutions undertaking research; relevant Ministry appointed committees; and, directly to identified Māori and Pacific health researchers. Each group was asked to distribute the invitation to participate via membership circulation lists. Invitations, poster displays and social media posts included introductory information, contact details and links and QR codes to the online questionnaire.
Participants eligible to contribute to the Audience Group were 16 years or older, living in Aotearoa New Zealand, and belonging to at least one of three groups:
1. Healthcare professionals involved in the provision of care for pregnant women and their offspring in Aotearoa New Zealand, ie midwives, obstetricians, neonatologists, paediatricians, neonatal nurses, obstetric physicians and other allied health professionals.
2. Consumers involved or recently involved with pregnancy, defined as: woman with current or recent experience (within the last five years) of maternity healthcare (including preconception, pregnancy, pregnancy loss, birth, postnatal and lactation care); partner, whānau and support persons of a woman with current or recent experience in maternity healthcare; caregiver of a newborn/infant/child born within the last five years; or, consumer representative of a group related to maternal and perinatal health.
3. Other stakeholders involved in maternal and perinatal health and healthcare research such as scientists, researchers, funders, policy-makers and members of Ministry appointed committees.
Qualtrics® software was used to host the online questionnaire. Demographic data was requested from each participant, together with open-ended questions to invite free text responses relating to knowledge gaps before pregnancy, during pregnancy and antenatal care, at time of birth, postnatal care and newborn health. Responses were not linked to individuals. Two members of the project management team undertook all interviews using the same approach, and these were then recorded and transcribed.
The Collection of Knowledge Gap Study nested within the Research Prioritisation Project was approved by The University of Auckland Human Participants Ethics Committee (024469), April 2020. Participants gave written, informed consent for interviews. Submission of the questionnaire provided a record of consent for the online questionnaire.
Knowledge gaps were also identified by a systematic review of all prioritisation studies that included women and/or their offspring published since 2009. Only those specific to an Aotearoa New Zealand setting were included.
Content analysis
Knowledge gaps generated by all three methods (survey, interview and systematic review) were reviewed by the project management team. The depth and breadth of research ideas were considered as research avenues, options and questions.[[12]] Using clinical and research knowledge, specific research questions were assigned as already answered or unanswered. Unanswered questions that still require new knowledge were formulated into practical research questions, assigned to one of four domains (delivery, development, discovery, description)[[12]] and assessed for their suitability to be answered in a randomised clinical trial or a large cohort study.
Research questions suitable for assessment by these two methods were further analysed to consider: to whom the “intervention” would be applied (mother, offspring, mother and offspring, family/whānau, healthcare professional and healthcare system); the timing at which the “intervention” would be applied (preconception, in pregnancy/foetal, during birth, neonatal—preterm baby, neonatal—unspecified or term baby, postpartum—maternal or, unspecified/crossover); the intended health beneficiary (mother, offspring, mother and offspring); and, the most suitable Advisory Group member disciplines for review (up to four disciplines per research question including anaesthetics, epidemiology/public health, Māori health, maternal mental health, midwifery, primary care/general practice, neonatology, neonatal nursing, obstetric medicine, obstetrics, Pacific health, science, reproductive health, other allied health). Each question was specifically considered with regards to relevance to Māori and Pacific health (if the research question only included Māori and Pacific people, or if the health condition included was one where Māori and Pacific people are over-represented).
Once the analysis of all research questions was complete, the results were reviewed by two principal study investigators, any discrepancies in judgement were resolved by discussion. Closely related research questions were combined where feasible without impacting on the original meaning of the knowledge gap. Research questions generated from the Audience Group by survey and interview were then grouped by discipline, and circulated to the appropriate Advisory Group members via email.
Advisory Group members were asked to review their allocated research questions and consider whether this remained a genuine knowledge gap, to provide a reference to the evidence if this was no longer a knowledge gap, and to review the wording to ensure clinical relevance and understanding. Responses were collated and reviewed by two principal study investigators. Where discrepancies were identified, additional literature searches were undertaken, and research questions were retained unless there was clear evidence that the knowledge gap no longer existed.
Ranking
Members of the Ranking Group received written information including project background and ranking methodology before an online meeting, in which an overview of maternity and perinatal healthcare in Aotearoa New Zealand, and some of the challenges faced, were presented alongside an overview of the prioritisation framework. Time was allowed for discussion, including smaller discipline-based breakout groups.
The ranking of research questions was completed in two steps by Multi-Criteria Decision Analysis (MCDA) using an established online decision-making tool, developed by 1000minds Ltd. through the University of Otago (https://www.1000minds.com). To first determine the weighting of the different criteria, five rating criteria were selected from the Child Health and Nutrition Research Initiative (CHNRI) Guidelines for Implementation of CHNRI Method for Setting Priorities in Global Child Health Research Investments,[[7]] and adapted to suit maternal and perinatal health research in Aotearoa New Zealand (View Figure 2). The Ranking Group completed the weighting survey which used the PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) method to determine the relative importance of each criterion.[[13]]
Ranking Group members were then asked to review each research question against the five rating criteria as high, medium-high, medium, low-medium or low, in importance. They were supplied with detailed descriptors of each criterion to support their assessments (Appendix 1). To limit bias and the effect of reviewer fatigue, the order in which questions were presented to each reviewer was allocated in a random manner. Rating of research questions was undertaken independently by each reviewer following the online meeting. The content of individual research questions was not discussed by the Ranking Group.
The 1000minds software used reviewer responses, and applied the relative weighting for each criterion assigned by the weighting survey to score and generate a ranked list of research questions. The ranked list was reviewed by the Core Group and the top 49 were grouped by themes (models of care/service provision, workforce, and specific interventions/conditions).
Results
The Research Prioritisation Framework (Figure 1) developed includes: key features of a transparent, structured approach; Māori participation at all levels and all steps;[[14]] an equity lens applied at each step;[[15]] wide engagement with diverse stakeholders; women and whānau (consumer) participation at all levels and all steps;[[16]] methods to accommodate cultural needs and location; and, use of a validated metric-based decision-making tool. The Framework includes 11 steps, in 4 phases, including 4 groups of participants.
Invitations to join the Audience Group were distributed to: 17 professional colleges and societies; 7 funding agencies; 5 Māori and Pacific specific health groups and agencies; 19 consumer groups, 6 academic units/departments; 16 individual Māori health contacts; and, via 3 national newsletters. The online questionnaire was accessible from 1 May to 1 August 2020; 305 questionnaires were submitted. The majority of responders were New Zealand European/European (235, 77%), 29 (9.5%) were Māori and 4 (1.3%) were Pacific. Responses were received from all geographical regions and included a variety of stakeholder groups including 147 (48.1%) healthcare professionals, 122 (40.0%) consumers and 36 (11.8%) other stakeholders (Table 1). Interviews were offered from 1 May to 14 August 2020; 62 interviews were completed. The majority of interviewees were New Zealand European/European (39, 62.9%) but a higher proportion were Māori (12, 19.4%) and Pacific (8, 12.9%), in comparison to the online questionnaire. Interviewees covered a variety of stakeholder groups including 34 (54.8%) healthcare professionals, 21 (33.9%) consumers and 7 (11.3%) other stakeholders (Table 1). More than half were from the Auckland Region (34, 54.8%).
A total of 3347 knowledge gaps were identified: 1610 from online questionnaires, 892 from interviews, and 845 from the systematic review of 62 eligible research prioritisation projects. After content analysis, 358 unanswered research questions were considered potentially able to be answered in a randomised clinical trial or large cohort study and entered the Ranking phase (Table 2). Of these 358 research questions 175 were identified by on-line questionnaires, 115 by interviews and 68 by the systematic review.
Twenty-nine individuals contributed to the Advisory Group and a further 23 agreed to join the Ranking Group (52 in total), of whom 47 (90.0%) completed the weighting survey and 46 (88.5%) completed the rating survey. Six Ranking Group members (11.5%) identified as Māori, but only two (4.3%) completed both surveys. All three Pacific Ranking Group members completed the weighting (5.8%) and rating surveys (6.5%) (Table 3). Representation by regions was focussed to areas including academic units and involved all the major professional health disciplines that provide maternal and perinatal care (Table 3).
View Tables 1, 2 & 3.
The mean weights generated for the five rating criteria were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. Ranking Group members reviewed a mean of 239 (range 2–358) of the 358 questions against the five rating criteria. The total score generated, and the ranking of each research question, are shown in Appendix 2. The top 49 ranked research questions grouped by themes within each ranked score group are shown in Table 4. Of the 20 research questions ranked highest, 13 related to models of care and service provision, five related to the health workforce and only two to specific interventions (smoking cessation programmes and maternal mental health support); 14 were directly specific to Māori, two to Pacific peoples and four to both Māori and Pacific peoples, covering interventions for healthcare systems (16), the mother (3) or, healthcare professionals (1); and all having both the mother and offspring as intended beneficiaries of the intervention.
View Table 4.
Discussion
We developed a new Aotearoa New Zealand specific Research Prioritisation Framework and used this to identify and answer the most important research questions through randomised clinical trials or large cohort studies in maternal and perinatal health and healthcare. This is the first systematic health-area-wide research prioritisation project undertaken for Aotearoa New Zealand. The developed Framework includes all the steps and recommended questions in a checklist for good practice[[10]] and enabling an equity-oriented approach,[[15]] as well as all the critical steps, which have since been identified in the recently published Australian Clinical Trials Alliance (ACTA) guideline for clinical trial networks interested in setting priorities for the conduct of clinical trials.[[2]]
We were successful in addressing the majority of key features planned in our framework. Key stakeholder groups contributed to the Core, Advisory and Ranking groups and a diverse group of stakeholders were included in the Audience Group. Consumer participation was included in all steps of the process with the exception of the Advisory Group consultation, as this was deemed to be an area where clinical practice and evidenced-based knowledge was required.
The use of two different methods for the collection of knowledge gaps was intended to accommodate cultural needs and different locations of potential participants. On-line questionnaires provided an efficient and cost-effective method of data collection. This allowed participation by a large number and variety of stakeholders, including those who may prefer to participate anonymously or are geographically distant, using their own choice of timing and location. The inclusion of face-to-face data collection has been demonstrated to be effective in enabling participation by healthcare consumers in research[[17,18]] and kanohi ki te kanohi hui were also expected to support and encourage participation by Māori.[[14]] Although this could not be undertaken due to COVID-19 restrictions, the higher rates of Māori and Pacific peoples’ participation in the collection of knowledge gaps via interview suggests that the use of video and telephone calls still provided some level of personal connection, thereby supporting their willingness to participate. However, it is also possible that a more targeted approach to include Māori and Pacific people was possible by using interview rather than survey methodology.
In designing the Research Prioritisation Framework, equity was considered a key feature, and essential to supporting the vision of the ON TRACK Network “to achieve greater equity, health outcomes, and quality of life for women and babies nationwide”. Consideration of an equity lens, as developed to ensure an equity-oriented approach to agenda and priority setting of Cochrane Reviews,[[15]] was utilised at each step of the project. The importance of equity was further supported by the inclusion of the rating criteria “effect on equity” as part of the ranking process, which was found to be the criterion most heavily weighted by the Ranking Group (26.1%), and hence made the most significant contribution to overall research question scores. This approach appears to have been effective in prioritising equity focussed research questions, since all of the top 20 prioritised research questions related specifically to Māori and/or Pacific peoples’ health and healthcare, and predominantly focussed on models of care and service provision. Within the top 49 prioritised research questions, a further 12 were specific to Māori and/or Pacific people. The remainder were mainly focussed on specific health conditions where Māori and/or Pacific people are disproportionately over-represented (mental health, obesity and diabetes in pregnancy) or inequities existing due to social disadvantage, poverty, access to care, rural location or high needs communities.
Through the ranking process it was highlighted that equity issues, and meeting the obligations of Te Tiriti o Waitangi, must be considered separately. We were able to achieve Māori participation at all levels and all steps of the process but we were not able to achieve equal partnership with Māori (50% of all participants), or indeed, reach levels that are representative of Māori within the Aotearoa New Zealand population (16.7%, NZ Stats 2020)[[19]] with only 11.2% of the Audience Group and 11.5% of the Ranking Group identifying as Māori, and only two of the six Māori participants in the Ranking Group completing both the weighting and rating surveys. This partly related to the lack of capacity for overstretched Māori healthcare workers and health researchers, but also to reservations relating to the framework, which by the nature of its design does not follow Kaupapa Māori research theory and practice. It was suggested that a Māori-led and -focused future research project should be considered. However, this may not provide opportunities to directly compare, measure and prioritise research for Māori against research for non-Māori or the general population, and would currently be significantly limited by capacity of those Māori working in this area. The prioritisation project itself identified the need for further research on how to develop and support our Māori healthcare and health research workforce, with two research questions within the top 12 relating to the Māori maternity and newborn health workforce.
The list of prioritised research questions will be promoted by the ON TRACK Network through peer-reviewed publication, presentation at scientific and public meetings, through the media and social media including the ON TRACK “Forum for Women and Whānau” Facebook page for consumers, and through the ON TRACK Network’s monthly national newsletter which is distributed across district health board members of the Network. The ON TRACK Network will support development of the top prioritised research questions in future activities including their annual trial development workshop. We expect that the list of prioritised research questions will be used by clinicians and researchers to develop clinical trials and large cohort studies in the future. It is also anticipated the list will attract more clinicians and researchers to this area and be of value to hospitals, consumer and advocacy groups, colleges and societies, funders and policy-makers when considering what research to support and where resources should be applied.
Further exploration of data will identify any differences in research priorities by subgroups, such as healthcare provider groups and consumers, and will provide opportunity to explore the methodology used including the impact of each criteria and their relative weightings. Furthermore, where knowledge gaps were suggested but evidence exists, there is scope for a number of research translation and implementation projects as well as national practice guidelines, with work focussed both for clinicians and consumers. A number of broader research avenues and options were also commonly identified, for example, preterm birth and gestational diabetes, and these findings support larger programmes of research, many of which may need randomised trials nested within them. Research questions identified under the domains of discovery (fundamental and inventive research) and description (epidemiological analysis) should be of interest to scientists and epidemiologists as well to clinicians and clinical researchers, funders, hospital managers and policy-makers. The use of a robust, systematic and transparent approach to prioritisation will allow later evaluation and updates.
This first systematic health-area-wide research prioritisation project, undertaken specifically for Aotearoa New Zealand, has identified the most important questions for future clinical trials and large cohort studies in maternal and perinatal health and healthcare. This Framework and methodology could be adapted for use in other areas of health and healthcare in Aotearoa New Zealand.
View Appendices 1 & 2.
Summary
Abstract
Aim
To identify priorities for clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Method
An Aotearoa New Zealand specific Research Prioritisation Framework was developed. Knowledge gaps were collected from an Audience Group via online questionnaires, video call interviews, and by systematic review. These were formulated into research questions. An Advisory Group reviewed questions suited to a clinical trial or large cohort study. A Ranking Group weighted the ranking criteria and ranked the research questions.
Results
A total of 305 online questionnaires, 62 interviews and 62 published prioritisation projects generated 3,347 knowledge gaps. After content analysis, 358 unanswered research questions were ranked. Rating criteria weightings were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. All of the top 20 prioritised research questions directly related to Māori and/or Pacific health and predominantly involved research into healthcare systems and workforce rather than disease conditions.
Conclusion
This project has identified the most important questions for future clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand. The Framework and methodology can be adapted for use across all areas of health.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) The Joint ACTA/ACSQHC Working Group: Tate R, Britton E, Webb S, et al. The value proposition of investigator-initiated clinical trials conducted by networks. Med J Aust. 2021; doi:10.5694/mja2.50935.
2) Australian Clinical Trials Alliance. Research Prioritisation Framework. January 2020. https://clinicaltrialsalliance.org.au/wp-content/uploads/2020/02/Research-Prioritisation-Framework-FINAL.pdf Accessed 7th April 2021.
3) Council on Health Research for Development (COHRED). National health research and priority setting: lessons learned; 1997. Available: http://www.cohred.org/downloads/586.pdf. Accessed 7th April 2021.
4) Hsu C, Sandford BA. The Delphi Technique: Making Sense of Consensus Practical Assessment. Research and Evaluation 2007: DOI: https://doi.org/10.7275/pdz9-th90 Accessed 7th April 2021.
5) Okello D, Chongtrakul P, COHRED Working Group on Priority Setting: A Manual for Research Priority Setting using the ENHR Strategy. Council on Health Research for Development, 2000. https://www.cohred.org/publications/library-and-archive/a_manual_for_researc_1_0/ Accessed 7th April 2021.
6) Ghaffar A, Collins T, Matlin SA, Olifson S: The 3D Combined Approach Matrix: An Improved Tool for Setting Priorities in Research for Health. Global Forum for Health Research. http://www.bvs.hn/Honduras/PIS/MEC3DEnglish.pdf Accessed 7th April 2021.
7) Rudan I, Gibson JL, Ameratunga S, et al. Setting Priorities in Global Child Health Research Investments: Guidelines for Implementation of CHNRI Method. Croat Med J. 2008; 49:720-33.
8) The James Lind Alliance Guidebook Version 10, March 2021. https://www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-Version-10-March-2021.pdf Accessed 7th April 2021.
9) The Health Research Council of New Zealand, the Ministry of Business, Innovation and Employment and the Ministry of Health. The New Zealand Health Research Prioritisation Framework. December 2019. https://hrc.govt.nz/sites/default/files/2020-01/NZ%20Prioritisation-Framework-FA-web_0.pdf Accessed 7th April 2021.
10) Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010; 8, 36. Doi:10.1186/1478-4505-8-36.
11) Duffy JMN, Adamson GD, Benson E, et al. Top 10 priorities for future infertility research; an international consensus development study. Hum Reprod 2020: doi:10.1093/humrep/deaa242.
12) Rudan I, Yoshida S, Chan K, et al. Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method. J Glob Health 2017:7 (1), 011004. doi: 10.7189/jogh.07.011004.
13) Hansen P, Ombler F. A new method for scoring additive multi‐attribute value models using pairwise rankings of alternatives. J Multi‐Crit Decis Anal 2008; 15(3‐4),87-107.
14) Reid P, Paine SJ, Curtis E, et al. Achieving health equity in Aotearoa: strengthening responsiveness to Māori in health research. NZ Med J. 2017:130(1465):96-103.
15) Nasser M, Ueffing E, Welch V, Tugwell P. An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. J Clin Epidemiol. 2013;66(5):511-521.
16) Coney S for the New Zealand Guidelines Group. Effective Consumer Voice and Participation for New Zealand. A systematic review of the evidence 2004. https://www.health.govt.nz/system/files/documents/publications/050511_consumer_report.pdf Accessed 7th April 2021.
17) Pollock A, George BS, Fenton M, et al. Development of a new model to engage patients and clinicians in setting research priorities. Journal Health Serv Res Policy. 2013:19(1):12-18.
18) MacFarlane A, Galvin R, O'Sullivan M, et al. Participatory methods for research prioritization in primary care: an analysis of the World Cafe approach in Ireland and the USA. Fam Pract. 2017: 34(3):278-284.
19) Stats NZ. Maori Population Data 2020. www.stats.govt.nz Accessed 7th April 2021.
Contact diana@nzma.org.nz
for the PDF of this article
The priorities for future clinical trials and large cohort studies addressing health and healthcare for mothers and babies in Aotearoa New Zealand
View Article PDF
Multicentre clinical trials and large cohort studies provide the most effective platforms to answer clinically important research questions that lead to improved health. The efficiency and cost-effectiveness of these types of research are significantly enhanced by well organised and co-ordinated clinical trials networks.[[1]] The ON TRACK Network: Better Health for Mothers and Babies, Te Awhi Rito was established in 2016 (http://ontrack.perinatalsociety.org.nz/) to be an Aotearoa New Zealand specific multidisciplinary maternal and perinatal clinical trials network. It aims to engage researchers, clinicians, consumers and other stakeholders in a systematic and constructive way across the whole pipeline of clinical research, with an ultimate goal to achieve greater equity, better health outcomes and improved quality of life for women and babies nationwide.
To enable future high quality multicentre clinical trials and large cohort studies, while ensuring the most efficient use of resources and maximum benefit, one of the first steps is to prioritise research questions.[[2]] Over recent decades, systematic approaches to research prioritisation have been developed.[[3–9]] However, no single gold standard research prioritisation method exists, as each method has advantages and disadvantages depending on the goals of the prioritisation project and the setting in which it takes place.
The aims of this project were to establish a Research Prioritisation Framework that was robust, transparent, systematic, equity-focused, and able to consider the unique social and geographic context of maternal and perinatal health and healthcare in Aotearoa New Zealand. The aim was then then to use this framework to establish the future priorities for clinical trials and large cohort studies in addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Methods
Development of a Research Prioritisation Framework
Established health research priority-setting methods were identified and reviewed, including: the Council of Health Research for Development;[[3]] Delphi studies;[[4]] Essential National Health Research;[[5]] 3D Combined Approach Matrix;[[6]] the Child Health Nutrition Research Initiative;[[7]] the James Lind Alliance;[[8]] and, within New Zealand, the Health Research Council’s ‘The New Zealand Health Research Prioritisation Framework’.[[9]] Each was assessed for quality using a recognised good practice tool,[[10]] the resources required per method, ability to engage with multiple disciplines and groups, applicability to Aotearoa New Zealand, and overall strengths and weaknesses. Considerations for the Aotearoa New Zealand context included the ability to be responsive to the needs of the Indigenous peoples, the ability to address equity, the opportunity for consumer participation, and the suitability for the New Zealand maternity care system in which the majority of pregnancy care is provided by independent practitioners. Using this information, we developed an Aotearoa New Zealand specific Framework that included 11 steps in four phases, involving four groups of participants: the Core Group, Audience Group, Advisory Group and Ranking Group (View Figure 1).
Information gathering
Maternal and perinatal health and healthcare was considered by the Core Group to include: before pregnancy (where there is impact on maternal and perinatal health outcomes); during pregnancy and antenatal care; at the time of birth; maternal postnatal care; and newborn health. Infertility was not included because a recent international consensus development study identifying top priorities for future infertility research, led by New Zealand investigators, has reported on issues relevant to Aotearoa New Zealand.[[11]] Knowledge gaps were collected from the Audience Group, and research priorities already identified were collected by a systematic review of published research prioritisation projects.
To enable a large and diverse group of individuals to participate in the Audience Group, we planned to use online semi-structured questionnaires and face-to-face (kanohi ki te kanohi) hui and focus groups. However, due to COVID-19-related restrictions, in-person group meetings were not feasible. We therefore used video and telephone semi-structured interviews for those who wished to interact in person. Some one-on-one in-person interviews were possible towards the end of the information gathering phase.
Invitations to contribute to the Audience Group were distributed via email and newsletters to: all ON TRACK Site Network leaders at each district health board participating in the Network; professional bodies of maternity and perinatal healthcare providers; consumer groups; Māori and Pacific maternity health groups; funders of research; academic institutions undertaking research; relevant Ministry appointed committees; and, directly to identified Māori and Pacific health researchers. Each group was asked to distribute the invitation to participate via membership circulation lists. Invitations, poster displays and social media posts included introductory information, contact details and links and QR codes to the online questionnaire.
Participants eligible to contribute to the Audience Group were 16 years or older, living in Aotearoa New Zealand, and belonging to at least one of three groups:
1. Healthcare professionals involved in the provision of care for pregnant women and their offspring in Aotearoa New Zealand, ie midwives, obstetricians, neonatologists, paediatricians, neonatal nurses, obstetric physicians and other allied health professionals.
2. Consumers involved or recently involved with pregnancy, defined as: woman with current or recent experience (within the last five years) of maternity healthcare (including preconception, pregnancy, pregnancy loss, birth, postnatal and lactation care); partner, whānau and support persons of a woman with current or recent experience in maternity healthcare; caregiver of a newborn/infant/child born within the last five years; or, consumer representative of a group related to maternal and perinatal health.
3. Other stakeholders involved in maternal and perinatal health and healthcare research such as scientists, researchers, funders, policy-makers and members of Ministry appointed committees.
Qualtrics® software was used to host the online questionnaire. Demographic data was requested from each participant, together with open-ended questions to invite free text responses relating to knowledge gaps before pregnancy, during pregnancy and antenatal care, at time of birth, postnatal care and newborn health. Responses were not linked to individuals. Two members of the project management team undertook all interviews using the same approach, and these were then recorded and transcribed.
The Collection of Knowledge Gap Study nested within the Research Prioritisation Project was approved by The University of Auckland Human Participants Ethics Committee (024469), April 2020. Participants gave written, informed consent for interviews. Submission of the questionnaire provided a record of consent for the online questionnaire.
Knowledge gaps were also identified by a systematic review of all prioritisation studies that included women and/or their offspring published since 2009. Only those specific to an Aotearoa New Zealand setting were included.
Content analysis
Knowledge gaps generated by all three methods (survey, interview and systematic review) were reviewed by the project management team. The depth and breadth of research ideas were considered as research avenues, options and questions.[[12]] Using clinical and research knowledge, specific research questions were assigned as already answered or unanswered. Unanswered questions that still require new knowledge were formulated into practical research questions, assigned to one of four domains (delivery, development, discovery, description)[[12]] and assessed for their suitability to be answered in a randomised clinical trial or a large cohort study.
Research questions suitable for assessment by these two methods were further analysed to consider: to whom the “intervention” would be applied (mother, offspring, mother and offspring, family/whānau, healthcare professional and healthcare system); the timing at which the “intervention” would be applied (preconception, in pregnancy/foetal, during birth, neonatal—preterm baby, neonatal—unspecified or term baby, postpartum—maternal or, unspecified/crossover); the intended health beneficiary (mother, offspring, mother and offspring); and, the most suitable Advisory Group member disciplines for review (up to four disciplines per research question including anaesthetics, epidemiology/public health, Māori health, maternal mental health, midwifery, primary care/general practice, neonatology, neonatal nursing, obstetric medicine, obstetrics, Pacific health, science, reproductive health, other allied health). Each question was specifically considered with regards to relevance to Māori and Pacific health (if the research question only included Māori and Pacific people, or if the health condition included was one where Māori and Pacific people are over-represented).
Once the analysis of all research questions was complete, the results were reviewed by two principal study investigators, any discrepancies in judgement were resolved by discussion. Closely related research questions were combined where feasible without impacting on the original meaning of the knowledge gap. Research questions generated from the Audience Group by survey and interview were then grouped by discipline, and circulated to the appropriate Advisory Group members via email.
Advisory Group members were asked to review their allocated research questions and consider whether this remained a genuine knowledge gap, to provide a reference to the evidence if this was no longer a knowledge gap, and to review the wording to ensure clinical relevance and understanding. Responses were collated and reviewed by two principal study investigators. Where discrepancies were identified, additional literature searches were undertaken, and research questions were retained unless there was clear evidence that the knowledge gap no longer existed.
Ranking
Members of the Ranking Group received written information including project background and ranking methodology before an online meeting, in which an overview of maternity and perinatal healthcare in Aotearoa New Zealand, and some of the challenges faced, were presented alongside an overview of the prioritisation framework. Time was allowed for discussion, including smaller discipline-based breakout groups.
The ranking of research questions was completed in two steps by Multi-Criteria Decision Analysis (MCDA) using an established online decision-making tool, developed by 1000minds Ltd. through the University of Otago (https://www.1000minds.com). To first determine the weighting of the different criteria, five rating criteria were selected from the Child Health and Nutrition Research Initiative (CHNRI) Guidelines for Implementation of CHNRI Method for Setting Priorities in Global Child Health Research Investments,[[7]] and adapted to suit maternal and perinatal health research in Aotearoa New Zealand (View Figure 2). The Ranking Group completed the weighting survey which used the PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) method to determine the relative importance of each criterion.[[13]]
Ranking Group members were then asked to review each research question against the five rating criteria as high, medium-high, medium, low-medium or low, in importance. They were supplied with detailed descriptors of each criterion to support their assessments (Appendix 1). To limit bias and the effect of reviewer fatigue, the order in which questions were presented to each reviewer was allocated in a random manner. Rating of research questions was undertaken independently by each reviewer following the online meeting. The content of individual research questions was not discussed by the Ranking Group.
The 1000minds software used reviewer responses, and applied the relative weighting for each criterion assigned by the weighting survey to score and generate a ranked list of research questions. The ranked list was reviewed by the Core Group and the top 49 were grouped by themes (models of care/service provision, workforce, and specific interventions/conditions).
Results
The Research Prioritisation Framework (Figure 1) developed includes: key features of a transparent, structured approach; Māori participation at all levels and all steps;[[14]] an equity lens applied at each step;[[15]] wide engagement with diverse stakeholders; women and whānau (consumer) participation at all levels and all steps;[[16]] methods to accommodate cultural needs and location; and, use of a validated metric-based decision-making tool. The Framework includes 11 steps, in 4 phases, including 4 groups of participants.
Invitations to join the Audience Group were distributed to: 17 professional colleges and societies; 7 funding agencies; 5 Māori and Pacific specific health groups and agencies; 19 consumer groups, 6 academic units/departments; 16 individual Māori health contacts; and, via 3 national newsletters. The online questionnaire was accessible from 1 May to 1 August 2020; 305 questionnaires were submitted. The majority of responders were New Zealand European/European (235, 77%), 29 (9.5%) were Māori and 4 (1.3%) were Pacific. Responses were received from all geographical regions and included a variety of stakeholder groups including 147 (48.1%) healthcare professionals, 122 (40.0%) consumers and 36 (11.8%) other stakeholders (Table 1). Interviews were offered from 1 May to 14 August 2020; 62 interviews were completed. The majority of interviewees were New Zealand European/European (39, 62.9%) but a higher proportion were Māori (12, 19.4%) and Pacific (8, 12.9%), in comparison to the online questionnaire. Interviewees covered a variety of stakeholder groups including 34 (54.8%) healthcare professionals, 21 (33.9%) consumers and 7 (11.3%) other stakeholders (Table 1). More than half were from the Auckland Region (34, 54.8%).
A total of 3347 knowledge gaps were identified: 1610 from online questionnaires, 892 from interviews, and 845 from the systematic review of 62 eligible research prioritisation projects. After content analysis, 358 unanswered research questions were considered potentially able to be answered in a randomised clinical trial or large cohort study and entered the Ranking phase (Table 2). Of these 358 research questions 175 were identified by on-line questionnaires, 115 by interviews and 68 by the systematic review.
Twenty-nine individuals contributed to the Advisory Group and a further 23 agreed to join the Ranking Group (52 in total), of whom 47 (90.0%) completed the weighting survey and 46 (88.5%) completed the rating survey. Six Ranking Group members (11.5%) identified as Māori, but only two (4.3%) completed both surveys. All three Pacific Ranking Group members completed the weighting (5.8%) and rating surveys (6.5%) (Table 3). Representation by regions was focussed to areas including academic units and involved all the major professional health disciplines that provide maternal and perinatal care (Table 3).
View Tables 1, 2 & 3.
The mean weights generated for the five rating criteria were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. Ranking Group members reviewed a mean of 239 (range 2–358) of the 358 questions against the five rating criteria. The total score generated, and the ranking of each research question, are shown in Appendix 2. The top 49 ranked research questions grouped by themes within each ranked score group are shown in Table 4. Of the 20 research questions ranked highest, 13 related to models of care and service provision, five related to the health workforce and only two to specific interventions (smoking cessation programmes and maternal mental health support); 14 were directly specific to Māori, two to Pacific peoples and four to both Māori and Pacific peoples, covering interventions for healthcare systems (16), the mother (3) or, healthcare professionals (1); and all having both the mother and offspring as intended beneficiaries of the intervention.
View Table 4.
Discussion
We developed a new Aotearoa New Zealand specific Research Prioritisation Framework and used this to identify and answer the most important research questions through randomised clinical trials or large cohort studies in maternal and perinatal health and healthcare. This is the first systematic health-area-wide research prioritisation project undertaken for Aotearoa New Zealand. The developed Framework includes all the steps and recommended questions in a checklist for good practice[[10]] and enabling an equity-oriented approach,[[15]] as well as all the critical steps, which have since been identified in the recently published Australian Clinical Trials Alliance (ACTA) guideline for clinical trial networks interested in setting priorities for the conduct of clinical trials.[[2]]
We were successful in addressing the majority of key features planned in our framework. Key stakeholder groups contributed to the Core, Advisory and Ranking groups and a diverse group of stakeholders were included in the Audience Group. Consumer participation was included in all steps of the process with the exception of the Advisory Group consultation, as this was deemed to be an area where clinical practice and evidenced-based knowledge was required.
The use of two different methods for the collection of knowledge gaps was intended to accommodate cultural needs and different locations of potential participants. On-line questionnaires provided an efficient and cost-effective method of data collection. This allowed participation by a large number and variety of stakeholders, including those who may prefer to participate anonymously or are geographically distant, using their own choice of timing and location. The inclusion of face-to-face data collection has been demonstrated to be effective in enabling participation by healthcare consumers in research[[17,18]] and kanohi ki te kanohi hui were also expected to support and encourage participation by Māori.[[14]] Although this could not be undertaken due to COVID-19 restrictions, the higher rates of Māori and Pacific peoples’ participation in the collection of knowledge gaps via interview suggests that the use of video and telephone calls still provided some level of personal connection, thereby supporting their willingness to participate. However, it is also possible that a more targeted approach to include Māori and Pacific people was possible by using interview rather than survey methodology.
In designing the Research Prioritisation Framework, equity was considered a key feature, and essential to supporting the vision of the ON TRACK Network “to achieve greater equity, health outcomes, and quality of life for women and babies nationwide”. Consideration of an equity lens, as developed to ensure an equity-oriented approach to agenda and priority setting of Cochrane Reviews,[[15]] was utilised at each step of the project. The importance of equity was further supported by the inclusion of the rating criteria “effect on equity” as part of the ranking process, which was found to be the criterion most heavily weighted by the Ranking Group (26.1%), and hence made the most significant contribution to overall research question scores. This approach appears to have been effective in prioritising equity focussed research questions, since all of the top 20 prioritised research questions related specifically to Māori and/or Pacific peoples’ health and healthcare, and predominantly focussed on models of care and service provision. Within the top 49 prioritised research questions, a further 12 were specific to Māori and/or Pacific people. The remainder were mainly focussed on specific health conditions where Māori and/or Pacific people are disproportionately over-represented (mental health, obesity and diabetes in pregnancy) or inequities existing due to social disadvantage, poverty, access to care, rural location or high needs communities.
Through the ranking process it was highlighted that equity issues, and meeting the obligations of Te Tiriti o Waitangi, must be considered separately. We were able to achieve Māori participation at all levels and all steps of the process but we were not able to achieve equal partnership with Māori (50% of all participants), or indeed, reach levels that are representative of Māori within the Aotearoa New Zealand population (16.7%, NZ Stats 2020)[[19]] with only 11.2% of the Audience Group and 11.5% of the Ranking Group identifying as Māori, and only two of the six Māori participants in the Ranking Group completing both the weighting and rating surveys. This partly related to the lack of capacity for overstretched Māori healthcare workers and health researchers, but also to reservations relating to the framework, which by the nature of its design does not follow Kaupapa Māori research theory and practice. It was suggested that a Māori-led and -focused future research project should be considered. However, this may not provide opportunities to directly compare, measure and prioritise research for Māori against research for non-Māori or the general population, and would currently be significantly limited by capacity of those Māori working in this area. The prioritisation project itself identified the need for further research on how to develop and support our Māori healthcare and health research workforce, with two research questions within the top 12 relating to the Māori maternity and newborn health workforce.
The list of prioritised research questions will be promoted by the ON TRACK Network through peer-reviewed publication, presentation at scientific and public meetings, through the media and social media including the ON TRACK “Forum for Women and Whānau” Facebook page for consumers, and through the ON TRACK Network’s monthly national newsletter which is distributed across district health board members of the Network. The ON TRACK Network will support development of the top prioritised research questions in future activities including their annual trial development workshop. We expect that the list of prioritised research questions will be used by clinicians and researchers to develop clinical trials and large cohort studies in the future. It is also anticipated the list will attract more clinicians and researchers to this area and be of value to hospitals, consumer and advocacy groups, colleges and societies, funders and policy-makers when considering what research to support and where resources should be applied.
Further exploration of data will identify any differences in research priorities by subgroups, such as healthcare provider groups and consumers, and will provide opportunity to explore the methodology used including the impact of each criteria and their relative weightings. Furthermore, where knowledge gaps were suggested but evidence exists, there is scope for a number of research translation and implementation projects as well as national practice guidelines, with work focussed both for clinicians and consumers. A number of broader research avenues and options were also commonly identified, for example, preterm birth and gestational diabetes, and these findings support larger programmes of research, many of which may need randomised trials nested within them. Research questions identified under the domains of discovery (fundamental and inventive research) and description (epidemiological analysis) should be of interest to scientists and epidemiologists as well to clinicians and clinical researchers, funders, hospital managers and policy-makers. The use of a robust, systematic and transparent approach to prioritisation will allow later evaluation and updates.
This first systematic health-area-wide research prioritisation project, undertaken specifically for Aotearoa New Zealand, has identified the most important questions for future clinical trials and large cohort studies in maternal and perinatal health and healthcare. This Framework and methodology could be adapted for use in other areas of health and healthcare in Aotearoa New Zealand.
View Appendices 1 & 2.
Summary
Abstract
Aim
To identify priorities for clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Method
An Aotearoa New Zealand specific Research Prioritisation Framework was developed. Knowledge gaps were collected from an Audience Group via online questionnaires, video call interviews, and by systematic review. These were formulated into research questions. An Advisory Group reviewed questions suited to a clinical trial or large cohort study. A Ranking Group weighted the ranking criteria and ranked the research questions.
Results
A total of 305 online questionnaires, 62 interviews and 62 published prioritisation projects generated 3,347 knowledge gaps. After content analysis, 358 unanswered research questions were ranked. Rating criteria weightings were: effect on equity 26.1%; potential to reduce disease burden 20.5%; effectiveness 20.0%; deliverability 17.9%; and answerability 16.0%. All of the top 20 prioritised research questions directly related to Māori and/or Pacific health and predominantly involved research into healthcare systems and workforce rather than disease conditions.
Conclusion
This project has identified the most important questions for future clinical trials and large cohort studies addressing maternal and perinatal health and healthcare in Aotearoa New Zealand. The Framework and methodology can be adapted for use across all areas of health.
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) The Joint ACTA/ACSQHC Working Group: Tate R, Britton E, Webb S, et al. The value proposition of investigator-initiated clinical trials conducted by networks. Med J Aust. 2021; doi:10.5694/mja2.50935.
2) Australian Clinical Trials Alliance. Research Prioritisation Framework. January 2020. https://clinicaltrialsalliance.org.au/wp-content/uploads/2020/02/Research-Prioritisation-Framework-FINAL.pdf Accessed 7th April 2021.
3) Council on Health Research for Development (COHRED). National health research and priority setting: lessons learned; 1997. Available: http://www.cohred.org/downloads/586.pdf. Accessed 7th April 2021.
4) Hsu C, Sandford BA. The Delphi Technique: Making Sense of Consensus Practical Assessment. Research and Evaluation 2007: DOI: https://doi.org/10.7275/pdz9-th90 Accessed 7th April 2021.
5) Okello D, Chongtrakul P, COHRED Working Group on Priority Setting: A Manual for Research Priority Setting using the ENHR Strategy. Council on Health Research for Development, 2000. https://www.cohred.org/publications/library-and-archive/a_manual_for_researc_1_0/ Accessed 7th April 2021.
6) Ghaffar A, Collins T, Matlin SA, Olifson S: The 3D Combined Approach Matrix: An Improved Tool for Setting Priorities in Research for Health. Global Forum for Health Research. http://www.bvs.hn/Honduras/PIS/MEC3DEnglish.pdf Accessed 7th April 2021.
7) Rudan I, Gibson JL, Ameratunga S, et al. Setting Priorities in Global Child Health Research Investments: Guidelines for Implementation of CHNRI Method. Croat Med J. 2008; 49:720-33.
8) The James Lind Alliance Guidebook Version 10, March 2021. https://www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-Version-10-March-2021.pdf Accessed 7th April 2021.
9) The Health Research Council of New Zealand, the Ministry of Business, Innovation and Employment and the Ministry of Health. The New Zealand Health Research Prioritisation Framework. December 2019. https://hrc.govt.nz/sites/default/files/2020-01/NZ%20Prioritisation-Framework-FA-web_0.pdf Accessed 7th April 2021.
10) Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010; 8, 36. Doi:10.1186/1478-4505-8-36.
11) Duffy JMN, Adamson GD, Benson E, et al. Top 10 priorities for future infertility research; an international consensus development study. Hum Reprod 2020: doi:10.1093/humrep/deaa242.
12) Rudan I, Yoshida S, Chan K, et al. Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method. J Glob Health 2017:7 (1), 011004. doi: 10.7189/jogh.07.011004.
13) Hansen P, Ombler F. A new method for scoring additive multi‐attribute value models using pairwise rankings of alternatives. J Multi‐Crit Decis Anal 2008; 15(3‐4),87-107.
14) Reid P, Paine SJ, Curtis E, et al. Achieving health equity in Aotearoa: strengthening responsiveness to Māori in health research. NZ Med J. 2017:130(1465):96-103.
15) Nasser M, Ueffing E, Welch V, Tugwell P. An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. J Clin Epidemiol. 2013;66(5):511-521.
16) Coney S for the New Zealand Guidelines Group. Effective Consumer Voice and Participation for New Zealand. A systematic review of the evidence 2004. https://www.health.govt.nz/system/files/documents/publications/050511_consumer_report.pdf Accessed 7th April 2021.
17) Pollock A, George BS, Fenton M, et al. Development of a new model to engage patients and clinicians in setting research priorities. Journal Health Serv Res Policy. 2013:19(1):12-18.
18) MacFarlane A, Galvin R, O'Sullivan M, et al. Participatory methods for research prioritization in primary care: an analysis of the World Cafe approach in Ireland and the USA. Fam Pract. 2017: 34(3):278-284.
19) Stats NZ. Maori Population Data 2020. www.stats.govt.nz Accessed 7th April 2021.
Contact diana@nzma.org.nz
for the PDF of this article
The priorities for future clinical trials and large cohort studies addressing health and healthcare for mothers and babies in Aotearoa New Zealand
The aims of this project were to establish a Research Prioritisation Framework that was robust, transparent, systematic, equity-focused, and able to consider the unique social and geographic context of maternal and perinatal health and healthcare in Aotearoa New Zealand. The aim was then then to use this framework to establish the future priorities for clinical trials and large cohort studies in addressing maternal and perinatal health and healthcare in Aotearoa New Zealand.
Subscriber Content
The full contents of this pages only available to subscribers.
Login, subscribe or email nzmj@nzma.org.nz to purchase this article.
