The recent publication of the interim report of the PHARMAC Review Panel[[1]] raises important questions of the role and processes of PHARMAC in securing equitable access to pharmaceuticals for all New Zealanders. The panel’s report has generated unfavourable media coverage,[[2]] with commentators highlighting the report’s observations that PHARMAC has a “fortress mentality that permits little transparency and openness.”[[1]] We consider it therefore both important and timely to report the key findings of a research project we carried out in 2017 assessing the fairness of decision-making in the New Zealand health system,[[3]] with a specific focus on PHARMAC and the district health boards (DHBs).

Our research assessed fairness of decision-making using the Decision-Making Audit Tool (DMAT) developed by Katharina Kieslich and Peter Littlejohns in the United Kingdom (UK).[[4]] Ethics approval was obtained from the University of Otago Human Ethics Committee (F16/008). We experienced difficulties in conducting this research with DHBs, due to a lack of publicly available documentation, the transparency on their websites and our inability to recruit appropriate staff for interviews (as reported in other studies).[[5]] In contrast, PHARMAC were supportive of the research. We were, therefore, able to review their publicly available documentation against the DMAT and conduct interviews with a number of their staff, as well as feeding back our findings to their Consumer Advisory Panel. We were also able to get input from them about the usefulness of DMAT and some of the issues around procedural justice and engagement with PHARMAC’s decision activities.

The DMAT draws on two frameworks for fair and legitimate priority setting in healthcare: accountability for reasonableness framework[[6,7]] and the social values framework.[[8]] The accountability for reasonableness framework is premised on the idea that it is easier to agree on fair process than on the fair principles for decision-making in priority setting and resource allocation activities.[[6,7]] Daniels and Sabin[[6,7]] describe four criteria that need to be met for procedural justice. They are:

• Transparent—open to public scrutiny
• Justifiable—supported by reasons considered relevant/appropriate
• Revisable—include a process to make changes or have the decision questioned
• Accountable—ensure that the above criteria are met

The social values framework developed by Clark and Weale[[8]] came from their work in health technology assessment and stipulated that there is a need to address content not just process. That is, that resource allocation and priority-setting decisions should be judged both on the way decisions are made and communicated and what accountability is shown for these decisions and related processes. In addition, the information that feeds into these decisions in terms of the clinical (evidence), the economic (cost) and values (public engagement) needs to be transparent and accessible. The DMAT, since publication of its first iteration,[[4]] has been refined through a variety of stakeholder engagement activities to eight domains with a total of 28 items to cover areas of process and content. The eight domains cover: Institutional Setting, Transparency, Accountability, Participation, Clinical Effectiveness, Cost-Effectiveness, Quality of Care, and Fairness (Table 1).

In 2017, we used PHARMAC’s and the DHBs’ websites to assess their performance against the DMAT items (as had previously been done in the UK with commissioning groups). This involved two team members (GR and ET) agreeing on working categorisations, reviewing documentation and webpages and cross-checking each other’s assessments.

Although we concluded that the DMAT would need further adaption for use in the New Zealand environment, we did find that it provided useful information. Seven of the eight domains of the DMAT applied to PHARMAC (Quality of Care was not relevant). Of the seven domains that did apply, PHARMAC received full points in five domains (Table 2). The two domains where it did not receive full scores were Accountability (13 of 15 points available) and Participation (21 of 25 points available). Overall, PHARMAC scored 119 points of possible 125 (excluding the one domain). This is a score of 95.2%. Where PHARMAC scored lower was tied to the lack of clarity of how stakeholder voices (across the spectrum) inform PHARMAC judgements using their Factors for Consideration.

Of note, as a comparator, DHBs scored significantly poorer with an average of 77.45 points of a possible 140 (range: 47–109). For most categories, a lack of information about decision-making and engagement activities was the confounding factor in understanding what it is that the DHBs may or may not be doing in terms of decision-making.

Our key findings from this research, carried out in 2017, are that PHARMAC has a clear decision-making process underpinned by values that are largely transparent. Important strengths are clear processes to communicate the basis of decisions on clinical and cost-effectiveness grounds determined by appropriate evidence. We consider PHARMAC stands alone in this regard when compared to other entities in the New Zealand health system.

The one area we considered there was room for improvement was around accountability (how open the organisation is about how it makes final decisions) and participation (consultation process and transparency around how the views of stakeholders influence final decisions). In these matters, our findings offer some support for the preliminary observations of the PHARMAC Review Panel[[1]] that there is scope to improve transparency around the weighing of Factors for Consideration and the engagement of public/patients in decision-making as well as issues of equity and Te Tiriti o Waitangi. It is important to note, however, that since we conducted this work PHARMAC has undertaken a review of its strategic direction[[9]] with a stated objective of improving stakeholder participation in decision-making.

Our conclusion is that PHARMAC’s decision-making framework is both fair and legitimate, noting that there is scope to further improve transparency around decision-making and stakeholder participation. More generally, we hope that the restructuring of the New Zealand health system,[[10]] with its abolition of DHBs, will lead to the proposed new health entities placing more focus on engagement, accountability and transparency when making decisions to achieve an equitable and sustainable healthcare system.[[11]]

Table 1: DMAT Domains and items. View Table 1.

Table 2: PHARMAC scores in DMAT framework. View Table 2.

The recent publication of the interim report of the PHARMAC Review Panel[[1]] raises important questions of the role and processes of PHARMAC in securing equitable access to pharmaceuticals for all New Zealanders. The panel’s report has generated unfavourable media coverage,[[2]] with commentators highlighting the report’s observations that PHARMAC has a “fortress mentality that permits little transparency and openness.”[[1]] We consider it therefore both important and timely to report the key findings of a research project we carried out in 2017 assessing the fairness of decision-making in the New Zealand health system,[[3]] with a specific focus on PHARMAC and the district health boards (DHBs).

Our research assessed fairness of decision-making using the Decision-Making Audit Tool (DMAT) developed by Katharina Kieslich and Peter Littlejohns in the United Kingdom (UK).[[4]] Ethics approval was obtained from the University of Otago Human Ethics Committee (F16/008). We experienced difficulties in conducting this research with DHBs, due to a lack of publicly available documentation, the transparency on their websites and our inability to recruit appropriate staff for interviews (as reported in other studies).[[5]] In contrast, PHARMAC were supportive of the research. We were, therefore, able to review their publicly available documentation against the DMAT and conduct interviews with a number of their staff, as well as feeding back our findings to their Consumer Advisory Panel. We were also able to get input from them about the usefulness of DMAT and some of the issues around procedural justice and engagement with PHARMAC’s decision activities.

The DMAT draws on two frameworks for fair and legitimate priority setting in healthcare: accountability for reasonableness framework[[6,7]] and the social values framework.[[8]] The accountability for reasonableness framework is premised on the idea that it is easier to agree on fair process than on the fair principles for decision-making in priority setting and resource allocation activities.[[6,7]] Daniels and Sabin[[6,7]] describe four criteria that need to be met for procedural justice. They are:

• Transparent—open to public scrutiny
• Justifiable—supported by reasons considered relevant/appropriate
• Revisable—include a process to make changes or have the decision questioned
• Accountable—ensure that the above criteria are met

The social values framework developed by Clark and Weale[[8]] came from their work in health technology assessment and stipulated that there is a need to address content not just process. That is, that resource allocation and priority-setting decisions should be judged both on the way decisions are made and communicated and what accountability is shown for these decisions and related processes. In addition, the information that feeds into these decisions in terms of the clinical (evidence), the economic (cost) and values (public engagement) needs to be transparent and accessible. The DMAT, since publication of its first iteration,[[4]] has been refined through a variety of stakeholder engagement activities to eight domains with a total of 28 items to cover areas of process and content. The eight domains cover: Institutional Setting, Transparency, Accountability, Participation, Clinical Effectiveness, Cost-Effectiveness, Quality of Care, and Fairness (Table 1).

In 2017, we used PHARMAC’s and the DHBs’ websites to assess their performance against the DMAT items (as had previously been done in the UK with commissioning groups). This involved two team members (GR and ET) agreeing on working categorisations, reviewing documentation and webpages and cross-checking each other’s assessments.

Although we concluded that the DMAT would need further adaption for use in the New Zealand environment, we did find that it provided useful information. Seven of the eight domains of the DMAT applied to PHARMAC (Quality of Care was not relevant). Of the seven domains that did apply, PHARMAC received full points in five domains (Table 2). The two domains where it did not receive full scores were Accountability (13 of 15 points available) and Participation (21 of 25 points available). Overall, PHARMAC scored 119 points of possible 125 (excluding the one domain). This is a score of 95.2%. Where PHARMAC scored lower was tied to the lack of clarity of how stakeholder voices (across the spectrum) inform PHARMAC judgements using their Factors for Consideration.

Of note, as a comparator, DHBs scored significantly poorer with an average of 77.45 points of a possible 140 (range: 47–109). For most categories, a lack of information about decision-making and engagement activities was the confounding factor in understanding what it is that the DHBs may or may not be doing in terms of decision-making.

Our key findings from this research, carried out in 2017, are that PHARMAC has a clear decision-making process underpinned by values that are largely transparent. Important strengths are clear processes to communicate the basis of decisions on clinical and cost-effectiveness grounds determined by appropriate evidence. We consider PHARMAC stands alone in this regard when compared to other entities in the New Zealand health system.

The one area we considered there was room for improvement was around accountability (how open the organisation is about how it makes final decisions) and participation (consultation process and transparency around how the views of stakeholders influence final decisions). In these matters, our findings offer some support for the preliminary observations of the PHARMAC Review Panel[[1]] that there is scope to improve transparency around the weighing of Factors for Consideration and the engagement of public/patients in decision-making as well as issues of equity and Te Tiriti o Waitangi. It is important to note, however, that since we conducted this work PHARMAC has undertaken a review of its strategic direction[[9]] with a stated objective of improving stakeholder participation in decision-making.

Our conclusion is that PHARMAC’s decision-making framework is both fair and legitimate, noting that there is scope to further improve transparency around decision-making and stakeholder participation. More generally, we hope that the restructuring of the New Zealand health system,[[10]] with its abolition of DHBs, will lead to the proposed new health entities placing more focus on engagement, accountability and transparency when making decisions to achieve an equitable and sustainable healthcare system.[[11]]

Table 1: DMAT Domains and items. View Table 1.

Table 2: PHARMAC scores in DMAT framework. View Table 2.