View Article PDF

Crisis resolution (CR) teams are now an established part of the mental health services in many parts of the world.1 Typically, CR teams aim to provide 24-hour access, rapid assessment, intensive short-term treatment and care of people at home where possible.2

Almost by definition, needing help from a CR team is a highly stressful time for consumers and their families. These situations may also be difficult for referrers, such as general practitioners, trying to organise urgent and appropriate help for people in crisis. Studies have evaluated the effectiveness, impact and service satisfaction of CR teams among consumers.3–6 However, few recent studies have examined the service satisfaction of family3,7 and referrers,8 despite the importance of partnerships with these people being recognised.1,9 Importantly, no known studies have reported the service satisfaction of all three key stakeholders (consumers, family and referrers) for the same case. Gaining more than one perspective on service satisfaction may provide insights on areas of agreement and disagreement among people who use CR.

The present study sought to systematically assess the service satisfaction of consumers, their families and referrers who have recently used crisis resolution services.

Method

Clinical context

In Christchurch (Canterbury, New Zealand), the Specialist Mental Health Services were re-configured in 2014 and the way that help was provided to people with urgent mental health needs was changed. The new model of care involved a CR function being built into the existing four Specialist Mental Health teams which cater for separate geographical areas within Canterbury. For each of these teams, CR means providing 24-hour access, seeing consumers where they want to be seen (such as in their own home) where feasible, providing a high level of integration between inpatient and outpatient care, and minimising meeting new staff and unnecessary assessments. Prior to the present study, only anecdotal evidence existed about how the people who use CR experience it.

Participants

Consecutive consumers recently discharged from CR over a five-week period in 2015 (last week of October 2015 until the end of November 2015) were recruited. For participation in the study, consumers were required to meet the following criteria.

Inclusion criteria:

  • Had face-to-face contact with CR within the previous six weeks;
  • Discharged from being a CR ‘case’ within the past seven days;
  • 18–65 years;
  • Contact information available;
  • Currently residing in New Zealand;
  • Able to adequately participate in telephone interview (eg, sufficient English language);
  • Able to provide informed consent (eg, not too unwell);
  • Consent given.

Exclusion criteria:

  • Consumer refused contact with mental health services;
  • Consumer opted out of being contacted about the study;
  • Participation in the study deemed to be potentially distressing, unhelpful or harmful to the consumer.

‘Family’ was broadly defined to include any people who had been involved in the consumer’s recent care with CR in a non-professional capacity (eg, partner, friend, Pastor, neighbour who facilitated involvement with CR and/or supported the consumer).

‘Referrer’was broadly defined to include any people who had been involved in the consumer’s recent care with CR in a professional capacity (eg, general practitioner, psychiatrist or counsellor who facilitated involvement with CR and/or worked alongside CR).

Design

As part of our ‘no surprises’ approach, consumers were made aware of the study at the outset of their contact with CR in a range of ways (eg, information sheet placed in their consumer pack and flyers placed in waiting rooms). However, they were not approached about the possibility of participation in the study until discharge from the service (ie, when they were not in current crisis). At discharge from CR, consumers were given the opportunity to opt out of being contacted about the study by their treating clinician, and via a letter from the clinical liaison (M Joan Taylor). Also, if clinicians felt concerned that participation may be potentially distressing, unhelpful or harmful to the consumer (see exclusion criteria), then they contacted the clinical liaison and made her aware of this, and these consumers were not approached about the study.

Consumers who met inclusion and exclusion criteria for the study were then phoned by the research coordinator (Madeline Weston), and verbal consent was sought for participation in a brief, structured telephone interview with her. At the completion of the interview, consumers were asked if there had been a family member or a referrer who had been involved in their care during their recent involvement with CR, and if they agreed to them being contacted and interviewed. These people were then contacted, and verbally consenting participants completed a brief, structured telephone interview with the research coordinator.

Measures

Brief structured interviews were conducted with participants over the telephone. Structured questions were asked, and the interviewer took care to ensure that the participant fully understood the questions before a rating was sought from the participant. Questions were designed to assess global satisfaction with care and satisfaction with specific aspects of care, and were developed following an examination of the literature on treatment satisfaction and discussions with staff around the aims of the re-configured service (eg, seeing consumers where they want to be seen, such as in their own home, where feasible). Nine forced-choice questions and two open-ended questions were asked. A deliberate attempt was made to word questions using plain English, and to keep ratings simple and straightforward. Table 1 shows the questions that were asked. Consumers and family were asked all 11 questions. Referrers were asked the first four questions, plus an additional question addressing communication. Consumers were also asked brief demographic questions assessing age category, ethnicity and gender.

Table 1: Structured interview questions for consumers.

Analysis

A low rate of missing data was found (>95% participants answered each question). Summary variables were calculated for some measures due to low numbers in some response categories (eg, New Zealand European: yes/no). Percentages were rounded to whole numbers using Swedish rounding, to avoid spurious specificity. ‘Not applicable’ ratings were excluded from the analyses. For agreement analyses, only participants who provided a definitive response were included (ie, people who said that they were ‘unsure’ were excluded), and positive response categories such as ‘good’ and ‘ok’ were combined to ease analysis. Analyses were primarily descriptive. Chi-square tests were conducted to see if any demographic variables (age category, ethnicity and gender) were associated with dissatisfaction on items assessing global satisfaction and specific aspects of care. Data were analysed using the statistical package, SPSS (version 22.0; Armonk, NY: IBM Corp).

For open-ended questions, themes were identified among responses, and the frequency with which these were reported was calculated. More formal analyses were not conducted for open-ended questions.

Ethical approval

This study was approved by the University of Otago Human Ethics Committee (Health). HD15/023

Results

Recruitment

Figure 1 summarises the recruitment of participants for the study. The response rates were 61% for consumers (75/123), 76% for family (22/29) and 76% for referrers (16/21).

Figure 1: Schematic summarising participant recruitment.

c

Description of participants

The 75 consumers were roughly evenly split by gender (female = 51%; male = 49%). They were most likely to be aged 18–24 years, although all age categories up to 64 years were represented (18–24 years = 34%; 25–34 years = 24%; 35–44 years = 19%; 45–64 years = 23%). Most consumers identified as being New Zealand European (81%), with the next most common ethnic groups being Māori (14%), ‘other’ (10%), Chinese (1%), Indian (1%), and Samoan (1%). The demographic characteristics of the sample were broadly consistent with people who use CR. (Taylor J, pers. comm., 2016).

The 22 family members were most likely to be the consumer’s partner (n=11) or mother (n=8), followed by father (n=1), sibling (n=1) or friend (n =1).

The 16 referrers were most likely to be the consumer’s general practitioner (n =13), followed by counsellor (n=2) and psychiatrist (n=1).

Forced-choice questions

Table 2 shows the responses of consumers, family and referrers who answered forced-choice questions assessing both global satisfaction and satisfaction with specific aspects of care, plus the agreement among raters.

Table 2: Consumer, family and referrer responses to forced-choice structured interview questions, plus agreement among raters.

c


c

* For all agreement analyses, only participants who provided a definitive response were included (ie, people who said they were ‘unsure’ were excluded), and positive responses such as ‘good’ and ‘OK’ were combined.
** Agreement indicates percentage agreement among consumer, family and referrer (n=34).
*** Agreement indicates percentage agreement between consumer and family, as referrers did not rate these questions (n=22).
ɸ For each cell where this symbol is used, one participant did not answer this question. In cells without this symbol, all participants answered the question.

Predictors of dissatisfaction

Consumers aged between 25-34 years were significantly more likely to rate their care as being ‘bad’ (overall rating) than people in other age categories (Chi-square value =9.6, df =1, p=.002). Out of the 74 consumers who answered this question, 18 were aged between 25-34 years, and 3 of these consumers said that their care had been ‘bad.’ No other consumers in any age category rated their care as being ‘bad’. No other demographic variables such as ethnicity or gender were significantly associated with dissatisfaction on either of the two global measures of service satisfaction. No demographic variables among consumers were significantly associated with dissatisfaction on any items assessing satisfaction with specific aspects of care.

Open questions

Consumers were less likely to comment on how the service could be improved (51% commented), than on what had been good or helpful about the service (75% commented). Family and referrers were roughly equally likely to comment on these two questions (65-69% commented on both). Table 3 shows comments that were made about how the service could be improved and what was especially helpful or good about the service (grouped according to themes identified), plus how frequently these comments were made by whom. These show that people were most likely to comment on the manner of staff (specific desirable attributes are described) and the treatment that had been received. A diverse range of other specific suggestions were also made. These findings will be discussed in the following section to avoid repetition.

Table 3: Summary of how frequently comments were made by consumers, family and referrers.

C = Consumer
F = Family
R = Referrer

Discussion

The present study sought to systematically assess the service satisfaction of consecutive consumers discharged from CR over a five-week period, plus family and referrers. The response rates for the study (61–76%) compare favourably with other similar studies, where response rates of less than 30% have been reported.3

High levels of satisfaction (86–96%) were reported by consumers (n=75), family (n=22) and referrers (n=16) on two forced-choice questions assessing global satisfaction with the service. If consumers were dissatisfied with their care on these global questions, they were significantly more likely to be aged 25–34 years.

The majority of participants (>75%) responded positively to questions assessing most specific aspects of care (exceptions were about being given a choice about where seen and involvement of family). These findings are broadly consistent with other studies that have examined satisfaction with CR (separately) among consumers, family or referrers.7,8

A unique feature of the present study was that the views of all three key stakeholders were assessed, which enabled agreement among them to be calculated. A high level of agreement was found among participants on global satisfaction (>85%) and on most specific aspects of care (>75%). The exception was for choice about where seen. Overall, these findings provide some level of reassurance that consumers, family and referrers tend to broadly agree about service satisfaction from CR.

For the open questions, consumers were more likely to comment on what had been good or especially helpful about CR, than about how the service could be improved. Family and referrers commented roughly equally on these questions. The themes that were identified as being important were: staff issues, access to the service, where consumers were seen, the interventions provided or facilitated, involvement of family, communication/liaison/records issues and transport). For example, it was especially important to participants that the consumer received effective treatment of sufficient duration, and that staff were warm, interested, empathetic, respectful, not rushed or dismissive, that they listened well, treated people as individuals and were positive and reassuring.

Not all of the issues that were raised by participants related to the wellbeing of consumers. Despite the fact that study participants were in the midst of managing their own or a family member’s mental health crisis, five participants (four consumers and one family member) commented that they had felt concerned about the safety of staff, and they suggested improved security for staff. One of these participants was a consumer who has been physically aggressive themselves in the context of being acutely unwell, and they were deeply troubled by this when their mental state improved.

Other challenges for staff were identified. For example, five consumers suggested a less rehearsed/protocol like/impersonal approach. However, assessing consumers in a systematic way covering all key areas such as current risk, without them feeling that they are simply being put through a protocol, may not always be easy to achieve. Also, some seemingly opposing comments were received from participants. For example, four participants (all family) said they wanted more opportunities for the consumer to be seen at home and/or to have family more involved, but three participants (all consumers) said that they wanted to be put under less pressure to have family involved or wanted more care taken to get permission for this. These findings highlight that it might be difficult sometimes for staff to get it right for everyone.

A diverse range of other issues were also raised, especially with regard to suggestions for improving the service. For example, a consumer who had had a panic attack while waiting to be seen suggested a more private waiting area be available, and a GP saw the need for more assistance to be available when consumers had a mental health problem, but were also intoxicated, and family did not feel equipped to manage these volatile situations.

Overall, many of the present findings have implications for funding (ie, sufficient funding is needed), and for how staff are trained, supported, supervised and kept safe. Also, it would seem important that service evaluation tools incorporate questions that assess how people experience staff (eg, warmth). Future research should re-evaluate the impact of demographic variables on service dissatisfaction with a larger sample size.

The key limitation of the study was the relatively small number of family (n=22) and referrers (n=16), which precluded formal statistical comparisons due to low power. The small sample size and low power may also have impaired our ability to detect differences on analyses involving demographic variables. At the same time, while only three demographic variables were examined (age, ethnicity and gender), tests were performed for all 10 survey questions. The use of multiple comparisons may have increased the chances of a spurious finding (only one statistically significant finding was obtained).

Not all consumers who had recently been discharged from the service were eligible for participation (eg, those who were too unwell), so some consumer groups will be under-represented in the present study. Also, it is possible that evaluations of CR within a different service delivery context may find that different issues are raised by participants about what is helpful and unhelpful. Finally, an important perspective that is missing from the current research is the view of staff. Future research could valuably focus on what staff think helps them to feel supported, effective and safe in their role.

Conclusion

High levels of satisfaction and agreement were found among consumers, family and referrers with CR. If consumers were dissatisfied with overall care, they were most likely to be aged 25–34 years. Open-ended questions identified which issues matter the most to key stakeholders, which may have implications for service evaluation tool.

Summary

Abstract

Aim

To systematically assess the service satisfaction of consumers, their families and referrers with crisis resolution (CR).

Method

Consecutive consumers discharged after receiving CR over a five-week period were potentially eligible for participation, together with their family and referrer (broadly defined). Structured telephone interviews were conducted and involved forced-choice questions assessing global satisfaction and satisfaction with specific aspects of care, plus two open-ended questions.

Results

Participants were 75 consumers, 22 family and 16 referrers. High levels of satisfaction were seen for all participants for both global (86-96%) and most specific aspects of care (>75%). If consumers were dissatisfied with their overall care, they were significantly more likely to be aged 25-34 years of age. High levels of agreement among raters were found for global satisfaction (>85%) and most specific aspects of care (>70%), which provides some level of reassurance for staff. Open-ended questions showed that having effective treatment of sufficient duration and staff manner were most important to participants.

Conclusion

High levels of satisfaction and agreement were found among consumers, family and referrers with CR. Open-ended questions identified which issues matter the most to key stakeholders, which may have implications for service evaluation tools.

Author Information

'- Frances A Carter, Clinical Psychologist, Scientific Officer, Senior Clinical Lecturer, University of Otago, Christchurch; M Joan Taylor, Clinical Liaison, Nurse Consultant, Specialist Mental Health Service, Canterbury District Health Board (CDHB), Chr

Acknowledgements

Andrea Bartram set up the research data base and provided supervision around data management, Vicki Dent and Warren Campbell-Trotter monitored consumer data bases for adverse events relevant to recruitment, Joseph Boden provided statistical advice and Virginia McIntosh provided comment on an early draft of the manuscript.

Correspondence

Dr Frances Carter, Clinical Research Unit, Department of Psychological Medicine, PO Box 4345, Christchurch.

Correspondence Email

frances.carter@cdhb.health.nz

Competing Interests

All authors were employed by the Canterbury District Health Board (CDHB) during the conduct of the study. The current project was in part funded by a Summer Studentship grant from the University of Otago.

  1. Lloyd-Evans B, Johnson F. Crisis resolution teams: how are they performing? Ment Health Today. 2014: 18–9.
  2. Wheeler C, Lloyd-Evans B, Churchard A, et al. Implementation of the Crisis Resolution Team model in adult mental health settings: a systematic review. BMC Psychiatry. 2015; 15:74.
  3. Barker V, Taylor M, Kader I, et al. Impact of crisis resolution and home treatment services on user experience and admission to psychiatric hospital. The Psychiatrist. 2011; 35:106–10.
  4. Carpenter R, Falkenburg J, White T, Tracy D. Crisis teams: systematic review of their effectiveness in practice. The Psychiatrist. 2013; 37:232–7.
  5. Irving CB, Adams CE, Rice K. Crisis intervention for people with severe mental illness (Review). Cochrane Database Syst Rev. 2006.
  6. Winness MG, Borg M, Kim HS. Service users’ experiences with help and support from crisis resolution teams. A literature review. J Ment Health. 2010; 19:75–87.
  7. Ampelas JF, Robin M, Caria A, et al. Patient and their relatives’ satisfaction regarding a home-based crisis intervention provided by a mobile crisis team. Encephale. 2005; 31:127–41.
  8. McCauley M, Bergin A, Bannon H, et al. How do GPs experience home-based treatment for acute psychiatric disorders? Primary Care Commun. 2005; 10:159–63.
  9. Worthington A, Rooney P. The Triangle of Care. London: National Mental Health Development Unit, 2010.

For the PDF of this article,
contact nzmj@nzma.org.nz

View Article PDF

Crisis resolution (CR) teams are now an established part of the mental health services in many parts of the world.1 Typically, CR teams aim to provide 24-hour access, rapid assessment, intensive short-term treatment and care of people at home where possible.2

Almost by definition, needing help from a CR team is a highly stressful time for consumers and their families. These situations may also be difficult for referrers, such as general practitioners, trying to organise urgent and appropriate help for people in crisis. Studies have evaluated the effectiveness, impact and service satisfaction of CR teams among consumers.3–6 However, few recent studies have examined the service satisfaction of family3,7 and referrers,8 despite the importance of partnerships with these people being recognised.1,9 Importantly, no known studies have reported the service satisfaction of all three key stakeholders (consumers, family and referrers) for the same case. Gaining more than one perspective on service satisfaction may provide insights on areas of agreement and disagreement among people who use CR.

The present study sought to systematically assess the service satisfaction of consumers, their families and referrers who have recently used crisis resolution services.

Method

Clinical context

In Christchurch (Canterbury, New Zealand), the Specialist Mental Health Services were re-configured in 2014 and the way that help was provided to people with urgent mental health needs was changed. The new model of care involved a CR function being built into the existing four Specialist Mental Health teams which cater for separate geographical areas within Canterbury. For each of these teams, CR means providing 24-hour access, seeing consumers where they want to be seen (such as in their own home) where feasible, providing a high level of integration between inpatient and outpatient care, and minimising meeting new staff and unnecessary assessments. Prior to the present study, only anecdotal evidence existed about how the people who use CR experience it.

Participants

Consecutive consumers recently discharged from CR over a five-week period in 2015 (last week of October 2015 until the end of November 2015) were recruited. For participation in the study, consumers were required to meet the following criteria.

Inclusion criteria:

  • Had face-to-face contact with CR within the previous six weeks;
  • Discharged from being a CR ‘case’ within the past seven days;
  • 18–65 years;
  • Contact information available;
  • Currently residing in New Zealand;
  • Able to adequately participate in telephone interview (eg, sufficient English language);
  • Able to provide informed consent (eg, not too unwell);
  • Consent given.

Exclusion criteria:

  • Consumer refused contact with mental health services;
  • Consumer opted out of being contacted about the study;
  • Participation in the study deemed to be potentially distressing, unhelpful or harmful to the consumer.

‘Family’ was broadly defined to include any people who had been involved in the consumer’s recent care with CR in a non-professional capacity (eg, partner, friend, Pastor, neighbour who facilitated involvement with CR and/or supported the consumer).

‘Referrer’was broadly defined to include any people who had been involved in the consumer’s recent care with CR in a professional capacity (eg, general practitioner, psychiatrist or counsellor who facilitated involvement with CR and/or worked alongside CR).

Design

As part of our ‘no surprises’ approach, consumers were made aware of the study at the outset of their contact with CR in a range of ways (eg, information sheet placed in their consumer pack and flyers placed in waiting rooms). However, they were not approached about the possibility of participation in the study until discharge from the service (ie, when they were not in current crisis). At discharge from CR, consumers were given the opportunity to opt out of being contacted about the study by their treating clinician, and via a letter from the clinical liaison (M Joan Taylor). Also, if clinicians felt concerned that participation may be potentially distressing, unhelpful or harmful to the consumer (see exclusion criteria), then they contacted the clinical liaison and made her aware of this, and these consumers were not approached about the study.

Consumers who met inclusion and exclusion criteria for the study were then phoned by the research coordinator (Madeline Weston), and verbal consent was sought for participation in a brief, structured telephone interview with her. At the completion of the interview, consumers were asked if there had been a family member or a referrer who had been involved in their care during their recent involvement with CR, and if they agreed to them being contacted and interviewed. These people were then contacted, and verbally consenting participants completed a brief, structured telephone interview with the research coordinator.

Measures

Brief structured interviews were conducted with participants over the telephone. Structured questions were asked, and the interviewer took care to ensure that the participant fully understood the questions before a rating was sought from the participant. Questions were designed to assess global satisfaction with care and satisfaction with specific aspects of care, and were developed following an examination of the literature on treatment satisfaction and discussions with staff around the aims of the re-configured service (eg, seeing consumers where they want to be seen, such as in their own home, where feasible). Nine forced-choice questions and two open-ended questions were asked. A deliberate attempt was made to word questions using plain English, and to keep ratings simple and straightforward. Table 1 shows the questions that were asked. Consumers and family were asked all 11 questions. Referrers were asked the first four questions, plus an additional question addressing communication. Consumers were also asked brief demographic questions assessing age category, ethnicity and gender.

Table 1: Structured interview questions for consumers.

Analysis

A low rate of missing data was found (>95% participants answered each question). Summary variables were calculated for some measures due to low numbers in some response categories (eg, New Zealand European: yes/no). Percentages were rounded to whole numbers using Swedish rounding, to avoid spurious specificity. ‘Not applicable’ ratings were excluded from the analyses. For agreement analyses, only participants who provided a definitive response were included (ie, people who said that they were ‘unsure’ were excluded), and positive response categories such as ‘good’ and ‘ok’ were combined to ease analysis. Analyses were primarily descriptive. Chi-square tests were conducted to see if any demographic variables (age category, ethnicity and gender) were associated with dissatisfaction on items assessing global satisfaction and specific aspects of care. Data were analysed using the statistical package, SPSS (version 22.0; Armonk, NY: IBM Corp).

For open-ended questions, themes were identified among responses, and the frequency with which these were reported was calculated. More formal analyses were not conducted for open-ended questions.

Ethical approval

This study was approved by the University of Otago Human Ethics Committee (Health). HD15/023

Results

Recruitment

Figure 1 summarises the recruitment of participants for the study. The response rates were 61% for consumers (75/123), 76% for family (22/29) and 76% for referrers (16/21).

Figure 1: Schematic summarising participant recruitment.

c

Description of participants

The 75 consumers were roughly evenly split by gender (female = 51%; male = 49%). They were most likely to be aged 18–24 years, although all age categories up to 64 years were represented (18–24 years = 34%; 25–34 years = 24%; 35–44 years = 19%; 45–64 years = 23%). Most consumers identified as being New Zealand European (81%), with the next most common ethnic groups being Māori (14%), ‘other’ (10%), Chinese (1%), Indian (1%), and Samoan (1%). The demographic characteristics of the sample were broadly consistent with people who use CR. (Taylor J, pers. comm., 2016).

The 22 family members were most likely to be the consumer’s partner (n=11) or mother (n=8), followed by father (n=1), sibling (n=1) or friend (n =1).

The 16 referrers were most likely to be the consumer’s general practitioner (n =13), followed by counsellor (n=2) and psychiatrist (n=1).

Forced-choice questions

Table 2 shows the responses of consumers, family and referrers who answered forced-choice questions assessing both global satisfaction and satisfaction with specific aspects of care, plus the agreement among raters.

Table 2: Consumer, family and referrer responses to forced-choice structured interview questions, plus agreement among raters.

c


c

* For all agreement analyses, only participants who provided a definitive response were included (ie, people who said they were ‘unsure’ were excluded), and positive responses such as ‘good’ and ‘OK’ were combined.
** Agreement indicates percentage agreement among consumer, family and referrer (n=34).
*** Agreement indicates percentage agreement between consumer and family, as referrers did not rate these questions (n=22).
ɸ For each cell where this symbol is used, one participant did not answer this question. In cells without this symbol, all participants answered the question.

Predictors of dissatisfaction

Consumers aged between 25-34 years were significantly more likely to rate their care as being ‘bad’ (overall rating) than people in other age categories (Chi-square value =9.6, df =1, p=.002). Out of the 74 consumers who answered this question, 18 were aged between 25-34 years, and 3 of these consumers said that their care had been ‘bad.’ No other consumers in any age category rated their care as being ‘bad’. No other demographic variables such as ethnicity or gender were significantly associated with dissatisfaction on either of the two global measures of service satisfaction. No demographic variables among consumers were significantly associated with dissatisfaction on any items assessing satisfaction with specific aspects of care.

Open questions

Consumers were less likely to comment on how the service could be improved (51% commented), than on what had been good or helpful about the service (75% commented). Family and referrers were roughly equally likely to comment on these two questions (65-69% commented on both). Table 3 shows comments that were made about how the service could be improved and what was especially helpful or good about the service (grouped according to themes identified), plus how frequently these comments were made by whom. These show that people were most likely to comment on the manner of staff (specific desirable attributes are described) and the treatment that had been received. A diverse range of other specific suggestions were also made. These findings will be discussed in the following section to avoid repetition.

Table 3: Summary of how frequently comments were made by consumers, family and referrers.

C = Consumer
F = Family
R = Referrer

Discussion

The present study sought to systematically assess the service satisfaction of consecutive consumers discharged from CR over a five-week period, plus family and referrers. The response rates for the study (61–76%) compare favourably with other similar studies, where response rates of less than 30% have been reported.3

High levels of satisfaction (86–96%) were reported by consumers (n=75), family (n=22) and referrers (n=16) on two forced-choice questions assessing global satisfaction with the service. If consumers were dissatisfied with their care on these global questions, they were significantly more likely to be aged 25–34 years.

The majority of participants (>75%) responded positively to questions assessing most specific aspects of care (exceptions were about being given a choice about where seen and involvement of family). These findings are broadly consistent with other studies that have examined satisfaction with CR (separately) among consumers, family or referrers.7,8

A unique feature of the present study was that the views of all three key stakeholders were assessed, which enabled agreement among them to be calculated. A high level of agreement was found among participants on global satisfaction (>85%) and on most specific aspects of care (>75%). The exception was for choice about where seen. Overall, these findings provide some level of reassurance that consumers, family and referrers tend to broadly agree about service satisfaction from CR.

For the open questions, consumers were more likely to comment on what had been good or especially helpful about CR, than about how the service could be improved. Family and referrers commented roughly equally on these questions. The themes that were identified as being important were: staff issues, access to the service, where consumers were seen, the interventions provided or facilitated, involvement of family, communication/liaison/records issues and transport). For example, it was especially important to participants that the consumer received effective treatment of sufficient duration, and that staff were warm, interested, empathetic, respectful, not rushed or dismissive, that they listened well, treated people as individuals and were positive and reassuring.

Not all of the issues that were raised by participants related to the wellbeing of consumers. Despite the fact that study participants were in the midst of managing their own or a family member’s mental health crisis, five participants (four consumers and one family member) commented that they had felt concerned about the safety of staff, and they suggested improved security for staff. One of these participants was a consumer who has been physically aggressive themselves in the context of being acutely unwell, and they were deeply troubled by this when their mental state improved.

Other challenges for staff were identified. For example, five consumers suggested a less rehearsed/protocol like/impersonal approach. However, assessing consumers in a systematic way covering all key areas such as current risk, without them feeling that they are simply being put through a protocol, may not always be easy to achieve. Also, some seemingly opposing comments were received from participants. For example, four participants (all family) said they wanted more opportunities for the consumer to be seen at home and/or to have family more involved, but three participants (all consumers) said that they wanted to be put under less pressure to have family involved or wanted more care taken to get permission for this. These findings highlight that it might be difficult sometimes for staff to get it right for everyone.

A diverse range of other issues were also raised, especially with regard to suggestions for improving the service. For example, a consumer who had had a panic attack while waiting to be seen suggested a more private waiting area be available, and a GP saw the need for more assistance to be available when consumers had a mental health problem, but were also intoxicated, and family did not feel equipped to manage these volatile situations.

Overall, many of the present findings have implications for funding (ie, sufficient funding is needed), and for how staff are trained, supported, supervised and kept safe. Also, it would seem important that service evaluation tools incorporate questions that assess how people experience staff (eg, warmth). Future research should re-evaluate the impact of demographic variables on service dissatisfaction with a larger sample size.

The key limitation of the study was the relatively small number of family (n=22) and referrers (n=16), which precluded formal statistical comparisons due to low power. The small sample size and low power may also have impaired our ability to detect differences on analyses involving demographic variables. At the same time, while only three demographic variables were examined (age, ethnicity and gender), tests were performed for all 10 survey questions. The use of multiple comparisons may have increased the chances of a spurious finding (only one statistically significant finding was obtained).

Not all consumers who had recently been discharged from the service were eligible for participation (eg, those who were too unwell), so some consumer groups will be under-represented in the present study. Also, it is possible that evaluations of CR within a different service delivery context may find that different issues are raised by participants about what is helpful and unhelpful. Finally, an important perspective that is missing from the current research is the view of staff. Future research could valuably focus on what staff think helps them to feel supported, effective and safe in their role.

Conclusion

High levels of satisfaction and agreement were found among consumers, family and referrers with CR. If consumers were dissatisfied with overall care, they were most likely to be aged 25–34 years. Open-ended questions identified which issues matter the most to key stakeholders, which may have implications for service evaluation tool.

Summary

Abstract

Aim

To systematically assess the service satisfaction of consumers, their families and referrers with crisis resolution (CR).

Method

Consecutive consumers discharged after receiving CR over a five-week period were potentially eligible for participation, together with their family and referrer (broadly defined). Structured telephone interviews were conducted and involved forced-choice questions assessing global satisfaction and satisfaction with specific aspects of care, plus two open-ended questions.

Results

Participants were 75 consumers, 22 family and 16 referrers. High levels of satisfaction were seen for all participants for both global (86-96%) and most specific aspects of care (>75%). If consumers were dissatisfied with their overall care, they were significantly more likely to be aged 25-34 years of age. High levels of agreement among raters were found for global satisfaction (>85%) and most specific aspects of care (>70%), which provides some level of reassurance for staff. Open-ended questions showed that having effective treatment of sufficient duration and staff manner were most important to participants.

Conclusion

High levels of satisfaction and agreement were found among consumers, family and referrers with CR. Open-ended questions identified which issues matter the most to key stakeholders, which may have implications for service evaluation tools.

Author Information

'- Frances A Carter, Clinical Psychologist, Scientific Officer, Senior Clinical Lecturer, University of Otago, Christchurch; M Joan Taylor, Clinical Liaison, Nurse Consultant, Specialist Mental Health Service, Canterbury District Health Board (CDHB), Chr

Acknowledgements

Andrea Bartram set up the research data base and provided supervision around data management, Vicki Dent and Warren Campbell-Trotter monitored consumer data bases for adverse events relevant to recruitment, Joseph Boden provided statistical advice and Virginia McIntosh provided comment on an early draft of the manuscript.

Correspondence

Dr Frances Carter, Clinical Research Unit, Department of Psychological Medicine, PO Box 4345, Christchurch.

Correspondence Email

frances.carter@cdhb.health.nz

Competing Interests

All authors were employed by the Canterbury District Health Board (CDHB) during the conduct of the study. The current project was in part funded by a Summer Studentship grant from the University of Otago.

  1. Lloyd-Evans B, Johnson F. Crisis resolution teams: how are they performing? Ment Health Today. 2014: 18–9.
  2. Wheeler C, Lloyd-Evans B, Churchard A, et al. Implementation of the Crisis Resolution Team model in adult mental health settings: a systematic review. BMC Psychiatry. 2015; 15:74.
  3. Barker V, Taylor M, Kader I, et al. Impact of crisis resolution and home treatment services on user experience and admission to psychiatric hospital. The Psychiatrist. 2011; 35:106–10.
  4. Carpenter R, Falkenburg J, White T, Tracy D. Crisis teams: systematic review of their effectiveness in practice. The Psychiatrist. 2013; 37:232–7.
  5. Irving CB, Adams CE, Rice K. Crisis intervention for people with severe mental illness (Review). Cochrane Database Syst Rev. 2006.
  6. Winness MG, Borg M, Kim HS. Service users’ experiences with help and support from crisis resolution teams. A literature review. J Ment Health. 2010; 19:75–87.
  7. Ampelas JF, Robin M, Caria A, et al. Patient and their relatives’ satisfaction regarding a home-based crisis intervention provided by a mobile crisis team. Encephale. 2005; 31:127–41.
  8. McCauley M, Bergin A, Bannon H, et al. How do GPs experience home-based treatment for acute psychiatric disorders? Primary Care Commun. 2005; 10:159–63.
  9. Worthington A, Rooney P. The Triangle of Care. London: National Mental Health Development Unit, 2010.

For the PDF of this article,
contact nzmj@nzma.org.nz

View Article PDF

Crisis resolution (CR) teams are now an established part of the mental health services in many parts of the world.1 Typically, CR teams aim to provide 24-hour access, rapid assessment, intensive short-term treatment and care of people at home where possible.2

Almost by definition, needing help from a CR team is a highly stressful time for consumers and their families. These situations may also be difficult for referrers, such as general practitioners, trying to organise urgent and appropriate help for people in crisis. Studies have evaluated the effectiveness, impact and service satisfaction of CR teams among consumers.3–6 However, few recent studies have examined the service satisfaction of family3,7 and referrers,8 despite the importance of partnerships with these people being recognised.1,9 Importantly, no known studies have reported the service satisfaction of all three key stakeholders (consumers, family and referrers) for the same case. Gaining more than one perspective on service satisfaction may provide insights on areas of agreement and disagreement among people who use CR.

The present study sought to systematically assess the service satisfaction of consumers, their families and referrers who have recently used crisis resolution services.

Method

Clinical context

In Christchurch (Canterbury, New Zealand), the Specialist Mental Health Services were re-configured in 2014 and the way that help was provided to people with urgent mental health needs was changed. The new model of care involved a CR function being built into the existing four Specialist Mental Health teams which cater for separate geographical areas within Canterbury. For each of these teams, CR means providing 24-hour access, seeing consumers where they want to be seen (such as in their own home) where feasible, providing a high level of integration between inpatient and outpatient care, and minimising meeting new staff and unnecessary assessments. Prior to the present study, only anecdotal evidence existed about how the people who use CR experience it.

Participants

Consecutive consumers recently discharged from CR over a five-week period in 2015 (last week of October 2015 until the end of November 2015) were recruited. For participation in the study, consumers were required to meet the following criteria.

Inclusion criteria:

  • Had face-to-face contact with CR within the previous six weeks;
  • Discharged from being a CR ‘case’ within the past seven days;
  • 18–65 years;
  • Contact information available;
  • Currently residing in New Zealand;
  • Able to adequately participate in telephone interview (eg, sufficient English language);
  • Able to provide informed consent (eg, not too unwell);
  • Consent given.

Exclusion criteria:

  • Consumer refused contact with mental health services;
  • Consumer opted out of being contacted about the study;
  • Participation in the study deemed to be potentially distressing, unhelpful or harmful to the consumer.

‘Family’ was broadly defined to include any people who had been involved in the consumer’s recent care with CR in a non-professional capacity (eg, partner, friend, Pastor, neighbour who facilitated involvement with CR and/or supported the consumer).

‘Referrer’was broadly defined to include any people who had been involved in the consumer’s recent care with CR in a professional capacity (eg, general practitioner, psychiatrist or counsellor who facilitated involvement with CR and/or worked alongside CR).

Design

As part of our ‘no surprises’ approach, consumers were made aware of the study at the outset of their contact with CR in a range of ways (eg, information sheet placed in their consumer pack and flyers placed in waiting rooms). However, they were not approached about the possibility of participation in the study until discharge from the service (ie, when they were not in current crisis). At discharge from CR, consumers were given the opportunity to opt out of being contacted about the study by their treating clinician, and via a letter from the clinical liaison (M Joan Taylor). Also, if clinicians felt concerned that participation may be potentially distressing, unhelpful or harmful to the consumer (see exclusion criteria), then they contacted the clinical liaison and made her aware of this, and these consumers were not approached about the study.

Consumers who met inclusion and exclusion criteria for the study were then phoned by the research coordinator (Madeline Weston), and verbal consent was sought for participation in a brief, structured telephone interview with her. At the completion of the interview, consumers were asked if there had been a family member or a referrer who had been involved in their care during their recent involvement with CR, and if they agreed to them being contacted and interviewed. These people were then contacted, and verbally consenting participants completed a brief, structured telephone interview with the research coordinator.

Measures

Brief structured interviews were conducted with participants over the telephone. Structured questions were asked, and the interviewer took care to ensure that the participant fully understood the questions before a rating was sought from the participant. Questions were designed to assess global satisfaction with care and satisfaction with specific aspects of care, and were developed following an examination of the literature on treatment satisfaction and discussions with staff around the aims of the re-configured service (eg, seeing consumers where they want to be seen, such as in their own home, where feasible). Nine forced-choice questions and two open-ended questions were asked. A deliberate attempt was made to word questions using plain English, and to keep ratings simple and straightforward. Table 1 shows the questions that were asked. Consumers and family were asked all 11 questions. Referrers were asked the first four questions, plus an additional question addressing communication. Consumers were also asked brief demographic questions assessing age category, ethnicity and gender.

Table 1: Structured interview questions for consumers.

Analysis

A low rate of missing data was found (>95% participants answered each question). Summary variables were calculated for some measures due to low numbers in some response categories (eg, New Zealand European: yes/no). Percentages were rounded to whole numbers using Swedish rounding, to avoid spurious specificity. ‘Not applicable’ ratings were excluded from the analyses. For agreement analyses, only participants who provided a definitive response were included (ie, people who said that they were ‘unsure’ were excluded), and positive response categories such as ‘good’ and ‘ok’ were combined to ease analysis. Analyses were primarily descriptive. Chi-square tests were conducted to see if any demographic variables (age category, ethnicity and gender) were associated with dissatisfaction on items assessing global satisfaction and specific aspects of care. Data were analysed using the statistical package, SPSS (version 22.0; Armonk, NY: IBM Corp).

For open-ended questions, themes were identified among responses, and the frequency with which these were reported was calculated. More formal analyses were not conducted for open-ended questions.

Ethical approval

This study was approved by the University of Otago Human Ethics Committee (Health). HD15/023

Results

Recruitment

Figure 1 summarises the recruitment of participants for the study. The response rates were 61% for consumers (75/123), 76% for family (22/29) and 76% for referrers (16/21).

Figure 1: Schematic summarising participant recruitment.

c

Description of participants

The 75 consumers were roughly evenly split by gender (female = 51%; male = 49%). They were most likely to be aged 18–24 years, although all age categories up to 64 years were represented (18–24 years = 34%; 25–34 years = 24%; 35–44 years = 19%; 45–64 years = 23%). Most consumers identified as being New Zealand European (81%), with the next most common ethnic groups being Māori (14%), ‘other’ (10%), Chinese (1%), Indian (1%), and Samoan (1%). The demographic characteristics of the sample were broadly consistent with people who use CR. (Taylor J, pers. comm., 2016).

The 22 family members were most likely to be the consumer’s partner (n=11) or mother (n=8), followed by father (n=1), sibling (n=1) or friend (n =1).

The 16 referrers were most likely to be the consumer’s general practitioner (n =13), followed by counsellor (n=2) and psychiatrist (n=1).

Forced-choice questions

Table 2 shows the responses of consumers, family and referrers who answered forced-choice questions assessing both global satisfaction and satisfaction with specific aspects of care, plus the agreement among raters.

Table 2: Consumer, family and referrer responses to forced-choice structured interview questions, plus agreement among raters.

c


c

* For all agreement analyses, only participants who provided a definitive response were included (ie, people who said they were ‘unsure’ were excluded), and positive responses such as ‘good’ and ‘OK’ were combined.
** Agreement indicates percentage agreement among consumer, family and referrer (n=34).
*** Agreement indicates percentage agreement between consumer and family, as referrers did not rate these questions (n=22).
ɸ For each cell where this symbol is used, one participant did not answer this question. In cells without this symbol, all participants answered the question.

Predictors of dissatisfaction

Consumers aged between 25-34 years were significantly more likely to rate their care as being ‘bad’ (overall rating) than people in other age categories (Chi-square value =9.6, df =1, p=.002). Out of the 74 consumers who answered this question, 18 were aged between 25-34 years, and 3 of these consumers said that their care had been ‘bad.’ No other consumers in any age category rated their care as being ‘bad’. No other demographic variables such as ethnicity or gender were significantly associated with dissatisfaction on either of the two global measures of service satisfaction. No demographic variables among consumers were significantly associated with dissatisfaction on any items assessing satisfaction with specific aspects of care.

Open questions

Consumers were less likely to comment on how the service could be improved (51% commented), than on what had been good or helpful about the service (75% commented). Family and referrers were roughly equally likely to comment on these two questions (65-69% commented on both). Table 3 shows comments that were made about how the service could be improved and what was especially helpful or good about the service (grouped according to themes identified), plus how frequently these comments were made by whom. These show that people were most likely to comment on the manner of staff (specific desirable attributes are described) and the treatment that had been received. A diverse range of other specific suggestions were also made. These findings will be discussed in the following section to avoid repetition.

Table 3: Summary of how frequently comments were made by consumers, family and referrers.

C = Consumer
F = Family
R = Referrer

Discussion

The present study sought to systematically assess the service satisfaction of consecutive consumers discharged from CR over a five-week period, plus family and referrers. The response rates for the study (61–76%) compare favourably with other similar studies, where response rates of less than 30% have been reported.3

High levels of satisfaction (86–96%) were reported by consumers (n=75), family (n=22) and referrers (n=16) on two forced-choice questions assessing global satisfaction with the service. If consumers were dissatisfied with their care on these global questions, they were significantly more likely to be aged 25–34 years.

The majority of participants (>75%) responded positively to questions assessing most specific aspects of care (exceptions were about being given a choice about where seen and involvement of family). These findings are broadly consistent with other studies that have examined satisfaction with CR (separately) among consumers, family or referrers.7,8

A unique feature of the present study was that the views of all three key stakeholders were assessed, which enabled agreement among them to be calculated. A high level of agreement was found among participants on global satisfaction (>85%) and on most specific aspects of care (>75%). The exception was for choice about where seen. Overall, these findings provide some level of reassurance that consumers, family and referrers tend to broadly agree about service satisfaction from CR.

For the open questions, consumers were more likely to comment on what had been good or especially helpful about CR, than about how the service could be improved. Family and referrers commented roughly equally on these questions. The themes that were identified as being important were: staff issues, access to the service, where consumers were seen, the interventions provided or facilitated, involvement of family, communication/liaison/records issues and transport). For example, it was especially important to participants that the consumer received effective treatment of sufficient duration, and that staff were warm, interested, empathetic, respectful, not rushed or dismissive, that they listened well, treated people as individuals and were positive and reassuring.

Not all of the issues that were raised by participants related to the wellbeing of consumers. Despite the fact that study participants were in the midst of managing their own or a family member’s mental health crisis, five participants (four consumers and one family member) commented that they had felt concerned about the safety of staff, and they suggested improved security for staff. One of these participants was a consumer who has been physically aggressive themselves in the context of being acutely unwell, and they were deeply troubled by this when their mental state improved.

Other challenges for staff were identified. For example, five consumers suggested a less rehearsed/protocol like/impersonal approach. However, assessing consumers in a systematic way covering all key areas such as current risk, without them feeling that they are simply being put through a protocol, may not always be easy to achieve. Also, some seemingly opposing comments were received from participants. For example, four participants (all family) said they wanted more opportunities for the consumer to be seen at home and/or to have family more involved, but three participants (all consumers) said that they wanted to be put under less pressure to have family involved or wanted more care taken to get permission for this. These findings highlight that it might be difficult sometimes for staff to get it right for everyone.

A diverse range of other issues were also raised, especially with regard to suggestions for improving the service. For example, a consumer who had had a panic attack while waiting to be seen suggested a more private waiting area be available, and a GP saw the need for more assistance to be available when consumers had a mental health problem, but were also intoxicated, and family did not feel equipped to manage these volatile situations.

Overall, many of the present findings have implications for funding (ie, sufficient funding is needed), and for how staff are trained, supported, supervised and kept safe. Also, it would seem important that service evaluation tools incorporate questions that assess how people experience staff (eg, warmth). Future research should re-evaluate the impact of demographic variables on service dissatisfaction with a larger sample size.

The key limitation of the study was the relatively small number of family (n=22) and referrers (n=16), which precluded formal statistical comparisons due to low power. The small sample size and low power may also have impaired our ability to detect differences on analyses involving demographic variables. At the same time, while only three demographic variables were examined (age, ethnicity and gender), tests were performed for all 10 survey questions. The use of multiple comparisons may have increased the chances of a spurious finding (only one statistically significant finding was obtained).

Not all consumers who had recently been discharged from the service were eligible for participation (eg, those who were too unwell), so some consumer groups will be under-represented in the present study. Also, it is possible that evaluations of CR within a different service delivery context may find that different issues are raised by participants about what is helpful and unhelpful. Finally, an important perspective that is missing from the current research is the view of staff. Future research could valuably focus on what staff think helps them to feel supported, effective and safe in their role.

Conclusion

High levels of satisfaction and agreement were found among consumers, family and referrers with CR. If consumers were dissatisfied with overall care, they were most likely to be aged 25–34 years. Open-ended questions identified which issues matter the most to key stakeholders, which may have implications for service evaluation tool.

Summary

Abstract

Aim

To systematically assess the service satisfaction of consumers, their families and referrers with crisis resolution (CR).

Method

Consecutive consumers discharged after receiving CR over a five-week period were potentially eligible for participation, together with their family and referrer (broadly defined). Structured telephone interviews were conducted and involved forced-choice questions assessing global satisfaction and satisfaction with specific aspects of care, plus two open-ended questions.

Results

Participants were 75 consumers, 22 family and 16 referrers. High levels of satisfaction were seen for all participants for both global (86-96%) and most specific aspects of care (>75%). If consumers were dissatisfied with their overall care, they were significantly more likely to be aged 25-34 years of age. High levels of agreement among raters were found for global satisfaction (>85%) and most specific aspects of care (>70%), which provides some level of reassurance for staff. Open-ended questions showed that having effective treatment of sufficient duration and staff manner were most important to participants.

Conclusion

High levels of satisfaction and agreement were found among consumers, family and referrers with CR. Open-ended questions identified which issues matter the most to key stakeholders, which may have implications for service evaluation tools.

Author Information

'- Frances A Carter, Clinical Psychologist, Scientific Officer, Senior Clinical Lecturer, University of Otago, Christchurch; M Joan Taylor, Clinical Liaison, Nurse Consultant, Specialist Mental Health Service, Canterbury District Health Board (CDHB), Chr

Acknowledgements

Andrea Bartram set up the research data base and provided supervision around data management, Vicki Dent and Warren Campbell-Trotter monitored consumer data bases for adverse events relevant to recruitment, Joseph Boden provided statistical advice and Virginia McIntosh provided comment on an early draft of the manuscript.

Correspondence

Dr Frances Carter, Clinical Research Unit, Department of Psychological Medicine, PO Box 4345, Christchurch.

Correspondence Email

frances.carter@cdhb.health.nz

Competing Interests

All authors were employed by the Canterbury District Health Board (CDHB) during the conduct of the study. The current project was in part funded by a Summer Studentship grant from the University of Otago.

  1. Lloyd-Evans B, Johnson F. Crisis resolution teams: how are they performing? Ment Health Today. 2014: 18–9.
  2. Wheeler C, Lloyd-Evans B, Churchard A, et al. Implementation of the Crisis Resolution Team model in adult mental health settings: a systematic review. BMC Psychiatry. 2015; 15:74.
  3. Barker V, Taylor M, Kader I, et al. Impact of crisis resolution and home treatment services on user experience and admission to psychiatric hospital. The Psychiatrist. 2011; 35:106–10.
  4. Carpenter R, Falkenburg J, White T, Tracy D. Crisis teams: systematic review of their effectiveness in practice. The Psychiatrist. 2013; 37:232–7.
  5. Irving CB, Adams CE, Rice K. Crisis intervention for people with severe mental illness (Review). Cochrane Database Syst Rev. 2006.
  6. Winness MG, Borg M, Kim HS. Service users’ experiences with help and support from crisis resolution teams. A literature review. J Ment Health. 2010; 19:75–87.
  7. Ampelas JF, Robin M, Caria A, et al. Patient and their relatives’ satisfaction regarding a home-based crisis intervention provided by a mobile crisis team. Encephale. 2005; 31:127–41.
  8. McCauley M, Bergin A, Bannon H, et al. How do GPs experience home-based treatment for acute psychiatric disorders? Primary Care Commun. 2005; 10:159–63.
  9. Worthington A, Rooney P. The Triangle of Care. London: National Mental Health Development Unit, 2010.

Contact diana@nzma.org.nz
for the PDF of this article

Subscriber Content

The full contents of this pages only available to subscribers.
Login, subscribe or email nzmj@nzma.org.nz to purchase this article.

LOGINSUBSCRIBE