Journal of the New Zealand Medical Association, 04-August-2006, Vol 119 No 1239
Comprehension of discharge information for minor head injury: a randomised controlled trial in New Zealand
Kim Yates, Andres Pena
Effective communication is an essential component of good medical care. Clear discharge information has been shown to improve compliance, improve patient satisfaction, and decrease unplanned representations to emergency departments (EDs).1–3
Past studies in American EDs have highlighted a disparity between the readability of written discharge instructions and patient literacy levels.4–6 For example, in one study, discharge instructions required reading levels of 8th to 14th grade, while 45% of patients had measured reading levels of 9th grade or below.6 Indeed, simplification or standardisation of discharge instructions has been shown to improve comprehension.1,7
Figure 1. ACC head injury advice sheet
Figure 2. North Shore Hospital head injury advice sheet
In New Zealand, the Accident Compensation Corporation (ACC), the nation’s no-fault accident insurance scheme, had developed a head injury advice sheet, which was commonly given out to patients discharged from EDs with a head injury (Figure 1).
Although the ACC form scored at 4th grade using the Flesch Reading Grade level (Microsoft Word), it was felt to be difficult to read by staff, and thus a simplified head injury advice sheet was developed (Figure 2).
The goals of this study were to investigate the health literacy of emergency medicine patients in our population, to compare the comprehension of the ACC head injury advice sheet with the simplified sheet, and to investigate factors affecting comprehension.
The study was a prospective randomised controlled trial, reviewed, and approved by Auckland Ethics Committee Y. A convenience sample of adult patients meeting inclusion and exclusion criteria that consented to participate were randomised, using an opaque envelope method, either to the group given ACC head injury advice sheet to read, or to the group given the simplified sheet developed by the Department of Emergency Medicine. Comprehension of the head injury advice was then assessed, an estimation of health literacy made, and demographic data collected.
The study was conducted at North Shore Hospital, an urban district hospital serving a population of around 470,000 in 2003. The annual Emergency Department census is approximately 42,000, with 27,000 self-referral patients presenting to Emergency Medicine and the rest, referrals by general practitioners to inpatient services.
From August to December 2003, Emergency Medicine patients aged 15 years or more, presenting on study shifts, were invited to participate. Study shifts were a mixture of days, afternoons, and weekends. Patients were excluded if they were unable to comprehend spoken or written English, if they had severe illness or pain, if they were triaged as needing to be seen immediately, if they had a significant eye condition or complaint, or if their corrected visual acuity was less than font size 10. During the consent process, words such as “literacy” and “test” were specifically avoided.
Following randomisation, participants were given 5–10 minutes to read their allocated head injury advice sheet, either the ACC form (Figure 1) or the simplified form (Figure 2). Readability scores (Readability Calculations, Micro Power & Light Co, Texas, USA) for both forms were similar (Flesch-Kincaid & Powers 4th grade, Dale-Chall 6th, FOG 7th, SMOG 8th)—but the ACC form had 750 words and the simplified form had 371 words.
Following the reading time, the participant was interviewed by a researcher, using a data collection sheet that included a script to standardise the interviews. Participants were asked 10 questions to assess comprehension of the advice sheet (Figure 3), and were able to refer to their sheet at any time.
Health literacy was estimated using the Rapid Estimate of Adult Literacy in Medicine (REALM), a validated word recognition test that takes 3–5 minutes to administer, which classified participants into 4 groups according to reading levels: 3rd grade or less, 4th–6th grade, 7th–8th grade, high school (9th grade) or above.8 Data was then collected on gender, age, years of schooling, and ethnicity—and finally, participants were shown the advice sheet they had not received and asked which advice sheet they preferred.
Figure 3. Comprehension assessment: questions with script and scoring guide
The main outcome of interest was the comprehension score for the advice sheet. Secondary outcomes included health literacy level, demographic factors, and form preference. A power calculation indicated 200 participants would be required to show a significant difference in comprehension scores.
Microsoft Excel and the Analyse-It general statistics module (Analyse-it Software Ltd, Leeds, England) were used for descriptive statistics with 95% confidence intervals, and for a Mann-Whitney U test of differences in comprehension scores between comparison groups. Logistic regression looking at factors affecting the comprehension score was performed by our statistician.
260 patients meeting inclusion and exclusion criteria were invited to participate; 60 declined giving reasons which included “too tired”, “not feeling well”, “have to go”, “headache”, “(family member) did not want me to” and “too dizzy”—thus leaving 200 study participants..
Table 1 summarises study group characteristics. Groups were well-matched for age, education, and literacy levels, and while less matched for gender and ethnicity, these differences were not significant.
Table 1. Study group characteristics
*Rapid Estimate of Adult Literacy in Medicine (test). †ACC=Accident Compensation Corporation.
Figure 4 shows the range of comprehension scores (questions correct out of 10) for the study groups. Median comprehension score for the ACC form was 9, and for the simplified form 10. The Mann-Whitney U test showed the simplified form group had significantly higher comprehension scores (p<0.0001).
Figure 4. Comprehension scores: number of participants with each score in the two study groups
Figure 5 shows REALM test results for the study groups. For the logistic regression analysis to investigate factors affecting comprehension, comprehension scores were condensed into 3 groups: 10 correct, 9 correct, <9 correct. Two factors had no effect on comprehension score: gender (p=0.6) and ethnicity (p=0.3).
Table 2 shows factors that had an effect on comprehension score. Logistic regression was also used to investigate the interaction of literacy levels and the form used, and there was no evidence of an effect of the form on comprehension scores for the different REALM groups (p=0.5), that is, whatever the REALM group, the simplified form improved comprehensions scores. The simplified sheet was preferred by both study groups: 94% of those in ACC advice sheet group, and 95% of those in simplified group.
Figure 5. Rapid Estimate of Adult Literacy in Medicine (REALM) test classification for the two study group
Table 2. Results of logistic regression: factors affecting comprehension scores
As for study limitations, there was poor representation of lower literacy groups in this study, particularly the lowest literacy group, so results may not apply to this group. With more than 20% of patients approached declining to participate for various reasons, selection bias is likely, and it is possible that patients with lower literacy levels may have been in this group. Population differences such as the high level of literacy in our population may mean that our results are less applicable to other ED populations.
Previous studies have reported discharge instructions with readability scores of 6th to 14th grade and have highlighted concerns that patients with lower literacy levels would be unable to understand the material.4–6,9 Although both head injury advice sheets in this study had similar readability scores, between 4th and 8th grade depending on the formula used, comprehension of the simplified sheet was significantly better whatever the participant’s literacy level. This suggests that when revising written discharge information to improve comprehension it would be unwise to rely solely on the readability score of the document.
The National Work Group on Literacy and Health highlight that, in the United States, health providers could be held liable if information is not presented in a way that is understandable to the patient, and that some national accrediting agencies require healthcare providers to ensure that patients understand the information they are given.10
The Work Group also point out that people of all literacy levels prefer (and have a better understanding of) simple written materials compared to complex material, and our study certainly supports this in the ED setting. Their recommendations for written material are that it should be at 5th grade level or lower, that common words should be used or difficult words explained, that short sentences and large font be used, and that the layout should have large blank spaces to make the text look easy to read.
These recommendations were followed when the advice sheet was revised at North Shore Hospital. The other goal when revising the advice sheet to a simple one-page document was to make it “internet friendly” so that the document could be stored on the ED intranet website and printed off anywhere in the department when required, to alleviate the problem of not being able to find advice sheets in a busy ED.
Factors associated with better comprehension (of medical information) in our study included:
Spandorfer and colleagues5 interviewed 217 patients discharged from the ED asking about their diagnosis, medications, and discharge instruction and found, as we did, that literacy level had a significant effect on comprehension, however age and education had no effect in their study.
Jolly and colleagues,7 who found improved comprehension when comparing scores for standard and simplified discharge instructions, also found that the higher education group (>12th grade) showed a greater improvement in average scores with the simplified instructions than the lower education group, although they did not use a regression model to study this improvement.
With 84.5% of our ED study population having a health literacy level of 9th grade or above, literacy levels appear higher than in other published studies, with Williams and colleagues finding that 45% of patients reading at or below 9th grade,6 and Spandorfer and colleagues finding a mean reading level of 6th grade, with 40% reading at 4th grade or below.5 According to the International Adult Literacy Survey literacy levels in the United States and New Zealand do not appear significantly different,11 so these differences are more likely to be due to study population differences, differences in testing and/or selection bias.
In summary, it appears that simplifying written discharge advice sheets can improve comprehension even when readability scores between the sheets are similar, and that even in a population with higher literacy levels the simplified advice sheet was preferred and better understood whatever the literacy level.
Author information: Kim M Yates, Emergency Medicine Specialist and Director of Emergency Medicine Research, Department of Emergency Medicine; Andres Pena Emergency Medicine Research Assistant (2003–2005), Emergency Care Centre; North Shore Hospital, Auckland
Acknowledgements: We thank Elizabeth Robinson (University of Auckland) for statistical advice and analyses; Terry Davis (Louisiana State University) for information on the REALM test; and Christine Woods and Peter Hughes (Auckland College of Education) for information on assessing readability.
Correspondance: Dr Kim Yates, Department of Emergency Medicine, Emergency Care Centre, North Shore Hospital, Private Bag 93 503, Takapuna, Auckland. Fax: (09) 486 8946; email: Kim.Yates@WaitemataDHB.govt.nz
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads)
Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals