Interns Overestimate the Effectiveness of Their Hand-off Communication

In 2003, the Accreditation Council for Graduate Medical Education restricted residency duty hours in response to increasingly complicated patients, data on the effects of sleep deprivation on residents, and growing media attention on the affects of long duty hours on patient safety and resident well-being.1 Since then, many voiced concerns about the increased frequency of hand-offs, whereby patient care responsibility is transferred from 1 resident who is leaving the hospital to another resident who will be staying and covering other residents’ patients.2,3 The frequency of hand-offs did in fact increase after the duty hour restrictions, with a member of the primary health care team being present in the hospital for less than half of a patient’s hospitalization.4 This emphasizes the importance of quality hand-offs.

A recent Pediatrics article5 found no significant change in the total hours of work or sleep before and after the duty hour restrictions. It is interesting that the authors found a significant increase in minor errors, which may result from an increase in hand-off frequency without a corresponding increase in hand-off education and improvement. In addition, in December 2008, the Institute of Medicine released recommendations to reduce work hours further and to train residents on transitions of patient care.6 Implementing these recommendations would further increase the frequency of hand-offs. Thus, undertaking formal hand-off education and improvement activities is becoming increasingly critical to ensuring safe patient care. To develop such programs, it is essential to understand the systemic reasons for hand-off communication failure.

The hand-off process, also known as “sign-out,” can be a written or verbal transfer of patient care information. Each time a hand-off occurs, the possibility for miscommunication arises. Hand-offs are often riddled with omitted or inaccurate information that could be critical to patient care, such as code status or allergies, resulting in uncertainty in the covering residents’ decisions for patients.7,8 The contribution of communication failures to adverse events has been estimated to be between 15% and 67%.9–11 To date, little is known about the hand-off process in general and in the field of pediatrics in particular. Although several studies focused on near-misses and adverse events,7,12,13 they lacked the theoretical foundation to explain why physician-to-physician communication is poor.

Theories from the psychology of communication may be applicable in understanding why hand-off communication is inherently problematic. Studies show that speakers systematically overestimate how well their messages are understood by listeners14 and that people in general believe that their thoughts are transparent to others.15–17 In addition, the more knowledge that people share, the worse they communicate new material because they overestimate the knowledge of the other.18 These psychological processes could systematically affect the effectiveness of communication during hand-offs. The aim of this study was to assess whether postcall pediatric interns who provide hand-offs can correctly estimate the information received by on-call interns at a hand-off communication.

METHODS

Participant Population

All interns, subinterns, and visiting interns who were rotating on the general pediatrics team at the University of Chicago Comer Children’s Hospital were eligible for the study. During June 2007, the study protocol was explained by pediatric resident and investigator (Dr Chang), and written consents were obtained. Participation was voluntary, and the institutional review board at the University of Chicago approved this study.

Study Setting

At the University of Chicago, there is 1 general pediatrics team that is composed of 1 attending, 2 senior residents (postgraduate year 3) who provide day coverage from 7 AM until 7 PM, 2 senior residents (postgraduate year 2 or 3) who provide night coverage from 7 PM until 7 AM, and 4 interns (postgraduate year 1). Interns rotate through general pediatrics for 1.5 to 2.0 months. One intern is on call each night and works 30-hour shifts every 4 days. The on-call intern admits general pediatric patients from 7 AM to 7 AM the next day. The verbal hand-off occurs at 11:30 AM daily in a dedicated conference room with the postcall intern communicating primarily to the on-call intern but with the other interns and senior residents also present. Senior residents are present in the room mainly to play a supervisory role and could interject if necessary. The attending is not present. The computerized written hand-off is a Microsoft Word document (Redmond, WA) updated by the postcall intern and given to the on-call intern.

Data Collection

Participant Recruitment

Interns were interviewed on the last 4 weekdays of their general pediatrics rotation. This allowed interns to establish their own hand-off practice patterns, minimizing the potential for the interview to act as an intervention, or the Hawthorne Effect.19

On-call and postcall interns were approached after the hand-off and asked to participate in an interview about the hand-off communication. The postcall intern was interviewed immediately after the handoff, allowing that intern to

leave the hospital within duty hour restrictions. The on-call intern was interviewed after a noon conference, at 1:00 PM (Fig 1). The interns had access to the written hand-off during the interview, which took place in a private room and was audiotaped. All interviews were conducted by trained research assistants (Ms Lev-Ari and Mr D’Arcy).

Postcall Interview Script

The interviewer asked the postcall interns to identify patients by room number to protect health information. They were told that the on-call intern will be asked to describe the most important pieces of information for each patient that was communicated during the hand-off and then asked to rank the information in order of importance (Table 1). The interviewer also explained that the on-call intern will be asked to report any rationale that he or she received for each item. The interviewer then asked the postcall interns to detail how they expect the on-call intern to answer these questions. This provided an estimate of what the postcall intern believed that the on-call intern received from the hand-off session.

The postcall intern was then asked to guess the on-call intern’s rating of how well the hand-off prepared him or her to take care of the patients, on a scale from 1 to 10, with 1 being “did not prepare me at all” and 10 being “prepared me very well.” Finally, the postcall intern guessed the on-call intern’s rating of how well the hand-off went in general, on a scale from 1 to 10, 1 being “badly” and 10 being “very well.”

On-call Interview Script

The interviewer referred to patients by the room numbers provided by the postcall intern. The interviewer asked the on-call intern to list the most important pieces of information, along with any rationale, about each patient that the postcall intern discussed during the hand-off and then to rank this information in order of importance. These questions were repeated for each patient. This provided an estimate of what the on-call intern actually received from the handoff session. Then, the on-call interns rated how well the hand-off prepared them to take care of the patients overnight and how well the hand-off went in general, both on a scale from 1 to 10.

Data Analysis

Audiotaped interviews were transcribed by Voss Transcription, Inc (Chicago, IL), and reviewed by 2 physician investigators (Drs Chang and Arora) for accuracy. Occasionally, when transcripts had missing information, the original audiotapes were consulted by physician investigators (Drs Chang and Arora) and blanks were filled in when possible. For example, “sickle-dex” was inaudible to transcribers, but given the investigators’ knowledge of the hospital’s qualitative screen for sickle hemoglobin, the missing name for the laboratory test was filled in.

Transcripts were stripped of identifiers. Postcall and on-call transcripts for the same hand-off were paired. For each patient, data were extracted into a Microsoft Excel spreadsheet, and important items that the postcall and on-call interns reported were juxtaposed. The items were then coded for agreement between the interns, with 1 being “agreement” and 0 being “no agreement.” To qualify as agreement, the item had to match in meaning, not necessarily in terminology. We then compared the rationale that the postcall intern expected the on-call intern to report with the rationale that the on-call intern actually reported. Last, we coded items with respect to importance: (1)the most important item about each patient and (2)the top 3 important items regardless of priority rank. The coding was initially conducted by 1 investigator (Dr Chang) and then reviewed by a second investigator (Dr Arora).

We used summary statistics to tabulate the mean agreement on the most important items about each patient, how well the hand-off prepared the on-call intern to take care of patients, and how well the overall hand-off went. We conducted subgroup analyses by using $\chi^{2}$ tests for type of intern and type of item being communicated. All statistical tests were performed by using Stata 10.0 (College Station, TX), with statistical significance defined as $P < .05$.

RESULTS

All 18 categorical pediatric interns and 5 combined medicine-pediatric interns (100%) agreed to participate in the study from July 2007 to May 2008. Ten of 28 visiting interns (family medicine interns rotating from other hospitals) or subinterns (fourth-year medical students from the study institution) also agreed to participate. We conducted 52 interviews, which constituted 59% of eligible interviewees. Of the 52 interviews, there were 19 dyad interviews, consisting of both the postcall and the on-call interns of the same hand-off. The majority of interviews that were included in the analysis were from categorical pediatric interns (63%). Sixteen percent of the analyzed interviews were from medicine-pediatric interns, 18% were from rotating interns, and only 1 was from a subintern (<2%). Fourteen interviews were discarded because only 1 of the 2 interns was interviewed and therefore could not be paired. Seventy-two patients were discussed during the interviews that were analyzed.

Postcall interns overestimated the effectiveness of their communication. For example, 1 postcall intern expected the on-call intern to have understood the following about a patient:

In contrast, the on-call intern actually understood:

This on-call intern mentioned only 1 of the 3 items that the postcall intern expected. This discrepancy was very common. On average, postcall interns expected on-call interns to mention 2.6 important items per patient, whereas on-call interns actually mentioned only 1.6 items on average ($P < .01$). For 69% of the patients, the on-call intern failed to note at least 1 of the important items that the postcall intern expected him or her to note.

We also looked at how interns ranked the items about each patient, in order of importance, as a measure of how well the gravity of each item was communicated. The postcall interns overestimated their ability to convey the information about the importance of each item. Overall, the item that postcall interns expected on-call interns to perceive as the most important was not perceived as such by the on-call interns for 60% of the patients. In fact, the most important item about a patient was not mentioned at all by the on-call intern for 40% of the patients.

We conducted subset analyses comparing categorical pediatric interns, combined medicine-pediatric interns, and rotating family medicine interns. There was no difference between categorical pediatric interns and combined medicine-pediatric interns in the percentage of items that were successfully communicated during the hand-off; however, when the postcall intern was a rotating family medicine intern, there was a significantly lower likelihood that the most important item about a patient was communicated (odds ratio: 0.16 [95% confidence interval: 0.04–0.75]; $P = .02$). In addition, the percentage of overall agreement was significantly lower compared with pediatric interns (95% confidence interval: 9.1%–49.0%; $P = .005$). There was no change in effective hand-off communication between the interns, with experience over time during their internship (using indicator variables representing 2- or 3-month intervals).

In addition, we categorized each important item by type: to-do, anticipatory guidance (if, then), and knowledge. We performed $\chi^2$ tests and found a statistically significant difference in the likelihood that an item would be effectively communicated when it was a to-do item (65%) or an item related to anticipatory guidance (69%) compared with knowledge items (38%; $P = .003$). The number of patients discussed during each hand-off ranged between 3 and 5, with an average of 3.8. We compared the effectiveness of hand-off communication between postcall interns who had fewer than the average number of patients (3 patients) and postcall interns who had more than the average (4 or 5 patients) and found no relationship between the number of hand-off patients and the agreement between the interns on the most important item about a patient (60% vs 55%; $P = .57$).

Postcall interns also overestimated the extent to which on-call interns appreciated the rationale behind the information. When the postcall intern provided a rationale, the on-call intern failed to mention that rationale 60% of the time. In some cases, postcall and on-call interns even provided very different rationales. For example, a postcall intern expected the on-call intern to say that the rationale behind “follow-up with case manager” was to “make sure she talked with patient’s [primary medical doctor],” but the on-call intern actually reported the rationale as “to ensure nothing holding up discharge.”

The average rating of how well prepared the on-call intern felt to take care of patients after the hand-off was 8.8 of 10.0 (SD: 1.0), and the postcall intern reported an average of 8.0 of 10.0 (SD: 1.0). The overall rating of hand-off was 8.3 (SD: 1.3) by the on-call intern and 7.6 (SD: 1.1) by the postcall intern.

DISCUSSION

This study found that pediatric interns overestimate the effectiveness of their hand-off communication, despite their failure to convey the most important information about a patient 40% of the time. This study ties in theories from communication psychology as a possible explanation for why hand-off communication is so poor. In that light, resident miscommunication is the result of a complex interplay among various factors. Because speakers know what they are trying to convey, they tend to think that what they say is clear to anyone.20 Moreover, because they overestimate how well they communicated, postcall interns are less likely to verify whether the on-call intern actually understood14,21; therefore, the inability of the postcall intern to gauge accurately the on-call intern’s understanding of patient information may greatly affect hand-off quality; not only are on-call interns failing to receive important patient information, but also the postcall interns are systematically failing to realize that breakdown of communication. These communication breakdowns occurred even with written hand-offs.

It is interesting that there was no change in effective communication over time. This could mean that increasing clinical knowledge and experience alone do not affect an intern’s ability to communicate effectively during hand-offs. Because there is no formal hand-off curriculum, this suggests that senior residents would not be much better than the interns. It is important to recognize that the literature has found that hand-offs in most residency programs are executed by interns alone with little supervision by senior residents or attendings.4 Our findings suggest that even with the presence of a senior resident, postcall interns still overestimate the effectiveness of their hand-off communication.

One possibility for the finding that to-do and anticipatory guidance items were more likely to be communicated compared with knowledge items is that they refer to high-priority items that are relevant for the on-call intern’s upcoming shift. In contrast, items related to knowledge may be less urgent and therefore not as likely to be remembered by the on-call intern. This is in concordance with a review by the National Aeronautics and Space Administration, where hand-off communication has been found to be most effective when it is driven by “problems, hypotheses, and intent” rather than long lists.22 It seems reasonable to train interns to communicate by using this framework and avoid unnecessary knowledge items that are unlikely to be remembered. This will prevent cognitive overload for on-call interns by tailoring information that is communicated. This is an area that requires additional studies.

There are several limitations to this study. It is a single-institution study with a small number of interns, making its generalizability unknown. It is also unclear whether our findings are generalizable to more senior residents and hand-offs in other subspecialties, yet it is reasonable to assume that our findings underestimate the extent of the problem. Hand-offs in our pediatric residency program receive high priority; they occur in a dedicated room and time, have both verbal and written information, and are supervised by senior residents. Despite such good conditions, we found overestimation of hand-off effectiveness by postcall interns. Although there is no survey of hand-off practices in pediatric residency programs, it is known that hand-offs in internal medicine do not always take place in such ideal setups4; therefore, a multisite study is likely to find more overestimation and miscommunication.

Unfortunately, some interviews were not conducted because either the postcall or the on-call intern was unavailable. Mainly postcall interns were unable to participate in interviews because they either did not answer a page as a result of workload or left the hospital to adhere to duty hour restrictions. There is no reason to believe that the excluded dyads had more effective hand-offs. If anything, given that the excluded dyads seemed to be under increased pressure, their hand-offs might have been even less effective.

CONCLUSIONS

This study shows that postcall interns overestimate the patient information that they convey in hand-offs, and it highlights the extent to which pediatric interns do not agree on the content, priority, or rationale communicated during hand-offs. In the era of restricted duty hours and increased frequency of hand-offs, it is important for educators to consider the role of systematic causes of miscommunication. Future studies should include hand-off improvement efforts, such as the development of specific hand-off curricula23 to include emphasis on important items, rationale, and a tailoring of information.24 At the very least, postcall interns should be aware of their “illusion of transparency.” They should appreciate that much of the important information that they thought they conveyed during the hand-off was never really received by the on-call intern.