The Extreme Illusion of Understanding

The single biggest problem in communication is the illusion that it has taken place.

—George Bernard Shaw

One morning, the University of Chicago's communications office received a message from the university hospital's public relations office that there was "a shooting on campus." When they anxiously called back, the hospital explained that doctors were running late because of a movie shooting on university grounds. The communication office staff was amused but mainly surprised by this miscommunication. Their job, after all, is communication. To avoid such misunderstanding, people routinely gauge whether they communicated successfully. This might allow speakers to detect when the listener misunderstood them, and listeners to detect when they misunderstood the speaker. Here we report our discovery that speakers and listeners systematically overestimate their success, even when communication is extremely unlikely to be successful.

Communication failure has its most dramatic consequences when the stakes are high. For example, Chang et al. (2010) reported surprising levels of miscommunication among physicians. They investigated patient "hand-offs" during shift changes in a hospital, where the departing intern communicated with the incoming intern about patients. The researchers asked the incoming intern to report the three most important pieces of information they received about each patient, and the departing intern to report the three most important things the incoming intern understood. The results were dramatic: for 60% of the patients the information that the departing intern perceived to be most important was not perceived as such by the incoming intern. Yet, the interns were unaware of their failure to communicate, perceiving the handoff to be of "high quality." We suggest that communication failures such as this are partly a result of an illusion of understanding.

There is evidence that speakers systematically overestimate the effectiveness of their communication (Keysar & Henly, 2002; Kruger et al., 2005; Savitsky et al., 2011). For example, when speakers attempt to convey the meaning of ambiguous sentences such as "The man is chasing a woman on a bicycle," most of them overestimated how well the listeners understood them. Such overestimation is even higher among close friends (Savitsky et al., 2011). The most prevalent explanation for this is egocentrism or curse of knowledge. When the speaker intends to communicate that the man is on the bicycle, they “hear” a prominent pause right after saying “woman,” but the pause is not heard by the listener (Keysar & Henly, 2002). When sending an e-mail with a sarcastic intent, the writer “hears” the sarcasm, making the message seem obviously sarcastic (Kruger et al., 2005). This egocentric interpretation makes the intention seem obvious for anyone. Since speakers know their intention, it is difficult for them to take the perspective of the uninformed listener and realize that the message is ambiguous.

Egocentrism or curse of knowledge might not be the only mechanism driving overestimation. First, Keysar and Henly (2002) compared speakers' overestimation to that of overhearers, who heard what the speaker said and knew the intended meaning. If curse of knowledge explains overestimation entirely, these overhearers should have shown the same overestimation, but they did not. Second, Kruger et al. (2005) found that message recipients also overestimated their understanding. Given that the recipients have no access to the intended meaning, egocentrism is a less satisfactory explanation of their overestimation.

Here we report an extreme case of this illusion of understanding. While speakers and listeners routinely overestimate their communication effectiveness, this overestimation should be drastically reduced when they realize that communication is extremely unlikely to succeed. For example, a listener who does not know the language of the speaker might assume that they do not understand the speaker, and therefore, might not overestimate their understanding. However, we find that communicators are insufficiently sensitive to such linguistic context. The illusion of understanding persists even under these extreme conditions for both speakers and listeners even when they do not share a language.

When people communicate, they can have different goals, such as to influence (e.g., Cialdini, 2021), to manage face (e.g., Goffman, 1967; Holtgraves, 1998), to develop relationships, to convey information (Yeomans et al., 2021), and so on. For each of these goals “success” is evaluated differently. For example, when using conversation for relational goals such as apologizing, success is about the interpersonal impact, not the accuracy of detecting linguistic intention. Here we focus on communication as an informational exchange, where success is defined by listeners correctly identifying the meaning that the speaker attempts to convey. In this study, speakers attempted to convey to listeners a specific meaning of an ambiguous utterance. The speaker then estimated whether the listener understood, while the listener estimated their own understanding. This allowed us to assess how accurately speakers and listeners gauged the success of the communication, and whether they over or underestimated success. Participants were incentivized; speakers to convey the intention effectively and listeners to understand accurately.

We varied whether the listener and speaker shared a language or not. Speakers communicated a message in their native language to listeners who shared their native language. Then, the message was conveyed via an audio recording to listeners who did not know that language. We found that speakers systematically overestimated listeners' understanding, regardless of whether listeners knew the language. Listeners also overestimated their own understanding, both when they did and when they did not know the language. The findings reveal a robust illusion of understanding.

Method

The study was preregistered on AsPredicted.org. Deidentified data are available on Open Science Framework (OSF). The University of Chicago IRB approved the research and informed consent was obtained.

Phase I

Following the advice of Simmons et al. (2018) to have more than 50 participants per cell, we recruited 240 native Mandarin Chinese speakers (78% female, 120 pairs). Most pairs participated in Beijing, while 5 pairs participated in Chicago.

We adapted 12 items from Savitsky et al. (2011) and translated them into Chinese (see online supplementary materials). Each item contained an ambiguous phrase that was accompanied by four possible meanings. For example, “What have you been up to?” had four possible meanings: (a)suggesting the other person may have been unfaithful, (b)suspicious that the other person is planning a surprise for you, (c)angry that the other person is 30 minutes late, and (d)wanted to know how the other person has been recently. The speaker attempted to convey one of the four randomly assigned meanings.

Within each pair, participants were randomly assigned to the role of “speaker” or “listener.” We were only interested in how people communicated through their voice, so participants sat back-to-back to prevent communication through cues such as facial expressions. In each of the twelve rounds, speakers and listeners received an ambiguous phrase along with its four possible meanings. For the speakers, one of the meanings was flagged as the meaning they had to communicate. The speaker was instructed to say each phrase such that the listener would be able to identify the target meaning from the four options. The listeners were informed that the meanings were randomly assigned to the speakers.

After each phrase, speakers guessed whether the Chinese listener correctly identified the meaning (1 = Yes, 2 = No), and rated their confidence in the listener's understanding (1 = Completely unconfident to 7 = Completely confident). Then speakers judged whether a listener who does not comprehend any Chinese could identify the intended meaning from the audio recording and rated their confidence in the non-Chinese listener's understanding. After hearing the speaker, Chinese listeners guessed the intended meaning. Listeners then judged whether they correctly identified the meaning (1 = Yes, 2 = No), and rated their confidence in their own understanding (1 = Completely unconfident to 7 = Completely confident; for details see online supplementary materials).

Phase II

We recruited 120 native English-speaking Americans as listeners. Each American listener was yoked to a Chinese speaker and was presented with an English version of the phrases and meanings. The procedure for the American listeners was identical to that of the Chinese listeners, except that they heard the speakers via audio recordings.

Results

For comparisons of estimated and actual success, we obtained the same pattern of results when we analyzed the data using mixed effects logistic regressions (see online supplementary materials).

Overestimation With a Shared Language

The Chinese listeners identified the intended meanings $44\%$ of the time which is significantly greater than chance $(25\%)$, $t(119) = 12.16$, $p < .001$, $d_{\mathrm{Cohen}} = 1.11$. Yet, both listeners and speakers overestimated the success of the communication. On average, Chinese listeners overestimated their understanding by 41 percentage points (pp), as they thought they identified the intended meanings $85\%$ of time, paired $t(119) = 19.74$, $p < .001$, $d_{\mathrm{Cohen}} = 1.80$. The speakers, in turn, overestimated the listeners' success by 26pp, as they thought the listeners understood them $70\%$ of the time, paired $t(119) = 12.30$, $p < .001$, $d_{\mathrm{Cohen}} = 1.12$. Figure 1 presents this overestimation graphically.

Overestimation does not necessarily entail miscommunication. If there are reliable cues that signal the potential for misunderstanding, perhaps miscommunication can be avoided even when people initially overestimate their success. For example, if confidence is strongly and positively correlated with accuracy, then low confidence might trigger a correction precisely when it is needed.

Yet there were only weak positive associations between confidence and accuracy over the 12 phrases ($\alpha_{\mathrm{adjusted}} = .0125$ for four comparisons). The mean of the correlations between confidence and accuracy was weak for Chinese listeners, $M_{\mathrm{Correlation}} = .17$, one-sample $t(118) = 6.34$, $p < .001$, and Chinese speakers, $M_{\mathrm{Correlation}} = .23$, one-sample $t(117) = 9.91$, $p < .001$. This suggests confidence would have provided little information to prevent miscommunication.

Another potential cue is feedback. Speakers might rely on cues from listeners, while listeners might seek confirmation from speakers. But such feedback would be less likely when both wrongly believe they communicated successfully. Indeed, this situation was common. When native Chinese listeners thought they identified the intended meaning, they were wrong about half the time (54%). When native Chinese speakers thought the listeners understood them, they were also wrong half the time (50%). The overlap of these errors was strikingly high. Speakers and listeners simultaneously believed they communicated successfully 61% of the time, yet in about half of these cases they were both wrong (48%)1. Given the frequent estimation errors and high overlap of these errors between speakers and listeners, the potential for miscommunication is noteworthy.

The results with a shared language replicate the illusion of transparency with speakers (e.g., Keysar & Henly, 2002). The overestimation by listeners is a new finding using this paradigm, and is related to a finding by Kruger et al. (2005). Next, we report the most surprising finding: the illusion of understanding persists even when the listener doesn't know the language.

Overestimation Without a Shared Language

Figure 2 shows the same illusion without a shared language. On average, American listeners who did not know Chinese identified the intended meanings $35\%$ of the time, which was better than chance $(25\%)$, $t(119) = 6.80$, $p < .001$, $d_{\mathrm{Cohen}} = .62$. Though American listeners were less accurate than Chinese listeners, $t(237.39) = 4.13$, $p < .001$, $d_{\mathrm{Cohen}} = .53$, they still overestimated their success by 30pp, believing that they succeeded $65\%$ of the time, paired $t(119) = 10.88$, $p < .001$, $d_{\mathrm{Cohen}} = .99$. The Chinese speakers overestimated here as well. While Chinese speakers indicated that the American listeners would understand less $(50\%)$ than the Chinese listeners $(70\%)$, paired $t(119) = 9.8$, $p < .001$, $d_{\mathrm{Cohen}} = .90$, they still overestimated the American listeners' understanding by 15pp, paired $t(119) = 5.28$, $p < .001$, $d_{\mathrm{Cohen}} = .48$. Hence, communicators overestimated their success even in the absence of a shared language.

Confidence was only weakly associated with accuracy in the absence of a shared language. The mean correlation between American listeners' accuracy and confidence over the 12 phrases was low, $M_{\text{Correlation}} = .15$, one-sample $t(117) = 5.25$, $p < .001$. The mean correlation between American listeners' accuracy and Chinese speakers' confidence in them was close to zero, $M_{\text{Correlation}} = .09$, one-sample $t(116) = 3.02$, $p = .003$. Thus, they could hardly rely on confidence to accurately gauge communication success.

Speakers and listeners often jointly believed that communication was successful, but they were frequently wrong. When listeners thought they were correct, they were wrong more than half the time (62%). When speakers thought the listeners were correct, they were also wrong more than half the time (62%). The prevalence of such error was high even when they simultaneously thought that the listener succeeded. When Chinese speakers and American listeners jointly thought the meaning was successfully communicated, they were wrong more than half the time (58%).

This study provides strong evidence that speakers and listeners systematically overestimate the listener’s understanding, regardless of whether they communicate in a mutually understood language or a language they do not share.

General Discussion

The illusion of understanding has important implications for theories of language production and comprehension. Listeners attempt to monitor for comprehension errors (Perfetti et al., 1996), but if they are convinced that they have understood their speakers, they might be less likely to check for errors. Speakers monitor their inner speech plan (e.g., Özdemir et al., 2007; Postma, 2000) and what they say, serving as their own listeners (Hartsuiker & Kolk, 2001; Levelt, 1983). Yet, if they are convinced that they communicated successfully, they might be less vigilant. Any theory of language use should consider this illusion, as it would impact the effectiveness of monitoring during the comprehension and production of speech.

The illusion of understanding is a complex phenomenon that can emerge from many sources, such as egocentrism or curse of knowledge (Keysar & Henly, 2002), and general overconfidence arising from incomplete feedback (Dunning et al., 2004). The persistence of the illusion under extreme circumstances might be related to a more general tendency of insensitivity to context. Speakers and listeners were insufficiently sensitive to the contextual cue of having no shared language, similar to how listeners were insensitive to the constraints of communication medium (Kruger et al., 2005).

American listeners who did not know Chinese identified the intended meanings significantly more often than chance. This suggests that the speakers’ utterances included nonverbal information about the meaning such as intonation and prosody. Indeed, there is evidence that some prosodic cues are similar across languages (Bryant & Barrett, 2007; Endress & Hauser, 2010). Our findings suggest that these cues can be somewhat useful in detecting speakers’ intentions even when the language is not understood.

It is important to consider the ecological validity of the experimental method. We suggest that with respect to noninteractive linguistic communication, the method is ecologically valid. With respect to interactive communication such as conversation, the applicability of the results is more tentative. Noninteractive linguistic communication is omnipresent, and takes many forms such as emails, notes, voicemails, text messages, ads, letters, lectures and webinars. Our results directly apply to such noninteractive communication as the method mimics such a situation. Therefore, the results suggest the existence of a robust illusion of understanding across domains in real life.

The applicability of our findings to interactive conversation might be more limited, because our method did not allow feedback or turn-taking. Face-to-face conversation provides a richer set of cues for disambiguating messages, such as gesture and facial expressions (Archer & Akert, 1977; Argyle et al., 1970; Depaulo & Friedman, 1998). Such cues were eliminated in our procedure. In addition, conversation is collaborative (e.g., Clark & Wilkes-Gibbs, 1986). Communicators use repairs to coordinate meaning (Healey et al., 2018) and create conceptual pacts to keep track of references (Brennan & Clark, 1996; Metzing & Brennan, 2003). Speakers actively try to avoid ambiguity (Ferreira et al., 2005), while listeners use politeness norms to infer speakers’ intentions (Holtgraves, 1997, 1998). Hence, interlocutors may discover misunderstandings and repair them, thereby avoiding the illusion of understanding.

It is still possible, we suggest, that this illusion might affect conversations under certain circumstances. First, by flagging all the utterances as ambiguous, our experimental procedure gave speakers and listeners an opportunity to avoid misunderstanding that a natural conversation does not afford. This raises the possibility that our results might underestimate the prevalence of this illusion in some cases. Second, the literature on turn-taking and repair is rarely concerned with accuracy. Hence, we do not know the extent to which turn-taking and repair actually rid conversations of systematic misunderstandings. In fact, there is evidence that interlocutors are blind to conversational incoherence (Galantucci & Roberts, 2014), show content deafness (Galantucci et al., 2018), and have difficulty in discerning others' conversational motives (Yeomans et al., 2021). Moreover, some experiments that involved conversations showed a high level of egocentrism in understanding (Keysar et al., 1998; Keysar et al., 2003), and a field study showed pervasive miscommunication even in high-stakes conversation between physicians (Chang et al., 2010). This suggests that conversations might not be immune to the illusion of understanding.

Linguistic feedback in conversation likely reduces misunderstanding. Yet it might also contribute to the illusion of understanding precisely because feedback itself can be ambiguous. Consider the true story of Y. Bassok who contacted a travel agent to reserve a ticket. When the agent asked for his first initial and last name, he said "Y," to which she replied, "Because we need it for the reservation." This turn-taking clarified the error for him and he said "Y is my first initial, B.A.S.S.O.K." She said OK and sent him a ticket for Y. Bass. This anecdote illustrates the complexity of the role of feedback. Hence, the illusion of understanding applies to noninteractive communication, and perhaps to conversation to some degree.

We documented the illusion of understanding with languages that are very different from each other. It would be interesting to investigate the illusion with languages that are closer to each other, such as French and Italian. On the one hand, similarity between languages could facilitate understanding and hence reduce the illusion. On the other hand, just as friends and spouses are more likely than strangers to overestimate their communication success (Savitsky et al., 2011), similarity between languages could induce a false sense of understanding, thereby enlarging the illusion. It would also be interesting to investigate the illusion in written communications, where paralinguistic cues are rarer. With shared languages, people seem to be insensitive to the limitations of written communication, overconfident in their ability to communicate sarcasm over e-mail (Kruger et al., 2005). With unshared languages, the challenges of written communication are extremely pronounced, so it is possible that people may calibrate their confidence and overcome the illusion. However, it is also possible that written unshared languages with cognates, especially false friends, may create a false impression of understanding, thereby exacerbating the illusion.

Miscommunication could be costly for individuals and society. Speakers and listeners might believe that they successfully communicated and rarely check this assumption, precisely because of the illusion of understanding. This leads to saying "there is a shooting on campus" without considering that it could involve either guns or cameras.

Footnotes (1)

1 Each instance of communication is weighed equally in the conditional probabilities.