Language modality influences risk perception: Innovations read well but sound even better

1 | INTRODUCTION

In 2019, the National Health Service (NHS) in the United Kingdom partnered with Amazon's Alexa to provide health information through a digital voice device that draws science-based information from the NHS website (Department of Health & Social Care, 2019). People can now ask their voice assistant for health advice such as information on a new medication, instead of getting it the "old fashioned" way by reading it on the NHS website. If the content of the information is the same, then people's evaluation of the new medication should be the same in the two language modalities. That is, it should be independent of whether people receive the information in a written or spoken form. This seems self-evident.

Modality independence is assumed in formal theories of language (e.g., Chomsky, 1965), which hold that language is independent of whether it is spoken or written. Once words are recognized, the modality or "form" of language is seen as irrelevant (but see Port & Leary, 2005, for a criticism of formal phonology). Broadly speaking in psychology, language is treated as a transcriptional medium by which thoughts and feelings become externally represented in linguistic form. This implicit assumption decouples language modality from meaning, and thus implies that language modality should not influence judgments and decisions.

Modality independence is also assumed by normative theories of decision making, which hold that alternative descriptions of a problem should elicit the same preferences (e.g., von Neumann & Morgenstern, 2007). This is known as the principle of invariance or extensionality (e.g., Tversky & Kahneman, 1986). Descriptive theories of risk perception, and more generally judgment and decision making, also implicitly assume that judgments are modality independent. Indeed, neuroimaging studies show that hearing or reading a given description evokes identical semantic representations in the human brain (Arana et al. 2020; Deniz et al. 2019).

Across three experimental studies, we challenge the assumption that risk perception is modality independent. We break new ground in two ways. First, we demonstrate that language modality systematically influences judgments of novel risks and benefits. When people evaluate a relatively novel technology, the perceived benefits outweigh the perceived risks by a bigger margin when they hear than when they read about it. Second, we demonstrate that modality impacts judgment through affect. Hearing about novel technologies induces more positive affect, which in turn leads to a more favorable risk perception. Therefore, the present findings inform theory development in the domain of risk research and judgment and decision making more generally.

1.1 | Theoretical background

Although there is evidence that spoken information tends to prompt better recognition and short‐term recall than written information, especially of presentation order (Glenberg, 1990; Glenberg & Fernandez, 1988; Greene, 2014; Metcalfe et al. 1981; Murdock & Walker, 1969; Penney, 1975; Unnava et al. 1994), and that people respond in a more socially desirable manner in interactive voice than text interviews (e.g. Dillman et al. 2009; Hochstim, 1967; Schober et al. 2015), the influence of language modality on risk perception has received limited attention. Here we examined the novel research question of how the language modality through which an unfamiliar technology is described impacts its perceived risk and benefit.

We relied on the dual‐process framework of risk perception (e.g. Loewenstein et al. 2001; Slovic et al. 2004). This framework emphasizes two modes of processing information: one that is fast and automatic and relies on feelings, and another that is slow and deliberate and involves analysis. The “risk as analysis” mode involves the calculations of the probability and utility of possible outcomes, and their integration based on normative rules such as expected utility theory (Slovic et al. 2004). The “risk as feelings” mode instead relies on the overall affective valence of a stimulus (e.g. Finucane et al. 2000; Fischhoff et al. 1978; Kahneman & Frederick, 2012). If the overall valence is positive, then the stimulus would be perceived as low in risk and high in benefit. If it is negative, it would be perceived as high in risk and low in benefit. Judging the attribute of a stimulus, such as its perceived risk, based on the affective valence it elicits is known as the “affect heuristic” (Finucane et al. 2000; Kahneman & Frederick, 2012).

One type of evidence for the affect heuristic in the context of risk perception is that people perceive risks and benefits as negatively correlated (Alhakami & Slovic, 1994; Finucane et al. 2000; Hadjichristidis et al. 2015; Keller et al. 2006; Savadori et al. 2004; Slovic et al. 2007). This finding is noteworthy because if actual risks and benefits of technologies are associated at all, they tend to be positively related (Finucane et al. 2000). Further support comes from the finding that learning about the benefits of a stimulus reduces the perceived risk, while learning about the risks reduces the perceived benefit (Finucane et al. 2000; for a replication see Efendić et al. 2021). This finding is consistent with the affect heuristic because risk information induces a negative affective reaction that reduces the perception of benefits, and benefit information induces a positive affective reaction that reduces the perception of risk. The evidence for the affect heuristic is typically correlational and does not show a causal link between affective valence and judgment of risk and benefit. The present research will provide such link (Study 3).

1.2 | Research overview

We report three experiments that challenge the modality independence assumption. We focused on novel technologies because people lack entrenched attitudes toward them, and therefore their attitudes might be malleable (Schwarz, 2007). In all three studies, we were interested in the difference between benefit and risk judgments across the two modalities, which is signified by an interaction between modality and judgment type (risk/benefit). In Study 1, we report our finding that risk perception is modality dependent. Judgments of benefit exceeded judgments of risk by a bigger margin when participants heard about a technology than when they read about it. This finding carries the signature of the affect heuristic. The modality effect could be the result of the spoken modality prompting more positive affect toward the novel technologies, thereby amplifying the difference between perceived benefit and risk. Studies 2 and 3 tested this theory by measuring affective reactions toward the technologies. As predicted, the spoken modality prompted more positive affect toward the novel technologies, and affect mediated the effect of modality on risk/benefit judgments. Study 3 included an additional test of the affect theory. Novel technologies are evaluated more in terms of affect than familiar technologies (e.g. Midden & Huijts, 2009; Van Giesen et al. 2015). Therefore, if modality acts through affect, its impact should be more marked for novel than for familiar technologies. As anticipated, modality influenced novel technologies but not familiar ones.

2 | STUDY 1: THE EFFECT OF MODALITY ON RISK AND BENEFIT JUDGMENT

Study 1 examined whether judgments of risk and benefit are modality independent. Participants received information about the advantages and disadvantages of five novel technologies through either spoken or written language, and were asked to judge their overall risk and benefit. We presented participants with both the advantages and disadvantages of the technologies because this is how people are usually informed. For example, medication comes with information about its benefits as well as its side‐effects, partly because of ethical and legal obligations to inform people about the risks (Fischhoff et al. 2011).

2.1 | Method

All data are published on the Open Science Framework (OSF). The materials are presented in the Supporting Information.

2.1.1 | Power analysis

An a priori power analysis for a 2 × 2 mixed‐factor analysis of variance (ANOVA) was conducted using G*Power3 (Faul et al. 2007) to determine a sufficient sample size using an alpha of 0.05, a power of 0.80, and a medium effect size (f = 0.20, estimated, using the SPSS effect size specification in the settings), number of groups = 2 (modality), number of measurements = 2 (judgment type), ɛ = 1. The minimum sample size is 202. As a precaution against possible data loss due to technical issues, we targeted about 10% more participants (estimated based on piloting).

2.1.2 | Participants

We recruited 223 native English speakers living in the United States (40.7% female, 55.7% male, 3.6% unknown, $M_{\mathrm{age}} = 32.1$ years, age range: 18–60 years) through Prolific (prolific.ac). Participants were eligible to participate only if they were American residents, native English speakers, and passed a captcha typing check. We excluded 21 participants (9.4%) because they failed on one or more of predetermined inclusion criteria (see Supporting Information, Table S1). The analyses were performed on the remaining 202 participants, 88 in the spoken modality condition and 114 in the written modality condition.

2.1.3 | Materials and procedure

We randomly assigned participants to either the spoken or the written condition. Participants in the spoken condition listened to descriptions, while those in the written condition read them. The recordings were created by a 22‐year‐old male native English speaker. In all three studies, we selected speakers from the Chicago area with a standard mid‐Western American English accent to avoid complications arising from the use of nonnative‐accented speech such as reduced comprehensibility, activation of stereotypes, or reduced trust (e.g. Lev‐Ari & Keysar, 2010). The speakers were asked to read the descriptions aloud in a neutral manner. The recordings were normalized in terms of loudness and peak amplitude using the software Audacity (Audacity Team, 2020). The speakers were unaware of the research question and hypotheses. Participants in both modality conditions heard the general instructions and received the same picture of the speaker. The aim was to decrease the likelihood that an eventual modality effect is due to a speaker's idiosyncratic characteristics such as physical attractiveness. The Supporting Information provides the instructions and the picture of the speaker.

Participants received descriptions of five novel technologies adapted from King and Slovic (2014) (see Table S2 in the Supporting Information). Each description included advantages and disadvantages of the technology, and was accompanied by a black and white picture of the technology. Following each description, we asked participants: “How risky do you perceive [technology] to be?” (1 = Not at all risky, to 9 = Very risky), and “How beneficial do you perceive [technology] to be?” (1 = Not at all beneficial, to 9 = Very beneficial). We counterbalanced the presentation order of advantages and disadvantages across conditions and randomized the presentation order of the technology descriptions separately for each participant. Presentation order did not interact with the modality effect in any of our studies, so we omitted it from the analyses.1

To confirm that participants listened to the instructions, we included an audio check. The speaker stated: “The answer to the question below is green as in the color green” and the participant had to respond to “What is the answer to the question?” by typing in a blank box “green.” To evaluate whether participants were paying attention we asked them two recognition questions after each description: one about the benefits of the technology and another about its risks (“Which of the following benefits/risks were mentioned in the description?”). Each question listed three possible answers, the correct one and two foils. The results of this measure are presented in the Supporting Information.

Finally, participants in both modality conditions answered questions concerning speaker characteristics: “Overall, how much did you like the speaker?”, “How much did you like the voice of the speaker?,” “Overall, how well did you understand the speaker?”, “How smart do you think the speaker is?,” “How much did you like the appearance of the speaker in the photo?” All questions were followed by a 7‐point scale (1 = Not at all, to 7 = Very much). Participants in the written condition also answered these questions because they heard the general instructions from the speaker and saw his picture. The results from the speaker characteristics measures are presented in the Supporting Information (Table S1 and Table S2).

2.2 | Results

2.2.1 | Judgments of risk and benefit

We averaged separately the judgments of risk and the judgments of benefit for each modality across the five technologies. Modality influenced judgments of risk and benefit (see Figure 1).

Compared to participants who read the descriptions, those who heard them perceived the novel technologies as less risky ($M_{\text{Spoken}} = 5.21$, $95\%$ CI [4.88, 5.53], $M_{\text{Written}} = 5.57$, $95\%$ CI [5.29, 5.86]) and more beneficial ($M_{\text{Spoken}} = 7.58$, $95\%$ CI [7.26, 7.89], $M_{\text{Written}} = 7.19$, $95\%$ CI [6.91, 7.47]).

We were interested in the difference between benefit and risk judgments, therefore following previous research (Hadjichristidis et al., 2015; King & Slovic, 2014), we submitted the mean judgments of risk and benefit to a 2 (Modality: spoken, written) $\times 2$ (Judgment type: risk, benefit) mixed-factor ANOVA, with judgment type as a within-participants factor.2

There was no main effect of modality ($F < 1$). Importantly, there was a significant Modality $\times$ Judgment type interaction, $F(1, 200) = 4.58$, $p = 0.034$, $\eta_p^2 = 0.02$.

Separate tests for risks and benefits, adjusted for multiple comparisons, did not show a significant modality effect for risk judgments, $F(1, 200) = 2.80$, $p = 0.096$, $\eta_p^2 = 0.01$, or benefit judgments, $F(1, 200) = 3.36$, $p = 0.068$, $\eta_p^2 = 0.02$. Finally, there was a main effect of judgment type, $F(1, 200) = 127.91$, $p < 0.001$, $\eta_p^2 = 0.39$. Novel technologies were judged to be more beneficial ($M = 7.38$, 95% CI [7.18, 7.59]) than risky ($M = 5.39$, 95% CI [5.17, 5.61]).

2.2.2 | Correlations between risk and benefit judgments

Technology	Spoken			Written
	R	p	n	r	p	n
Study 1
Vaccine strips	-0.446	<0.001	88	-0.587	<0.001	114
Flu drug	-0.452	<0.001	88	-0.334	<0.001	114
GPS	-0.080	0.459	88	-0.231	0.013	114
Power mat	-0.510	<0.001	88	-0.479	<0.001	114
Nano Bottle	-0.643	<0.001	88	-0.247	0.008	114
Overall	-0.484	<0.001	88	-0.281	0.002	114
Study 2
Vaccine strips	-0.512	<0.001	194	-0.516	<0.001	195
Flu drug	-0.568	<0.001	194	-0.428	<0.001	195
GPS	-0.436	<0.001	194	-0.393	<0.001	177
Power mat	-0.341	<0.001	194	-0.483	<0.001	195
Nano Bottle	-0.605	<0.001	194	-0.555	<0.001	195
Overall	-0.434	<0.001	194	-0.429	<0.001	195
Study 3 (Novel)
Vaccine strips	-0.528	<0.001	194	-0.552	<0.001	199
Power mat	-0.440	<0.001	194	-0.494	<0.001	199
Nano Bottle	-0.470	<0.001	194	-0.484	<0.001	199
Overall	-0.403	<0.001	194	-0.463	<0.001	199
Study 3 (Familiar)
Nuclear energy	-0.560	<0.001	194	-0.468	<0.001	199
Pesticides	-0.328	<0.001	194	-0.484	<0.001	199
GMO food	-0.673	<0.001	194	-0.678	<0.001	199
Overall	-0.574	<0.001	194	-0.529	<0.001	199

According to previous research, negative correlations between perceived risks and benefits signify that participants use the affect heuristic to judge risk and benefit (e.g., Alhakami & Slovic, 1994; Finucane et al., 2000; Fischhoff et al., 1978; Skagerlund et al., 2020). So, we examined the correlation between risk and benefit judgments within each modality condition. Within each modality condition, and for each technology, we computed the correlation between risk and benefit judgments across participants (see Table 1). Results show that all but one of these correlations were negative and statistically significant. Further analyses showed that the magnitude of the correlations did not differ across modalities (Spoken: $r[86] = -0.484$ , $p < 0.001$ ; Written: $r[112] = -0.281$ , $p = 0.002$ , $z = -1.66$ , $p = 0.096$ ).

2.3 | Discussion

Contrary to the assumption that risk and benefit judgments are modality independent, we found that modality exerts a systematic effect. Spoken descriptions of novel technologies increased the gap between judgments of benefit and judgments of risk compared to written descriptions. The correlations between risk and benefit judgments were negative and significant in both modality conditions. This is consistent with the hypothesis that risk and benefit evaluations were based on the affect heuristic, and is in line with previous research examining the same technologies (King & Slovic, 2014). Therefore, these results raise the possibility that modality impacts risk and benefit judgment by changing affective reactions toward the technologies. Study 2 investigated this idea.

3 | STUDY 2: EXPLAINING THE MODALITY EFFECT ON RISK AND BENEFIT JUDGMENTS

Study 2 investigates the underlying mechanism for the modality effect. The results of Study 1 point to such a mechanism in the crossover interaction between modality and risk and benefit judgments. Modality did not just alter risk perception, but it influenced it in a particular direction. The gap between benefit and risk judgment was greater in the spoken than in the written condition, showing a more favorable perception toward the technologies in the spoken modality. This is exactly the pattern one would expect if risk and benefit judgments are based on the affect heuristic, and spoken descriptions induce more positive affect than the written descriptions (Alhakami & Slovic, 1994; Siegrist & Sutterlin, 2014). We therefore predicted that modality might influence risk perception by inducing more positive affect in the spoken modality than in the written modality. If we are correct, then we should find that the spoken modality induces more positive affect than the written modality, and that this mediates the effect of modality on risk and benefit judgments. To test this prediction, Study 2 replicated Study 1 and collected measures of affective reactions.

3.1 | Method

3.1.1 | Power analysis

An a priori power analysis for a 2 (Modality: spoken, written) $\times 2$ (Judgment type: risk, benefit) mixed-factor ANOVA was conducted using an alpha of 0.05, a power of 0.80, a small effect size ($f = 0.143$, based on Study 1, using the SPSS effect size specification), number of groups $= 2$ (modality), number of measurements $= 2$ (judgment type), and $\varepsilon = 1$ ($1/2 - 1$). The analysis revealed a minimum sample of 388 participants.

As a precaution against possible data loss due to technical issues, we targeted about $15\%$ more participants.

3.1.2 | Participants

We recruited 447 native English speakers (46.5% female, 51.9% male, 1.6% other, $M_{\mathrm{age}} = 31.9$ years, age range: 18–64 years) through Prolific Academic (prolific.ac). We excluded 58 participants (13%) because they failed at least one of the predetermined inclusion requirements (for details see Supporting Information, Table S1).3

We performed analyses on the remaining 389 participants, 194 in the spoken condition and 195 in the written condition.

3.1.3 | Materials and procedure

The materials and procedure were identical to those of Study 1 with the following exceptions. In addition to the 22-year-old male native English speaker of Study 1, we included a 25-year-old female native English speaker in order to increase generalizability. Importantly, we added three questions to evaluate the participants' affective reactions toward each technology, each presented on a separate screen (adapted from King & Slovic, 2014; see also Batra & Ahtola, 1991): "How much do you like or dislike [technology]?" (1 = I like it, to 11 = I dislike it), "How good or bad do you find [technology]?" (1 = Good, to 11 = Bad), "How favorable or unfavorable is your opinion about [technology]?" (1 = Favorable, to 11 = Unfavorable). The presentation order of these questions was randomized separately for each participant.

3.1.4 | Results

3.1.5 | Judgments of risk and benefit

The results are illustrated in Figure 2. Replicating Study 1, the spoken modality resulted in lower judgments of risk ($M_{\mathrm{Spoken}} = 5.48$, $95\%$ CI [5.27, 5.69], $M_{\mathrm{Written}} = 5.82$, $95\%$ CI [5.61, 6.03]) and higher judgments of benefit ($M_{\mathrm{Spoken}} = 7.51$, $95\%$ CI [7.33, 7.69], $M_{\mathrm{Written}} = 7.32$, $95\%$ CI [7.14, 7.50]). Speaker and order had no impact on the modality effect and hence were omitted from the analyses.4

We submitted the mean risk and benefit judgments to a 2 (Modality: spoken, written) $\times 2$ (Judgment type: risk, benefit) mixed-factor ANOVA, with repeated measures on judgment type. While there was no main effect of modality, $F(1, 387) = 1.04$, $p = 0.309$, $\eta_p^2 < 0.01$, modality interacted with judgment type, $F(1, 387) = 5.18$, $p = 0.023$, $\eta_p^2 = 0.013$ (see Figure 2). We then conducted separate tests adjusted for multiple comparisons for each type of judgment and found a significant modality effect for risk judgments, $F(1, 387) = 5.27$, $p = 0.022$, $\eta_p^2 = 0.013$, but not for benefit judgments, $F(1, 387) = 2.24$, $p = 0.135$, $\eta_p^2 = 0.01$. As in Study 1, there was also a significant main effect of judgment type, $F(1, 387) = 220.32$, $p < 0.001$, $\eta_p^2 = 0.36$, whereby the technologies were perceived to be more beneficial ($M = 7.41$, $95\%$ CI [7.29, 7.54]) than risky ($M = 5.65$, $95\%$ CI [5.50, 5.80]).

3.1.6 | Correlations between risk and benefit judgments

The correlations were all negative and statistically significant (see Table 1). We then compared the magnitude of these correlations across modalities. No difference was detected (spoken: $r[192] = -0.434$, $p < 0.001$, written: $r[193] = -0.429$, $p < 0.001$; $z = -0.06$, $p = 0.96$). Next, we consider whether modality impacted risk and benefit judgments via the affect it induced.

3.1.7 | Affect ratings

The three affective valence measures across the five technologies had a McDonald's $\omega = 0.73$ and a Cronbach's $\alpha = 0.82$, so we averaged them into one index. As predicted, participants who received spoken descriptions rated them more positively than participants who received written descriptions ($M_{\text{Spoken}} = 4.90$, $95\%$ CI [4.70, 5.11]), $M_{\text{Written}} = 5.21$, $95\%$ CI [5.03, 5.39]; lower ratings represent more positive affect). A one-way ANOVA on the affect index confirmed a significant main effect of modality, $F(1, 387) = 5.03$, $p = 0.025$, $\eta^2 = 0.01$; Kruskal-Wallis Test $H(1) = 4.01$, $p = 0.045$.

3.1.8 | Explaining the modality effect on risk/benefit judgments

We then examined whether the modality effect on risk and benefit judgments was mediated by affect. As outcome variable, we used the difference between risk and benefit for each technology, which we then averaged across the technologies. This is known as the risk-benefit index (e.g., Skagerlund et al., 2020), and is useful as it encapsulates the modality effect which is the interaction between modality and judgment type (see Figure 2). The higher the risk-benefit index, the greater the judged risk and the lower the judged benefit.

We conducted a test of an indirect effect using 10,000 bootstrapped samples and the $95\%$ confidence interval (Hayes, 2018). There was a significant indirect effect from modality $(0 = \text{Written}, 1 = \text{Spoken})$ to affect to the risk-benefit index (indirect effect $= -0.44$, $95\%$ CI $[-0.829, -0.058]$, partially standardized indirect effect $= -0.19$, $95\%$ CI $[-0.347, -0.025]$ ). The path from modality to affect was significant, $b = -0.31$, $95\%$ CI $[-0.577, -0.041]$, $\beta = -0.23$, and so was the path from affect to the risk-benefit index, $b = 1.42$, $95\%$ CI [1.325, 1.523], $\beta = 0.82$. The effect of modality on the risk-benefit index was reduced when controlling for affect (from $b = -0.54$, $95\%$ CI $[-1.011, -0.231]$ to $b = -0.10$, $95\%$ CI $[-0.371, 0.165]$; see Figure 3). This suggests that affect is associated with the modality effect on the risk-benefit index.

3.2 | Discussion

Both studies 1 and 2 found a crossover interaction between modality and type of judgment. Describing the advantages and disadvantages of relatively unfamiliar technologies in the spoken modality increased the gap between judgments of benefit and judgments of risk, indicating a more positive perception of the technologies in the spoken modality. Study 2 suggests that the underlying mechanism for this effect is the differential affect that the modalities induce. The spoken modality induced more positive affect toward the unfamiliar technologies than the written modality and this mediated the effect as suggested by the affect heuristic. Study 3 provides a further test of the affective mechanism for the modality effect on risk perception by testing familiar and unfamiliar technologies.

4 | STUDY 3: THE MODALITY EFFECT IS UNIQUE TO NOVEL TECHNOLOGIES

The first goal of Study 3 was to replicate the modality effect on risk and benefit judgments for relatively novel and unfamiliar technologies as well as the mediating role of affect. Its second goal was to investigate whether this effect is less pronounced for familiar technologies. We predicted that the modality effect on affect would diminish with familiar technologies because people have stable affective attitudes toward them (Ajzen, 2001; Chaiken et al., 1999; Fazio, 2007; see also Xie et al., 2011). In contrast, when people hear about novel technologies, they form judgments on the spot based on immediately available information (Fazio, 2007; Schwarz, 2007). If this is true, then the modality effect should replicate for relatively novel and unfamiliar technologies but diminish for more familiar technologies because modality would induce differential affect only for unfamiliar technologies. We tested this prediction by using familiar and unfamiliar technologies and measuring participants' affective reactions toward them as well as their risk and benefit evaluations.

4.1 | Method

The study design, sample size, exclusion criteria, and data analysis were preregistered on https://aspredicted.org/jb5dj.pdf.

4.1.1 | Power analysis

To determine the desired sample size, we used the same a priori power analysis for a 2 (Modality: spoken, written) $\times 2$ (Judgment type: risk, benefit) $\times 2$ (Technology type: familiar, unfamiliar) mixed-factor ANOVA ($f = 0.143$, based on Study 1; using the SPSS effect size specification and $\eta_p^2 = 0.02$), power $= 0.80$, alpha $= 0.05$, number of groups $= 2$ (modality), number of measurements $= 4$ (2 [judgment type] $\times 2$ [technology type]), $\varepsilon = 0.34$ (1/4 - 1). The analysis suggested a minimum sample of 384 participants. We requested 400 participants using Prolific but received seven more responses; we had no control over that.

4.1.2 | Participants

Native English speakers ($N = 407$; $50.1\%$ female, $49.1\%$ male, $0.7\%$ others, $M_{\mathrm{age}} = 33.9$ years, age range: 18–63 years) participated through Prolific. Participants were eligible to participate only if they passed an initial audio test. From those who were eligible, we excluded 14 participants ($3.4\%$; for details see Supporting Information). The final sample included 393 participants, 199 (50.3% female, 49.7% male) in the written condition and 193 (50.8% female, 47.6% male, 1.6% other) in the spoken condition.

4.1.3 | Materials and procedure

We presented participants with three novel technologies from Studies 1 and 2 and with three more familiar technologies: genetically modified food, nuclear energy, and pesticides (see Table S2 and Table S3 in the Supporting Information). In addition to the 22-year-old male native English speaker of Study 1, we included another 22-year-old male native English speaker. Participants in the spoken condition heard the descriptions from one speaker. The novel and familiar technologies were presented in separate blocks, and the presentation order of the two blocks was counterbalanced across participants. The presentation order of the three technologies within each block was randomized separately for each participant.

The procedure and measures of Study 3 were identical to those of Study 2. The only difference was that for the affect questions, we reversed the labels of the scales so that in Study 3 higher ratings indicated more positive affect (1 = I dislike it to 11 = I like it, 1 = Bad to 11 = Good, 1 = Unfavorable to 11 = Favorable). This scale appeared more natural to us, and we wanted to ensure that the results of Study 2 were not due to the less intuitive scale.

Following the presentation of the six technologies and the associated risk, benefit, and recognition questions, participants were asked to indicate whether they experienced any technical problems. For exploratory purposes, we also measured subjective feelings of task involvement by asking participants to indicate their agreement or disagreement with the following statements (for the results of this involvement measure see Supplementary Results): "While reading (listening to) the technology descriptions, I had the impression that time was passing quickly", "In general, I found the task extremely rewarding", "While reading (listening to) the technology descriptions, I was fully absorbed", "I enjoyed reading (listening to) the technology descriptions", "I felt totally involved in reading (listening to) the technology descriptions" (5-point scale: $1 = \text{Strongly disagree}$, $2 = \text{Disagree}$, $3 = \text{Neither agree nor disagree}$, $4 = \text{Agree}$, $5 = \text{Strongly agree}$). We randomized the presentation order of these statements separately for each participant. The results of these measures are presented in the Supporting Information. As a check for familiarity, participants then indicated how familiar they were with each technology before participating in the study (5-point slider scale: $1 = \text{Not at all familiar}$ to $5 = \text{Very familiar}$).

4.2 | Results

4.2.1 | Familiarity check

Participants indicated less familiarity with the novel technologies than with the familiar ones ($M_{\text{Novel}} = 2.24$, $95\%$ CI [2.16, 2.32], $M_{\text{Familiar}} = 3.92$, 95% CI [3.84, 3.99]); $F(1, 389) = 1161.99$, $p < 0.001$, $\eta_p^2 = 0.75$; see Figure 4). Paired $t$-tests, adjusted for multiple comparisons, showed that each unfamiliar technology was rated as significantly less familiar than each familiar technology ($ts > 9.40$, $ps < 0.001$). There was no main effect of modality on familiarity, $F(1, 389) = 0.41$, $p = 0.523$, $\eta_p^2 < 0.01$, nor a Modality × Technology type interaction, $F(1, 389) = 0.02$, $p = 0.892$, $\eta_p^2 < 0.01$.

4.2.2 | Judgments of risk and benefit

The results are presented in Figure 5. For novel technologies, the results replicated Studies 1 and 2. Participants judged their risks as lower in the spoken than in the written modality ($M_{\text{Spoken}} = 5.79$, $95\%$ CI [5.56, 6.02], $M_{\text{Written}} = 6.37$, $95\%$ CI [6.15, 6.60]), and their benefits as higher ($M_{\text{Spoken}} = 7.07$, $95\%$ CI [6.84, 7.31], $M_{\text{Written}} = 6.67$, $95\%$ CI [6.44, 6.89]). For familiar technologies, modality made no difference, with comparable ratings of risk ($M_{\text{Spoken}} = 7.28$ vs $M_{\text{Written}} = 7.31$) and benefit ($M_{\text{Spoken}} = 7.35$ vs $M_{\text{Written}} = 7.37$).

As preregistered, we submitted the mean risk and benefit ratings to a 2 (Modality: spoken, written) $\times 2$ (Technology type: novel, familiar) $\times 2$ (Judgment type: risk, benefit) mixed-factor ANOVA with repeated measures on the last two factors. The analysis revealed the predicted three-way interaction between modality, technology type, and judgment type, $F(1,391) = 8.18$, $p = 0.004$, $\eta_p^2 = 0.02$ (see Figure 5). As expected, there was a significant Modality $\times$ Judgment type interaction for novel technologies, $F(1,391) = 12.71$, $p < 0.001$, $\eta_p^2 = 0.03$, but not for familiar technologies ($F < 1$).

Pairwise tests, adjusted for multiple comparisons, showed that in the spoken modality risk ratings for novel technologies were significantly lower than in the written modality, $F(1,391) = 12.66$, $p < 0.001$, $\eta_p^2 = 0.03$, while benefit ratings were higher, $F(1,391) = 6.15$, $p = 0.014$, $\eta_p^2 = 0.02$. For familiar technologies, all pairwise comparisons were not statistically significant (all $Fs < 1$). Finally, as in Study 2, speaker and presentation order of the advantages and disadvantages had no impact on the effect of modality.5

4.2.3 | Correlations between judgments of risk and benefit

All correlations were negative and statistically significant. As in Study 2, for novel technologies the correlations between risk and benefit were comparable across conditions (spoken: $r[192] = -0.403$, $p < 0.001$; written: $r[197] = -.0463$, $p < 0.001$; $z = -0.73$, $p = 0.468$). The same was true for familiar technologies (spoken: $r[192] = -0.574$, $p < 0.001$; written: $r[197] = -0.529$, $p < 0.001$; $z = -0.64$, $p = 0.522$).

4.2.4 | Affect ratings

We submitted the mean affect ratings across the three measures for novel technologies (McDonald's $\omega = 0.78$, Cronbach's $\alpha = 0.83$) and familiar technologies (McDonald's $\omega = 0.88$, Cronbach's $\alpha = 0.91$) to a 2 (Modality: spoken, written) $\times$ 2 (Technology type: novel, familiar) mixed-factor ANOVA, with repeated measures on the last factor. There was a main effect of technology type, $F(1, 391) = 10.07$, $p = 0.002$, $\eta_p^2 = 0.03$, which was qualified by a significant Modality $\times$ Technology type interaction, $F(1, 391) = 4.76$, $p = 0.030$, $\eta_p^2 = 0.01$.

Pairwise tests, adjusted for multiple comparisons, revealed that for novel technologies the spoken modality induced more positive affect ($M_{\text{Spoken}} = 6.42$, $95\%$ CI [6.19, 6.65]) than the written modality ($M_{\text{Written}} = 6.00$, $95\%$ CI [5.78, 6.23]), $F(1, 391) = 6.52$, $p = 0.011$, $\eta_p^2 = 0.02$. This replicates Study 2. In contrast, there was no difference in affective reaction between the spoken and written modalities for the familiar technologies ($M_{\text{Spoken}} = 5.82$, $95\%$ CI [5.54, 6.10]), $M_{\text{Written}} = 5.89$, $95\%$ CI [5.61, 6.17]), $F(1, 391) = 0.13$, $p = 0.720$, $\eta_p^2 < 0.01$. Finally, there was no main effect of modality, $F(1, 391) = 1.43$, $p = 0.233$, $\eta_p^2 < 0.01$.

4.2.5 | Explaining the modality effect on risk and benefit judgments

As in Study 2, we evaluated whether affect mediated the modality effect on risk and benefit judgments for novel technologies (see Figure 3). We conducted this analysis only for novel technologies because familiar technologies showed no modality effect. We found that there was a significant indirect effect from modality $(0 = \text{Written}, 1 = \text{Spoken})$ to affect to the risk-benefit index (indirect effect $= -0.57$ , $95\%$ CI $[-1.015, -0.143]$ ; partially standardized indirect effect $= -0.20$ , $95\%$ CI $[-0.352, -0.052]$ ). The effect of modality on the risk-benefit index was reduced when controlling for affect (from $b = -0.99$ , $95\%$ CI $[-1.543, -0.446]$ to $b = -0.43$ , $95\%$ CI $[-0.766, -0.092]$ ), suggesting that affect is associated with the modality effect.

In sum, replicating Study 2, the modality effect on risk and benefit judgments was in part because spoken descriptions prompted a more positive affective reaction toward the novel technologies compared to written descriptions. Consistent with this, modality had an effect only when it induced differential affect. With familiar technologies there was no difference in affect ratings and thus no modality effect.

5 | GENERAL DISCUSSION

The results of three studies challenge the assumption that the modality in which information is delivered is inconsequential for judgment and decision making. In Study 1 we found that modality influences judgments of risk and benefit. Spoken descriptions of novel technologies led to a more positive perception in terms of overall risks and benefits. We replicated this modality effect in Studies 2 and 3 which also investigated its psychological underpinnings and found a mediating role of affect. Spoken descriptions induced more positive affect, which increased the gap between judged benefit and judged risk. Study 3 further supported the affect account and found that the modality effect is unique to relatively unfamiliar technologies. For familiar ones, for which people are likely to have stable affective attitudes, spoken and written information prompted similar affective reactions, and consequently, risk and benefit judgments were unaffected. Therefore, our studies show that the judgment of risk and benefit of relatively unfamiliar technologies is modality dependent, and that this effect is associated with the differential affective reactions the modalities induce.

5.1 | Possible mechanisms

5.1.1 | Spoken modality increases perceptual fluency

One could hypothesize that the link between modality and affect is due to differences in fluency: Processing spoken information might be associated with a higher metacognitive feeling of ease compared to processing written information. This might be because spoken language contains more explicit prosodic information such as rhythm, pauses and pitch variations than does written language (Breen & Clifton, 2011; Liberman & Whalen, 2000; Rayner & Clifton, 2009), which facilitates syntactic (Snedeker & Trueswell, 2003) and referential processing (Dahan et al. 2002). Research suggests that perceived fluency prompts positive affective reactions toward a stimulus (Winkielman et al. 2003). This might explain the more positive affective reaction in the spoken condition and—through the affect heuristic—the more favorable evaluation in terms of overall risks and benefits. However, our data do not support some predictions that are related to fluency. This account predicts greater task enjoyment in the spoken condition (Labroo & Pocheptsova, 2016), but we found the opposite (Study 3, exploratory measure, “I enjoyed reading [listening to] the technology descriptions.”). Furthermore, this account predicts higher ratings of familiarity in the spoken condition (Whittlesea et al. 1990), but we found no such modality differences.

5.1.2 | Spoken modality increases cognitive effort

One could also imagine that the modality effect is due to spoken information being more ephemeral and thus requiring more cognitive effort to retain the information. In this account, spoken language resembles written language under cognitive load. Existing evidence challenges this possibility. First, King and Slovic (2014) tested the same novel technologies in the written modality with or without additional cognitive load, and found that load does not influence affective reactions. In the present studies the spoken modality induced more positive reactions. Furthermore, time pressure tends to strengthen the negative correlation between risk and benefit judgments for familiar technologies (Finucane et al. 2000). However, in the present studies modality left these correlations unaffected (see last rows of Table 1).

5.1.3 | Spoken modality reduces attentional engagement

A third possibility is that the modality effect results from differential attentional engagement. Listeners might have attended less to the technology descriptions compared to readers. This is consistent with findings that listening requires fewer attentional resources than reading, and increases thoughts and feelings unrelated to the task (Kopp & D'Mello, 2016; Sousa et al. 2013). Research on information processing suggests that positively valenced information is relatively faster and easier to process than negatively valenced information (e.g. Unkelbach et al. 2020). In the present study, participants received technology descriptions that mentioned both positive features, advantages, and negative features, disadvantages. To the extent that processing spoken information reduces attentional engagement and positive information is faster and easier to process than negative information, participants in the spoken condition might have weighed less the disadvantages than participants in the written condition. This, in turn, could explain the overall more positive attitudes in the spoken condition.

This attentional engagement account is partly consistent with our data. The exploratory analyses on the involvement measure in Study 3 show that listeners feel less involved than readers, and that perceived involvement partially mediates the modality effect on the risk‐benefit index (see Supporting Information). Affect remained a significant predictor, and thus this finding does not undermine our main hypothesis. However, because our involvement measure only indirectly captures attentional engagement and because we found no consistent modality differences in recognition performance (see Supporting Information), it is difficult to draw this conclusion firmly.

5.2 | Theoretical contribution

Our findings have important implications for judgment and decision making and language understanding. They provide strong evidence that the impact of language on an audience depends on communication modality, suggesting that form of language is more fundamental for judgment and decision making than has been previously assumed. The present study further demonstrates that the affective reaction to novel stimuli changes based on language modality, thereby influencing judgment. The way in which we receive language, impacts our perception of a situation and the judgments we make. This raises the possibility that modality influences other affect‐based judgments such as the willingness to invest in novel products.

Our findings might also contribute to the understanding of the relationship between language and thought broadly defined. According to rational choice theory, risk and benefit perception should follow the invariance criterion. Rational agents should form judgments based on the content of the information, not on how they receive it. Kahneman and Tversky (1984) demonstrated a violation of the invariance criterion by showing that descriptions lead to different choices when they are framed in terms of gains or losses. Here we demonstrate that descriptions with identical wording lead to different risk and benefit judgments when the language modality changes.

Beyond the broad theoretical contribution, the current study presents strong evidence for the affect heuristic. The main evidence for the affect heuristic has been correlational, namely that judgments of risk and benefit are negatively associated. In Study 3, we offer causal evidence. When the modality manipulation influenced affect (novel technologies), it influenced the risk‐benefit index in the direction predicted by the affect heuristic. When it did not (familiar technologies), it did not influence the risk‐benefit index.

5.3 | Implications

The choice of modality in communication tends to be a matter of convenience. Here we show that it is consequential. The importance of this should be evident for surveys and opinion polls about novel issues, which are crucial tools in academia, politics, and commerce. For example, using a voice or a written survey to conduct a poll on the acceptance of a novel technology, such as the much‐debated 5G wireless technology (Timmers, 2020), could lead to different results. An opinion poll conducted by voice could minimize the perceived risks, augment the perceived benefits, and consequently increase the public acceptance of the technology (cf. Schober et al. 2015). This is in line with research showing that survey respondents tend to provide more extreme positive ratings when asked questions about their satisfaction with certain consumer services by voice rather than text (e.g. Dillman & Christian, 2005; Tourangeau et al. 2002).

5.4 | Limitations

The present research measured affective attitudes via self‐reports. Physiological measures of affect, such as skin conductance, pupil dilation, or heart rate could be further explored in order to better understand how modality influences affective reactions. Furthermore, we used unfamiliar technologies toward which participants have an overall positive affective reaction. It is possible that spoken modality polarizes affective attitudes rather than increases positive affect. Therefore, it would be fruitful to investigate how modality influences risk perception of unfamiliar technologies toward which most people have a negative affective reaction, such as insect‐based food (Geipel et al. 2018). It is possible that in this context spoken modality would reduce positive affect and increase perceived risk.

6 | CONCLUSION

Novel technologies, such as a new medication or the use of novel nanotechnology in food packaging, promise to promote health, safety, and sustainability. However, the public is frequently hesitant to accept them. The ongoing digital transformation gives people the choice to inform themselves about novel technologies by voice or text. Our findings suggest that learning about novel technologies by voice as compared to text can prompt a more positive overall perception toward them, which could influence the public's willingness to adopt them. Returning to the opening example, the patients in the United Kingdom who receive information about a new medication via their voice assistant system would likely view the medication more favorably.

Footnotes (5)

1 Besides being standard practice, counterbalancing the presentation order of the advantages and disadvantages of the technologies is important in our studies to rule out memory explanations. Research suggests better recall of temporal order with spoken information than with written information (e.g. Glenberg, 1990; Glenberg & Fernandez, 1988; Metcalfe et al. 1981), as well as a stronger primacy effect with spoken information (Unnava et al. 1994). In all our studies, presentation order did not interact with the effect of modality suggesting that the modality effect is not memory based.

2 Including presentation order in the model did not reveal a main effect of order, $F(1, 198) = 0.71, p = 0.401, \eta_{p}^{2} < 0.01$, an Order × Modality interaction, $F(1, 198) = 0.04, p = 0.848, \eta_{p}^{2} < 0.01$, or an Order × Modality × Question type interaction, $F(1, 198) = 0.01, p = 0.914, \eta_{p}^{2} < 0.01$.

3 The results were not affected by the exclusions. The Modality × Question type interaction is significant even if we include participants who did not pass the attention check or participants who indicated that they did not read or listened carefully.

4 Order and speaker had no impact on the effect of modality on risk/benefit judgments or affect ratings. For risk/benefit judgments, there was no Modality × Judgment type × Speaker type interaction, $F(1, 381) = 0.11, p = 0.740, \eta_{p}^{2} < 0.01$, nor a Modality × Judgment type × Order interaction, $F(1, 381) = 3.31, p = 0.070, \eta_{p}^{2} = 0.01$. For affect ratings, there was no Modality × Speaker × Order interaction, $F(1, 381) = 0.86, p = 0.354, \eta^{2} < 0.01$, nor a Modality × Speaker interaction, $F(1, 381) = 0.04, p = 0.838, \eta^{2} < 0.01$, or Modality × Order interaction, $F(1, 381) = 1.90, p = 0.168, \eta^{2} = 0.01$.

5 Order and Speaker had no impact on the effect of modality on risk/benefit judgments or affect ratings. For risk/benefit judgments, there was no Modality × Speaker interaction, $F(1, 385) < 0.01, p = 0.978, \eta_{p}^{2} < 0.01$, nor a Modality × Order interaction, $F(1, 385) < 0.01, p = 0.955, \eta_{p}^{2} < 0.01$, or a three‐way Modality × Speaker × Order interaction, $F(1, 385) = 0.12, p = 0.733, \eta_{p}^{2} < 0.01$. For affect ratings, there was no Modality × Speaker interaction, $F(1, 385) = 2.28, p = 0.132, \eta_{p}^{2} = 0.01$, nor was there a Modality × Order interaction, $F(1, 385) = 1.54, p = 0.215, \eta_{p}^{2} = 0.01$, or a three‐way Modality × Speaker type × Order interaction, $F(1, 385) = 0.13, p = 0.718, \eta_{p}^{2} < 0.01$.