2021

By Asher van der Schelde, Marina Tulin, and Jay Lee, Erasmus University Rotterdam How do you stay informed about what is happening in the world? Chances are social media play a crucial role. This is not entirely surprising as science communication increasingly occurs via platforms such as YouTube and Twitter. Nonetheless, we know little about how viewers perceive, trust, or judge, this type of science communication. In this study, we tried to bridge this knowledge gap by running a survey experiment using a science communication video by the popular animation studio Kurzgesagt (In a Nutshell). The experiment focused on climate change. Despite the vast majority of scientists agreeing that humans have a (lasting) impact on climate change, a fraction of the population does not believe in the existence of the phenomenon, or its man-made cause. As a consequence, science communication experts face the critical, yet difficult, task of increasing public understanding, and stimulating engagement within this group of skeptics. The Kurzgesagt video we used is called “Who is responsible for climate change? – Who needs to fix it?”. As the title suggests, the video revolves around climate change and which countries should take responsibility in countering this worrying development. Figure 1: Screenshot from the Kurzgesagt video “Who Is Responsible For Climate Change? – Who Needs To Fix It?” We split up this video in three chapters to which we made several manipulations (e.g., the gender of the narrator, textual changes). After watching one chapter, respondents answered how they perceived the video in terms of trustworthiness, reliability, engagement, and entertainment. This allowed us to understand whether small changes in the video affect the overall perceptions of the video. Respondents also answered how they perceived the narrator, the overall production and what they thought the primary aim of the video was, which produced interesting insights as well. Positive perceptions First, it is important to mention that the video was perceived very positively. Respondents deemed the video to be trustworthy, reliable, engaging and entertaining (Figure 2). They were also positive towards the female and male narrator. The narrators were perceived to be similar in terms of warmth and trustworthiness, but the male narrator was perceived to be more competent (Figure 3). This result can reflect some underlying gender bias and would require further analysis to disentangle cultural and individual effects. Figure 2: Video perceptions. 1 = strongly disagree, 5 = strongly agree (with 95% CI). Figure 3: Narrator perceptions by gender (F = female, M = male). 1 = strongly disagree, 5 = strongly agree (with 95% CI). Trustworthiness and entertainment go hand-in-hand Science communicators can be hesitant to communicate insights in an entertaining way as they fear this might harm the trustworthiness of the message. Our analysis indicates this fear is not grounded in reality. Respondents who perceived the video to be entertaining, were more likely to say the video was also trustworthy, reliable and engaging. As a consequence, quality and entertainment should not be treated as a trade-off. On the contrary: they go hand-in hand. All correlations between the video and narrator perceptions are illustrated in table 1. Table 1: Correlations between video and narrator perceptions Perceived aim matters One of the most striking findings is the effect of perceived aim on the overall perception of the video. When respondents believe the aim of the video is to blame, they perceive the video to be less trustworthy. Some of these respondents commented that the video was ‘quite brain washing’ or ‘seemed to be aimed at children’. Therefore, science communicators should make sure their message does not come across as such. On the contrary, respondents that indicated the video aimed to inform, rated the video higher on trustworthiness than those who did not. The same goes for changing behaviour, which is a remarkable finding, as this suggests that science communicators do not have to obscure their beliefs if they feel strongly about a certain topic and want to make a change. The significant effects of perceived aim on trustworthiness are displayed in figure 4. Figure 4: Perceived trustworthiness by perceived aim. 1 = strongly disagree, 5 = strongly agree (with 95% CI). Production value serves a proxy for overall perception When production value was perceived to be high, the video and narrator were perceived more positively. By perceived production value we mean the extent to which viewers believe that the video is of high quality regarding technical aspects, such as quality of resolution, professional voice-over, recording quality, sound design, detail of illustrations, or smoothness of animation. Perceived production value possibly serves as a heuristic that viewers use as a proxy for their overall perception of the video. Viewers project good intentions onto the creators and give them the benefit of the doubt. Figure 5: Perceived trustworthiness, engagement and entertainment value separated by perceived production value. 1 = strongly disagree, 5 = strongly agree (with 95% CI). Effects of manipulations We incorporated several manipulations to see if they would affect perceptions of the video. Through a check in the survey, we found that a substantial part of the participants had not noticed or remembered the manipulations. In hindsight, this is unsurprising as most manipulations were made in the narrative while humans tend to be more focused on visual information. Running a separate analysis in which we only included respondents who did notice and remember the manipulation did result in more significant findings. For example: participants who remembered that negative consequences of climate change for Europe (instead of worldwide) were mentioned, were more engaged with the video (leaving a comment about the video was coded as engagement). Since all respondents reside in the UK, this illustrates the human tendency to be more concerned by local developments. Similarly, engagement was greater among the respondents who watched (and remembered) the video that contained a fearful message than among those who were provided with a hopeful message. Figure 6: Differences in engagement by remembered manipulation. X-axis corresponds to proportion of respondents in group who engaged (i.e., left a comment) with the video (with 95% CI). Besides more engagement, the ‘fearful message’ also resulted in a decrease in the narrator’s perceived warmth. Thus, ending on a positive note results in a warmer perception of the narrator. This difference is depicted in Figure 7. Figure 7: Differences in perceived warmth by message. 1 = strongly disagree, 5 = strongly agree (with 95% CI). Communicate uncertainty! Science is often uncertain. However, science communicators might fear that communicating these uncertainties will confuse people. We did not find significant negative effects of using uncertain terms (e.g., approximately and presenting numbers in ranges) on the video’s perceived trustworthiness, reliability, engagement, and entertainment. This indicates that science communicators can be honest about presenting uncertain findings. Audience matters Finally, we found that science communicators should be well aware who they are targeting as climate change attitudes affect how the videos are perceived. For example: the inclusion of prominent sources enhances engagement, but this effect is muted for those who ‘believe’ more in climate change. This leads us to believe that this group is not fond of “over-the-top” stimuli. Similarly, climate change deniers are more likely to perceive the video as less trustworthy when the narrator uses uncertain terms in comparison to believers. In sum, the public tends to be informed (sometimes falsely) about many scientific topics. As such, people have already formed their opinions. Consequently, science communicators need to be aware they are reaching out to an already informed, and critical audience. The next TRESCA objective will be to produce a new video with Kurzgesagt on the history of rationality, and different aspects of human reasoning. Valuable insights that were gained in this experiment will be used to improve the video. Keep an eye on the TRESCA website and the YouTube channel of Kurzgesagt-In a nutshell for the video release (expected in September 2021).