Sensitivity to argument quality: Adding Turkish data to the question of cultural variability versus universality

Empirical research has shown that high-quality arguments according to criteria from argumentation theory lead to higher claim acceptance than low-quality arguments. However, this relationship was not observed in some cultural settings. This leads to the question whether criteria for high-quality arguments are culturally variable or universal. Therefore, adding to existing research on sensitivity to quality criteria for the argument from authority and the argument from generalization conducted mainly in Western cultural contexts, an experiment was run in Turkey (N = 307). Results showed that Turkish participants were sensitive to the quality of arguments: claim acceptance was higher when high-quality variants were used than when low-quality variants were used. While not neglecting potential cultural variability, these data add to the findings that there might be some level of universality in sensitivity to criteria for argument quality.

Demir, Y., & Hornikx, J. (2022). Sensitivity to argument quality: Adding Turkish data to the question of cultural variability versus universality. Communication Research Reports, 39 (2), 104-113. [link]

Combining anecdotal and statistical evidence in real-life discourse

The persuasiveness of anecdotal evidence and statistical evidence has been investigated in a large number of studies, but the combination of anecdotal and statistical evidence has hardly received research attention. The present experimental study therefore investigated the persuasiveness of this combination. It also examined whether the quality of anecdotal evidence affects persuasiveness, and to what extent people comprehend the combination of anecdotal and statistical evidence. In an experiment, people read a realistic persuasive message that was relevant to them. Results showed that anecdotal evidence does not benefit from the inclusion of statistical evidence, nor from its intrinsic quality. The analysis of readers’ cognitive thoughts showed that only a minority of participants comprehended the relationship between anecdotal and statistical evidence.

  • Hornikx, J. (2018). Combining anecdotal and statistical evidence in real-life discourse: Comprehension and persuasion. Discourse Processes, 55 (3), 324-336.

Cultural differences in the perceptions of strong and weak arguments

The process of arbitration requires human reasoning and decision-making. Parties evaluate the evidence that is available to them and decide how to best present their case. Arbitrators aim to resolve a dispute by weighing the evidence and the legal arguments that are presented by each side. Researchers have underlined the importance of strong evidence in legal deliberations, but what exactly characterizes strong arguments? This chapter addresses this question as a first point. The characteristics of arbitrators, such as age, gender, and cultural background, may affect how arbitrators process arguments. Yet given the aim of arbitration to be an objective and neutral process, it is important to consider how such characteristics may impact the ultimate outcome of a case. This chapter examines the last of these characteristics, namely the role of culture in this decision-making process. More precisely, this chapter reviews the research evidence on how members of different cultures evaluate strong and weak arguments.

  • Hornikx, J.  (2017). Cultural differences in the perceptions of strong and weak arguments. In T. Cole (ed.), The role of psychology in international arbitration (pp. 72-92). Alphen aan den Rijn: Wolters Kluwer. [link]

An experimental investigation of the role of distraction and dilution

PaglieriStudies on persuasive arguments have generally found that claims supported by high-quality evidence are better accepted than claims supported by low-quality evidence. However, an experiment by Hoeken and Hustinx (2007) demonstrated that this effect was only observed in short texts (a claim with evidence), but not in longer texts (where information unrelated to the evidence was added at the end of the text). The present experiment was conducted to examine whether this effect of text length could be explained by distraction (the additional text at the end distracts the reader) or by dilution (the additional text makes the fragment less diagnostic for claim evaluation). Participants (N = 629) read two texts with a claim supported by high-quality or low-quality (anecdotal, statistical, or expert) evidence. The text was presented in one of the three versions: (1) short, (2) long with additional information at the end, or (3) new in comparison to Hoeken and Hustinx (2007) – long with additional information at the start. The data found support for the distraction explanation. An effect of evidence quality on claim acceptance was observed in two conditions: in the short text, and in the longer text with additional information at the start. The effect of evidence quality was not found in the longer text with additional information at the end.

  • Hornikx, J. (2016). Evidence quality variations and claim acceptance: An experimental investigation of the role of distraction and dilution. Paglieri, F., Bonelli, L., & Felletti, S. (eds.), The psychology of argument: Cognitive approaches to argumentation and persuasion (pp. 211-222). London: College Publications. [paper book]

Het effect van evidentiekwaliteit op de beoordeling van standpunten: de rol van toegevoegde tekst

TvT coverClaims supported by high-quality evidence have been found to be more persuasive than claims supported by low-quality evidence. However, Hoeken and Hustinx (2007) showed that this effect was only observed in short texts (a claim with evidence), but not in longer texts (where information unrelated to the evidence was added at the end of the text). The current experiment was conducted to examine whether this effect of text length could be explained by distraction (the additional text at the end distracts the reader) or by dilution (the additional text makes the fragment less diagnostic for claim evaluation). Participants (N = 629) read two texts with high/low-quality evidence. The text was presented in three versions: short, long with additional information at the end, or – new in comparison to Hoeken and Hustinx (2007) – long with additional information at the start. The data found support for the distraction explanation: an effect of evidence quality on persuasiveness was observed in the short text, and in the longer text with additional information at the start, but not in the longer text with additional information at the end.

  • Hornikx, J. (2014). Het effect van evidentiekwaliteit op de beoordeling van standpunten: de rol van toegevoegde tekst. Tijdschrift voor Taalbeheersing, 36 (1), 107-125 [link].

Germans are not sensitive to the quality of statistical evidence

JOCC coverFor a long time, research in communication and argumentation has investigated which kinds of evidence are most effective in changing people’s beliefs in descriptive claims. For each type of evidence, such as statistical or expert evidence, high-quality and low-quality variants exist, depending on the extent to which evidence respects norms for strong argumentation. Studies have shown that participants are sensitive to such quality variations in some, but not in all, cultures. This paper expands such work by comparing the persuasiveness of high- and low-quality statistical and expert evidence for participants from two geographically close cultures, the Dutch and the German. Study 1, in which participants (N = 150) judge a number of claims with evidence, underscores earlier findings that high-quality is more persuasive than low-quality evidence for the Dutch, and – surprisingly – also shows that this is less the case for the Germans, in particular for statistical evidence. Study 2 with German participants (N = 64) shows again they are not sensitive to the quality of statistical evidence, and rules out that this finding can be attributed to their understanding of the rules of generalization. Together, findings in this paper underline the need to empirically investigate what norms people from different cultures have for high-quality evidence, and to what extent these norms matter for persuasive success.

  • Hornikx, J., & Haar, M. ter (2013). Evidence quality and persuasiveness: Germans are not sensitive to the quality of statistical evidence. Journal of Cognition and Culture, 13 (5), 483-501. [linkpdf upon request]

Persuasive evidence in India

Research on persuasive evidence types has been limited to Western cultures. Because Western systems of thought are claimed to be fundamentally different from Eastern systems of thought, the persuasiveness of evidence types was explored in one Eastern culture. Indians (N = 183) judged claims supported by different evidence types.  Statistical, expert, and causal evidence were found to be equally persuasive as support for claims.  Indians also appeared sensitive to evidence quality that was manipulated according to Western norms for reasonable argumentation: normatively strong evidence was more persuasive than normatively weak evidence.  Findings are compared to results from studies conducted in Western cultures.

  • Hornikx, J. & Best, J. de (2011). Persuasive evidence in India: An investigation of the impact of evidence types and evidence quality. Argumentation and Advocacy, 47 (4), 246-257. [pdf]

De overtuigingskracht van normatief sterke en normatief zwakke anekdotische evidentie

Vanuit normatief oogpunt zou de kwaliteit van evidentie invloed moeten hebben op de overtuigingskracht ervan. In experimenten lijkt normatief sterke evidentie ook overtuigender dan normatief zwakke evidentie, maar wanneer de boodschap langer (Hoeken & Hustinx, 2007) of natuurlijker is (Hoeken & Van Wijk, 1997) blijken ontvangers niet gevoelig voor evidentiekwaliteit. In deze studie is onderzocht of de kwaliteit van anekdotische evidentie wel effect heeft op overtuigingskracht als ook statistische evidentie aanwezig is. Nijmegenaren (N = 125) beoordeelden een brief van de gemeente Nijmegen over afvalverwerking waarin normatief sterke of zwakke anekdotische evidentie stond, die wel of niet was aangevuld met statistische evidentie. Zonder statistische evidentie had de kwaliteit van anekdotische evidentie geen effect op de overtuigingskracht, maar mét statistische evidentie was sterke anekdotische evidentie overtuigender dan zwakke anekdotische evidentie. Deze studie laat daarmee zien dat de kwaliteit van evidentie ook in een natuurlijke, realistische setting de overtuigingskracht ervan kan beïnvloeden.

  • Hornikx, J., & Houët, T. (2009). De overtuigingskracht van normatief sterke en normatief zwakke anekdotische evidentie in het bijzijn van statistische evidentie. In W. Spooren, M. Onrust, & J. Sanders (Eds.), Studies in taalbeheersing, vol. 3 (pp. 125-133). Assen: Van Gorcum. [pdf]

Comparing the actual and expected persuasiveness of evidence types

Whereas there are many publications in which argumentation quality has been defined by argumentation theorists, considerably less research attention has been paid to lay people’s considerations regarding argument quality. Considerations about strong and weak argumentation are relevant because they can be compared with actual persuasive success. Argumentation theorists’ conceptions have to some extent been shown to be compatible with actual effectiveness, but for lay people such compatibility has yet to be determined. This study experimentally investigated lay people’s expectations about the persuasiveness of anecdotal, statistical, causal, and expert evidence, and compared these expectations with the actual persuasiveness of these evidence types. Dutch and French participants (N = 174) ranked four types of evidence in terms of their expected persuasiveness for eight different claims. Both cultural groups expected statistical evidence to be the most persuasive type of evidence to other people, followed by expert, causal, and, finally, anecdotal evidence. A comparison of these rankings with the results of Hornikx and Hoeken (2007, Study 1) on the actual persuasiveness of the same evidence types reveals that people’s expectations are generally accurate: How relatively persuasive they expect evidence types to be often corresponded with their actual persuasiveness.

  • Hornikx, J. (2008). Comparing the actual and expected persuasiveness of evidence types: How good are lay people at selecting persuasive evidence? Argumentation, 22 (4), 555-569. [pdf]

Cultural differences in the persuasiveness of evidence types and evidence quality

Cultural differences in reasoning and persuasion have mainly been documented for the East – West divide. Nisbett (2003) expects such differences to be absent for Western cultures because of their shared Grecian inheritance. The results of two experiments, however, show that France and the Netherlands, both Western-European countries, differ with respect to the persuasiveness of different evidence types. In Study 1 (N = 600), cultural differences occurred between the relative persuasiveness of anecdotal, statistical, causal and expert evidence. In Study 2 (N = 600), the quality of statistical and expert evidence was manipulated. For the Dutch, but not for the French, normatively strong evidence was more persuasive than normatively weak evidence for both evidence types. Implications and possible explanations are discussed.

  • Hornikx, J., & Hoeken, H. (2007). Cultural differences in the persuasiveness of evidence types and evidence quality. Communication Monographs, 74 (4), 443-463. [pdf]