Cultural differences in the perceptions of strong and weak arguments

The process of arbitration requires human reasoning and decision-making. Parties evaluate the evidence that is available to them and decide how to best present their case. Arbitrators aim to resolve a dispute by weighing the evidence and the legal arguments that are presented by each side. Researchers have underlined the importance of strong evidence in legal deliberations, but what exactly characterizes strong arguments? This chapter addresses this question as a first point. The characteristics of arbitrators, such as age, gender, and cultural background, may affect how arbitrators process arguments. Yet given the aim of arbitration to be an objective and neutral process, it is important to consider how such characteristics may impact the ultimate outcome of a case. This chapter examines the last of these characteristics, namely the role of culture in this decision-making process. More precisely, this chapter reviews the research evidence on how members of different cultures evaluate strong and weak arguments.

  • Hornikx, J.  (2017). Cultural differences in the perceptions of strong and weak arguments. In T. Cole (ed.), The role of psychology in international arbitration (pp. 72-92). Alphen aan den Rijn: Wolters Kluwer. [link]

An experimental investigation of the role of distraction and dilution

PaglieriStudies on persuasive arguments have generally found that claims supported by high-quality evidence are better accepted than claims supported by low-quality evidence. However, an experiment by Hoeken and Hustinx (2007) demonstrated that this effect was only observed in short texts (a claim with evidence), but not in longer texts (where information unrelated to the evidence was added at the end of the text). The present experiment was conducted to examine whether this effect of text length could be explained by distraction (the additional text at the end distracts the reader) or by dilution (the additional text makes the fragment less diagnostic for claim evaluation). Participants (N = 629) read two texts with a claim supported by high-quality or low-quality (anecdotal, statistical, or expert) evidence. The text was presented in one of the three versions: (1) short, (2) long with additional information at the end, or (3) new in comparison to Hoeken and Hustinx (2007) – long with additional information at the start. The data found support for the distraction explanation. An effect of evidence quality on claim acceptance was observed in two conditions: in the short text, and in the longer text with additional information at the start. The effect of evidence quality was not found in the longer text with additional information at the end.

  • Hornikx, J. (2016). Evidence quality variations and claim acceptance: An experimental investigation of the role of distraction and dilution. Paglieri, F., Bonelli, L., & Felletti, S. (eds.), The psychology of argument: Cognitive approaches to argumentation and persuasion (pp. 211-222). London: College Publications. [paper book]

Het effect van evidentiekwaliteit op de beoordeling van standpunten: de rol van toegevoegde tekst

TvT coverClaims supported by high-quality evidence have been found to be more persuasive than claims supported by low-quality evidence. However, Hoeken and Hustinx (2007) showed that this effect was only observed in short texts (a claim with evidence), but not in longer texts (where information unrelated to the evidence was added at the end of the text). The current experiment was conducted to examine whether this effect of text length could be explained by distraction (the additional text at the end distracts the reader) or by dilution (the additional text makes the fragment less diagnostic for claim evaluation). Participants (N = 629) read two texts with high/low-quality evidence. The text was presented in three versions: short, long with additional information at the end, or – new in comparison to Hoeken and Hustinx (2007) – long with additional information at the start. The data found support for the distraction explanation: an effect of evidence quality on persuasiveness was observed in the short text, and in the longer text with additional information at the start, but not in the longer text with additional information at the end.

  • Hornikx, J. (2014). Het effect van evidentiekwaliteit op de beoordeling van standpunten: de rol van toegevoegde tekst. Tijdschrift voor Taalbeheersing, 36 (1), 107-125 [link].

Germans are not sensitive to the quality of statistical evidence

JOCC coverFor a long time, research in communication and argumentation has investigated which kinds of evidence are most effective in changing people’s beliefs in descriptive claims. For each type of evidence, such as statistical or expert evidence, high-quality and low-quality variants exist, depending on the extent to which evidence respects norms for strong argumentation. Studies have shown that participants are sensitive to such quality variations in some, but not in all, cultures. This paper expands such work by comparing the persuasiveness of high- and low-quality statistical and expert evidence for participants from two geographically close cultures, the Dutch and the German. Study 1, in which participants (N = 150) judge a number of claims with evidence, underscores earlier findings that high-quality is more persuasive than low-quality evidence for the Dutch, and – surprisingly – also shows that this is less the case for the Germans, in particular for statistical evidence. Study 2 with German participants (N = 64) shows again they are not sensitive to the quality of statistical evidence, and rules out that this finding can be attributed to their understanding of the rules of generalization. Together, findings in this paper underline the need to empirically investigate what norms people from different cultures have for high-quality evidence, and to what extent these norms matter for persuasive success.

  • Hornikx, J., & Haar, M. ter (2013). Evidence quality and persuasiveness: Germans are not sensitive to the quality of statistical evidence. Journal of Cognition and Culture, 13 (5), 483-501. [linkpdf upon request]

Persuasive evidence in India

Research on persuasive evidence types has been limited to Western cultures. Because Western systems of thought are claimed to be fundamentally different from Eastern systems of thought, the persuasiveness of evidence types was explored in one Eastern culture. Indians (N = 183) judged claims supported by different evidence types.  Statistical, expert, and causal evidence were found to be equally persuasive as support for claims.  Indians also appeared sensitive to evidence quality that was manipulated according to Western norms for reasonable argumentation: normatively strong evidence was more persuasive than normatively weak evidence.  Findings are compared to results from studies conducted in Western cultures.

  • Hornikx, J. & Best, J. de (2011). Persuasive evidence in India: An investigation of the impact of evidence types and evidence quality. Argumentation and Advocacy, 47 (4), 246-257. [pdf]

De overtuigingskracht van normatief sterke en normatief zwakke anekdotische evidentie

Vanuit normatief oogpunt zou de kwaliteit van evidentie invloed moeten hebben op de overtuigingskracht ervan. In experimenten lijkt normatief sterke evidentie ook overtuigender dan normatief zwakke evidentie, maar wanneer de boodschap langer (Hoeken & Hustinx, 2007) of natuurlijker is (Hoeken & Van Wijk, 1997) blijken ontvangers niet gevoelig voor evidentiekwaliteit. In deze studie is onderzocht of de kwaliteit van anekdotische evidentie wel effect heeft op overtuigingskracht als ook statistische evidentie aanwezig is. Nijmegenaren (N = 125) beoordeelden een brief van de gemeente Nijmegen over afvalverwerking waarin normatief sterke of zwakke anekdotische evidentie stond, die wel of niet was aangevuld met statistische evidentie. Zonder statistische evidentie had de kwaliteit van anekdotische evidentie geen effect op de overtuigingskracht, maar mét statistische evidentie was sterke anekdotische evidentie overtuigender dan zwakke anekdotische evidentie. Deze studie laat daarmee zien dat de kwaliteit van evidentie ook in een natuurlijke, realistische setting de overtuigingskracht ervan kan beïnvloeden.

  • Hornikx, J., & Houët, T. (2009). De overtuigingskracht van normatief sterke en normatief zwakke anekdotische evidentie in het bijzijn van statistische evidentie. In W. Spooren, M. Onrust, & J. Sanders (Eds.), Studies in taalbeheersing, vol. 3 (pp. 125-133). Assen: Van Gorcum. [pdf]

Comparing the actual and expected persuasiveness of evidence types

Whereas there are many publications in which argumentation quality has been defined by argumentation theorists, considerably less research attention has been paid to lay people’s considerations regarding argument quality. Considerations about strong and weak argumentation are relevant because they can be compared with actual persuasive success. Argumentation theorists’ conceptions have to some extent been shown to be compatible with actual effectiveness, but for lay people such compatibility has yet to be determined. This study experimentally investigated lay people’s expectations about the persuasiveness of anecdotal, statistical, causal, and expert evidence, and compared these expectations with the actual persuasiveness of these evidence types. Dutch and French participants (N = 174) ranked four types of evidence in terms of their expected persuasiveness for eight different claims. Both cultural groups expected statistical evidence to be the most persuasive type of evidence to other people, followed by expert, causal, and, finally, anecdotal evidence. A comparison of these rankings with the results of Hornikx and Hoeken (2007, Study 1) on the actual persuasiveness of the same evidence types reveals that people’s expectations are generally accurate: How relatively persuasive they expect evidence types to be often corresponded with their actual persuasiveness.

  • Hornikx, J. (2008). Comparing the actual and expected persuasiveness of evidence types: How good are lay people at selecting persuasive evidence? Argumentation, 22 (4), 555-569. [pdf]

Cultural differences in the persuasiveness of evidence types and evidence quality

Cultural differences in reasoning and persuasion have mainly been documented for the East – West divide. Nisbett (2003) expects such differences to be absent for Western cultures because of their shared Grecian inheritance. The results of two experiments, however, show that France and the Netherlands, both Western-European countries, differ with respect to the persuasiveness of different evidence types. In Study 1 (N = 600), cultural differences occurred between the relative persuasiveness of anecdotal, statistical, causal and expert evidence. In Study 2 (N = 600), the quality of statistical and expert evidence was manipulated. For the Dutch, but not for the French, normatively strong evidence was more persuasive than normatively weak evidence for both evidence types. Implications and possible explanations are discussed.

  • Hornikx, J., & Hoeken, H. (2007). Cultural differences in the persuasiveness of evidence types and evidence quality. Communication Monographs, 74 (4), 443-463. [pdf]

Is anecdotal evidence more persuasive than statistical evidence?

Recent reviews of communication studies on the persuasiveness of evidence types have concluded that statistical evidence is more persuasive than anecdotal evidence. Cognitive psychological studies on the representativeness heuristic, however, have shown a large impact of anecdotal evidence (individuating information), and a small impact of statistical evidence (base rate information) on judgements. The difference between these conclusions can be explained by the research design of the psychological studies, which was in favour of anecdotal evidence. This article discusses more recent studies in cognitive psychology, and demonstrates that statistical evidence has more impact than the classic cognitive psychological studies suggested. This discussion brings back some consistency in results on the persuasiveness of anecdotal and statistical evidence, and also presents areas for future research.

  • Hornikx, J. (2007). Is anecdotal evidence more persuasive than statistical evidence? A comment on classic cognitive psychological studies. Studies in Communication Sciences, 7 (2), 151-164. [pdf]

Hoe goed zijn taalgebruikers in het selecteren van overtuigende evidentie?

Argumentatietheoretici hebben normatieve, kritische vragen geformuleerd om argumentkwaliteit te beoordelen. Deze vragen kunnen worden vertaald naar criteria voor hoge argumentkwaliteit. In recente onderzoeken is deze normatieve benadering van argumentkwaliteit (wat zou overtuigend moeten zijn?) vergeleken met descriptieve benaderingen (wat is overtuigend?). In dit artikel is deze vergelijking gemaakt voor gewone taalgebruikers. Argumentkwaliteit is daarbij onderzocht door te kijken naar soorten evidentie. De centrale vraag is dan: hoe goed zijn taalgebruikers in het selecteren van overtuigende evidentie? Deze vraag is beantwoord door de resultaten van Hornikx en Hoeken (2005) over de daadwerkelijke overtuigingskracht van evidentietypen af te zetten tegen een experiment waarin de verwachte overtuigingskracht van dezelfde evidentietypen voor dezelfde soort standpunten is gemeten. Proefpersonen maakten rangschikkingen van evidentietypen op basis van de verwachte overtuigingskracht om een ander van een aantal standpunten te overtuigen. Proefpersonen bleken behoorlijk goed in het selecteren van overtuigende evidentietypen: de evidentie waarvan ze verwachten dat deze (niet) overtuigend is, was meestal ook daadwerkelijk (niet) overtuigend.

  • Hornikx, J. (2007). Hoe goed zijn taalgebruikers in het selecteren van overtuigende evidentie? Tijdschrift voor Taalbeheersing, 29 (3), 224-236. [pdf]