The importance and use of normative criteria to manipulate argument quality

Argument quality plays an important theoretical and methodological role in persuasion research. Researchers frequently refrain from employing independent normative criteria to manipulate argument quality. Instead they use pretests to qualify arguments that evoke predominantly favorable thoughts as strong, and arguments that predominantly evoke unfavorable thoughts as weak. In this paper, we analyze weak arguments as they have been used in actual studies. These weak arguments ranged from arguments referring to less favorable consequences compared to their strong counterparts, to consequences that are irrelevant to the participants, or even to undesirable consequences thereby essentially functioning as counterarguments. We discuss the implications of this practice for our understanding of the persuasion process and our ability to provide evidence-based guidelines for message designers. We also provide guidelines on how to manipulate argument quality using normative criteria.

Hoeken, H., Hornikx, J., & Linders, Y. (2020). The importance and use of normative criteria to manipulate argument quality. Journal of Advertising, 49 (2), 195-201.

An experimental investigation of the role of distraction and dilution

PaglieriStudies on persuasive arguments have generally found that claims supported by high-quality evidence are better accepted than claims supported by low-quality evidence. However, an experiment by Hoeken and Hustinx (2007) demonstrated that this effect was only observed in short texts (a claim with evidence), but not in longer texts (where information unrelated to the evidence was added at the end of the text). The present experiment was conducted to examine whether this effect of text length could be explained by distraction (the additional text at the end distracts the reader) or by dilution (the additional text makes the fragment less diagnostic for claim evaluation). Participants (N = 629) read two texts with a claim supported by high-quality or low-quality (anecdotal, statistical, or expert) evidence. The text was presented in one of the three versions: (1) short, (2) long with additional information at the end, or (3) new in comparison to Hoeken and Hustinx (2007) – long with additional information at the start. The data found support for the distraction explanation. An effect of evidence quality on claim acceptance was observed in two conditions: in the short text, and in the longer text with additional information at the start. The effect of evidence quality was not found in the longer text with additional information at the end.

  • Hornikx, J. (2016). Evidence quality variations and claim acceptance: An experimental investigation of the role of distraction and dilution. Paglieri, F., Bonelli, L., & Felletti, S. (eds.), The psychology of argument: Cognitive approaches to argumentation and persuasion (pp. 211-222). London: College Publications. [paper book]

Een kritische analyse van de manipulatie van argumentkwaliteit in reclameonderzoek

demachtvantaalOver het algemeen raken mensen meer overtuigd door sterke dan door zwakke argumenten. In de empirische literatuur bestaan er echter inconsistente bevindingen voor het effect van argumentkwaliteit. In dit paper wordt onderzocht of de manipulatie van argumentkwaliteit een verklaring kan bieden voor deze tegenstrijdige bevindingen. Een analyse van dergelijke manipulaties in 32 uitgevoerde empirische onderzoeken leidt tot twee belangrijke inzichten. Ten eerste manipuleren onderzoekers voornamelijk pragmatische argumentatie, waarbij het standpunt over de wenselijkheid van de aanschaf van een product wordt ondersteund door te verwijzen naar de wenselijke gevolgen van het gebruik van dat product. Ten tweede blijkt de wijze van manipulatie sterk uiteen te lopen: bij de zwakke argumenten wordt soms naar minder wenselijke gevolgen verwezen, in andere gevallen naar neutrale gevolgen (waardoor er eigenlijk geen sprake is van een argument), en soms zelfs naar onwenselijke gevolgen (waardoor er geen sprake is van een zwak pro-argument maar van een tegenargument). Deze verschillende manieren waarop argumentkwaliteit in de 32 onderzochte studies gemanipuleerd werd, zouden een verklaring kunnen bieden voor de soms tegenstrijdige bevindingen in onderzoek naar argumentkwaliteit.

  • Weerman, A., Hoeken, H., & Hornikx, J. (2016). Een kritische analyse van de manipulatie van argumentkwaliteit in reclameonderzoek. In D. Van de Mieroop, L. Buysse, R. Coesemans, & P. Gillaerts (Red.),  De macht van de taal: Taalbeheersingsonderzoek in Nederland en Vlaanderen (pp. 309-321). Leuven: Acco. [link]

Germans are not sensitive to the quality of statistical evidence

JOCC coverFor a long time, research in communication and argumentation has investigated which kinds of evidence are most effective in changing people’s beliefs in descriptive claims. For each type of evidence, such as statistical or expert evidence, high-quality and low-quality variants exist, depending on the extent to which evidence respects norms for strong argumentation. Studies have shown that participants are sensitive to such quality variations in some, but not in all, cultures. This paper expands such work by comparing the persuasiveness of high- and low-quality statistical and expert evidence for participants from two geographically close cultures, the Dutch and the German. Study 1, in which participants (N = 150) judge a number of claims with evidence, underscores earlier findings that high-quality is more persuasive than low-quality evidence for the Dutch, and – surprisingly – also shows that this is less the case for the Germans, in particular for statistical evidence. Study 2 with German participants (N = 64) shows again they are not sensitive to the quality of statistical evidence, and rules out that this finding can be attributed to their understanding of the rules of generalization. Together, findings in this paper underline the need to empirically investigate what norms people from different cultures have for high-quality evidence, and to what extent these norms matter for persuasive success.

  • Hornikx, J., & Haar, M. ter (2013). Evidence quality and persuasiveness: Germans are not sensitive to the quality of statistical evidence. Journal of Cognition and Culture, 13 (5), 483-501. [linkpdf upon request]

Reasoning and argumentation

Although argumentation plays an essential role in our lives, there is no integrated area of research on the psychology of argumentation. Instead research on argumentation is conducted in a number of separate research communities that are spread across disciplines and have only limited interaction. Cognitive psychological research on argumentation has focused mostly on argument as a reason, and argument as structured sequence of reasons and claims. A third meaning of argument has been neglected: argument as a social exchange. All meanings are integral to a complete understanding of human reasoning and cognition. In this special issue, we present work that is relevant to all these three meanings of argument. The papers by Heit and Rotello (on the effect of argument length on inductive reasoning), by Harris, Hsu and Madsen (on a Bayesian test of the ad Hominem fallacy), and by Thompson and Evans (on belief bias in informal reasoning tasks) focus on arguments as reasons. By contrast, the contributions by Van Eemeren, Garssen, and Meuffels (on the reasonableness of the disguised abusive ad hominem fallacy), by Hoeken, Timmers, and Schellens (on argument quality and convincing arguments), by Mercier and Strickland (on how arguments can be evaluated from audience reactions), and by Bonnefon (on generating consequential arguments) deal intrinsically with situations where there are multiple protagonists in a communicative exchange. By including these papers, by researchers from a range of theoretical backgrounds, this special issue underlines the breadth of argumentation research as well as stresses opportunities for mutual awareness and integration.

  • Hahn, U., & Hornikx, J. (2012). Reasoning and argumentation [A special issue of Thinking and Reasoning]. London: Psychology Press. [link]

Reasoning and argumentation: Towards an integrated psychology of argumentation

Although argumentation plays an essential role in our lives, there is no integrated area of research on the psychology of argumentation. Instead research on argumentation is conducted in a number of separate research communities that are spread across disciplines and have only limited interaction. With a view to bridging these different strands, we first distinguish between three meanings of the word “argument”: argument as a reason, argument as a structured sequence of reasons and claims, and argument as a social exchange. All three meanings are integral to a complete understanding of human reasoning and cognition. Cognitive psychological research on argumentation has focused mostly on the first and second of these meanings, so we present perspectives on argumentation from outside of cognitive psychology, which focus on the second and third. Specifically, we give an overview of the methods, goals, and disciplinary backgrounds of research on the production, the analysis, and the evaluation of arguments. Finally, in introducing the experimental studies included in this special issue, which were conducted by researchers from a range of theoretical backgrounds, we underline the breadth of argumentation research as well as stress opportunities for mutual awareness and integration.

  • Hornikx, J., & Hahn, U. (2012). Reasoning and argumentation: Towards an integrated psychology of argumentation. Thinking and Reasoning, 18 (3), 225-243. [pdf upon request; publisher]

De overtuigingskracht van normatief sterke en normatief zwakke anekdotische evidentie

Vanuit normatief oogpunt zou de kwaliteit van evidentie invloed moeten hebben op de overtuigingskracht ervan. In experimenten lijkt normatief sterke evidentie ook overtuigender dan normatief zwakke evidentie, maar wanneer de boodschap langer (Hoeken & Hustinx, 2007) of natuurlijker is (Hoeken & Van Wijk, 1997) blijken ontvangers niet gevoelig voor evidentiekwaliteit. In deze studie is onderzocht of de kwaliteit van anekdotische evidentie wel effect heeft op overtuigingskracht als ook statistische evidentie aanwezig is. Nijmegenaren (N = 125) beoordeelden een brief van de gemeente Nijmegen over afvalverwerking waarin normatief sterke of zwakke anekdotische evidentie stond, die wel of niet was aangevuld met statistische evidentie. Zonder statistische evidentie had de kwaliteit van anekdotische evidentie geen effect op de overtuigingskracht, maar mét statistische evidentie was sterke anekdotische evidentie overtuigender dan zwakke anekdotische evidentie. Deze studie laat daarmee zien dat de kwaliteit van evidentie ook in een natuurlijke, realistische setting de overtuigingskracht ervan kan beïnvloeden.

  • Hornikx, J., & Houët, T. (2009). De overtuigingskracht van normatief sterke en normatief zwakke anekdotische evidentie in het bijzijn van statistische evidentie. In W. Spooren, M. Onrust, & J. Sanders (Eds.), Studies in taalbeheersing, vol. 3 (pp. 125-133). Assen: Van Gorcum. [pdf]

Cultural differences in the persuasiveness of evidence types and evidence quality

Cultural differences in reasoning and persuasion have mainly been documented for the East – West divide. Nisbett (2003) expects such differences to be absent for Western cultures because of their shared Grecian inheritance. The results of two experiments, however, show that France and the Netherlands, both Western-European countries, differ with respect to the persuasiveness of different evidence types. In Study 1 (N = 600), cultural differences occurred between the relative persuasiveness of anecdotal, statistical, causal and expert evidence. In Study 2 (N = 600), the quality of statistical and expert evidence was manipulated. For the Dutch, but not for the French, normatively strong evidence was more persuasive than normatively weak evidence for both evidence types. Implications and possible explanations are discussed.

  • Hornikx, J., & Hoeken, H. (2007). Cultural differences in the persuasiveness of evidence types and evidence quality. Communication Monographs, 74 (4), 443-463. [pdf]

Is anecdotal evidence more persuasive than statistical evidence?

Recent reviews of communication studies on the persuasiveness of evidence types have concluded that statistical evidence is more persuasive than anecdotal evidence. Cognitive psychological studies on the representativeness heuristic, however, have shown a large impact of anecdotal evidence (individuating information), and a small impact of statistical evidence (base rate information) on judgements. The difference between these conclusions can be explained by the research design of the psychological studies, which was in favour of anecdotal evidence. This article discusses more recent studies in cognitive psychology, and demonstrates that statistical evidence has more impact than the classic cognitive psychological studies suggested. This discussion brings back some consistency in results on the persuasiveness of anecdotal and statistical evidence, and also presents areas for future research.

  • Hornikx, J. (2007). Is anecdotal evidence more persuasive than statistical evidence? A comment on classic cognitive psychological studies. Studies in Communication Sciences, 7 (2), 151-164.

Normatively strong and normatively weak expert evidence

I will give an overview of studies that investigated the persuasiveness of expert evidence as well as other types of evidence. One of these studies demonstrated that the persuasiveness of expert evidence was not the same in two different cultures. Section 3 will therefore discuss the relationship between expert evidence and the cultural background of people who judge expert evidence. Special attention will be paid to the question whether people from different cultures may vary in the persuasiveness of expert evidence that is normatively strong or normatively weak according to criteria from argumentation theory. The second part of this article will report on an experiment that investigated the persuasiveness of normatively strong or normatively weak expert evidence in France and the Netherlands.

  • Hornikx, J. (2007). Cultural differences in the persuasiveness of normatively strong and normatively weak expert evidence. In F. H. van Eemeren, J. A. Blair, C. A. Willard, & B. Garssen (Eds.), Proceedings of the sixth conference of the International Society for the Study of Argumentation (pp. 645-650). Amsterdam: Sic Sat. [pdf]