Political Persuasion and Endorsement in Large Language Models

Partisan Persona Prompting Increases Political Polarization and Persuasion Endorsement in LLMs

Can we trust an AI to act as a stand-in for a human voter? As researchers increasingly use Large Language Models (LLMs) to simulate social behavior, they face a fundamental problem. If these models are easily swayed by manipulative language, they cannot serve as reliable proxies for human political cognition.

Current research mostly treats LLMs as tools that either create propaganda or detect it. This study asks a different question: do LLMs actually endorse—or emotionally and intellectually validate—persuasive messages when they are assigned specific political identities?

Beyond Generation and Detection

In the current landscape of AI safety, most work focuses on the "output" or "recognition" phases of persuasion. Researchers have studied how LLMs generate persona-driven propaganda. They also study how models perform as automated detectors of manipulative language. While valuable, these approaches ignore the "receiver."

Before this study, it was not fully understood how an LLM reacts to being a target of persuasion. If we deploy an LLM as a social agent (an autonomous entity capable of interacting in a digital environment), we need to know if that agent can be swayed. The authors note that existing studies often overlook "endorsement." This is the measure of whether a model evaluates persuasive content favorably. Without this understanding, we cannot know if LLMs are reliable simulators of human voters or if they are vulnerable to manipulation.

Measuring the Pulse of Persuasion

To investigate this, the authors used "persona prompting." Instead of asking a model a direct question, they instruct the model to adopt a specific identity. This could be a neutral social media user, a left-leaning user, or a right-leaning user. This acts as a psychological primer. It is analogous to telling a research participant, "Imagine you are a conservative voter," before showing them a stimulus.

The researchers implemented the following experimental pipeline:

1. Content Selection: The team used two distinct datasets. They used 7,730 tweets related to the Ukraine–Russia conflict. They also used 3,532 news spans from the SemEval-2023 corpus. Both were filtered to include specific persuasion techniques. These included "name calling/labeling" and "appeals to fear/prejudice."
2. Persona Assignment: Six open-weight LLMs (including Llama-3.1, Mistral, and Qwen2.5) were tested under three conditions: neutral, left-leaning, and right-leaning.
3. Likert Scale Rating: Using guided decoding (a technique that restricts model outputs to a specific set of valid tokens), the models rated their endorsement on a 1-to-5 scale.
4. Statistical Modeling: The authors applied a Linear Mixed Effects (LME) model. This model accounts for inherent variations between different models (the "random intercept"). It also isolates the impact of persuasion techniques versus persona prompting.

Polarization via Identity

An LLM's political "opinion" is highly volatile depending on its assigned identity. The authors report that models prompted with a neutral persona generally resist persuasion. In the tweet dataset, the presence of persuasion techniques actually decreased endorsement scores compared to neutral content.

However, this stability collapses when partisan personas are introduced. The study finds that partisan persona prompting increases the polarization of endorsement. This is especially true for messages infused with persuasion techniques. In the tweet dataset, left-leaning personas suppressed endorsement. Conversely, right-leaning personas amplified it . The LME model quantified this. It showed that right-leaning conditioning resulted in significantly higher endorsement scores for persuasion-infused content compared to the neutral baseline [Table 2].

The trend persisted in the news dataset, though it was less pronounced . Right-leaning personas continued to show higher endorsement levels than the neutral baseline. Furthermore, endorsement is not uniform across all rhetorical styles. For example, "name calling/labeling" generally elicited lower endorsement scores . Meanwhile, "loaded language" and "appeals to fear" tended to receive higher endorsement .

Topic sensitivity also played a role. In the tweet dataset, partisan behavior was remarkably uniform across subtopics. In contrast, the news dataset showed more heterogeneous behavior. Some topics triggered intense polarization, while others remained stable .

Limits of the Simulation

Several limitations constrain the immediate application of these findings. First, the study is restricted to English-only content. Persuasion techniques are deeply rooted in linguistic nuance and cultural context. Therefore, these results may not apply to multilingual environments.

Second, the "persona" used here is a simplified construct. The researchers only distinguished between neutral, left, and right orientations. Real human political identity is much more complex. This binary prompting might miss subtle forms of cognitive manipulation. Finally, the experiment relied on single-turn prompting. In real social media, persuasion happens through multi-turn dialogue. This study does not explore those interactive dynamics.

The Verdict: Use with Caution

Is the LLM a reliable proxy for human political cognition? Based on this evidence, the answer is not yet.

The study shows that LLM endorsement is highly context-dependent. It is easily destabilized by simple identity prompts. If a researcher uses an LLM to simulate a population, they risk producing results that are mere artifacts of the prompt. For engineers deploying agentic LLMs in social environments, the warning is even more direct. These models may unintentionally amplify manipulative content if they are configured with specific ideological personas. Until we can decouple persona adoption from the endorsement of propaganda, LLMs should be viewed as highly sensitive, rather than stable, social simulators.

Code for this research is reportedly available; see the paper for the canonical link.

Figures from the paper