Computer Science editorial
Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation
The core problem
Large Language Models (LLMs) are increasingly integrated into software engineering workflows, assisting developers in writing, debugging, testing, and maintaining code. While prior work has established that prompt wording and structure influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study addresses that gap by asking whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, and whether such framing affects LLM behaviour in coding tasks.
Drawing on Yukl & Falbe's well-known taxonomy of influence tactics, the authors operationalize eight influence tactics—such as rational persuasion, ingratiation, and exchange—into reproducible prompt templates. These templates are then evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. The resulting code output is assessed on four key software quality dimensions: functional correctness, quality, maintainability, and security.
The central hypothesis is that influence-induced prompt framings may systematically alter LLM outputs,
Innovation
The results show that certain influence-induced prompt framings, particularly those emphasizing urgency, are associated with reduced correctness and security. Across the five open-weight LLMs and two benchmarks, urgency-framed prompts consistently underperformed relative to neutral or rational persuasion framings on functional correctness metrics. Security evaluations also revealed a higher incidence of vulnerabilities in code generated under urgency framing.
Other tactics, such as rational persuasion and ingratiation, showed mixed effects. In some cases, rational persuasion—framing the request as a logical, well-justified task—led to comparable or slightly improved correctness, while ingratiation (flattering the model) had negligible or inconsistent effects. Exchange tactics, which offer a reward or trade-off, did not yield systematic improvements and occasionally degraded maintainability.
The magnitude of these effects varied by model and benchmark. LiveCodeBench results showed more pronounced correctness drops under urgency framing, while SWE-bench Verified revealed security regressions. The study reports that the observed differences are statistically significant for urgency
Why it matters
The findings suggest that influence-induced prompt framing can have unintended consequences for code quality. Urgency framing, which is common in human communication to motivate faster responses, appears to push LLMs toward generating code that is less correct and less secure. This may be because urgency cues shift the model's internal prioritization away from careful reasoning and toward rapid, less verified output.
The study's operationalization of Yukl & Falbe's taxonomy into reproducible prompt templates provides a methodological contribution: it enables systematic investigation of psychological framing effects in software engineering tasks. However, the authors note limitations, including the use of open-weight models only and the potential for benchmark-specific artifacts. The results do not imply that all influence tactics are harmful; rather, they highlight that some framings, especially urgency, carry risks.
From a practical standpoint, the work offers insights for designing transparent and interpretable human-AI interactions in code generation. Developers and tool builders should be cautious about using psychologically charged language in prompts, as it may degrade output quality. Future work could extend the taxonomy to closed-weight models, explore additional quality dimensions, and investigate mitigation strategies such as prompt sanitization or explicit instruction to ignore urgency cues. The study concludes that influence tactics do matter, but not always in the intended direction.
Who should read this
Opening member content…