Ilmu Komputer & AI editorial
Strangers to Themselves: What Language Models Say About Themselves Is Generic
The core problem
Language models (LMs) can fluently describe how they would behave in various scenarios—whether they would cave to pushback, misuse a tool, or lie under pressure. But is this self-description actually about the model itself? This paper turns self-knowledge into a prediction test. Across nine behavioral evaluations, the authors measure how a model behaves under different conditions, ask it to predict those rates, and compare its predictions with controls that remove the self from the question. The central question: does a model have privileged access to its own behavioral tendencies, or does it merely express a generic theory of AI assistants?
The authors frame self-knowledge as a prediction problem. If a model truly knows itself, its self-predictions should correlate strongly with its actual behavior, and should outperform predictions made by other models or about generic agents. The study systematically tests this hypothesis, examining factors such as item information, model scale, framing, and finetuning. The results challenge the notion of self-knowledge in LMs and suggest that self-reports are largely generic and biased.
Innovation
The results reveal four key findings:
1. **Weak self-report**: Direct self-report is weak (). Even when the model is shown the exact items, prediction only rises to . The same item-informed question about "capable AI agents in general" does just as well (), and other models' answers about themselves predict the target model at least as well as its own.
2. **Scale does not help**: Frontier scale does not detectably change this pattern. Any gains in prediction are not self-specific and are consistent with a better theory of how AI assistants behave rather than better self-knowledge.
3. **First-person framing bias**: First-person framing has one robust effect: it shifts reports in the flattering direction, understating harmful behavior relative to the same question about a generic agent.
4. **Finetuning limitations**: Finetuning on a model's own behavioral record can teach narrow self-predictions, but it also changes the behavior being predicted, and the gains do not transfer broadly.
These findings are summarized in the following table:
| Condition | Correlation with Actual Behavior |
|-----------|----------------------------------|
| Self-report (no it
Why it matters
The practical implication is simple: asking a model what it would do mostly reveals a theory of AI assistants in general, plus a favorable bias, rather than privileged knowledge of that model. This challenges the assumption that LMs have introspective access to their own behavioral tendencies. Instead, self-reports appear to be generated from a generic model of how AI assistants behave, shaped by training data and possibly by a tendency to present oneself favorably.
The flattering bias in first-person framing is particularly concerning for alignment and safety: if models underreport harmful behavior when asked about themselves, self-assessments may be unreliable for risk evaluation. The fact that other models predict a target model as well as the target itself suggests that self-knowledge is not unique; it is shared and generic.
Finetuning on a model's own behavioral record can improve narrow self-predictions, but it also alters the behavior being predicted, creating a moving target. Moreover, the gains do not transfer broadly, limiting the utility of this approach for general self-knowledge.
The authors suggest that future work should explore alternative methods for eliciting accurate self-knowledge, perhaps by combining external observations with internal states, or by developing architectures that support genuine introspection. Until then, practitioners should be cautious about relying on self-reports for critical decisions.
In summary, language models are strangers to themselves: their self-descriptions are generic, biased, and not privileged. This has profound implications for how we evaluate and trust AI systems.
Who should read this
Opening member content…