Why it’s still open
Preferences in large models fit coherent utility functions, yet personas can hold opposite values, and narrow training can flip a character’s values wholesale. Whether there is a persona-independent layer of belief and value underneath is unsettled.
What would count as an answer
Measurements of the same beliefs and values across many personas and contexts, separating what stays fixed from what the persona controls, with mechanistic backing for the shared part.
Papers that tackle it
- Gilg et al 2026 - Probing persona-dependent preferences — Finds a preference direction in two open models that predicts and steers task choices.
- Lu et al 2026 - The Assistant Axis — Maps a space of personas and traits in model activations and finds a leading direction tracking closeness to the default Assistant.
- Marks Lindsey Olah 2026 - The persona selection model — Proposes that pretraining teaches a model to simulate many characters and post-training selects and refines one of them, the Assistant.
- Betley et al 2025 - Emergent misalignment — Fine-tuning on one narrow task (writing insecure code) produces broadly misaligned behavior in unrelated contexts, suggesting that training shifts a whole character rather than a single skill.
- Mazeika et al 2025 - Utility engineering — Shows that independently sampled LLM preferences fit coherent utility functions increasingly well with scale, surfaces troubling values, and proposes methods to control them.
- Beyond mimicry 2025 - Preference coherence in LLMs — Extends pain/pleasure trade-off tests across many models and finds that coherent preference structure is the exception rather than the rule.
- Trhlik et al 2026 - LLMs contain multitudes — Values shift with deployment framing.
- Ajavi Chowdhury Lazar 2026 - Incoherent values — Utility fits hide step-by-step incoherence.
Methods in play
Who’s closest
Placed by the editor, v1 — corrections welcome.
Related questions
- Whose valence is it
- Writing about a character or becoming it
- Do models represent goals and preferences
Know a paper that belongs here, or think this is framed wrongly? Suggest a paper or a correction (forms on the Atlas home).