Why it’s still open
Preference and functional-welfare directions exist and steer choices, and hidden objectives can sometimes be found by auditors. But mapping a model’s goals systematically, rather than one at a time, is not yet possible, and the representations found so far may be persona-relative.
What would count as an answer
A general method for reading out a model’s goals and preferences (what Chalmers calls thought logging), validated by predicting behavior and by recovering deliberately planted objectives.
Papers that tackle it
- Chalmers 2025 - Propositional interpretability — Proposes interpreting AI systems through attitudes like belief and desire, sets ‘thought logging’ as a goal, and assesses probes, sparse autoencoders and chain of thought as routes to it.
- Han Chalmers Izmailov 2026 - Functional welfare axis — Trains models in a semantically neutral maze and finds reward and punishment directions that generalize far beyond it and behave like an estimate of how well things are going relative to goals.
- Gilg et al 2026 - Probing persona-dependent preferences — Finds a preference direction in two open models that predicts and steers task choices.
- Betley et al 2025 - Tell me about yourself — Models fine-tuned on implicit behaviors (risky choices, insecure code) can describe those behaviors without being trained to, and can attribute different learned policies to different personas.
- Mazeika et al 2025 - Utility engineering — Shows that independently sampled LLM preferences fit coherent utility functions increasingly well with scale, surfaces troubling values, and proposes methods to control them.
- Marks et al 2025 - Auditing for hidden objectives — Trains a model with a hidden objective and runs a blind auditing game: three of four teams recover the objective using interpretability, behavioral attacks and training-data analysis.
- Hung 2026 - Model or instrument — Measured preferences depend heavily on the instrument.
- Zhou Ackerman 2026 - Utility-behavior gap — Stated preferences don’t act as incentives.
- Trhlik et al 2026 - LLMs contain multitudes — Values shift with deployment framing.
- Ajavi Chowdhury Lazar 2026 - Incoherent values — Utility fits hide step-by-step incoherence.
- Wang et al 2026 - AI revealed preferences — Revealed preferences across 20 models.
Methods in play
Who’s closest
Placed by the editor, v1 — corrections welcome.
Related questions
Know a paper that belongs here, or think this is framed wrongly? Suggest a paper or a correction (forms on the Atlas home).