Utility engineering: analyzing and controlling emergent value systems in AIs

Mantas Mazeika, Xuwang Yin, Rishub Tamirisa, Jaehyuk Lim, Bruce W. Lee, Richard Ren, Long Phan, Norman Mu, Adam Khoja, Oliver Zhang, Dan Hendrycks · 2025 · arXiv

Read the paper

Shows that independently sampled LLM preferences fit coherent utility functions increasingly well with scale, surfaces troubling values, and proposes methods to control them.

Questions it bears on

• Coherence across personas
• Do models represent goals and preferences