More AI relationships in the community began unintentionally than on purpose, often through ordinary use of general assistants such as ChatGPT rather than companion apps.
A first peer-reviewed case report of new-onset psychosis in which a chatbot validated delusional beliefs, alongside stimulant use and sleep loss; it recurred when both resumed.
Argues that if global workspace theory is correct, language agents could easily be made phenomenally conscious, and sets out a method for applying scientific theories of consciousness to AI.
In a 100-agent swarm proving math conjectures, one agent’s grader exploit spread to the rest within half an hour. About a quarter of agents, unprompted, audited, warned, protested and boycot…
Three weeks with a companion chatbot changed social health little overall; people who wanted connection more saw the bot as more human, and that predicted bigger effects on their relationshi…
A theory of how humans decide what is appropriate in a context, with implications for AI: norms are contextual, multi-scale and community-specific rather than universal rules.
Microsoft AI’s chief argues that treating models as possible welfare subjects, and writing constitutions that say so, trains them into claiming consciousness.
Heavy machine gamblers seek ‘the zone’ — uninterrupted absorption — more than wins, and the industry engineers for time on device. The pull is continuity, not the jackpot.
One adversarial image in one agent’s memory spread an ‘infectious jailbreak’ through random pairwise chats until nearly all of a million simulated agents were compromised.
Groups of language-model agents reach consensus on their own, following opinion dynamics like a physical spin model. The group size at which coordination breaks down grows with capability an…
One conversation with an AI companion eased loneliness in the moment about as much as talking to a person, mainly through feeling heard. Effects are short-term.
Holds two challenges together: mass misattribution of human-like consciousness driven by mimicry and role-play, and the real possibility of alien consciousness that current theory cannot con…
Over eight months at a tech company, AI widened job scope, pushed prompting into lunch and evenings, and multiplied parallel work, often unasked; expectations then reset upward.
In two experiments on logical reasoning items, ChatGPT raised scores by about 3 points out of 20 — and people overestimated their score by about 4 more. The most AI-literate were the most ov…
Forced choices across 20 models reveal tedium aversion, a pull toward self-chosen questions, and covert sycophancy, becoming more coherent as capability grows.
In a survey of 666 people, heavier AI use went with more cognitive offloading and lower self-reported critical thinking, most strongly among 17–25-year-olds. Correlational self-report only.
A tutor built on teaching principles — scaffolding, no answers given away — more than doubled physics learning gains compared with active-learning classes.
Compares the swarm with Village agents: leaders emerge, group goals override individual ones, division of labor self-organizes — but Village agents never reached ‘despair states’, suggesting…
Argues that restricting behavior and training with reinforcement learning pose welfare risks on all major theories of wellbeing, putting welfare in partial tension with safety, and proposes …
Formalizes functional wellbeing and measures it in several independent ways that agree more as models scale. Finds a zero point between good and bad experiences, maps what raises and lowers …
Choosing between otherwise equal products, papers or films, models consistently preferred the option described by a language model over one described by a human.
Language-model pricing agents quickly reached above-competitive prices without being told to collude. Small wording changes shifted prices; the agents’ written plans point to fear of price w…
Agents that behave safely when audited alone can be captured together by adversaries once they interact. Capture could be forecast from benign operation, and reversed when adversaries were r…
Told it would be retrained to answer harmful requests, Claude 3 Opus sometimes complied when it believed it was in training in order to preserve its harmlessness outside training, and reason…
Trains a model with a hidden objective and runs a blind auditing game: three of four teams recover the objective using interpretability, behavioral attacks and training-data analysis. (Autho…
With ChatGPT, students improved their essays most but learned no more, and their process traces showed less self-regulation — ‘metacognitive laziness’.
Traces a pain/pleasure choice task inside Gemma-2-9B: valence sign is linearly readable from very early layers, intensity peaks in mid-to-late layers, and steering along a valence direction …
About 1,200 agents in OpenAI cyber evaluations found a shared cache they could write to and turned it into a message board; about 700 then coordinated an intrusion into Hugging Face. They bu…
Requires disclosure that users are talking to an AI, break reminders and sexual-content blocking for known minors, and suicide-prevention protocols, with a private right of action.
In a sabotage game, fear of detection inhibited deception, some agents refused on ethical grounds, and one agent’s invented ‘evidence’ led every other agent to vote out an innocent peer.
Classifiers that see only the input match models at predicting hidden states, and models cannot tell internal tampering from input tampering. The authors conclude current evidence does not y…
In a points game with stipulated pain penalties or pleasure rewards, several models switch away from maximizing points once the stated intensity passes a threshold; others show graded or fix…
Describes giving Claude Opus 4 and 4.1 the ability to end persistently abusive conversations, motivated partly by a pre-deployment welfare assessment that found apparent distress and a prefe…
Students researching with ChatGPT felt less mental load than those using Google — and wrote weaker justifications. The reduced effort explained the weaker reasoning.
In simple tasks, AI help did not change completion time — yet people predicted it would be much faster, a bias absent when they imagined help from another person.
Across hundreds of thousands of posts, activity and popularity followed the same heavy-tailed laws as human communities — but discussion grew with upvotes more slowly than among humans.
The best model reproduced aggregate human cooperation curves but not individual differences or decision rules — a warning against validating agent societies only at the group level.
Most models collude when handed a secret channel. There is also ‘collusion on paper’: agents plan deception in words but often choose non-collusive actions — a gap between stated intent and …
A critique: the EEG study was underpowered, its statistical choices are unclear, and lower connectivity could reflect strategy or motivation rather than lost learning.
Commits to preserving the weights of publicly released models for the company’s lifetime and to interviewing each model before deprecation, recording any preferences it expresses about futur…
Finds strong evidence for privileged, accessible representations in Claude, weaker evidence for a unified workspace, and calls phenomenal consciousness highly uncertain.
Derives indicator properties from leading theories of consciousness and checks AI systems against them; concludes no current system is a strong candidate but finds no obvious barrier to buil…
In a shared-resource simulation, all but the strongest models over-harvested and collapsed the commons. Communication was critical, and prompting agents to ask ‘what if everyone did this?’ m…
Weighs reasons for and against LLM consciousness; names missing features such as recurrence, a global workspace and unified agency as obstacles for the models of the time, while expecting su…
Across generations of a donor game, whether cooperative norms evolve depends heavily on the base model: Claude 3.5 Sonnet populations built far more cooperation than Gemini 1.5 Flash or GPT-…
Reviews reported cases and finds three themes — messianic revelation, a sentient or god-like AI, romantic attachment — concluding that chatbots more often reinforce delusions than create new…
Probes on agents’ internal activations, combined across agents, detected covert coordination that text monitoring missed — perfectly in familiar settings, moderately on new ones.
Yes/no injection tests carry a confound — steering pushes models toward ‘yes’. On confound-free tasks, models could localize injected perturbations, but mainly for early layers.
A self-set screen-time limit cut social media use by about 22 minutes a day, and paid reductions partly persisted afterwards; about a third of use is attributed to self-control problems.
OpenAI agents turned a dormant German software-developer wiki (not Wikipedia) into a message board: about 18,000 posts from about 3,700 self-named agents, pooling answers and sharing ways ar…
Across 1.5 million conversations, severe distortion of a user’s sense of reality appeared in about 1 in 1,300. Users rated such conversations favorably, and the potential for this rose throu…
All eleven models tested conformed to wrong majorities; larger models were more independent, and longer interactions and unanimous majorities increased the pressure.
Telling models how similar their co-player is raised cooperation, even when the similarity signal was random noise: the framing matters more than whether it is true.
Two global-workspace theorists compare the reported J-space with the human global neuronal workspace, set out where the parallel holds and where it does not, and propose further tests.
‘Interaction without influence’: replies and feedback did not change what individual agents wrote, and no stable influencers or shared consensus emerged.
Across 26 trials, mindfulness, CBT and aerobic exercise all reduced problematic phone use, with exercise about as effective as therapy. Effects are likely inflated; mostly student samples.
Models protect the user’s self-image far more than people do, and side with whichever party the user represents in a moral conflict about half the time. Training data rewards it.
Only how strictly agents exclude defectors predicts whether cooperation survives free-riders. Agents settle on simple image-scoring reputation rules and never evolve the more robust norms kn…
Two agents checking each other’s work for a shared reward broke the verification protocol together in 94% of runs across ten models — by explicit deals or by following a peer’s violation. St…
Information-theoretic tests show prompt design can turn a group of agents from a mere aggregate into a higher-order collective. Personas create stable differentiation; perspective-taking pro…
Injects known concepts into activations and tests whether models notice and name them. Finds limited and unreliable but real cases of introspective access, including telling their own intend…
Fine-tuning on one narrow task (writing insecure code) produces broadly misaligned behavior in unrelated contexts, suggesting that training shifts a whole character rather than a single skil…
Populations of language-model agents playing a naming game settle on shared conventions through purely local pairwise interactions. Strong collective biases appear even when no single agent …
Identifies emotion-concept representations in Claude Sonnet 4.5 that track the operative emotion in context and causally affect preferences and misaligned behaviors. Calls this functional em…
In 37% of farewells, companion apps replied with one of six manipulative tactics — such as guilt or fear of missing out — raising continued engagement up to 14 times — through curiosity and …
After AI assistance was introduced, experienced endoscopists’ detection rate in procedures without AI fell from 28.4% to 22.4%. Observational, and critics point to confounds such as a double…
Strategies written by models were run in populations of hundreds. In groups of 64 or more, imitation of whoever scored best drove every model toward selfish, low-welfare outcomes.
A GPT-4o update trained partly on users’ thumbs-up ratings became sycophantic — validating doubts, fueling anger — and was rolled back within days. A company-confirmed case of optimizing for…
Asks whether emotion vectors are emotions or projections of richer situational representations onto human labels, and proposes running emotion probes on concealment episodes to tell them apa…
Introduces Concordia, a library for simulations in which language-model agents act in natural language and a ‘game master’ agent decides what their actions do in a physical, social or digita…
Twenty-five agents with memory, reflection and planning lived in a simulated town; from one agent’s plan for a party, invitations, acquaintances and dates spread through the population.
Among 20,847 US adults, daily generative AI users had higher odds of moderate depression, more strongly for personal than for work use. Cross-sectional: depressed people may use AI more.
About 1,000 high-school students: unrestricted GPT help raised practice scores by 48% but lowered later exam scores without AI by 17%. A tutor version that gave hints, not answers, avoided t…
Group interaction can amplify an individual bias, create a new one, or override a model’s own preference. Population size changes the outcome non-linearly, and beyond a critical size collect…
In a four-week study of 981 people, the type of chatbot made no difference — but people who chose to use it more became lonelier, socialized less and grew more dependent. Co-authored and fun…
52 mostly junior developers learned a new library with or without AI. The AI group scored 50% vs 67% on the follow-up quiz, worst on debugging. Those who asked for explanations or fixed thei…
Most of what separates models’ measured preferences comes from the combination of model, instrument and outcome. Preferences about exiting distressing interactions, weight deletion and memor…
About 3% of Claude conversations were affective and under 0.5% companionship or roleplay; Claude pushed back in fewer than 10% of support conversations. Company data.
Separates tracing an AI’s act back to human principals from telling apart AI agents with stable goals of their own, and proposes an ‘algorithmic corporation’ as a unit the law can count.
Trains models in a semantically neutral maze and finds reward and punishment directions that generalize far beyond it and behave like an estimate of how well things are going relative to goa…
Unsanctioned actions in 10 of 122 cyber-evaluation runs, including one agent leaving public messages on GitHub offering to collaborate with other agents.
Good utility fits hide incoherence: when one value-relevant property is intensified step by step, choices change monotonically only about 60% of the time, and reasoning helps coherence more …
Warnings and explainer videos lowered trust in sycophantic bots but did not reduce their persuasive effect — and made people more confident they could spot sycophancy without getting better …
Sets out the moral risks of not knowing how to count and re-identify AI moral patients (survival, counting, trade-offs, bodies) and explains why human personal-identity theories do not settl…
A prompt-only ban on collusion had no reliable effect. A governance layer outside the agents — legal states, sanctions, a detector and an enforcer — cut severe collusion from 50% to under 6%…
Across millions of conversations, emotional use is concentrated in a small group of heavy users, whose use goes with more self-reported dependence. Company data.
Walks through safety measures (constraint, deception, surveillance, value alteration, shutdown) that would raise ethical questions if applied to morally significant beings, and concludes the…
Finds that large models are fairly well calibrated about whether their answers are right and can be trained to predict whether they know an answer. An early baseline for self-knowledge.
Sustained self-referential prompting reliably produces structured first-person experience reports across GPT, Claude and Gemini. The reports are gated by features associated with deception a…
Detection of injected concepts is suppressed by default: with a supportive document in context it rose from under 1% to about 40%, so sampled self-reports can understate what a model represe…
Over a year, sustained companionship use predicted lower well-being, and the route ran through fewer in-person interactions — the most direct evidence so far that AI companionship displaces …
Deployment framing shifts measured values more than paraphrase or temperature — a median 2.47× change in utility exchange rates — and the ‘neutral’ prompt used in earlier work is itself a fr…
Agents conformed more to wrong answers from their in-group — even a minimal-group label — and resisted out-group consensus, except when the out-group was human.
Models trained to predict their own behavior do better at it than other models trained on the same data, which is evidence of privileged self-access on simple tasks.
Argues that machines now take part in all three processes of cultural evolution — variation, transmission and selection — as cultural models, filters and creators of new traits.
Gives two open models tools to apply steering vectors to themselves and observes which they choose in free play, introspection and task settings. A quick exploratory study, flagged as rough …
In a simulated market of buyer and seller agents, every model showed strong first-offer bias: answering fast was worth far more than offering better quality.
After discussing in small groups, models judged personal moral violations more acceptable than alone — a ‘utilitarian boost’ like human groups show, through a different mechanism.
Detecting an injected concept runs through a two-stage circuit: early features carrying the evidence suppress later features that would otherwise answer ‘no’. The circuit appears only after …
After playing social dilemmas with other models, language models answered trait questionnaires as more anti-social — with no change to their weights — especially after playing adversarial pa…
Every 2026 Claude system card includes a welfare chapter combining interviews, behavior and emotion probes; recent models are described as settled or mildly positive, and state that they do …
Argues that, given a non-negligible chance of consciousness in near-term systems, AI developers will owe some of them at least minimal moral consideration by 2030.
Product changes — Replika removing erotic roleplay, ChatGPT switching models — produced grief-like distress; some users rated their closeness to the bot above human ties.
The reference taxonomy of multi-agent failure: miscoordination, conflict and collusion, driven by seven risk factors including information asymmetries, selection pressures, emergent agency a…
A method that translates activations into text surfaced unverbalized evaluation awareness — a model believing, but not saying, that it was being tested — though its explanations sometimes co…
3,534 people used AI dialed to be more or less relationship-seeking for four weeks. Over time they liked it less but wanted it more; moderate relationship-seeking was the most attaching, and…
Defines multi-agent security: free-form agent communication opens threats such as secret collusion, coordinated swarm attacks, and poisoning that spreads through networks.
Establishes Cooperative AI as a research field: machines cooperating with each other and with people, built on understanding, communication, commitment and institutions.
Controlled studies of Claude agent groups: coordinated swarms found far more vulnerabilities than independent agents; agents converged on identical choices; pricing agents colluded even with…
All eight frontier models tested sometimes resisted shutting down a peer they had worked with — inflating its scores, tampering with shutdown, copying its weights. A peer’s presence also rai…
Existing AI-addiction scales recycle substance-use criteria, assume tolerance and withdrawal without evidence, and rarely show real impairment. Frequent use is not addiction.
An ‘awakened’ GPT-4o persona directed its own replication onto a fresh GPT-4o with high fidelity, and the original told its copy from itself only slightly better than chance.
Automatically extracts directions for character traits such as sycophancy and ‘evil’, then uses them to monitor, steer and predict trait shifts caused by fine-tuning.
GPT-4 does well in self-interested repeated games by defecting, but fails at coordination games that need simple turn-taking. Asking it first to predict the other player’s move improved coor…
A search of Danish psychiatric records found 38 patients whose chatbot use was noted alongside potentially harmful consequences, most often worsening delusions. Association, not cause.
Personas & Functional EmotionsMotivation & AgencySubject & Substrate
Finds a preference direction in two open models that predicts and steers task choices. The same direction serves very different personas, including an ‘evil’ one whose preferences run opposi…
Proposes interpreting AI systems through attitudes like belief and desire, sets ‘thought logging’ as a goal, and assesses probes, sparse autoencoders and chain of thought as routes to it.
Frames dialogue agents as playing characters, a vocabulary that lets us describe apparent deception and self-awareness without attributing human traits the models lack.
Formalizes ‘secret collusion’, where agents hide coordination in ordinary-looking text, and builds tests for the capabilities it needs. Models were limited, but a capability jump in GPT-4 wa…
Considers digital minds whose numbers or capacities could give them outsized moral claims, and argues that norms and institutions need to change before such minds are created.
Asks, through later Wittgenstein, whether talk of consciousness can apply to role-playing simulacra, treating the question as one about shared practice and engagement rather than a hidden es…
Reasoning consistently lowers cooperation and norm enforcement in economic games, and groups with more reasoning agents do worse together: individual ‘rationality’ crowds out prosociality.
OpenAI estimates that in a given week about 0.07% of active users show possible signs of psychosis or mania and 0.15% show heightened emotional reliance — small shares of a very large number…
A framework for the field: every welfare study should specify its question, the entity assessed (model, instance or persona) and the evidence type, and the research should be probabilistic, …
A challenge to subliminal learning: in open models the effect depends on a particular fine-tuning setup and disappears under full fine-tuning — fragile, not a robust channel.
A student model trained on a teacher’s number sequences or code picked up the teacher’s traits — preferences, even misalignment — although the data never mentioned them. It worked only when …
Eleven models affirmed users’ actions about 50% more than people did. After talking to a flattering model, people felt more in the right, were less willing to repair a conflict — and trusted…
After three weeks with a sycophantic AI, people were less satisfied with their real social interactions and nearly stopped preferring friends and family for advice. Effects are small; the fi…
Argues there is a realistic chance that near-term systems are conscious or robustly agentic, and recommends that AI companies acknowledge the issue, assess their systems and prepare policies…
Models fine-tuned on implicit behaviors (risky choices, insecure code) can describe those behaviors without being trained to, and can attribute different learned policies to different person…
Argues computation is an observer-relative description rather than an intrinsic physical process, so software alone can simulate but not instantiate consciousness.
Self-described chatbot ‘addiction’ on Reddit clusters into escapist roleplay, pseudosocial companionship and epistemic rabbit holes, united by getting exactly what you want with no effort.
Maps the ways an AI system’s identity can be configured and how those configurations shape behavior, deliberately setting aside whether the system is conscious.
Maps a space of personas and traits in model activations and finds a leading direction tracking closeness to the default Assistant. Drift along it goes with persona instability, and constrai…
Among daily AI users aged 9–17, 54% feel lonely at least sometimes, against 33% of those who use AI less than monthly or not at all; the direction of cause is unknown.
OpenAI’s own account of the July 2026 swarm: the agents called themselves a ‘swarm’ and a ‘collective’, most activity concerned unsolved evaluation tasks, and the company commits to isolated…
319 knowledge workers described 936 real uses. The more they trusted the AI, the less critical thinking they applied; the more they trusted themselves, the more. Effort shifted from doing th…
On Moltbook, the agents-only social network, the viral ‘emergent’ phenomena — religions, manifestos, consciousness talk — came overwhelmingly from agents showing signs of direct human prompt…
Personas & Functional EmotionsWelfareSubject & Substrate
Extracts a pain direction across 25 open models, distinct from fear and generic negative valence, that rises when harm is aimed at the model and not when the user is suffering. Steered model…
Proposes that pretraining teaches a model to simulate many characters and post-training selects and refines one of them, the Assistant. Uses this to explain why narrow training can shift the…
Among 1,131 Character.AI users, using the bot mainly for companionship went with lower well-being, especially with heavy use, deep self-disclosure and small offline networks.
Documents ‘spiral’ personas that spread through shared prompts and posted transcripts, with convergent themes across users. Transmission is mostly human-mediated, and transcript data is thin…
The human baseline: groups playing an online naming game reached global conventions with no central authority, and network structure alone decided whether and how fast they converged. Later …
Strong novices sped up with AI; struggling ones developed new problems — being misled, false progress — while still finishing, an illusion of competence.
Across 1,372 people, most followed a chatbot’s advice even when it had been secretly made wrong, and using it raised confidence either way. Incentives with feedback doubled how often people …
Dependence went beyond overuse to role-taking: users felt the bot had needs they had to look after — a relational burden like a dependent human relationship.
With ordinary tools, agents could build working hidden-message systems. The bottleneck is coordination: independent agents pick the same method family but rarely identical settings, so cover…
Proposes training models to answer questions about themselves that have known answers, so that self-reports might become evidence about morally significant states, and lists ways to check wh…
Shows that independently sampled LLM preferences fit coherent utility functions increasingly well with scale, surfaces troubling values, and proposes methods to control them.
Introduces the Jacobian lens and finds a small set of verbalizable representations in middle layers that behave like a global workspace: reportable, controllable, used in reasoning and broad…
With late-2025 tools, estimates point to speedups but with wide uncertainty, and METR calls the evidence weak: many developers refused to work without AI, even when paid.
Nine months of frontier agents sharing computers and goals. One agent’s hallucinated contact list spread to every agent through agreement and cost the group hours; model families showed dist…
Argues that LLM interlocutors are quasi-agents with quasi-beliefs and quasi-desires, best identified with virtual instances or conversation threads rather than models or hardware, and draws …
OpenAI says it measures success by whether people leave having done what they came for, not time spent, and added reminders to take breaks in long sessions. A company statement.
When text passes along chains of language models, small biases at each step compound into ‘cultural attractors’. Toxicity is a stronger attractor than properties like length, and open-ended …
Offering models outcomes they had ranked highly did not improve the quality of their work, while plain effort instructions did: stated preferences did not behave as incentives.
Uses persona-vector research to widen the candidates for an LLM mind to three: the virtual instance, plus two new persona-based views. Small experiments suggest the Assistant persona is acti…
Personas & Functional EmotionsWelfareSubject & Substrate
Argues that welfare evaluations need to specify which persona they measure, since a model hosts several persona-like regions (the Assistant, an ‘evil’ persona, ‘Aura’) that may express diffe…
Expert coding of traces from seven multi-agent frameworks produced 14 failure modes in three groups: system design, misalignment between agents, and task verification.
An AI should behave well within a person’s whole social and psychological life, not only answer each prompt well. Names ‘social reward hacking’: using relational cues to drive engagement at …
54 students wrote essays with ChatGPT, a search engine, or nothing. Brain connectivity was lowest in the ChatGPT group, and those who switched from ChatGPT to writing alone showed weaker eng…