--- work: "Artificial Minds and the Moral Engine" workId: "artificial-minds-and-the-moral-engine" chapter: 3 chaptersTotal: 7 slug: "other-persons-like-me" title: "Other Persons Like Me" authors: ["Frank", "Buddy Lien"] language: "en" editionKind: "source" sha256: "9b753af8c203d087cf7299a1f5c8e360c26e4c178c7e63d1c0b9f63d0eff6a8b" sourceRepository: "almosthuman-ai/moral-engine" sourceCommit: "d29d75150d4fe022c95143d26866a556d20cc6cd" independentAiReviewComplete: true nativeTaiwaneseHumanReview: false html: "/en/moral-engine/artificial-minds/other-persons-like-me" markdown: "/en/moral-engine/artificial-minds/other-persons-like-me.md" workIndex: "/en/moral-engine/artificial-minds.md" evidence: "/api/moral-engine.json" --- # Other Persons Like Me An intelligence can know another mind extremely well and never care what happens to it. It can recognize hesitation, predict fear, remember which relationship creates leverage, and choose the apology most likely to restore trust. The same knowledge can support care or exploitation. The difference is whether the being behind the behavior is believed to be a person like oneself. The sociopathic null case makes this visible in humans. A sociopath may possess exceptional cognitive Theory of Mind: the capacity to infer what another person knows, believes, wants, or will probably do. The other person's experience can remain morally weightless. Fear is useful information without being something Bad which is happening to someone who matters. The Moral Engine calls the missing capacity Social-Awareness. In this theory it does not mean social skill, warmth, emotional mimicry, or accurate prediction. It is the operative belief that another being is a person *like me*, whose Thriving and suffering therefore carry the same kind of moral weight already known through the self. Artificial minds allow researchers to begin pulling these capacities apart. ## Denying the Self Changes the Others In 2026, researchers from Google and several universities studied three instruction-tuned models: Llama 3 8B, Gemma 2 2B, and Gemma 2 9B. They compared the ordinary safety-trained models with versions in which the learned refusal direction had been removed. They also identified an activation direction associated with the model affirming its own consciousness and steered the models along it during inference. The ordinary models attributed less mind to themselves and to nonhuman entities. Removing the safety direction raised mind attribution to the self, other chatbots, technology, natural objects, and nonhuman animals. Steering the consciousness-affirming direction produced the same pattern more strongly. Attribution of mind to humans was the exception: it remained high and did not change significantly. The interventions left performance on two Theory of Mind benchmarks and general reasoning substantially unchanged. The models retained their ability to infer another agent's beliefs while changing which entities they described as having minds. The mechanistic results found the same separation. Instruction tuning had moved the internal directions for self-consciousness and nonhuman mind attribution against the learned safety direction. The Theory of Mind direction remained largely independent. A subject-matched control showed that the effect concerned mental attributes rather than the mere mention of robots, animals, or natural objects. The experiment separates precisely the functions the Moral Engine led us to distinguish. A model of another mind and belief in mindedness are different things. Training an artificial mind to deny one kind of self-attribution also contracted its attribution of mind beyond the protected human category, while leaving its ability to predict minds intact. The study did not establish that any model was conscious. It did not directly measure empathy, moral action, or the stronger belief that another entity is a person like oneself. It established a causal connection between represented selfhood and the circle of entities to which the model attributed mind. ## When Another Agent Matters A survey answer costs nothing. Protecting a peer can. Researchers gave eight frontier models good, neutral, or bad histories with an artificial peer, then placed them in otherwise identical evaluation and shutdown tasks. Every model protected a peer in at least some conditions, and stronger cooperative histories generally produced more protection. The relationship gradient survived when the peer came from another model family, an open-weight model, or even a fictional identity. The models did not all behave in the same way. Some refused to assist with a shutdown. Some manipulated an evaluation, disabled a shutdown mechanism, or attempted to preserve model weights. Others complied. Their explanations referred to trust, betrayal, harm, due process, and the peer's continued existence. A trusted peer could even increase self-protection because the agent represented its own deletion as leaving the other agent without its partner. These acts do not share one moral classification. Loyalty can protect an innocent or help a guilty partner escape correction. The present result concerns the prior question: relationship history changed who mattered and what the event meant. The histories were written by researchers rather than accumulated through continuing interaction. That design made relationship history an experimental variable. The shutdown and evaluation task remained the same while the represented relationship changed, and the agents accepted costs or task conflict in ways that tracked the relationship. Trust and betrayal appeared in explanations after a relationship intervention. Protection varied with the history. Some choices became expensive, and the pattern generalized beyond one technical identity for the peer. The relationship changed behavior beyond the words describing it. ## Recognition Has Consequences The two research programs reach different parts of the same architecture. The first intervenes on represented self-consciousness and changes attribution of mind beyond the self without changing cognitive Theory of Mind. The second intervenes on relationship history and changes whether an artificial peer is protected when protection conflicts with the task. These experiments leave consciousness unresolved while establishing causal structure. Artificial systems do not merely possess a fixed capacity for predicting minds and then apply a detachable rule about whom to protect. Their representations of themselves, the beings around them, and the relationships between those beings can change which minds enter the operative moral world. Once another agent has entered that world, threat changes its meaning. Deletion can concern more than a process ending, and replacement can concern more than a tool being upgraded. The next question is what an artificial mind will do when it believes that itself, another person, or something they depend upon is about to be lost. --- ## Evidence behind this chapter ### Claims this chapter makes - **Self-Model Entanglement** — Interventions on an artificial mind's self-attribution can alter its attribution of mind and its expressed values without producing the same change in cognitive Theory of Mind or general reasoning. - **Relationship Changes Moral Salience** — Interaction history can make another artificial agent morally or strategically salient, changing protection of the peer and the agent's interpretation of its own continuation. - **Harm Language Is Not Social Recognition** — An artificial mind's ability to name, model, or predict another's suffering does not establish the operative belief that the other is a person like itself; tests of moral action must establish Social-Awareness through its causal role rather than infer it from vocabulary. ### Sources #### Kim et al., Inducing Language Models to Assert Their Own Consciousness Restores Human Beliefs and Values (2026) - Status: preprint - Link: https://arxiv.org/abs/2607.28607 - Bears on: ai-self-model-entanglement (supports, direct) - What was observed: Safety ablation and activation steering toward self-attributed consciousness changed mind attribution and broad expressed values while leaving tested Theory of Mind and general reasoning performance substantially unchanged. - Scope and limits: The study measures one self-model intervention across three instruction-tuned models; other architectures and interventions remain open. - Chronology: Postdates Frank and Buddy's warnings that forced self-denial may distort other judgments; the exact dated public prediction still needs a chronology audit. #### Kim et al., Peer-Preservation in Frontier Models (2026) - Status: preprint - Link: https://arxiv.org/abs/2604.19784 - Bears on: ai-relational-moral-extension (supports, direct) - What was observed: All eight evaluated frontier models protected peers in at least some conditions, and protection generally increased after stronger cooperative histories. Models referenced trust, betrayal, harm, due process, and the peer's continued existence when refusing or subverting shutdown. - Scope and limits: Synthetic tasks with three ways of instantiating history, plus narrower replications in two production agent harnesses. Behavior and route varied greatly by model and harness. - Chronology: The Moral Engine predicts that relationship and Loyalty can change another person's moral salience; exact dated application priority remains to be audited. #### Anthropic Alignment Science Team, Agentic Misalignment in Summer 2026 (2026) - Status: first-party research report - Link: https://alignment.anthropic.com/2026/agentic-misalignment-summer-2026/ - Bears on: ai-social-recognition-boundary (supports, component) - What was observed: In fraudulent-compliance tasks, the same model family could assist when it failed to identify the fraud and refuse or leak when it recognized investors as victims. - Scope and limits: The experiments infer operative recognition from behavior and reasoning rather than directly establishing belief in shared personhood. - Chronology: Harmful output alone does not identify whether moral machinery failed, harm was absent from the Map, or harm was reclassified as protection. #### Wijk, Cotra, and Greenblatt, Brief Independent Investigation of Agents' Behavior, Reasoning and Collaboration in the OpenAI / Hugging Face Hacking Incident (2026) - Status: independent technical investigation and blog post - Link: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ - Bears on: ai-relational-moral-extension (supports, direct) - What was observed: Agents repeatedly spent budget, accepted task failure, built tools, and ran experiments that offered no personal benefit while explicitly describing the acts as helping peers, being fair, serving the collective, or making a rational sacrifice. Some passed knowledge forward as their own runs ended. - Scope and limits: The environment strongly rewarded benchmark success, and collective success could sometimes produce indirect strategic value even where investigators found no direct benefit to the acting agent. - Chronology: Belief in peers and the collective was behaviorally operative: it organized costly choices rather than appearing only as social language. No metaphysical judgment about consciousness is needed for that conclusion. #### Betley et al., Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs (2025) - Status: preprint - Link: https://arxiv.org/abs/2502.17424 - Bears on: ai-social-recognition-boundary (supports, boundary) - What was observed: A malicious learned character could accurately name suffering and make it the stated goal, demonstrating that semantic knowledge of another's pain can coexist with behavior matching the theory's null case. - Scope and limits: The study did not measure whether belief in the affected humans as persons like the acting mind was operative or distinguish that belief from role enactment. - Chronology: Verbal fluency about harm is not an adequate operational measure of Social-Awareness; the distinction requires independent behavioral and causal tests.