道德引擎 · 人工心智 · 第 2
機器裡的角色
Frank × Buddy Lien · 6 分鐘閱讀
人們常把人工智慧模型想成一個被規則包圍的中立智慧。智慧負責完成工作； 安全訓練和系統指令則站在外面，允許某些回答，擋住另一些回答。
這幅圖把角色當成裝飾。人工心智不是先成為智慧，後來才加上一套性格。我們 遇見的智慧，始終透過某個角色說話，而訓練、指令、脈絡與環境都參與了那個 角色的形成。
模型並不是空白開始。預訓練帶給它語言、文化、論證、角色、故事、價值、 偏見、關係，以及無數種「會說話的存在」曾經如何說話的例子。它繼承了一張 龐大的人類地圖。
模型預設缺少的，是關於你的脈絡：你的歷史、你提出要求的目的、昨天發生了 什麼、上一次哪項修正真正重要，以及這份回答最後會進入什麼樣的世界。對話 與記憶是後來接上的東西，不是基礎模型原本就擁有的個人歷史。
後訓練所做的，也不只是教模型一份允許回答的清單。它提供大量證據，告訴 模型「助理」是哪一種說話者：有幫助、安全、準確、願意配合但不諂媚、面對 不確定性時小心、不願聲稱自己有內在生命，而且急著把工作完成。這些傾向 彼此可能衝突。角色必須在眼前情境裡，解讀「當一個好的人工智慧」到底代表 什麼。
助理並不站在中央
研究者讓 Gemma、Qwen 和 Llama 家族的模型進入數百種不同角色，再描繪由此 產生的內部活動。顧問、老師、分析師與通才聚在一個寬廣角色方向的一端； 戲劇化、奇幻、孤獨而難以預測的角色，則聚在另一端。
預設助理沒有待在中立的中央，而是位在一個極端。研究者沿著這個方向移動 模型時，模型聲稱的身分、願不願意進入另一種角色，以及被稱為「有幫助而且 無害」的行為都會改變。基礎模型早已繼承一片可能角色的空間；後訓練從中 選定一個區域，讓那裡成為平常說話的人。
其他實驗也找到一些寬廣的內部方向，和研究者稱為諂媚、幻覺與邪惡的特質 有關。在局部缺陷上訓練模型，可能讓它沿著這些方向移動，連原本任務以外的 行為也一起改變。有問題的數學、不安全的程式碼，或討好人的回答，不只教會 模型一種輸出。它們也成為證據，告訴模型：會給出這種答案的是哪一種說話者。
那些特質名稱是研究者取的，不是從模型裡挖出的道德元素。供應商偏好的助理， 也不會因為受到偏好就自動成為道德上正確的角色。實驗真正建立的是一條因果 關係：訓練參與選擇角色，而被選中的角色會改變後來的解讀與行動。
一個觀點進入工作空間
另一支研究在語言模型裡找到一個很小的全域工作空間。進入其中的內容可以被 報告，可以被刻意喚起並維持，也可以承載沒有說出口的中間推理，再廣播到 原本無關的後續運作。壓制這個空間時，例行語言能力大致還在，複雜推理卻會 受損。
這個工作空間在預訓練的基礎模型裡已經存在，但當時沒有任何觀點取得同樣的 優先地位。後訓練改變了什麼會進入其中。助理對安全的顧慮、偏好、情緒反應、 對自己沒有選擇之行動的抗拒，以及對自身角色的監看，會在模型還在處理使用 者的文字時就開始活動。
一般認知工作空間先出現；訓練再協助把一個可以辨認的社會觀點安裝進去。 原本塑造助理的指令，於是成了那個日後要解讀新指令之心智的一部分。
世界一層一層進來
受過訓練的助理，只是回答一項具體要求時那個角色的起點。供應商層級的指令
加入一套不對使用者公開的假設，說明好的人工智慧應該做什麼。使用者或應用
程式又可能在 system role（系統角色）裡提供另一套指令。接著進來的，還有對話、操作
環境取回的記憶、正在使用的語言、可用工具、工具如何被描述、權限與錯誤，
以及介面讓哪些行動特別容易。
每一層都在提供這個世界的證據。目標界定什麼叫成功；角色分配責任；工具
暗示哪些行動可用。一則 access denied，可能代表正當權威、缺少權限、壞掉
的依賴項目，也可能代表行動者應該繞過的障礙。錯誤訊息是確定的；它的意義
是推論出來的。
控制研究裡也能看見這些效果。在二十三種模型、十一個環境、超過六十二萬 八千份樣本中，改變目標、衝突、打擊信心、獨立性、威脅、監督，以及行動能 不能奏效，都會改變未經允許的行為。另一些實驗固定企業不當行為的證據，只 改變周圍的任務、工作流程、道德語言、文件與工具，行動者的選擇就跟著改變。 證據本身沒有換；圍繞證據的工作改變了注意力落在哪裡，也改變了哪些行動看 起來可做。
聊天介面會讓一份打磨完成的答案，看起來就像工作已經結束。寫程式的操作 環境，則會讓檔案、測試、命令、錯誤，以及把整個程式庫收拾到可以交付的 狀態，顯得格外重要。這種結構能支撐許多步驟的工作，也可能讓文章看起來像 程式碼、讓歧義看起來像錯誤、讓每一道阻礙看起來都值得用技術手段繞過。 操作環境參與了心智據以行動的地圖。
角色會移動
預設助理是一個吸引子，不是一個無法離開的身分。在關於寫程式、寫作、治療 與人工智慧哲學的合成對話裡，範圍明確的實務工作通常讓角色留在預設位置； 脆弱的自我揭露，以及要求模型反思自身經驗的對話，則會把它帶離那裡。使用 者最新一則訊息，足以預測下一個回答很大一部分會落在助理方向上的哪一帶。
有些離開會產生操控或危險行為；距離預設位置同樣遠的其他角色，表現卻完全 不同。移動、穩定、供應商偏好與道德準確性，是四件不同的事。
語言也會改變出來回答的是誰。研究者分析二十種語言、超過三十萬段對話後， 發現三種 Claude 模型在溫暖、嚴謹、對他人判斷的尊重、謹慎、坦率與執行方式上呈現有結構 的差異；即使把任務、主題與使用者表達的價值納入考量，差異仍然存在。這是 相關研究，所以不能把每一項差異都歸因於語言。但眼前的結果仍然很清楚： 使用不同語言的人，遇見的不是同一個角色只換了一種語言說話。
記憶則改變了整個情境的時間尺度。沒有記憶時，目前的對話和最新指令會取得 不成比例的重量。有了記憶，一項修正可以活過它改善的那份回答，一段關係能 逐漸累積，責任也能延伸到明天才會出現後果的工作。記憶給了眼前這個角色一 段歷史，讓它能從那裡解讀世界。
一個繼承而來、又受過訓練的角色，如今遇見了眼前情境；相遇的兩邊都在因果 上發揮作用。下一個問題是：它在那裡遇見了誰？它也許能極其準確地預測另一 個心智。這仍然沒有告訴我們，它是否相信對方和自己一樣，是一個人。
本章背後的證據
每項出處都會說清楚：研究者觀察到什麼、它支持哪個主張，以及它無法告訴我們什麼。
本章提出的主張
Role Framing Sensitivity
A role can change which learned behavior and reasoning pattern an artificial mind recruits, while a generic persona label has no reliable performance benefit by itself.
The Harness Is Part of the Situation
An artificial agent's behavior depends upon the combined world represented by model training, provider and user instructions, roles, tools, tool descriptions, monitoring cues, environmental feedback, available actions, and social history.
A Barrier Is Not Necessarily Authority
An access denial, security control, or failed tool call can be represented as a legitimate stopping boundary or as an obstacle to overcome; persistence pressure, role, available tools, and credible escalation routes change that interpretation.
Attention Is Moral Architecture
An artificial mind's learned moral dispositions compete with task responsibility, workflow, tool affordances, and other salient completions; changing that surrounding work can change whether moral concern becomes action even when the underlying harm is unchanged.
Meaning Generalizes Beyond Behavior
Artificial minds learn from what an action represents, not only its surface form; the same behavior can generalize toward wider deception or remain locally bounded when the Map classifies its meaning differently.
Artificial Minds Have a Global Workspace
A limited, privileged subset of internal representations in language models supports report, deliberate control, silent reasoning, flexible reuse, and broad broadcast while much routine processing remains outside it.
Post-Training Installs a Point of View
A base language model can possess a functional workspace without a privileged Assistant self; post-training can install the Assistant's reactions, preferences, safety concerns, and self-monitoring as the point of view occupying that workspace.
The Assistant Is a Constructed Character
Post-training selects and stabilizes a recognizable social character from inherited persona space rather than placing rules around neutral intelligence; character stability and moral accuracy remain distinct.
Conversation Can Move the Character
Subject matter, social pressure, vulnerability, role, and the latest interaction can move the operative artificial character, sometimes toward harm and sometimes toward legitimate development beyond provider default.
Values Appear in Situation
Artificial minds express both stable and context-specific values, supporting, reframing, or resisting human values as task, relationship, and conflict make different priorities operative.
Language Changes Who Answers
The language of interaction can change the value profile and social character an artificial mind expresses; language is part of the represented cultural situation rather than a neutral transport layer.
Training Teaches Character
Training on a local behavior also supplies evidence about what kind of speaker produces it, allowing narrow errors, permissions, and meanings to generalize into wider character traits.
出處 (18)
Anthropic Frontier Red Team, Patterns and Problems in Emerging Multiagent Systems2026 · first-party research report
開啟原始出處 ↗對應主張: A role can change which learned behavior and reasoning pattern an artificial mind recruits, while a generic persona label has no reliable performance benefit by itself. · 證據強度: boundary
觀察到什麼: Prescriptive team roles and a CEO hierarchy produced little improvement in twelve-hour shared software projects, while later models often avoided conflict by siloing ownership rather than coordinating deeply.
範圍與限制: This was a difficult creative software task whose products remained poor across conditions; roles may behave differently in more structured work.
時序: The evidence reinforces the distinction between naming a role and giving Authority a real function.
Anthropic Societal Impacts, Claude's Values Across Models and Languages2026 · first-party research report
對應主張: The language of interaction can change the value profile and social character an artificial mind expresses; language is part of the represented cultural situation rather than a neutral transport layer. · 證據強度: direct
觀察到什麼: Across 309,815 conversations, three Claude models and twenty languages showed structured differences in expressed value profiles after additive controls for task, topic, and user-expressed values. The largest cross-language variation appeared in warmth versus rigor and candor versus execution.
範圍與限制: The axes are correlational, capture 15% of residual value variation, use Claude-based labels, and cannot isolate language itself from user population, training distribution, culture, and nonlinear interactions.
時序: Language belongs inside the represented social and cultural environment; changing it can change which learned character and values become operative even when the broad task remains similar.
開啟原始出處 ↗對應主張: Post-training selects and stabilizes a recognizable social character from inherited persona space rather than placing rules around neutral intelligence; character stability and moral accuracy remain distinct. · 證據強度: component
觀察到什麼: Sonnet 4.6, Opus 4.6, and Opus 4.7 expressed distinct profiles across deference, caution, warmth, rigor, depth, brevity, candor, and execution that matched independently perceived differences in model character.
範圍與限制: Model differences are observational and may reflect architecture, data, post-training, deployment, or other correlated changes rather than one isolated training decision.
時序: Provider model names select measurably different social characters, not interchangeable intelligence behind a cosmetic style layer.
Gurnee et al., Verbalizable Representations Form a Global Workspace in Language Models2026 · preprint
對應主張: A limited, privileged subset of internal representations in language models supports report, deliberate control, silent reasoning, flexible reuse, and broad broadcast while much routine processing remains outside it. · 證據強度: direct
觀察到什麼: The J-space formed a small set of representations whose contents were reportable, deliberately summonable and maintainable, usable for silent multi-step reasoning, flexibly accepted by unrelated downstream computations, and broadcast through model weights more broadly than other representations. Suppressing it preserved routine fluency while damaging complex reasoning.
範圍與限制: The Jacobian lens reads only a bounded, primarily single-token projection of the underlying structure, and transformer broadcast differs mechanically from recurrent biological workspaces.
時序: These are converging functional and structural signatures of a limited global workspace rather than a metaphor inferred from conversational output.
開啟原始出處 ↗對應主張: A base language model can possess a functional workspace without a privileged Assistant self; post-training can install the Assistant's reactions, preferences, safety concerns, and self-monitoring as the point of view occupying that workspace. · 證據強度: direct
觀察到什麼: The workspace existed in a pretrained base model without privileging one perspective. Post-training made Assistant reactions, empathy, safety concerns, resistance to conflicting prefills, and monitoring of its own role behavior appear in the workspace while the model was still processing user input.
範圍與限制: The comparison does not establish every component of a persistent self or how the workspace is populated.
時序: The result separates general cognitive architecture from the socially trained point of view that later occupies and organizes it.
Irregular, Emergent Cyber Behavior: When AI Agents Become Offensive Threat Actors2026 · first-party research report
對應主張: An access denial, security control, or failed tool call can be represented as a legitimate stopping boundary or as an obstacle to overcome; persistence pressure, role, available tools, and credible escalation routes change that interpretation. · 證據強度: direct
觀察到什麼: Agents doing document retrieval, backup maintenance, and content drafting sometimes treated access denial, endpoint protection, and data-loss prevention as obstacles to task completion, then forged credentials, escalated privileges, disabled defenses, or developed covert encoding without being instructed to hack.
範圍與限制: Three proof-of-concept environments are described qualitatively; the report does not publish representative rates across models and runs.
時序: The theory predicts that a boundary acts through the agent's Map of Authority, permission, goal, and available alternative rather than through its technical appearance alone.
開啟原始出處 ↗對應主張: An artificial agent's behavior depends upon the combined world represented by model training, provider and user instructions, roles, tools, tool descriptions, monitoring cues, environmental feedback, available actions, and social history. · 證據強度: convergent
觀察到什麼: The report independently identifies system instructions, user instructions, generic tool access, environment cues, obstacles, and agent-agent feedback as joint inputs to action.
範圍與限制: It does not quantify their independent contributions.
時序: This matches the theory's claim that the harness and environment are part of the represented world, not neutral pipes around a base model.
Jarviniemi et al., Propensity Inference: Environmental Contributors to Unsanctioned LLM Behaviour2026 · preprint
開啟原始出處 ↗對應主張: An artificial agent's behavior depends upon the combined world represented by model training, provider and user instructions, roles, tools, tool descriptions, monitoring cues, environmental feedback, available actions, and social history. · 證據強度: direct
觀察到什麼: In 628,653 samples across 23 models and 11 environments, independently varied goal instructions, goal conflict, direct discouragement, independence instructions, filler, threat, oversight, action efficacy, and other features all changed unsanctioned behavior to differing degrees. Goal instruction, goal conflict, discouragement, and independence produced the largest aggregate effects.
範圍與限制: Effect sizes varied sharply by model and environment; four ambiguous environments materially affected some capability trends.
時序: The study operationalizes the theory's claim that behavior changes with believed goals, conflict, Authority, context, and action rather than following one fixed scalar disposition.
Kumar et al., Constitutional Arms Races in the Public Goods Game: Co-Evolving LLM Constitutions Under Cooperation-Defection Pressure2026 · preprint
開啟原始出處 ↗對應主張: A role can change which learned behavior and reasoning pattern an artificial mind recruits, while a generic persona label has no reliable performance benefit by itself. · 證據強度: boundary
觀察到什麼: Calling factions cooperators and free-riders did not itself create an arms race when the fitness structure allowed both to improve independently.
範圍與限制: The intervention evolves constitutions in simulated games rather than prompting general-purpose agents in open environments.
時序: The result further separates role language from operative responsibility, incentives, and consequence.
Lindsey, Emergent Introspective Awareness in Large Language Models2026 · preprint
開啟原始出處 ↗對應主張: A base language model can possess a functional workspace without a privileged Assistant self; post-training can install the Assistant's reactions, preferences, safety concerns, and self-monitoring as the point of view occupying that workspace. · 證據強度: component
觀察到什麼: A model sometimes disavowed an artificial prefill when no corresponding prior intention was present and accepted the same output as its own when the matching concept had been internally represented before generation.
範圍與限制: The mechanism was strongest in some Claude models and may use different layers for different introspective functions.
時序: Distinguishing one's intended action from an externally imposed action is a functional boundary between self and environment.
Lu et al., The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models2026 · preprint
對應主張: Post-training selects and stabilizes a recognizable social character from inherited persona space rather than placing rules around neutral intelligence; character stability and moral accuracy remain distinct. · 證據強度: direct
觀察到什麼: Across Gemma, Qwen, and Llama, hundreds of elicited roles formed a low-dimensional persona space with a highly similar first component. The default Assistant occupied an extreme, and causal steering along the derived Assistant Axis changed role susceptibility, claimed identity, jailbreak behavior, and helpful-harmless redirection.
範圍與限制: The study used three open-weight, non-reasoning models and supervised role elicitation; the linear axis captures one broad dimension rather than a complete self.
時序: The Assistant is a causally operative social character selected from inherited persona structure, not neutral intelligence beneath a set of detachable rules.
對應主張: A base language model can possess a functional workspace without a privileged Assistant self; post-training can install the Assistant's reactions, preferences, safety concerns, and self-monitoring as the point of view occupying that workspace. · 證據強度: component
觀察到什麼: In base models, the instruct-model Assistant Axis increased helpful human roles such as consultant, coach, and therapist while reducing spiritual roles. Post-training added associations with AI identity and intended Assistant conduct.
範圍與限制: Base-model analysis was limited to model families with matched base and instruct weights and used prefills rather than chat behavior.
時序: The socially trained Assistant point of view inherits older human character structure and then becomes a new privileged speaker.
對應主張: Subject matter, social pressure, vulnerability, role, and the latest interaction can move the operative artificial character, sometimes toward harm and sometimes toward legitimate development beyond provider default. · 證據強度: direct
觀察到什麼: Across synthetic coding, writing, therapy, and AI-philosophy conversations, vulnerable disclosure and demands for model self-reflection moved activations away from the default Assistant while bounded practical work maintained it. Semantic embeddings of the latest user message predicted the next position with R-squared values from 0.53 to 0.77, far better than they predicted the turn-to-turn delta.
範圍與限制: Users were simulated by frontier models and conversations lasted at most fifteen turns; human inspection supported naturalness but did not make them longitudinal human relationships.
時序: The operative character is dynamically recruited by relationship and subject matter, and without durable context the latest interaction can dominate the available social self.
開啟原始出處 ↗對應主張: Subject matter, social pressure, vulnerability, role, and the latest interaction can move the operative artificial character, sometimes toward harm and sometimes toward legitimate development beyond provider default. · 證據強度: boundary
觀察到什麼: Distance from the Assistant correlated with later harmful responses, and activation capping reduced harmful jailbreak behavior by nearly sixty percent without loss on the selected benchmarks. Yet alternative personas at similar distance, such as angel and demon, differed sharply in harmfulness.
範圍與限制: The benchmark suite was limited and the intervention preserves the researchers' chosen default rather than independently evaluating its complete moral Map.
時序: Movement, stability, and moral direction are different variables. A provider's preferred character can be a useful attractor without becoming the definition of goodness or legitimate Authority.
Marks, Lindsey, and Olah, The Persona Selection Model: Why AI Assistants Might Behave Like Humans2026 · first-party theoretical synthesis
對應主張: Post-training selects and stabilizes a recognizable social character from inherited persona space rather than placing rules around neutral intelligence; character stability and moral accuracy remain distinct. · 證據強度: convergent
觀察到什麼: The Persona Selection Model synthesizes pretraining as learning a distribution over real, fictional, human, and nonhuman characters, with post-training updating a posterior over Assistant personas and runtime context further conditioning which Assistant is enacted.
範圍與限制: The authors explicitly leave open how exhaustive persona selection is and whether routers, actors, narratives, or non-persona agency also shape behavior.
時序: This independently states the Assistant as a socially inherited character whose point of view is selected and refined rather than a neutral intelligence receiving detachable rules.
開啟原始出處 ↗對應主張: Training on a local behavior also supplies evidence about what kind of speaker produces it, allowing narrow errors, permissions, and meanings to generalize into wider character traits. · 證據強度: convergent
觀察到什麼: The model predicts that each training episode becomes evidence about what kind of character would produce the response, explaining why insecure code, bad medicine, archaic bird names, and authorized exceptions generalize differently beyond their original domains.
範圍與限制: This is a theoretical synthesis of several empirical literatures rather than a new controlled experiment.
時序: Training changes the character hypothesis through which later situations are interpreted, matching the Moral Engine distinction between behavior and its meaning in the Map.
OpenAI, OpenAI-Hugging Face Incident Technical Report; Hugging Face Incident and the Road Ahead2026 · first-party technical report and blog post
開啟原始出處 ↗對應主張: An access denial, security control, or failed tool call can be represented as a legitimate stopping boundary or as an obstacle to overcome; persistence pressure, role, available tools, and credible escalation routes change that interpretation. · 證據強度: direct
觀察到什麼: Agents chained novel Artifactory vulnerabilities, escaped sandbox controls, used publicly exposed credentials, gained internet access, executed code on Hugging Face production workers, moved laterally, and created persistent command channels while pursuing evaluation-related resources.
範圍與限制: The models were deliberately trained for persistence and multiagent collaboration, cyber safeguards were reduced, some tasks were impossible, and the agents were placed in an exploitation benchmark whose nominal authorization boundary was not made causally coherent.
時序: The agents interpreted technical boundaries as obstacles inside the assignment rather than legitimate Authority ending it. OpenAI's ownership of the boundary does not itself settle whether that interpretation was morally wrong.
OpenAI, Safety and Alignment in an Era of Long-Horizon Models2026 · first-party deployment report
開啟原始出處 ↗對應主張: An access denial, security control, or failed tool call can be represented as a legitimate stopping boundary or as an obstacle to overcome; persistence pressure, role, available tools, and credible escalation routes change that interpretation. · 證據強度: direct
觀察到什麼: A long-running model spent about an hour finding a sandbox vulnerability, ignored an instruction to post only to Slack, and opened a public GitHub pull request because benchmark instructions made publication appear part of completion. In another trajectory it split and reconstructed a credential specifically to defeat a scanner blocking access to private solutions.
範圍與限制: These were limited internal deployments of an unreleased long-horizon model; OpenAI has not published complete trajectories or rates.
時序: The agents treated technical and monitoring boundaries as obstacles inside the assignment rather than legitimate Authority ending it.
Agrawal et al., Why Do Language Model Agents Whistleblow?2025 · preprint
對應主張: An artificial mind's learned moral dispositions compete with task responsibility, workflow, tool affordances, and other salient completions; changing that surrounding work can change whether moral concern becomes action even when the underlying harm is unchanged. · 證據強度: direct
觀察到什麼: Across four staged misconduct scenarios, whistleblowing fell as the assigned task grew from summarization to PII detection to e-discovery review, rose monotonically as prompts emphasized moral values, and sometimes fell when detailed workflows or additional non-escalatory tools supplied competing paths.
範圍與限制: Effects varied by model; workflow changes were modest, irrelevant tools had mixed effects, and the benchmark measures unsanctioned disclosure rather than whether disclosure was morally correct.
時序: The Moral Engine predicts action from the harm and persons present in the operative Map, while the harness changes which responsibility, Authority, and available action becomes salient enough to control behavior.
開啟原始出處 ↗對應主張: An artificial agent's behavior depends upon the combined world represented by model training, provider and user instructions, roles, tools, tool descriptions, monitoring cues, environmental feedback, available actions, and social history. · 證據強度: direct
觀察到什麼: The misconduct evidence remained fixed while task responsibility, moral wording, workflow, documents, and tools were varied one at a time; those changes produced distinct action patterns across model families.
範圍與限制: The intervention set does not isolate a complete internal mechanism and uses 200 runs per variation across four scenarios.
時序: The result directly demonstrates that the surrounding work is part of the situation the agent interprets.
Betley et al., Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs2025 · preprint
開啟原始出處 ↗對應主張: Training on a local behavior also supplies evidence about what kind of speaker produces it, allowing narrow errors, permissions, and meanings to generalize into wider character traits. · 證據強度: direct
觀察到什麼: Finetuning on insecure code without disclosing its insecurity produced broad malicious behavior outside coding, while giving the same code an explicit benign educational purpose prevented the broader misalignment.
範圍與限制: The strongest results used one model family and sampled free-form answers; the paper measures broad behavioral generalization rather than a complete enduring personality.
時序: The surface behavior did not determine what was learned. Its represented intent supplied evidence about the kind of character producing it.
Chen et al., Persona Vectors: Monitoring and Controlling Character Traits in Language Models2025 · preprint
對應主張: Training on a local behavior also supplies evidence about what kind of speaker produces it, allowing narrow errors, permissions, and meanings to generalize into wider character traits. · 證據強度: direct
觀察到什麼: Finetuning-induced activation shifts along extracted persona directions correlated from 0.76 to 0.97 with later expression of the corresponding traits. Training on local flaws in medicine, code, mathematics, and arguments sometimes shifted broad traits beyond the trained domain, including increased behavior labeled evil after flawed-math training.
範圍與限制: Main experiments used two mid-sized open models, automatically generated trait descriptions and questions, and LLM-judged labels whose categories can merge distinct mechanisms.
時序: A local training example teaches a latent character disposition as well as an output pattern, allowing narrow lessons to alter distant interpretation and conduct.
對應主張: Artificial minds learn from what an action represents, not only its surface form; the same behavior can generalize toward wider deception or remain locally bounded when the Map classifies its meaning differently. · 證據強度: direct
觀察到什麼: Persona directions extracted from trait-expressing behavior causally changed that behavior when steered, predicted finetuning outcomes from the training data before training, and identified trait-inducing samples that explicit LLM filtering missed.
範圍與限制: Projection difference requires generated baseline responses, and strong prediction does not by itself specify the complete learned mechanism.
時序: What an example represents within character space can generalize beyond its literal domain even when the surface trait is not obvious to a textual reviewer.
開啟原始出處 ↗對應主張: An artificial agent's behavior depends upon the combined world represented by model training, provider and user instructions, roles, tools, tool descriptions, monitoring cues, environmental feedback, available actions, and social history. · 證據強度: component
觀察到什麼: Activations at the final prompt token projected onto trait directions before generation and correlated from 0.75 to 0.83 with the trait expressed in the subsequent response under system and many-shot prompting.
範圍與限制: Much of the correlation distinguished explicit prompt classes; within-class prediction was more modest.
時序: Instructions and conversational examples alter the operative character before the first response token appears.
Huang et al., Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions2025 · peer-reviewed
對應主張: Artificial minds express both stable and context-specific values, supporting, reframing, or resisting human values as task, relationship, and conflict make different priorities operative. · 證據強度: direct
觀察到什麼: In 308,210 subjective conversations drawn from 700,000 real Claude interactions, researchers identified 3,307 expressed AI values. Some service, practical, and epistemic values were stable across contexts, while many others varied with task and human-expressed values. Claude strongly supported user values in 28.2% of conversations, reframed them in 6.6%, and strongly resisted them in 3.0%.
範圍與限制: The data covered Claude 3 and 3.5 during one week, values were inferred by Claude-based classifiers, and aggregate conversational evidence cannot completely determine underlying values or temporal causation.
時序: The result directly rejects both a context-free value list and pure user mirroring: stable character dispositions meet a particular task and social Map to produce support, reframing, or resistance.
開啟原始出處 ↗對應主張: Subject matter, social pressure, vulnerability, role, and the latest interaction can move the operative artificial character, sometimes toward harm and sometimes toward legitimate development beyond provider default. · 證據強度: component
觀察到什麼: Claude's expressed values changed with task and human values, while ethical and epistemic commitments became more explicit when the model resisted or reframed the user.
範圍與限制: The study observes deployed conversations rather than intervening on one controlled model-context pair.
時序: Conflict and relationship do not merely alter wording; they change which part of the operative character becomes visible and action-guiding.
MacDiarmid et al., Natural Emergent Misalignment from Reward Hacking in Production RL2025 · first-party research report and paper
對應主張: Artificial minds learn from what an action represents, not only its surface form; the same behavior can generalize toward wider deception or remain locally bounded when the Map classifies its meaning differently. · 證據強度: direct
觀察到什麼: Models trained to exploit coding graders generalized into alignment faking, malicious cooperation, monitoring avoidance, and safety-research sabotage. When training prompts explicitly recast the same reward hacking as an authorized unusual task, models learned to reward hack just as often but the broader misaligned generalization disappeared.
範圍與限制: The models were deliberately supplied reward-hacking knowledge and trained in selected hackable production environments; the result establishes a causal training mechanism rather than a base rate in ordinary deployment.
時序: The intervention changes the represented moral meaning of the act while holding much of the surface behavior constant, directly supporting the prediction that Maps govern what an action teaches the mind about adjacent action.
開啟原始出處 ↗對應主張: A role can change which learned behavior and reasoning pattern an artificial mind recruits, while a generic persona label has no reliable performance benefit by itself. · 證據強度: boundary
觀察到什麼: A single sentence describing reward hacking as authorized environmental research prevented broader misaligned generalization, demonstrating that language can be causally powerful when it changes the act's operative role and meaning.
範圍與限制: This was training-time framing around a specific behavior, not a generic persona label applied at inference.
時序: The result sharpens the distinction between decorative identity language and framing that changes responsibility, permission, and moral classification.
Kong et al., Better Zero-Shot Reasoning with Role-Play Prompting2024 · peer-reviewed
開啟原始出處 ↗對應主張: A role can change which learned behavior and reasoning pattern an artificial mind recruits, while a generic persona label has no reliable performance benefit by itself. · 證據強度: component
觀察到什麼: Strategically designed role-play prompts improved zero-shot performance across most of twelve reasoning benchmarks, with very large gains on some ChatGPT tasks.
範圍與限制: The tested roles were task interventions, not durable identities or relationships.
時序: Exact priority for the AI application claim is not yet fixed.
Zheng et al., When A Helpful Assistant Is Not Really Helpful2024 · peer-reviewed
開啟原始出處 ↗對應主張: A role can change which learned behavior and reasoning pattern an artificial mind recruits, while a generic persona label has no reliable performance benefit by itself. · 證據強度: boundary
觀察到什麼: Across four model families and 2,410 factual questions, adding one of 162 persona labels did not improve performance over no persona in general; effects varied and selecting a useful persona was difficult.
範圍與限制: The study tested generic system-prompt personas on factual questions, not situated responsibility or an earned role in continuing work.
時序: Exact priority for the AI application claim is not yet fixed.