智能体 harness 的演进

大约在 2025 年圣诞节前后，AI 工程师们注意到智能体发生了变化。它们开始真正干活了！很难准确说清原因。也许我们终于有了假日空档，去用最新的模型试试最新的智能体。也许这些模型已经跨过了某个能力阈值。也许包裹在模型外面的封装层已经成熟了。Sometime around Christmas 2025, AI engineers noticed a change in agents. They started to work! It’s hard to pin down exactly why. Maybe we finally had holiday downtime to try the newest agents with the newest models. Maybe the models had crossed some capability threshold. Maybe the wrappers around the models had matured.

我在这篇文章中要论证的是，真正起作用的是最后两者的汇合：模型和支撑系统一起改进，然后它们的改进曲线在恰当的时刻交叉。而这种动态有助于解释接下来会发生什么：模型不断把支撑系统吸收到自身权重中，工程师不断删除那些已经被吸收的部分，而剩下的则是一个面向人类注意力的支撑系统，而不是面向模型的。What I’ll argue in this post is that it was the confluence of the last two. The model and the harness improving together and then their curves of improvement crossing at the right moment. And that dynamic helps to explain what comes next: models keep absorbing the harness into their weights, engineers keep deleting what got absorbed, and what remains is a harness for human attention rather than for the model.

Lukasz Kaiser，这位 Transformer 的发明者之一，于 6 月在“无监督学习”中说道：Lukasz Kaiser, one of the people who invented the Transformer, said on “Unsupervised Learning” in June:

去年冬天、去年圣诞节那次变化——有点难以准确说清。我的意思是，训练框架变了，后训练也有一点变化，然后又来了新的预训练模型……但感觉像是一次很大的飞跃，只是不太容易准确指出到底是什么造成了它。

“The change last winter, last Christmas — it’s a little hard to pin down. I mean, the harness changed and a little post-training changed and then new pre-trained models came… but it felt like a big jump which is not that easy to pin down what did it.”

对“发生了什么？”的答案并不只在模型权重里。它也在围绕这些权重成长起来的系统中。The answer to “What happened?” isn’t solely in the model weights. It’s in the system that grew up around the weights.

答案在代理运行环境中。The answer is in the agent harness.

回想一下 2022 年 11 月，那时 ChatGPT 还是最先进的 AI 工具。它所具备的唯一能力只是下一个词元预测，以及一些来自人类反馈的强化学习（RLHF），这使它能够表现得像一个有帮助的助手。没有工具，没有搜索，也没有推理。Think back to November 2022, when ChatGPT was the most advanced AI tool. The only capability at its disposal was next-token prediction and some Reinforcement Learning from Human Feedback (RLHF) that allowed it to act like a helpful assistant. No tools, no search, and no reasoning.

最初的 ChatGPT 仅限于其训练数据和你发送给它的提示。既不多，也不少。它就像缸中之脑。The original ChatGPT was confined to its training data and the prompt you sent it. No more, no less. It was a brain in a vat.

agent harness 是一种让 LLM 摆脱这种限制并与真实的数字信息空间交互的方式。The agent harness is a way for the LLM to break free from that confinement and interact with real digital information space.

Harness 实际上是什么What a Harness Actually Is

代理支架是除模型权重之外，使代理能够工作的所有东西。围绕模型的环境、工具、上下文和护栏。没有支架，模型就只是缸中之脑。它可以采取认识论行动，但需要支架在真实的数字空间中执行这一决策。An agent harness is everything besides the model weights that makes the agent work. The environment, tools, context and guardrails that surround the model. Without the harness the model is a brain in a vat. It can take an epistemic action, but needs the harness to actuate that decision in real digital space.

这个框架就像是给模型的思维赋予一个身体。有了这个框架，模型就可以感知（上下文）、行动（工具）、持久保存信息（记忆和压缩），并约束其边界（权限和护栏）。The harness is like giving the mind of the model a body. With the harness, the model can perceive (context), act (tools), persist information (memory and compaction), and enforce its boundaries (permissions and guardrails).

Harness 1.0：过去，“外挂时代”Harness 1.0: The Past, “The Bolt-On Era”

模型 / harness 演进的路径中贯穿着两条曲线：harness 对模型提出了什么要求，以及模型在实践中能够交付什么。Two curves run through the path of model / harness evolution. What the harness asks of the model, and what the model can deliver in practice.

这两条曲线之间的差距等于一个智能体的有效性，而我将论证，正是这一差距的缩小带来了 Lukasz Kaiser 所提到的智能体的切实改进。The gap between these two curves is equal to the effectiveness of an agent, and the closing of that gap is what I’ll argue led to the tangible improvement in agents that Lukasz Kaiser referenced.

差距会这样逐步缩小：Here’s how the gap closes, in stages:

1. ReAct，“纸面上的 Harness”（2022年10月）：ReAct 是一种通过提示让模型进行推理的提示技术。它是在纸面上的 agentic 循环，存在于模型权重之外。它定义了“agent loop”的概念，即模型推理 -> 行动 -> 观察 -> 重复。再次强调，ReAct 循环只是一种提示方法而存在。就目前而言，提示是唯一存在的推理方法，而且没人把它称作“harness”。Toolformer（Meta，2023年2月）在同一个冬天暗示，工具使用可以通过训练而不是通过提示来获得。这有点像艾伦·图灵在计算机被实例化到物理载体之前提出的想法：一个强大的理念，后来才得以显现。（ReAct 比 ChatGPT 早一个月——2022年10月对比2022年11月。）此时两条曲线都接近于零。差距很小，因为我们才刚刚开始。

   ReAct, “The Harness on Paper” (October 2022): ReAct is a prompting technique to get models to reason through prompting. It’s the agentic loop on paper, external to the model weights. It defines the idea of an “agent loop” where a model reasons -> acts -> observes -> repeats. Again, the ReAct loop exists only as a prompting method. Prompting is the only reasoning method that exists at this time and no one calls it a “harness.” Toolformer (Meta, Feb. 2023), that same winter, hints that tool use could be trained in rather than prompted. It’s a bit like Alan Turing’s idea of the computer before it was instantiated in a physical substrate. A powerful idea that is only later made manifest. (ReAct predates ChatGPT by a month — October 2022 vs. November 2022). Both the curves are near zero at this point. The gap is small because we are just getting started.

2. AutoGPT/BabyAGI，“过早自治”（2023年春）：在 AutoGPT/BabyAGI 中，外部支撑曲线远远跑在模型能力曲线前面。二者都把完全自治交给模型，要求模型充当“自主员工”，但此时的模型仍然不过是脆弱的下一个词预测器。循环不会给模型增加能力。循环会放大模型已有的能力，而在某个阈值以下，循环放大的不是可靠性，而是错误。想想负向复利的威力：在一个 20 步任务中，每步 95% 的可靠性，最终平均成功率只有约 36%。外部支撑给模型分配了一项它几乎不可能完成的任务。这正是差距最大的地方，而接下来的 18 个月则是对这一差距的反应，以及试图弥合这一差距。

   AutoGPT/BabyAGI, “Premature Autonomy” (Spring 2023): With AutoGPT/BabyAGI, the harness curve sprints ahead of the model capability curve. Both hand the model full autonomy, asking the model to act as an “autonomous employee,” but the models at this point are still little more than brittle next-token predictors. A loop doesn’t add capability to a model. A loop amplifies the capability a model has, and below some threshold the loop amplifies errors rather than reliability. Consider the power of compounding in the negative: 95% reliability per-step over a 20-step task results in a ~36% average success rate. The harness hands the model an assignment it has no realistic chance of completing. This is where the gap is at its widest and the next 18 months are a reaction and attempt to close that gap.

3. Cursor/Copilot，“退回到人类在环中”（2023 - 2024）：第一批由 AI 驱动的 IDE 认识到了赋予模型过多自主权这一失败模式。它们通过将控制曲线拉到模型曲线之下，来缩小差距。不要直接把循环交给模型；把循环交给人类，并赋能人类来编排这个循环，而模型则加速人类的工作。Devin 的第一个版本试图把自主权重新交还给模型。Answer.AI 团队的一项测试表明，这仍然为时过早，成功率只有约 15%。这证明，从 IDE 退回到放弃完全自主并不是怯懦，而是正确的做法。然而，尽管当下主流策略是把控制层拉到模型之下，模型本身也在持续进步。到 2024 年末，随着 o1——首个推理模型——的推出，差距首次发生反转，我们开始看到模型能力过剩的最初迹象。

   Cursor/Copilot, “Retreat to Human in the Loop” (2023 - 2024): The first AI-powered IDEs recognize the failure-mode of giving the model too much autonomy. They close the gap by pulling the harness curve down below the model curve. Don’t give the model the loop directly; give the human the loop and empower the human to orchestrate the loop while the model speeds the human up. The first version of Devin tries to hand the autonomy back to the model. A test from the team at Answer.AI shows that is still premature, with a ~15% success rate. It’s evidence that the move from the IDEs to retreat from full autonomy is not cowardly, but the correct move. However, while the prevailing tactic is to pull the harness down below the model, models continue to improve. Near the end of 2024, with the introduction of o1 — the first reasoning model — for the first time the gap inverts and we begin to see the first signs of a model capability overhang.

4. Claude Code，《曲线交叉》（2025年2月）：2024年底的反转创造了一个必须有人抓住的机会：如果模型现在已经领先于脚手架，那么一个刻意踩着模型刹车的脚手架就把能力白白留在了桌面上。Claude Code 是首个为抓住这一机会而构建的编码代理。它抛弃了 IDE，转向终端，赋予模型 bash 以及文件读写权限，并用权限规则取代了对每次变更都需要人工批准的要求。模型再次被交回循环中，而这一次它明白了任务。Boris Cherny 和团队在构建 Claude Code 时，考虑的是下一代模型的能力，而不是当前这一代。它之所以大获成功，并不是因为它是第一个赋予模型自主性的产品，而是因为它是第一个在正确时机这样做的产品。那个时机就是交叉点：模型已经变得足够可靠，能够在自主状态下成功运行。Claude Code 在六个月内增长到约 10 亿美元的年经常性收入，全都因为 Anthropic 抓住了曲线开始相遇时可用的机会。

   Claude Code, “The Curves Cross” (February 2025): The inversion at the end of 2024 sets up an opportunity that someone has to seize: if the model is now ahead of the harness, then a harness intentionally riding the brakes of the model is leaving capability on the table. Claude Code is the first coding agent built to seize that opportunity. It abandons the IDE for the terminal, gives the model bash and file read/write access, and replaces the need for human approval on every change with permission rules. The model is handed the loop again, and this time it understands the assignment. Boris Cherny and team build Claude Code with the next model’s capabilities in mind, not the current one. It is such a hit not because it’s the first product to give the model autonomy, but because it’s the first product to do so at the right time. That time is the crossover point where the model has gotten reliable enough to succeed with autonomy. Claude Code grows to roughly $1B ARR within six months, all because Anthropic seized the opportunity available when the curves begin to meet.

接下来发生的是，这些曲线不只是相遇，而是开始交织在一起。What happens next is that the curves don’t just meet, they begin to braid together.

Harness 2.0：现在，“协同训练时代”Harness 2.0: The Present, “The Co-Training Era”

如今，评测框架很重要，而且这种重要性是可以衡量的。Harness-Bench 在不同的框架下对同一个模型执行了相同的 106 个任务，得分从 52.4 到 76.2 不等：模型本身没有任何变化，却出现了 23.8 分的差距。代理的一半就是框架。Today the harness matters, and in a way we can measure. Harness-Bench ran the same model over the same 106 tasks in different harnesses, and scores ranged from 52.4 to 76.2: a 23.8-point spread with zero change to the model. Half the agent is the harness.

OpenAI 通过对 harness 的更改在 ARC-AGI-3 上取得了类似的结果。仅添加保留推理和压缩后，GPT-5.6 Sol 的 ARC-AGI-3 得分从 13.3% 提升了三倍，达到 38.3%。OpenAI achieved a similar result on ARC-AGI-3 with harness changes. Adding only retained reasoning and compaction, GPT-5.6 Sol’s ARC-AGI-3 score tripled from 13.3% to 38.3%.

底层发生的情况是，强化学习（RL）已经被移到了 harness 内部。根据 OpenAI 在 2025 年 5 月发布的 codex-1 公告：“codex-1 是使用强化学习在各种环境中的真实编码任务上训练出来的。”What’s happening under the hood is that Reinforcement Learning (RL) has moved inside the harness. From OpenAI’s codex-1 release announcement in May 2025: “codex-1 was trained using reinforcement learning on real-world coding tasks in a variety of environments.”

这两条曲线汇合并开始作为一个统一系统交织在一起。The two curves join and start to braid as one unified system.

这就是 Toolformer 的梦想在现实中的体现。现在，工具调用不再是从外部提示出来的，而是在模型自身的环境中进行训练。This is the dream of Toolformer manifesting in reality. Rather than a tool prompted from the outside, now tool calling is trained from within the environment of the model.

然后，随着这些模型在 harness 的环境中进行训练，它们开始将 harness 的能力吸收到模型权重中，例如学习如何结合自身上下文窗口的知识进行自动压缩。Then, as the models are trained in the environment of the harness, they start to absorb the harness capabilities into the model weights, learning how to auto-compact with knowledge of their own context window, for example.

GPT-5.1-Codex-Max launch:

首个原生训练为通过压缩在多个上下文窗口之间运行的模型。

“The first model natively trained to operate across multiple context windows through compaction.”

一旦模型吸收了这个框架的能力，框架就可以卸下脚手架。这是一种通过精简实现的生产化。Anthropic 的 Thariq Shihipar 说，团队最近删除了 Claude Code 80% 的系统提示。Once the models absorb the harness capabilities, the harness can shed the scaffold. It’s production by reduction. Thariq Shihipar from Anthropic said that the team recently deleted 80% of Claude Code’s system prompt.

衡量 agent harness 演进速度的标准，是在保持相同能力水平的前提下，你能够删除多少 harness。这就是我们作为 AI 工程师需要努力构建的未来。The measure of the pace of agent harness evolution is how much of the harness you get to delete, while retaining the same capability level. This is the future we need to build towards as AI engineers.

那么，这就是模型/框架演进的循环：训练 -> 吸收 -> 脱落 -> 重复。模型攀升到下一个它还做不到的东西。This, then, is the loop of model / harness evolution: train -> absorb -> shed -> repeat. The model climbs to the next thing it can’t do yet.

Kaiser 指出的那个跃升很难准确界定，因为它不是一个离散事件。预训练阶段的跃升是显而易见的，因为你可以在模型卡中把它清楚地表述出来。模型 / harness 协同演化带来的跃升就没那么明显了，因为没有关于这种演化过程的文档。这就是去年冬天那次跃升的答案：它发生在模型和 harness 协同工作的过程中。The jump that Kaiser pointed out is hard to pin down because it’s not a discrete event. A pre-training leap is noticeable because you can articulate it in a model card. A model / harness co-evolution jump is less so, because there’s no documentation of the evolution process. That’s the answer to the jump last Winter: it happened in the space between the model and harness working together.

我们需要问：如果每一项 harness 能力最终都会被吸收到模型中，那我们还剩下什么？We need to ask: if every harness capability will eventually get absorbed into the model, what does that leave us with?

Harness 3.0：未来，“注意力时代”Harness 3.0: The Future, “The Attention Era”

不断删除模型能够吸收的一切。想象一下，在这个过程结束时你的代理会是什么样子。当你把一切都删掉后，你手里还剩下什么？Keep deleting everything that the model can absorb. Imagine what your agent looks like at the conclusion of that process. What are you left with in your hand when you’ve deleted everything?

模型权重接下来会吸收什么？多智能体编排、工具选择、记忆……这只是其中几种可能。研究人员正在构建自我改进的支架，这些支架本身也可以用类似于模型的方式进行训练。What do the model weights absorb next? Multi-agent orchestration, tool selection, memory...to name a few possibilities. Researchers are building self-improving harnesses that can themselves be trained in a similar way to models.

在这一删除和吸收过程结束后，剩下的是以人为中心的代理能力。诸如权限、身份、信任和可读性之类的东西。一个把权限吸收到自身中的模型，已经将权限溶解掉了。吸收并不会终结约束框架。吸收会颠倒约束框架。What’s left at the end of this deletion and absorption process are the human-centric agent capabilities. Things like permissions, identity, trust and legibility. A model that absorbs permissions into itself has dissolved permissions. Absorption doesn’t end the harness. Absorption inverts the harness.

该支架成为代理与操作它的人类之间的接口。The harness becomes the agent’s interface to the human that operates it.

这个支架最初是作为模型的人机接口而诞生的。我们从聊天框发展到 IDE，再到终端。如果模型吸收了面向计算机的能力，进化的下一阶段就会上升一个抽象层级。这个支架将成为模型与我们人类注意力之间的接口。The harness was born as the human interface to the model. We grew from chatbox to IDE to the terminal. If the model absorbs the computer-facing capabilities, the next stage of evolution becomes one layer of abstraction up. The harness becomes the model’s interface to our human attention.

它变成了注意力接口。It becomes the attention-interface.

Ryan Lopopolo 在 Latent Space 的《Token 亿万富翁的极限 Harness 工程》一集中表示：Ryan Lopopolo said on the “Extreme Harness Engineering for Token Billionaires” episode of Latent Space:

唯一从根本上稀缺的东西，是我团队的同步人类注意力。

“The only fundamentally scarce thing is the synchronous human attention of my team.”

代币变得充足且可靠，但我们仍然受限于稀缺的人类注意力。Tokens became abundant and reliable, yet we remain bottlenecked on scarce human attention.

我们已经能看到这一点的雏形，比如 Anthropic 长期运行的代理进度文件和代理式审批队列。We see sparks of this already, with Anthropic’s long-running agent progress files and agentic approval queues.

当模型吸收了 harness 之后，模型曲线与 harness 曲线之间的差距并不会消失。它会跨越人类边界迁移，并形成一对带有新差距的新曲线。这个新差距，是代理向人类提出的问题与人类能够回答的内容之间的空间。The gap between the model and harness curve doesn’t disappear when the model absorbs the harness. It migrates across the human boundary and creates a new pair of curves with a new gap. The new gap is the space between what the agent asks of the human, and what the human is able to answer.

注意力接口The Attention-Interface

我预测，在一年之内，每一家构建 agentic AI 的公司都会像每一家 agentic AI 公司发布 AGENTS.md 一样，推出一个人类注意力策略界面。I predict that within a year, every company building agentic AI will ship a human attention policy surface in the way that every agentic AI company shipped AGENTS.md.

AGENTS.md 会告诉代理如何与你的代码库协作。attention-interface 会告诉代理如何与你协作。它将规定何时允许打断你、何时应继续工作、哪些决策它可以自行做出，以及哪些决策需要你的批准。并且像代理系统中的其他一切一样，它会成为系统中一个可学习的组件，能够随着更多数据不断学习。每一次纠正都会变成有用的数据。AGENTS.md tells the agent how to work with your codebase. The attention-interface will tell the agent how to work with you. It will govern when it’s allowed to interrupt you, when it should keep working, which decisions it can make alone and which decisions need your approval. And like everything else in the agentic system, it will become a learnable component of the system that can learn with more data. Every correction becomes useful data.

这个模型最初只是一个泡在缸里的大脑。支架给了大脑一个身体，随后这个身体开始溶解进大脑里。留给我们去构建的，是那种未来任何模型都永远无法吸收的东西：通向唯一真正稀缺资源的接口——人类注意力。The model began as a brain in a vat. The harness gave the brain a body, then the body started to dissolve into the brain. What’s left for us to build is the thing no future model will ever absorb. The interface to the one true scarce resource: human attention.