LinkedIn 的上下文工程：如何用 MCP 为 AI 智能体构建组织级上下文层

文字记录Transcript

Ajay Prakash：我叫 Ajay Prakash，是 LinkedIn 的一名软件工程师。今天我将介绍 LinkedIn 的上下文工程。设想一下，你是一家软件公司的工程师，而且是一家大型软件公司的工程师，恰好轮到你为团队值班。你的团队负责维护一组非常关键的服务，这些服务被数百万用户使用。突然，你收到寻呼机警报，提示其中一项服务的延迟出现峰值，导致大量用户受到影响。在你努力弄清楚如何处理这个问题、如何更快地缓解问题时，你复制了该警报的链接，并将其交给一个编码代理，例如 GitHub Copilot。编码代理获取了相关说明，了解如何调试这一问题，或如何在公司内调试警报。它识别出，这个问题源自你们负责的一项服务。Ajay Prakash: My name is Ajay Prakash. I'm a software engineer at LinkedIn. I'll be talking about context engineering at LinkedIn. Imagine you're an engineer in a software company, a big software company, and you happen to be on call for your team. Your team owns a set of very critical services, which is used by millions of users. All of a sudden, you get a pager alert about a latency spike in one of your services. It is causing disruption for a lot of your users. While you're trying to figure out how to deal with this issue, and how to mitigate it faster, you take the link to that alert and give it to a coding agent, like GitHub Copilot. The coding agent, it fetches the instructions on how to figure out how to debug this issue, or debug alerts in your company. It identifies that this issue is coming from one of your services.

然后，它还会获取有关如何调试该服务的说明。根据这些说明，它会获取日志、指标以及最近的更改和已部署的内容。它发现问题源于另一个服务，也就是一个下游服务，而该服务正在产生大量错误。接着，编码代理还会获取有关如何调试下游服务中该问题的说明。它根据这些说明，以同样的方式获取日志和指标，并发现最近合并了一次有问题的 PR 代码更改，该更改已经部署上线，正是导致所有这些问题的原因。令人印象深刻的是，在你还在设法处理这个问题时，代理已经能够找出所有这些情况。获取到相关信息后，它还会为你创建一份详细摘要，说明它发现了什么、根本原因是什么，以及需要采取哪些步骤或行动来缓解该问题。Then it also fetches instructions on how to debug that service. Based on those instructions, it fetches the logs, metrics, and recent changes, and deployments that went in. It identifies that the issue is coming because of a different service, which is a downstream service, which is throwing a lot of errors. Then the coding agent also goes and fetches the instruction on how to debug the issue in the downstream service. It does the same thing based on the instructions on how to fetch the logs, metrics, and it identifies that there was a recent PR code change that went in, which was buggy, and it got deployed, and it is causing all of the issues. The impressive thing is the agent was able to figure out all these things, while you were figuring out how to deal with that issue. Once it finds the information, it also creates a detailed summary for you on what it found and what was the root cause, and what are the steps, or actions needed to be taken to mitigate the issue.

一旦你确认并完成验证，它还会继续采取措施来缓解问题。这一切只需几分钟即可完成。编码代理并不会止步于此。它还会创建一份详细报告，并为你更新事件管理系统。此外，它还会进一步获取有关如何对出现漏洞的受影响服务进行编码修复的指令，并为你创建一个 PR。正如我所说，整个过程只需几分钟即可完成。若没有编码代理，这项工作很容易就需要耗费数小时。这不是虚构的。LinkedIn 的团队正是这样使用编码代理的：把它们当作非常高效的同事，它们深入了解 LinkedIn 的系统，能够自动化工作流程并显著提升生产力。这只是一个例子。我们有 600 多个这样的工作流程，以及数千种帮助团队实现工作流程自动化的工具。这之所以能够实现，是因为我们构建了一个系统，并将其称为 Contextual Agent Playbooks and Tools。今天我将介绍我们为什么构建它、如何构建这一系统，以及我们在构建过程中和系统取得成功后学到了什么。Once you confirm that and verify, it will also go ahead and take the action to mitigate the issue. This happens in a matter of a few minutes. The coding agent doesn't stop there. It will also go and create a detailed report and updates the incident management system for you. Also, it goes beyond where it fetches instructions on how to make the coding on the affected service where the bug was happening, and it also creates a PR for you. As I said, it all happens in a matter of a few minutes. This would have easily taken, without coding agents, a few hours of time. This is not fiction. This is how teams at LinkedIn are using coding agents as very productive coworkers with a deep knowledge of LinkedIn systems to automate their workflows and be very productive. This is just one example. We have over 600 such workflows and thousands of tools helping teams automate their workflows. This is possible because of a system that we built, we are calling it Contextual Agent Playbooks and Tools. Today I'll be going over why we built it and how we built the system, and what did we learn by building the system, and from its success.

编码代理Coding Agents

要理解我们为什么构建这个系统，就必须回到 AI 编程助手早期的发展阶段。早期的编程助手主要是基于 LLM 的智能自动补全工具，会根据已有上下文帮助你补全一个句子、一行代码或一个代码块。随后，代理模式在各种编程工具中陆续推出。代理模式非常强大，因为如今由 AI 驱动的代理可以使用工具来编辑文件和运行终端命令。它们不仅能够找出应该如何进行修改，还可以直接执行这些修改。它们可以在你的工作区内编辑多个文件，这一点非常强大。2025 年初，Andrej Karpathy 将这种方式称为 vibe coding。其理念是，你不必查看代码，只需持续向代理提示你想要实现的目标，代理就会自行判断需要进行哪些修改，并完成这些修改。To understand why we built this system, we have to go back in time of the early days of AI coding assistants. The early coding assistants were mainly LLM based smart autocomplete, where it will help you autocomplete a sentence, or a line, or a code block based on the context it has. Then the agent mode was introduced across different coding tools. The agent mode was really powerful because the agent now with AI powered agents now have access to tools to edit the files and also run terminal commands. Now they could figure out how to make the changes and not only suggest those changes, they can also go ahead and make those changes. They can edit multiple files within your workspace. This was really powerful. In early 2025, Andrej Karpathy famously termed this vibe coding. The idea is you don't have to look at the code. You can just keep prompting the agent on what you want and the agent will figure out what changes to make and it will make the changes.

由于它可以访问终端，因此还会运行一些命令来验证自己所做的更改。你只需查看输出即可。这个想法非常强大，在整个行业引发了大量热议。和其他公司一样，即使是在 LinkedIn，也有很多人对使用这些编码代理感到兴奋。我们很早就向所有工程师提供了这些编码工具的访问权限。我们也希望每个人都能使用它，从而真正提高生产力。我们中的很多人都尝试使用编码代理，在 LinkedIn 的代码库中进行氛围编程，或实现自动化编码。但这并不奏效。氛围编程没有奏效。你能猜到为什么它没有奏效吗？主要问题在于，LinkedIn 的代码库非常成熟，而且包含大量内部上下文。这些代理并不了解应如何正确进行更改所需的上下文。Because it has access to the terminal, it will also run some commands to verify the changes it makes. You just have to watch the output. This idea was really powerful and it created a lot of hype around the industry. Just like any other company, even at LinkedIn, there was a lot of excitement to use these coding agents. We provided access to the coding tools early on to all the engineers. We also wanted everyone to use it and be really productive. A lot of us tried the coding agents to do vibe coding or to become automated coding in our LinkedIn codebase. It was not working. The vibe coding was not working. Can you guess why it was not working? The main issue was the LinkedIn's codebase is so mature and there is a lot of internal context. The agents didn't have the context of how to make the changes properly.

尽管这个想法确实非常强大，但当你在公司中尝试使用它时，智能体会遗漏大量上下文，要么只能生成质量欠佳的结果，要么完全产生幻觉，编写出错误的代码。工程师们不得不不断地向智能体提供正确的提示和上下文，引导它们朝着正确的方向工作。这与人们的期望完全相反：你本来是想提高生产力，但现在却得一直“照看”这个智能体，确保它做正确的事，并理解公司的代码库。许多工程师最终又回到了常规的手动编码，因为每个人都有截止期限。这并没有像我们预期的那样奏效。Even though the idea is really powerful, but when you try it in your company, the agents miss a lot of the context and either produce subpar results or it will completely hallucinate and create wrong code. The engineers had to constantly give the right prompts and give the right context to the agents to guide them in the right direction. This was opposite to what you hope for, because you want to be productive, but now you're babysitting this agent to do the right thing and understand your company's codebase. A lot of the engineers just went back to regular manual coding because everyone has deadlines to it. It was not working as we expected.

LinkedIn 的庞大技术栈LinkedIn's Massive Stack

为了更具体地理解这个问题，大家可以看看 LinkedIn 的技术栈，它的规模非常庞大。我们有数千个代码仓库、数千个相互依赖的微服务和应用程序。它们在每一层都使用了大量内部框架。此外，我们还构建了许多定制化的基础设施。例如，我们有自研的数据库、自定义的追踪与实验系统、可观测性系统以及配置管理系统，这些系统都非常成熟。它们的设计是为了应对 LinkedIn 这一规模所特有的挑战。 无论是谁加入 LinkedIn，工程师都需要参加为期一周以上的严格训练营，仅仅为了尝试理解这些内部系统并熟悉它们。他们还需要几周时间才能真正具备生产力，因为需要学习和正确完成的事情实在太多了。编码代理也面临同样的问题。你不可能把一个编码代理直接丢进这样庞大的系统中，然后指望它立刻高效工作。To understand this problem more concretely. If you take a look at the LinkedIn stack, it's massive. We have thousands of repos, thousands of microservices and apps, which depend on each other. They're all built using a lot of internal frameworks at every layer. Also, we have a lot of custom infra built. For example, we have our custom databases built. We have a custom tracking and experimentation system, observability, and configuration management, which are really mature. These are built to handle the unique challenges of LinkedIn scale. Engineers, whoever joins LinkedIn, they spend over one week through a rigorous bootcamp, just to try to understand the internal systems, get familiar with them. It also takes a few weeks for them to be really productive because there is so much to learn and do things right. The same problem is with the coding agents as well. You just cannot throw a coding agent at such a massive system and expect them to be productive.

我们曾问自己这样一个问题：如何让这些编码代理如此深入地理解 LinkedIn 的内部系统，从而能够交付出我们工程师可以信赖的代码？编码代理面临的问题之一是，工程师无法真正信任它们生成的代码，因为我们对代码质量以及系统可靠性都有非常高的要求。由于代理缺乏正确的上下文，我们确实无法信任它生成的代码。于是我们问自己：如何让这些代理生成的代码真正值得信赖？We asked this question to ourselves, how do we make these coding agents understand LinkedIn's internal systems so well that they can ship the code our engineers can trust? One of the problems with coding agents also was the engineers couldn't really trust the code that was getting produced because we have a really high quality bar for code and also the reliability of systems. Since the agents didn't have the right context, we cannot really trust the code that it was generating. We asked ourselves, how do we make the code that is generated by these agents really trustworthy?

模型上下文协议Model Context Protocol

大约在同一时期，Anthropic 发布并开源了模型上下文协议（Model Context Protocol），简称 MCP。它迅速流行起来，并被采纳为一项开放标准。它成为了连接工具与智能体的开放标准。所有编码智能体都开始支持 MCP。对我们来说，这是一个非常好的时机，因为现在我们可以连接编码智能体，并通过添加工具来扩展它们的能力，使它们能够访问我们的内部系统并理解我们的内部系统。 我们添加的第一组工具是代码搜索工具。我们当时已经有一个非常先进的代码搜索系统，工程师们一直在使用它。这个代码搜索引擎基本上会摄取 LinkedIn 内部 1,000 个代码仓库中的全部代码，使工程师能够使用关键词或正则表达式，以及文件类型和编程语言等高级筛选条件来搜索代码。现在，借助 MCP，我们将这个代码搜索工具封装起来，用于查询相关代码片段，同时获取相应文件的文件内容，并将其连接到编码智能体。Around the same time, Anthropic released and open-sourced Model Context Protocol, or MCP. It became really popular and also got adopted as an open standard. It became an open standard for connecting the tools to the agents. All of the coding agents started supporting MCP. This was a really good moment for us, because now we can connect the coding agents and extend their capability by adding the tools so that they can access our internal system and understand our internal system. The first set of tools that we added was code search. We had a very sophisticated code search system already existing, which engineers were using. This code search engine, it basically ingests all of the code within LinkedIn across 1,000 repos and allows the engineers to search for code using keywords or regex and advanced filters like file types and language. Now using MCP, we wrap this code search tool to query the code for relevant code snippets, and also get the file content of the file, and connected it to the coding agents.

这是一个非常重大的突破，因为从现在开始，代码代理不再只能依赖其预训练数据。要知道，所有这些 LLM 都只是基于开源代码进行训练的。它们非常擅长生成代码，但并不知道 LinkedIn 中的代码是什么样的。通过加入代码搜索能力，代码代理现在可以使用工具，通过执行查询并获取相关示例，来了解 LinkedIn 中的代码是什么样的。此外，如果有需要，它们还可以进一步读取整个文件。LLM，或者说代码代理，通常都非常擅长调用工具。即使用户给出的是自然语言查询，例如“给我展示一个如何使用 LangChain 的示例”，即便查询非常模糊且使用的是自然语言，代码代理也可以反复使用代码搜索工具，来判断应该使用哪些查询。This was a really huge unlock because now the coding agents not just have to rely on their pre-trained data. Because, remember, all of these LLMs are just trained on open-source code. They're really good at generating the code, but they don't know what the code in LinkedIn looks like. By adding code search capability, now the coding agents were able to use the tools to understand how the code looks like in LinkedIn by performing a query and getting the relevant examples. Also, if they need, they can also go and read the entire file. The LLMs, or the coding agents generally, they are really good at tool calling. The users, even if you give a natural language query like, show me how to do a LangChain example. Even if the query is really vague and in natural language, the coding agents can repeatedly use the code search tool to figure out what queries to use.

他们可以反复调用代码搜索，使用不同类型的查询，灵活查找相关上下文，然后对其进行总结。这一功能非常强大，解锁了许多可能性。在此基础上，我们也开始不断添加更多工具。例如，我们让编码代理能够搜索文档和 Wiki；此外，你还可以提供任何文档或 Wiki 的链接，他们能够阅读这些内容并将其纳入上下文。同样地，我们还添加了用于读取功能开关、任务管理系统和数据平台的工具。每添加一个工具，所解锁的能力就是：代理现在可以获取更好、更丰富的上下文来完成工作。例如，借助文档功能，你现在可以提供产品需求文档、架构文档，以及团队创建的一些 Wiki 页面，其中包含大量内部上下文。现在，编码代理也可以访问这些内容了。They can repeatedly call the code search with different kinds of things and improvise and find the relevant context, and then summarize that. This was really powerful and it unlocked a lot of possibilities. With that, we also started adding more and more tools. For example, we made it possible for the coding agents to search for the docs and wikis, and also, you can give any link to the doc or the wiki, they can read that and get that into the context. Similarly, we added tools to read the feature flags, task management, and data platform. Every tool that we added, what it unlocked was now the agents can get better and richer context to do their work. For example, using docs, you can now give product requirement documents and also architecture documents and some wiki pages, which were created by the team, which has a lot of internal context. Now the coding agents have access to it.

上下文缺口The Context Gaps

这一切都很好。现在，智能体能够获取所有上下文，但事情仍然不太奏效。尤其是当任务变得哪怕稍微复杂一些时，智能体也无法端到端地完成任务。你不能指望智能体自行弄清楚所有事情，并端到端地完成一个稍微复杂的任务。主要问题包括，首先是部落知识。要完成任何编码任务，不仅仅是找出相关的代码示例并进行编码。还需要弄清楚如何安装依赖项，以及编写用于编译或测试代码的命令。这些都是细微而关键的知识，散落在大量文档、Wiki 和 Slack 讨论串中。这就是部落知识，存在于某些资深工程师的脑海里。即使有了所有这些工具，编码智能体仍然无法找到相关的上下文。This is all good. Now the agents are able to get all the context, but still, it was not quite working. Especially if the task becomes even slightly complex, the agents were not able to do it end-to-end. You couldn't rely on agents to figure out everything and do a slightly complex task end-to-end. The main problems were, the first is the tribal knowledge. To do any coding task, it's not just about figuring out how to find the relevant code examples and just doing the coding. It also involves figuring out how to install the dependencies and write commands to compile the code or test the code. These are all the nuanced knowledge. These were scattered across a lot of docs and wikis and Slack threads. It's just tribal knowledge, which lives in some senior engineer's head. Even with all these tools, the coding agents were not able to find the relevant context.

这就像在大海捞针。这是一个问题。第二个问题是上下文过载。每当代理使用特定工具时，工具输出也会占用 LLM 上下文中的空间。所有 LLM 都有有限的上下文窗口。它们的上下文窗口越来越大，但仍然是有限的。当编码代理试图寻找执行某项任务所需的相关上下文时，会发生什么？它们使用的工具越多，就越容易被上下文信息压垮。最终，到了一定程度，它们会通过压缩来总结所有这些信息。一旦上下文窗口被压缩，就会丢失大量信息。然后，它们不得不再次执行所有工具调用，以找到相关上下文。这也是编码代理有时会陷入奇怪循环的原因之一：它们执行大量工具调用，弄清楚该如何做，但突然间，这些内容被总结了，它们会忘记自己正在做什么，然后又开始把整个过程重做一遍。It's like finding the needle in the haystack. This is one problem. The second problem is context overload. Whenever an agent uses a specific tool, the tool output also takes space in the LLMs context. All of the LLMs have a limited context window. They're getting bigger and bigger, but it is still limited. What happens when the coding agents try to look for the relevant context to do a certain task? The more tools they use, they get overloaded with the context. Finally, at a certain point, they summarize all of that information using compaction. Once the context window is compacted, it loses a lot of the information. Then it has to do all of the tool calls again to find that relevant context. This is one of the reasons sometimes coding agents get into a weird loop where they do a bunch of tool calls, figure out how to do things, but all of a sudden, they get summarized, and they will forget what they were doing, and they start to do that all over again.

这就是上下文过载问题。另一个大问题是，智能体没有长期记忆。如今，我们已经在编码智能体中看到了一些形式的记忆，但那时还处于早期阶段，并不存在持久的长期记忆。问题在于，即使智能体通过所有工具调用以及你手动提供的全部上下文弄清楚了这些事情，下次你想执行相同的任务时，智能体仍然必须把这一切重新弄清楚。这非常耗时，因为如果你是一名工程师，正在执行一项任务，你希望尽快完成它。如果智能体每次都从头开始，试图再次弄清楚所有这些事情，那么体验就不太好。此外，这也非常昂贵，因为会消耗大量 token。That's the context overload problem. Another big problem is the agent doesn't have a long-term memory. The coding agents, now we are seeing some form of memory, but these were the early days, and there was no durable long-term memory. The issue with this is even if the agent figures out by all of the tool calls and all of the context you give by hand, the next time you want to do the same task, the agent has to figure out all of these things all over again. This is really time-consuming, because if you're an engineer and you are doing a task, you want to get it really fast. If the agent is always starting from scratch and trying to figure out all of these things over again, it is not really a good experience. Also, this is very expensive because it takes up a lot of tokens.

程序性记忆 - 操作手册Procedural Memory - Playbooks

这一切的解决方案是什么？当然，正如我所说，解决方案就是提供记忆。我们如何让智能体记住如何做事的上下文？我们必须理解程序性记忆。程序性记忆就是关于如何做事的上下文。例如，智能体每次接到任务时，都会摸索出完成工作所需的全部上下文，并成功达到最终状态，那么你就可以将这些上下文——所有相关的上下文——存储为程序性记忆。当智能体下次想要执行同样的任务时，你就向它提供这份程序性记忆，包括所有步骤、背景信息和细微的知识，从而再次完成同样的任务。通过提供这种程序性记忆，如果你能高效地做到这一点，就可以让智能体记住相关内容，完成整个任务，而不必花费太多时间，也不会消耗大量令牌。What is the solution to all of this? Of course, as I said, the solution is to provide memory. How do we make the agents remember a context on how to do things? We have to understand procedural memory. Procedural memory is just the context on how to do things. If the agent, for example, whenever you give a task, it had figured out all of the context it needs to perform a job and it was able to get to the final state, you store that context, store all of the relevant context as a procedural memory. When the agent wants to do the same task next time, you serve that procedural memory on all the steps and all the background information and nuanced knowledge to the agents to do that same task again. By giving this procedural memory, if you're able to do this efficiently, then you can make the agents remember things and do the entire task without taking too much time and also without burning a lot of tokens.

这就是程序性记忆的全部概念。程序性记忆也是 Anthropic 开源的 agent skills 背后的概念。在 LinkedIn，我们也有类似的实现。当时 agent skills 甚至还不存在，所以我们没有使用它。我们开发了自己的程序性记忆概念，称之为 playbooks。 什么是 playbooks？还记得我们构建了一个 MCP 并连接了各种工具吗？我们使用同一个 MCP 来提供程序性记忆，并将其称为 playbooks。每当你想要执行某项特定任务时，我们都有一个对应的 playbook，其中包含名称、描述和指令。名称和描述会向编码代理提示该特定任务是什么。现在，编码代理可以像调用常规工具一样调用 playbooks。调用时，所有指令都会作为工具输出返回给编码代理。例如，有一个用于使用 Airflow 编写离线流水线的 playbook。That's the whole concept of procedural memory. This procedural memory is also the concept behind agent skills, which is open sourced by Anthropic. At LinkedIn, we had a similar implementation. We didn't use agent skills because it was not even a thing back then. We developed our own concept of procedural memory, we called it playbooks. What are playbooks? Remember, we built a MCP and connected the tools. We use the same MCP to serve the procedural memory. We call it playbooks. Whenever you want to do a certain task, we have a playbook for that, which consists of name, description, and instructions. The name and description give the hint to the coding agent on what that specific task is. Now the coding agents can invoke the playbooks as regular tools. When it invokes, all of the instructions are returned as a tool output back to the coding agent. For example, there is a playbook for writing offline pipelines using Airflow.

如果一名工程师提出请求：“你能帮我设置一个 Airflow 流水线吗？”那么编码代理会意识到，它可以通过 MCP 访问 Airflow 操作手册，并调用该手册来获取相关上下文。它还可以从用户那里了解需要具体执行哪些更改。它会利用所有这些上下文来完成任务。LinkedIn 的任何人都可以创建操作手册，并将其提交到中央代码仓库。这样，LinkedIn 的其他所有人都可以使用它。这是一个非常强大的概念，也使得共享提示词和上下文成为可能。If an engineer asks, can you set up an Airflow pipeline for me? Then the coding agent realizes that it has access to the Airflow playbook via MCP and it invokes that to get the relevant context. It can also get the context from the user on what specific changes it needs to do. It uses all of this context to perform the task. Anyone at LinkedIn can create a playbook and check it into a central repository. Now it is available for everyone else at LinkedIn. This is a really powerful concept, and it made it possible to share the prompts and share the context.

什么样的操作手册才算优秀？此外，每当我们创建操作手册时，都必须牢记以下两点。这些是我们努力贯彻的设计理念。 第一项理念是，操作手册应当自包含，也就是说，一个操作手册应该只完成一件明确且具体的事情。例如，如果它用于创建 Airflow，就应该包含所有相关概念、说明以及其他一切必要内容。如果某个操作手册要完成多项不同的任务，那么最好将其拆分成多个操作手册。代理可以自行决定选择使用哪个操作手册。 第二项设计理念是可组合性。这意味着，如果一个操作手册非常长，其中有些任务可以提取出来，那么最好创建更小的操作手册，并由较大的操作手册引用它们。这类似于代码复用：将代码拆分到多个位置。What does a good playbook look like? Also, whenever we are creating playbooks, we have to keep these two things in mind. These are the design philosophies we try to enforce. The first philosophy is the playbook should be self-contained, which means a playbook should do the exactly one specific thing. For example, if it is for creating the Airflow, it should have all of the concepts and instructions and everything. If there is a playbook which is doing multiple different things, it is better to break it up into multiple playbooks. The agent can always decide to choose which playbook to use. The second design philosophy is composability, which means if you have a very long playbook, and there are some tasks which can be extracted out, then it is better to create smaller playbooks, which can be referenced by the bigger playbooks. This is similar to how you do code reusability, where you break the code into multiple places.

这种可组合性有两个主要优势。第一个是复用。正如我所说的，就像代码一样，playbook 也可以复用，多个 playbook 可以引用较小的 playbook 来执行某项操作。第二个更大的优势是上下文的渐进式披露。我的意思是，当编码代理想要执行某项任务、获取相关上下文时，如果它们一次性加载所有上下文，就会造成上下文过载。这对代理的性能也不利。通过渐进式披露上下文，当一个大型 playbook 引用较小的 playbook，并以此类推时，代理只有在需要时才可以选择读取较小的 playbook。如果不需要，就可以跳过它们。这是一种高效的上下文管理方式。基于这两种理念，我们现在可以在 LinkedIn 内部创建组织级上下文：几乎可以将 LinkedIn 中的所有上下文捕获为一个由 playbook 构成的图，代理可以在其中导航，以了解如何在 LinkedIn 中执行各种操作，并完成运行时间越来越长的任务。There are two main advantages of this composability. The first one is reuse. As I said, just like code, the playbooks can also be reused, where multiple playbooks can reference the smaller playbooks to do a certain action. The second biggest advantage is the progressive disclosure of context. What I mean by that is, when the coding agents want to perform a task, want to get a context, if they load all of the context up in one go, it overloads the context. It's not also good for the performance of the agent. With progressive disclosure of context, when a big playbook references small playbooks, and so on, the agents can choose to read the smaller playbooks only if they need to. If they don't need to, then they can skip those. This is efficient management of context. With these two philosophies, we can now create an organizational context within LinkedIn, where you can almost capture all of the context within LinkedIn as a graph of playbooks, which the agent can now navigate to figure out how to do things at LinkedIn, and do longer and longer running tasks.

随着我们创建越来越多的操作手册，会出现一个常见问题。任何知识库都会过时，因为这些操作手册是在某个时间点创建的，当时记录了特定的操作说明。随着系统不断演进，这些操作手册中的说明也会过时。编码代理的一大优点是它们能够即兴应变，这意味着当它们使用操作手册时，如果发现其中存在过时的信息，不会就此停下，而是可以利用现有工具进行即兴处理，并找出正确的解决方案。一旦找到正确的解决方案，它们还可以识别出信息已经过时，并尝试查找最新信息；或者，它们也可以随时要求编码代理的用户提供相关信息。这同样适用于各种边缘情况。操作手册不必从第一天起就做到尽善尽美。As we are creating more and more playbooks, there is one common problem. With any knowledge base, it gets outdated, because these playbooks are created at some point in time, at that time you captured certain instructions. As the systems evolve, the instructions in those playbooks get outdated. Great thing about coding agents is they can improvise, which means when they use a playbook, and if they detect that there is outdated information, they don't just stop there, they can improvise and figure out what is the right solution using the tools that we have. Once they find the right solution, and also, they can find that the information is outdated, and they can try to find what is the latest information, or they can always ask the user of the coding agent to provide the relevant information. This also works for any edge cases. The playbook doesn't have to be really perfect from day one.

你可以从一套基本的指令开始，但总会存在边界情况，而代理非常擅长发现其中一些边界情况。在我们的系统提示中，我们鼓励编码代理始终记录这些经验、过时的信息或任何边界情况，并在会话结束时进行总结。然后，我们还会鼓励编码代理查看操作手册的代码，并记录它们掌握的所有信息，以便更新操作手册；现在，这些更新可以提交到代码库中。这就形成了一个非常好的飞轮：操作手册被使用得越多，代理就越能用新的经验对其进行更新。程序性记忆使用得越多，就会变得越来越好。You can start with a basic set of instructions, and there are always edge cases, and the agents, they are really good at finding some edge cases. We have in our system prompt, which encourages the coding agents to always capture these learnings, outdated information or any edge cases, and summarize that at the end of the session. Then we also encourage the coding agents to check out the code for the playbooks, and also capture all of their information to update the playbook, which can be now checked in. This creates a really good flywheel where more and more the playbooks are used, the agents, they can update it with the new learnings. The procedural memory, the more it is used, it becomes better and better.

架构Architecture

从架构角度来看，整体是什么样的？我们有一个本地 MCP 服务器。所有工具和操作手册都通过这个 MCP 服务器提供。操作手册分为两类：中心操作手册和本地操作手册。这只是上下文工程的另一个方面：某些操作手册仅在特定代码仓库中使用或需要。只有当用户在该代码仓库中工作时，才需要使用相应的操作手册。 由于我们使用的是本地 MCP 服务器，因此可以访问文件系统。MCP 服务器还知道自己正在处理哪个目录。当 MCP 服务器启动并确定其工作的工作区后，如果该工作区包含操作手册目录，它就可以加载该目录中的所有操作手册。团队的所有操作手册以及特定于代码仓库的操作手册都可以放在这里。它们也会与所在的代码仓库一起提交，这样团队就可以创建所有需要的操作手册，而不必担心系统负载过重。How does it all look like from an architecture point of view? We have one local MCP server. All of the tools and playbooks are served via this MCP server. We have two kinds of playbooks, central playbooks and local playbooks. This is just another context engineering aspect where certain playbooks are only used or only needed in a specific repository. Only when the user is working in that repository, you would need that playbook. Because we are using a local MCP server, we have access to the file system. The MCP server also has the awareness of which directory it is working in. When the MCP server starts, and it identifies which workspace it is working in, and if it has the playbooks directory, then it can load all of the playbooks in that directory. All of the team's playbooks and repository specific playbooks can go here. They also get checked in with the repository where they exist in, so that the teams can create all of their playbooks without having to worry about overloading the system.

接下来是中央 playbooks，这些 playbooks 适用于多个代码仓库。我们要确保只有适用于多个代码仓库，或甚至不依赖代码仓库的 playbooks 才放在这里。 然后是工具。所有工具都需要身份验证。我们有一个无缝的身份验证库，每个工具都会连接到我们的内部系统或外部系统，因此需要某种形式的身份验证。这个过程通过无缝身份验证来完成：用户首次使用工具时，如果系统识别出该工具需要身份验证，就会启动身份验证流程，通常是 OAuth，用户即可完成身份验证。身份验证层会捕获令牌，并将其存储在安全密钥链中。随后，工具执行会继续进行。下次调用该工具时，它可以获取相同的令牌、刷新令牌，并执行工具操作。Then we have the central playbooks, these are the playbooks which are applicable across multiple repositories. We want to make sure that only playbooks which are applicable across the repositories or even without repositories, they go in here. Then we have tools. All of the tools, they require authentication. We have a seamless authentication library where every tool, they connect to either our internal system or our external system, and they would need some kind of authentication. This happens using a seamless auth where whenever the user is using the tool for the first time, and it identifies that it requires authentication, it launches the authentication flow, generally OAuth, and the user can authenticate. The auth layer, it captures the tokens and stores it in a secure keychain. Then the tool execution continues. The next time the tool is called, it can fetch the same tokens, refresh the tokens and perform the tool execution.

这个 MCP 服务器也预装在所有 LinkedIn 笔记本电脑上。每当有更新时，它还会每小时自动更新一次。这意味着任何人都可以创建一个操作手册或工具，而一旦部署完成，其他所有人都能在一小时内使用它。我们创建的每个工具都必须经过信息安全审查。由于 MCP 还比较新，可能会引发大量安全问题，因此将所有内容集中在一个地方，有助于我们的团队确保每个工具都经过审计，并且每个工具都是安全的。This MCP server, it is also pre-installed on all LinkedIn laptops. It is also auto-updated every hour whenever there is an update. This means that anyone can create a playbook or a tool, and once it gets deployed, it is available for everyone else within one hour. Every tool that we create has to go through our InfoSec review. Because the MCP is fairly new, and it can open up a can of worms for a lot of security issues, having everything in one place helps our team to make sure every tool is audited and also every tool is secure.

你们有多少人会想知道，这个 MCP 服务器在性能下降之前究竟能容纳多少个工具？有人在思考工具数量的问题吗？我认为这其实是 MCP 本身的一个普遍问题，因为我说的是创建数千个工具。如果你熟悉 MCP，或者通常所说的工具，就会知道每个工具都会占用上下文空间；随着提供给编码代理的工具越来越多，它们会占用上下文窗口中的大量空间，并降低代理本身的性能。例如，如果你添加了很多工具，即使并不需要它们，也会导致代理更频繁地进行压缩。通常来说，工具数量超过 30 个后，代理就会变慢，其性能也会像我刚才说的那样下降。我们该如何将规模扩展到超过 30 个工具？当然，我们希望创建大量操作手册和工具，而这是一个核心系统。How many of you are wondering, how many tools can this MCP server possibly fit before it degrades the performance? Anyone wondering about how many tools? I think this is a general problem with MCP itself, where I'm talking about creating thousands of tools. If you are familiar with the MCP or in general tools, every tool takes up space in the context, and as more and more tools are given to the coding agent, it takes up a lot of space in their context window, and it degrades the performance of the agent itself. For example, if you add a lot of tools and even if you don't need them, it causes the agent to perform compactions more often. Generally, beyond 30 tools, it makes the agents slow down and also degrades their performance as I said. How do we scale this beyond 30 tools? Of course, we want to create a lot of playbooks and a lot of tools, and this is a central system.

我们的做法是：不将所有这些工具和操作手册直接暴露给 MCP，而是用三个工具替代它们。第一个工具用于搜索工具和操作手册。代理每当需要执行某项操作或查找某个操作手册时，都可以使用这个工具，首先根据关键词以及领域、工具执行的操作等标签进行搜索。这个搜索工具会返回相关的工具和操作手册结果，然后代理可以根据名称和描述决定使用哪个工具，接着获取该工具的 schema。这样做是为了获取输入参数、了解如何构造该工具的参数，之后便可以执行。通过这种方式，我们能够将工具和操作手册目录扩展到数千个工具和操作手册。那么，谁来维护所有这些工具、操作手册和服务器呢？因为每个人都在不断添加工具，也都在不断添加操作手册。What we do is, instead of exposing all of these tools and playbooks directly to the MCP, we replace them with three tools. The first one is search for the tools and playbooks. The agent, whenever it needs to perform an action or find a playbook, it can use this tool to first do the search based on keywords and also tags like domain and what action the tool does. This search tool will surface relevant results of the tools and playbooks, and then the agent can decide based on the name and description, which tool to use, and then it can get the schema of the tool. This is to get the input arguments, how to construct the arguments for that tool, and then it can execute. This way we are able to scale the tools and playbook catalogs to thousands of tools and playbooks. Who maintains all these tools and playbooks and server, because everyone keeps adding the tools and everyone keeps adding the playbooks?

如果每个人都只是不断添加工具，就应该有人负责维护它们。我们采用这样一种所有权模型：由一个中央团队负责维护 MCP 服务器本身，同时也维护一组核心工具。这样可以确保，如果 MCP 服务器或任何核心工具出现问题，我们负责值班的中央平台团队会提供帮助并修复问题。 此外，我们还采用开放贡献模式。正如我所说，任何人都可以进行贡献，当然需要经过信息安全审查；但无论哪个团队添加了工具或操作手册并将其提供给所有人，他们也应该负责维护这些工具和操作手册。对于任何未被使用或未得到维护的操作手册或工具，我们都会积极且果断地将其弃用。我们的总体目标是确保每个工具和所有内容都有明确的归属和管理，从而维护整个系统的健康。If everyone just keeps adding tools, there should be someone to maintain them. We follow this ownership model where there is a central team which is responsible for maintaining the MCP server itself, and they also maintain a set of core tools. This ensures that if there is any issue with the MCP server or any of the core tools, the central platform we have on call, they will help and fix any issues. Then we have an open contribution model where, as I said, anyone can go and contribute, of course with the InfoSec review, but whichever team adds the tools or playbooks and makes it available for everyone, they also should be maintaining those tools and playbooks. Any playbooks or tools which are not used or not maintained, we'll actively and aggressively deprecate them. The whole point is we want every tool and everything to be accounted for to maintain the health of the entire system.

我们有超过 8,000 名用户每天使用 MCP 服务器及其中的各种工具。这些用户不只是工程师。我们也向其他职能部门开放了编码工具或编码代理，例如产品经理、设计师和 TPM。他们会在工作流程中使用全部或部分工具。再说一次，编码代理已经不再只是用于编码了。你还可以用它们做更多事情，例如调试。很多人使用这些编码工具，是因为它们可以访问文档和 wiki；他们可以使用 MCP 中的工具来创建文档或撰写文档。 我们还研究了这些 playbook 主要用于哪些方面。我们有超过 600 个 playbook，用于自动化各种工作流程。我们发现，团队创建并使用这些 playbook 时，主要围绕以下五大主题。第一个主题很有意思，是问题的调试与调查。We have over 8K users who use the MCP server and any of the tools on a daily basis. This is not just engineers. We gave access to coding tools or coding agents to other functions as well, like product managers, designers, TPMs. They use all or some of the tools for their workflows. Again, the coding agents are not just for coding anymore. You can do a lot more things, for example, debugging. A lot of people use the coding tools, because it has access to docs and wiki, they can go and create docs or write the docs using the tools from the MCP. We also looked at what are these playbooks primarily used for. We have over 600 playbooks, which are automating the workflows. We realized that these are the top five broad themes which the teams are creating for the playbooks and using them. The first one is, interestingly, debugging and investigation of issues.

这与编码有关，但并不完全就是编码。我们意识到，现在所有团队都在将他们的运行手册转换为操作手册。当你为智能体提供一套良好的指令，告诉它们去哪里查找数据并展开调查时，它们非常擅长摄取所有这些数据，并从中找出隐藏在大海捞针般信息中的关键线索。这也是人类不喜欢做的事情之一。对于编码智能体来说，这是一个非常有趣的方面。 第二个方面当然就是整个操作手册的预期用途：编码。如果你看看 LinkedIn 这类大型公司的编码工作，其中有大量样板代码，因为我们有很多内部框架，还需要进行数据库配置。这些工作会占用大量样板代码编写，而这非常手动。只要提供正确的指令，智能体就能很好地为你创建这些样板代码，还能运行命令并验证配置是否正确完成。It is related to coding, but this is not exactly coding. What we realized is all of the teams, now they are converting their runbooks into playbooks. The agents, when you give them a good set of instructions on where to look for the data and go and investigate, they are really good at ingesting all of this data and figuring that needle in the haystack. This is one of the areas where humans also hate doing this. This is a really interesting aspect of the coding agents. The second one, of course, which was the intended purpose of this whole playbooks was the coding. If you look at coding in big companies like LinkedIn, it is a lot of boilerplate code, because we have a lot of internal frameworks and setting up the databases. It takes up a lot of boilerplate code, which is very manual. Given the right set of instructions, the agents can do a really good job of creating those boilerplate code for you and also running the commands and verifying that it is set up properly.

当你需要设置大量样板代码时，可以执行一个运行时间更长的任务。这样一来，工程师就能非常轻松地省去设置样板代码时所有枯燥且繁琐的工作，转而专注于真正有趣的事情：要构建什么产品、要进行哪些更改，以及如何更快地迭代。第三大使用场景则是代码清理和迁移。任何在大公司工作过的人都会知道，这是一个令人头疼的问题：迁移工作持续不断，而且并不有趣，但你必须承担这项成本。这正是编码代理能够真正提供帮助的地方：当你有大量迁移工作需要相关团队完成时，可以创建一个操作手册，然后让编码代理执行迁移并创建一个 PR。You can do a longer running task when you have a lot of boilerplate to go set up. This makes it really easy for engineers to cut out all of the boring stuff and manual stuff of setting up the boilerplate code and focus instead on really interesting things, which is, what products to build and what changes to make and how to iterate faster. Then the third top most use case is also code cleanup and migrations. If anyone is working in a big company, they will know that this is one of the pains, which is, there is constant migrations, and these are not fun, but you have to pay the tax. This is where coding agents can really help where you have a lot of migration that you want the dependent teams to do. You can create a playbook and you can ask the coding agents to go and do the migrations and create a PR.

由于迁移操作手册确实非常完善，我们实际上可以依赖编码代理来完成正确的工作。所有操作手册还需要包含良好的验证步骤，让代理能够自行检查其工作成果。然后，正如我所说，我们有大量定制化基础设施，需要投入很多管理工作。此外，我们还有许多流水线，例如离线流水线和 AI 训练流水线，而这些操作手册确实能为编码代理提供很大帮助。这些都是编码代理可以协助完成的任务：无需工程师亲自管理这些流水线、获取日志并尝试调试，或管理一些长时间运行的任务；你可以将这些工作委派给编码代理，并通过操作手册向它们提供指令，它们就能完成这些工作。第五项是环境和代码仓库的设置。正如我所说，任何新加入的工程师现在都可以使用这些操作手册来设置本地开发环境，从而更快进入高效工作状态，而不必自己从 README 中摸索所有事项并逐步完成这一流程。Because the migration playbooks are really good, we can actually rely on the coding agents to do the right job. All of the playbooks also need to have good verification steps where the agent can go and verify its own work. Then, as I said, there is a lot of custom infra and it requires a lot of management. Also, we have a lot of pipelines, for example, offline pipelines and AI training, and the playbooks really help the coding agents. These are the tasks which the coding agents can help with, where instead of an engineer going and managing these pipelines and fetching the logs and trying to debug them or trying to manage some of the long-running tasks, you can delegate that to coding agents and give them instructions via playbooks, and they can do that job. The fifth one is environment and repo setup. As I said, any new engineer joins, now they can use these playbooks to set up their local development environment and get productive sooner instead of figuring out all of the things in the README and going through this process.

接下来是什么？What's Next?

接下来呢？我们现在已经有一个运行良好的系统。接下来我们要做什么？正如我之前提到的，到目前为止，工程师负责创建操作手册。当然，我们也有用于编写操作手册的操作手册，这意味着工程师并不是手动编写这些操作手册，而是利用编码代理来确定正确的上下文，并将其记录为操作手册。我们希望将这一过程自动化：让一个后台编码代理查看 PR 中的所有信息，以及代理会话和遥测数据，从而了解哪些常见工作流可以实现自动化，并由它们自行编写操作手册。这样，我们就能更快地扩展 LinkedIn 整个组织的上下文。 我们正在开展的另一项工作是：现在我们已经有很多操作手册。当然，我们有办法更新它们并使其保持最新，但这仍然是一个手动过程，需要有人记得或指示代理根据新的经验教训去更新操作手册。我们希望将这一过程自动化，让一个后台编码代理同样以离线任务的形式，查看所有代理会话以及遥测数据中的任何新发现，再检查操作手册本身，看看是否存在遗漏，然后自行更新相应的操作手册。What next? We have a good system which is working. What are we working on next? So far, as I mentioned, the engineers were responsible for creating the playbooks. Of course, we have playbooks to write playbooks, which means the engineers are not manually writing the playbooks. They are using the coding agents to figure out the right context and capturing that as a playbook. We want to automate that, where we can have a background coding agent, which looks at all of the information from the PRs and also the agent sessions and the telemetry to understand what are the common workflows that can be automated, and they can write the playbooks themselves. This allows us to scale the entire organizational context at LinkedIn much faster. Another thing we are working on is also, we have a lot of playbooks. Now, of course, we have a way to update them and keep them up to date, but it is still a manual process where somebody has to remember or direct the agent to go and update the playbooks with the new learnings. We want to automate that where there is a background coding agent, again, like an offline job, which looks at all of the agent sessions and any new learnings from the telemetry, and takes a look at the playbooks themselves, and see if there is any gaps, and it can go and make updates to that playbook.

关键要点Key Takeaways

最后，我想给大家留下两个关键要点。第一，仅仅把 AI 编码工具交给团队和公司的工程师，并期待他们因此提高生产力，是不够的。首先，从第一天起，你就需要思考如何围绕质量设置足够的防护措施，以及如何确保编码代理生成高质量的输出。这对你的公司非常重要。第二个要点是，它所产生的下游影响意味着，你需要确保编码代理生成的代码不会降低系统的可靠性。随着代码产出速度提升，这会引发一些下游影响，这是一个非常常见的问题。你需要从第一天起就思考如何维持质量和可靠性。第二点是，编码代理、支撑框架、模型以及所有其他组件都会不断更新。I'll leave you all with two key takeaways. The first one is, it's not enough to give AI coding tools to the engineers in your team and in your company and expect them to be productive. The first one is, you need to think from day one, how to put enough guardrails around the quality, how to ensure that the coding agents are creating quality output. This is really important in your company. The second one, the downstream effect of it is you need to make sure whatever the code your coding agents are generating, it's not degrading the reliability of your systems. This is a very common issue where, as the code velocity goes up, it will cause some downstream effects. You need to think from day one on how to maintain the quality and the reliability. The second one is, the coding agents and the harnesses, models, everything keeps getting updated.

有一件事不会改变：编码代理仍然需要访问你的内部上下文和“部落知识”，这一点不会改变。你需要搭建某种基础设施，让编码代理能够通过技能或像我们所做的那样的操作手册，轻松访问你的内部部落知识。你还需要为代理搭建某种基础设施，让它们能够在你的公司中找到方向；只有这样，它们才能取得成功。第三点是开发者体验。要推动编码代理的采用，你需要让使用编码代理的体验真正顺畅。否则，生产力提升会非常困难。问题在于，这些编码工具还很新，人们可能不会马上喜欢编码代理生成的代码，或者他们根本不想使用这些代码。作为一家公司，你需要持续投入，不断改善他们生成代码时的使用体验。One thing that doesn't change is the coding agents still need access to your internal context, your tribal knowledge, that doesn't change. You need to have some infra where you make it easy for the coding agents to access your internal tribal knowledge via skills or playbooks like we did. You need to have some kind of infra for the agents to navigate your company, only then they will be successful. The third one is the developer experience. To get the adoption of the coding agents, you need to make the experience of using the coding agents really seamless. Then the productivity is really hard. The problem is these coding tools are really new and people may not right away like the code that the coding agents generate or they don't want to use that. You need to put constant effort as a company to make that experience better for them to generate the code.

问题与答案Questions and Answers

参与者 1：你已经获得了过去几个月大量用户使用该操作手册的数据，目前有 8,000 人。你能否提供一些关于生产力和质量随时间变化的见解？Participant 1: You got the data point for a lot of users using the playbook over the past few months, right now 8,000 people. Can you provide some insights about the productivity and quality over time?

Ajay Prakash：他们不只是使用操作手册，很多人还在自行使用这些工具。目前有 8,000 名用户正在使用这些工具。正如我提到的，我们从第一天起就为质量设置了防护措施。质量并没有下降，这是一件非常好的事情。另外，我们还发现生产力也得以保持。生产力有所提升，大约提高了 20%。我们还会根据系统的可靠性来衡量生产力。系统的可靠性完全没有下降。Ajay Prakash: They're not just using the playbooks, but a lot of people are using the tools by themselves. There are 8K users who are using the tools. Over the quality, as I mentioned, we put the guardrails on quality from day one. It has not decreased, which is a really good thing. Also, what we realized is we also maintain the productivity. The productivity has gone up, about a 20% increase in productivity. We also measure the productivity against the reliability of our system. The reliability has not gone down at all.

参与者 2：当你搜索工具时，也就是先进行搜索，然后返回架构，接着实际执行，那么在进行搜索时，工具调用的哪些部分是相关的？只是名称和描述之类的信息，还是也包含了架构中的某些部分，或者其他内容？Participant 2: When you're doing the searching for the tools, so you're doing the searching, and then returning the schema and then actually executing, what pieces of the tool call were relevant when you were doing the search? Was it just like the name and description, or did you include like pieces of the schema or pieces of something else in there?

Ajay Prakash：代理如何根据我们在结果中呈现的内容来判断应使用哪个工具？首先是工具本身的名称和描述。这些信息足以让代理决定使用哪个工具。另外，我们还有一种称为简短描述的内容。我们同时提供详细描述和简短描述。我们发现，仅凭简短描述本身，编码代理就足以判断应使用哪个工具。Ajay Prakash: How does the agent figure out the right tool to use based on what we present in the results? The first one is the name and the description of the tool itself. It is enough for the agent to decide which tool to use. Also, we have something called a short description. We have a long-form description and a short description. What we realized is just the short description itself is enough for the coding agent to figure out which tool to use.

参与者 3：您提到代理能够识别过时的操作手册。具体来说，代理究竟如何识别某些内容已经过时，并根据基础设施或其他架构变更自动通知需要更新操作手册？您会如何实现这一点？Participant 3: You mentioned that agents would be able to identify outdated playbooks. How exactly is the agent able to identify that anything has gotten outdated and automatically notify that the playbook needs to be updated based on any infrastructure or any other architectural changes? How would you do that?

Ajay Prakash：智能体如何检测操作手册中的过时信息，并进行即兴处理？其工作方式是：即使没有操作手册，智能体也可以尝试推断该做什么。有了操作手册后，当它发现其中存在过时信息时，比如你提供的命令无法运行，它会先实际尝试执行该命令，但执行失败。然后，它会获取输出并进行推理：好吧，这是操作手册中给出的命令，但它无法运行，我需要尝试其他方法。当然，基于预训练所获得的知识，它们可以尝试不同的做法。此外，如果找不到相关信息，它们始终可以访问工具。例如，我可以去 Google Docs 中搜索正确的信息，或者使用代码搜索去查找正确的信息，然后进行即兴处理。Ajay Prakash: How do the agents detect the outdated information in the playbooks and how do they improvise? The way it works is, so even without the playbook, also the agents can try to figure out what to do. With the playbooks, when it finds any outdated information, suppose the command that you gave is not working, it actually first tries the command and it fails. It takes the output and reasons about it that, ok, so this is the command given in the playbook, but it is not working. I need to try something else. Of course, based on their pre-training, they can try different things. Also, if they cannot find it, then they always have access to the tools. For example, let me go and search in the Google Docs on the right information, or they can use the code search to go and look for the right information and they can improvise.

这些编码代理或像 Claude 这样的模型的特点是，它们不会那么轻易放弃。它们会不断尝试新的方法，直到耗尽所有选项。通常情况下，它们不会停下来，而是会一直继续尝试。我们还在代理中设置了这样的指令：如果它发现走进了死胡同，并且已经使用了大量令牌，它可以随时向用户寻求指引，请用户提供一些缺失的信息。然后，用户可以找到相关信息，并将它引导到正确的方向。这些就是让它们获得更新信息的机制。The thing about these coding agents or the models like Claude, they don't give up that easily. They keep going and trying new things until they exhaust all of the options. They generally don't, they just keep going. We also have those instructions in the agent that if it finds a dead end and it has used up a lot of tokens, they can always go and ask the user to direct them, provide some missing information. Then the user can find the information and point it in the right direction. These are the mechanisms to make them updated.

参与者 4：我想了解一下，贵公司控制可拥有的剧本数量的流程是什么，同时又该如何删除或防止创建重复的剧本？那么，系统是如何对此进行控制的？Participant 4: I wanted to know what is the process to control how many playbooks you can have in your company, but also how to remove or prevent duplication of playbooks that are created. What is the system then to control that?

Ajay Prakash：主要机制当然是，我们创建的每个 playbook 都必须经过代码审查。通常会由人工在合并之前进行审查。如果发现重复内容，就会被标记出来。此外，我们还为系统设置了自动化代码审查，并提供了相应的指令。例如，每当有人创建 PR 时，我们也会让负责审查的编码代理判断它是否重复、是否没有必要；如果是，它们就可以将这些问题指出来。还有一点，正如我提到的，编写 playbook 的并不是人工手动完成的，人们会使用编码代理来创建 playbook。我们还制定了关于如何编写 playbook 的最佳实践指令，并告知用户或工程师某个 playbook 可能没有必要或与当前情况无关。我认为这个 MCP 服务器的另一个重要方面是，我们还能够利用指标来衡量所有工具和 playbook 的使用情况。我们会查看每个 playbook 的指标，并将那些未被使用的内容弃用。Ajay Prakash: The main mechanism is, of course, every playbook that we create has to go through code review. Generally, a human will review before it gets merged. If there is any duplicate, it gets flagged. Also, we have automated code reviews for the system, where we have instructions. For example, whenever someone creates a PR, we also ask the coding agent which is doing the reviews to figure out if it is a duplicate or if it is not needed, then they can surface that. Also, another thing is like the coding, as I mentioned, the people are not doing the playbooks manually, they are using coding agents to create the playbooks. We also have instructions on what is the best practice of writing a playbook and also telling the user, or the engineer that this might not be needed or irrelevant. I think another important aspect of this MCP server is also we are able to use metrics to measure the usage of all the tools and playbooks. We go and look at the metrics for each of the playbooks, and we can deprecate the things that are not used.

 

查看更多带有文字稿的演示文稿See more presentations with transcripts