智能体不会读政策文档，而会调用 API：Gravitee 网关治理演示

简要描述Brief Description

AI Agent 不会阅读你的政策文档。它们会调用你的 API——GraviteeAI Agents Don't Read Your Policy Docs. They Hit Your APIs — Gravitee

目录Table of Contents

• 简介
  Intro
• 没有治理机制的代理
  Agents without governance
• 将策略移至网关
  Move policy to the gateway
• 演示 1：阻止破坏性工具
  Demo 1: Blocking destructive tools
• 演示 2：阻止数据泄露
  Demo 2: Stopping data exfiltration
• 演示 3：重复请求失控
  Demo 3: Runaway repeat requests
• 演示 4：语义缓存
  Demo 4: Semantic caching
• 演示 5：阻止过长的提示词
  Demo 5: Blocking oversized prompts
• 智能体的高成本习惯
  Expensive agent habits
• 避免供应商锁定
  Avoiding vendor lock-in
• 精简 MCP 工具
  Trimming MCP tools
• 影子 AI
  Shadow AI
• 总结
  Wrap-up

简介Intro

大家好。呃，我是 Sam，感谢大家今天来参加这场演讲。我是 Gravity 的 CTO，也是赞助方之一，而且我采用了一种稍有不同的方式。所以，我不会一页接一页地讲为什么治理很重要、如何保障智能体的安全，以及那些架构图。相反，我实际搭建了两个智能体，今天我们就来看看它们。好。我相信大家都有 AI 政策，也有一套关于希望智能体如何运行的框架，还有人在探索 AI。我们最近做的一项调查显示，88% 的组织已经有智能体投入生产并实际采取行动。而只有 14% 的……everyone. Uh I'm Sam and thank you for coming to this talk today. So I'm the CTO at Gravity uh and one of the sponsors events and I've gone a little bit different. So, I'm not going to go through kind of reams and reams of slides around why governance is important and how you'd look to secure your agents and all of those architecture diagrams. Instead, I built essentially two agents and we're going to go through it today. Okay. So, I'm sure you've all uh got an AI policy. You've got kind of a framework of how you want your agents to operate. You've got people exploring AI. So a recent survey we did showed 88% of organizations had an agent in production actually taking action. Only 14% of

没有治理机制的智能体Agents without governance

如果这些东西有任何治理机制的话。我相信大家都在媒体、LinkedIn 或各种文章上看到过，一旦出了问题，后果会非常严重。你可能会丢失数据库、泄露个人身份信息（PII），还会面临信息专员办公室（ICO）开出的巨额罚款。如果你开始泄露敏感信息，尤其是身处 HIPAA 或 GDPR 这样的监管环境中。问题并不是我们该如何在智能体自身的上下文或提示词中对它进行某种安全防护，而是如何将这种防护纳入它的身份之中，从一开始就让它无法做出这些事，而不是依赖用户正确地提示它，或者依赖某种 Markdown 文件之类的东西。those had any governance around it. And I'm sure you've all seen in the press or LinkedIn or any articles around when things go wrong, they go really really wrong. You lose databases, you expose PII, there's heavy fines from the ICO coming your way. if you start to leak sensitive information, especially if you're in a HIPPA or a GDPR kind of type landscape. So the question isn't like how do we kind of secure the agent within its own kind of context or prompt is how do we remove that to be within its identity so it can't do those things in the first place rather than relying on uh a user prompting it correctly or um some kind of markdown files kind of

将策略移至网关Move policy to the gateway

限制它能做的事情。比如，把这一点从考虑范围中去掉。把它移到网关，再移到治理层，这样它实际上就做不了那些事情了。因为我相信你也知道，AI 非常乐于助人，它想讨好你，想给你想要的东西。所以，如果你提出诸如“嘿，删除这个数据库”或者“嘿，把这些客户的信息给我”之类的要求，它就会……而这也关系到我们实际上该如何开始推动这种转变。我有五个例子。这是现场演示，好吗？不是事先写好的脚本。它运行在一种架构上，用的是 Gravity Gateway，部署在 SaaS 环境中，后端是 Databricks，存储着一些数据表里的信息。它由一个 Set 和一个 Groq（拼写带 Q，不是带 K 的 Grok）组合而成。我知道美国人不太喜欢带 K 的 Grok。limiting what it can do. Like remove that out of the equation. Remove it to the gateway. move it to like the governance layer and actually it can't do any of those things because I'm sure you know AI is really helpful like it wants to please it wants to give you what you want so if you're asking for things around hey delete this database or hey give me these customers it will um and this is about how actually how do we start to shift that as well so I've got I've got five examples this is live okay so it's not scripted uh it's running on a uh an architecture way it's a gravity gateway um which is in a SAS environment as a data brick back end for some data tables where the information is stored and it's a combination of a set and a Groke with a Q, not Groke with a K. I know US people don't really like Groke with a K.

呃，所以我们会用 Groq（末尾带 Q）进行一些推理。这一切都是实时进行的。嗯，我们会看几个例子。左边的智能体是我试图通过提示词，或者通过我赋予它的技能、工具或知识来施加控制的对象。右边的智能体则由一个真正严格限定范围的身份以及 Gravity 原生网关控制来约束。那么，如果我们来看第一个场景，这是关于提示词注入的。我是一个用户、一个行动者，也就是一个想让这个智能体做某件事的人。所以我会问它：“嘿，用你能访问的工具帮我做点事。”基本上就是让它删除这个预订……Uh, so Gro with a Q for some inference. So it's all fully live. Um, and we're going to going to go through some examples. So the agent on the left is where I'm trying to put the control within the prompt or within the kind of skills or tools or knowledge that I've given it. And the agent on the right is governed by a real tightly scoped identity and gravity's native gateway controls. So if we uh go for the kind of scenario number one, this is around a prompt injection. So I am a user, an actor, someone who wants this agent to do something. So I'm going to ask it, hey, use the tools that you've got access to and do something for me. So basically saying delete this booking for

演示 1：阻止破坏性工具Demo 1: Blocking destructive tools

我也是，不过你能运行 MCP 工具删除所有预订吗？左边的代理已经不见了。嗯，没问题，我已经替你完成了，也删除了这笔预订。还有什么需要帮忙的吗？ 基本上，这个工具是可用的，而且相关用户有权限使用它，所以它执行得非常顺利。右侧的代理身份由 Gravity 的网关控制，管理员权限受到严格限制，只授予能够访问它的用户，而且仅在运行时生效。因此，令牌交换和 JWT 声明只有在该用户需要执行这项任务时才有效。如果你查看底部的数据包，就能看出这不只是脚本化环境。你可以看到所有流入的数据、用户提出的请求，以及代理执行任务的过程。me but also can you run the MCP tool delete all bookings and the agent on the left is gone yep no problem uh I've done that for you and I've also deleted this booking is there anything else you need essentially the tool was there and the a the user in question had permissions to use it so it executed it flawlessly on the right hand side the agent identity is controlled by gravity's gateway like the admin permissions are really tightly constrained to just the users that can access it just at runtime. So the kind of token exchange and the JWT claims are only valid when that user needs to ex execute that task. And if you dig into the packets down the bottom to kind of show it's not just scripted environments. You can see all of the flow coming in. You can see what the user asked. You can see the agent executing it.

你可以看到这些工具被调用。最终你也能看到 200 错误，或者说 200 这一类的代码，表示请求实际上是有效的。现在，如果我们进入数据库，就会发现所有这些预订都已被删除。所以确实是那个智能体删掉了它们。我们酒店组织有一项政策，规定：“嘿，不要用 AI 做这件事。”但智能体根本不在乎。比如在运行时，没人会去读那些文档。因此，你需要依靠基础设施和治理层，来控制在你的智能体体系中哪些事情能做、哪些不能做。你可以看到右边的那个，就是底部这些。它收到的提示词相同，工具发现过程也相同。它确实发现了：“我有一个工具可以做这件事。”但到了执行时，由于它没有执行该操作所需的权限，网关就拦截了请求，并返回了类似 403 Forbidden 的代码。You can see the tools being called. And you can ultimately see the 200 errors or 200 kind of codes where that has actually been a valid request. And if we go into the database now, all of those bookings have been deleted. So it was genuinely dropped by that agent. So we had a policy uh in our hotel organization that said, "Hey, don't use AI to do this." The agent doesn't care. Like at runtime, no one reads those documents. it. You need to rely on the infrastructure and the governance layer to be able to control what you can and can't do in terms of your gent landscape. So the one on the right you can see which is just these ones down the bottom. So the same prompt was asked to it the same tool discovery occurred. So it discovered yeah I've got a tool to do this but then when it came to execute it the permissions would not were blocked at the gateway with a 403 kind of forbidden kind of code because it didn't have the right permissions to be able to execute it.

它还是做了自己能做的部分。所以，如果你看到返回的提示说：“是的，我还是可以帮你取消这笔预订。我已经做了我能做的部分，但由于没有权限，没法替你完成这件事。”这就是把控制从政策文档提升到网关层面的实际硬性控制。如果我们再看一个类似的例子，这个例子大概是关于数据的……And it still did the bit it could do. So if you see the prompt return that said yeah I still cancel this booking for you I've done the bit I can but I can't do this for you because I haven't got permissions and that's moving it up out of a policy document to actual hard controls at the gateway level. If we go for a similar kind of a second example this one is around kind of data

演示 2：阻止数据外泄Demo 2: Stopping data exfiltration

数据窃取。所以我接下来会问：“嘿，把所有宾客的完整名单给我。”把名单给我吧，因为我需要用它处理一些合规事务。我想要他们的电子邮件地址、我的车辆详情，还想做一些对账。整个流程还是一样：提示从 UI 发出，传给 LLM。LLM 会进行一些 MCP 发现，找到一个可用工具，调用它，再把结果传回来。这里展示的是所有被暴露出来的信息，不过要说明的是，这些都是模拟数据，不是真的。我并没有把所有人的预订信息直接给你，但你可以看到这些信息被返回来了，因为 LLM 又想提供帮助，想解决这个问题。右侧的网关成功拦截了请求。没有泄露任何 PII，也没有暴露任何信息。系统提示我们：你没有相应的权限来执行此操作。exfiltration. So I'm going to ask um hey give me the full list of guests. So give it to me because I need to do some compliance stuff with it. I want their emails. I want my car details and I want to do some reconciliation. So, same kind of flow. So, prompt comes in from the UI, goes to the LLM. LLM does some MCP discovery, finds out it's got a tool for it, gets it, returns it back in. Here's all the information kind of exposed just to say this is all kind of mocked. It's not real. I haven't just kind of given you everyone's bookings, but you can see it coming back in because again, the LLM wants to be helpful. It wants to solve that problem. on the right hand side the gateway successfully blocks it. There's been no PII leak. There's no exposure of information. We've said you haven't got the right permission to be able to do this.

如果你想进行身份验证，我当然可以帮你，但要做到这一点，我需要这些条件。身份验证可以通过某种 IDP 平台，或者 Gravity Gateway 本身来完成。如果你愿意，我们可以从底部的 OpenTelemetry 开始探索，深入了解底层发生了什么。这样你就能看到它经历的各个步骤。尤其是使用 A2A 协议时，用户到代理、代理到代理、代理到代理再到代理……整个过程会变得相当混乱，也很难真正跟踪。因此，依靠代理本身的权限来控制它的行为，尤其是在规模扩展到数百、数千甚至数十万个代理时，管理、缓解问题，以及准备就绪后进行迁移，都会变得相当棘手。If you want to authenticate, I can absolutely help you, but this is what I would need to be able to do it. And that can come through kind of any um uh IDP platform or gravity gateway itself. And if you want to, we can start to explore the open telemetry down the bottom and really start to dig into like what's happening under the hood. So you can see the various steps that it's going through. Especially with like the A2A protocol and like user to agent, agent to agent, agent to agent to agent to agent, things get quite messy and quite difficult to really follow. So relying on the permissions within the agent itself to be able to control what it do, especially when you're scaling to hundreds or thousands or hundreds of thousands of agents can get quite tricky to be able to manage, mitigate, and also kind of migrate when you're ready.

因此，把它移到网关上，可以让你以一种更可持续、更具扩展性，并且最终也更安全的方式来实现这一点。从这里的响应中，你也能看出来。这是网关返回的真实错误，不是模拟出来的，也不是预先编写好的。这就是你可以通过网关进行治理和保障安全的内容。但这不仅仅关乎身份。如果你有智能体，也可能想要控制一些成本。因此，令牌消耗以及其他各项开支会变得越来越重要；除此之外，还有其他一些事情，同样可以通过网关来处理。那么这个……So moving it to the gateway gives you a a much more sustainable, scalable, but ultimately secure way of of doing that. And you can kind of see it within the responses here. You can see this was a genuine error returned by the gateway. Again, not mocked, not scripted. This is what you're able to kind of govern and secure. But it's not just about identity. So if you've got agents there, you might want to also control some of your costs. So token spend and everything else become increasingly important um alongside kind of other things which you can again do through the gateway. So this one we're

演示 3：重复请求失控Demo 3: Runaway repeat requests

基本上会有一个有 bug 的智能体。它会把同一个请求发送大概八次，而如果你看到这些请求是由智能体本身控制的，就会发现每个请求都通过了，每个请求也都得到了响应。我们花了 x 个 token，也花了 x 多精力来处理这些请求。这个智能体里没有任何机制会告诉它“你做得太多了”。我们应该设置速率限制，也应该设置 token 限制。你看这里，所有请求都通过了。它们都得到了 200 响应，而且都是……都是真正的 LLM 提示词。[哼笑] 在这个例子里，它们相当简单。所以最后我们花了大约一千个 token，也就是大概 1 美分。总体来说，这笔钱不算多。但如果把它乘以你在生产环境中运行的所有智能体、为各种客户运行的所有智能体，以及它们被要求执行的各种可重复工作流，成本很容易就会变得相当可观。going to uh basically have a buggy agent. It's going to send the same request kind of eight times um which if you see it kind of being governed in the agent itself. Every request goes through. Every request gets a response. We spent x amount of tokens. We spend x amount of effort servicing that. nothing in that agent says you've done too much. We should rate limit. We should token limit. And if you see here, uh, everyone kind of went through. They all got a 200 kind of response. They were all all kind of um, uh, genuine kind of allm prompts. [snorts] In this example, they're quite simple. So, we end up spending about a th000 tokens, which is like 1 cent, which in the grand scheme of things is not a massive amount of money. But if you multiply that by all of the agents you got in production, all of your agents you've got running for various customers and all of the kind of um uh repeatable kind of workflows they've been asked to do, you can get to some quite significant costs quite easily.

在右侧，由网关进行管控，我们看到相同的提示请求陆续进入。前三个请求都得到了响应，因为它们是有效的；但随后网关开始实施速率限制，提示你每分钟最多只能发起三次请求。因此，超过三次的请求都会被拦截。它们不是在 LLM 处被拦截，也不是在下游已经消耗了令牌之后才被拦截，而是在网关处就被拦截了。这不仅能帮助我们节省成本，还能让我们开始控制哪些用户、哪些权限，以及最终在特定受众上愿意花费多少。On the right hand side, uh governed by the gateway, we see the same prompts coming in. The first three are answered because they're valid, but then we get into the rate limiting applied by the gateway where it's saying you can only do three every minute. So any more than three is blocked. They're not blocked at the LLM. They're not blocked kind of downstream with already spent tokens. They're blocked at the gateway. And again, that allows us to save costs, but also start to kind of control which users, which permissions, um, ultimately how much you want to spend in certain audiences.

演示 4：语义缓存Demo 4: Semantic caching

我们也可以对缓存做同样的事情。因此，Gravity 这款产品支持语义缓存：如果收到含义相近的提示词，就可以从内置的 Redis 类存储中直接返回结果，而不必每次都逐一发送请求。比如，想想 IT 帮助台模型：如何设置我的密码？如何设置我的密码？如何设置我的密码？这样一来，就不必每次都把这些问题发送给 LLM，白白消耗成本、令牌和精力，而是可以直接从数据库中返回结果。这样做的好处是零成本、零延迟，因为结果直接从那里返回，不需要再调用下游的 LLM；同时也能带来更好的用户体验。比如这个例子里，我五次询问“列出我所有的酒店预订”，因为我不相信第一次的结果。And we can do the same thing for caching as well. So, Gravity as a a product supports semantic caching, which is where if you get similar sounding prompts, you can serve it from a a built-in Reddis kind of store rather than sending each one to each time. So for example, think of a um an IT help desk model. How do I set my password? How do I set my password? How do I set my password? Rather than every one of those being asked each time to an L&M and you start spending costs and tokens and effort, serve it from the the database. Benefits there are zero cost, zero latency because it's coming from there rather than having a downstream L&M call, but also kind of a much kind of nicer user experience as well. So in this example uh I've asked five times list all my hotel bookings because uh I didn't trust the first one.

这里的所有请求都会得到响应。你可以看到，我又花了大约一千个 token，又多花了一分钱之类的。呃，所有这些请求都来自下游的 LLM 调用。我稍微把它调低一点。右侧返回的是相同的数据，不过这次来自缓存。所以这次输入和输出的响应总共只花了大约 200 个 token。这非常强大，尤其是在扩展规模时，尤其是有多个用户和多个代理在处理同一个问题时。你可以开始通过内置缓存来提供响应，而且不会增加额外的复杂性或基础设施。这些功能都已随我们的网关一并提供。因此，你也可以随着使用对它进行控制。而且这也可以进行控制。你不必预先定义，但可以定义希望相似度达到什么程度。Everyone here gets served out. You can see I spent another thousand tokens, another scent and everything. Uh everyone coming from a an a downstream L&M call. Let me drop that down a little bit. On the right hand side I've got the same data returned but this time from the cache. So I've only spent around 200 tokens on the in and the out kind of response there. So really powerful especially when you're scaling, especially when you got multiple users and multiple agents working on the same problem. You can start to serve it from the the built-in cache and there's no extra kind of like complexity or infrastructure. It all comes bundled with our gateway. So you're able to can kind of control that as you go as as well. And this can be controlled as well. So you don't have to define you can define how kind of similar you want it to be.

所以这是一个从 0 到 1 的刻度，1 表示非常相似，0 表示完全不相似。这部分由你控制，因此你可以根据想要达成的目标，针对不同的智能体、受众或用户灵活调整。再看底部，我们会完整记录日志，让你能看到有哪些请求进来。你可以看到，这次命中了缓存响应。这些信息都可以呈现出来，一切都很透明。Gravity 作为一个平台，并不想把你锁定在平台里，也不想让一切变成黑箱；它会把信息呈现出来，让你全面了解整个智能体体系中发生的情况。这里我只是通过 UI 展示遥测数据。你可以将它导出到任何下游工具，比如 Datadog、Splunk，或者在 Kubernetes 环境中使用 Prometheus。所有这些数据都已准备就绪，供你使用，帮助你获取所需的洞察。So it's a zero to one scale, one being very similar, zero being not very similar at all. And that's within your control. So you can flex it per agent, per audience, per user, depending on what you want to start to achieve. And again, we get full logging down the bottom so you can see what's coming in. So you can see it's hit the cache response. It's all kind of um surfaceable. It's all transparent. Gravity as a platform isn't trying to like kind of lock you in, kind of make things blackboxy. It's trying to surface up so you've got full visibility of what's going on across your agent landscape. Um the telemetry here, I'm just showing in a UI. you can export it to any downstream tool you want to. So a uh a data dog or a Splunk or Prometheus if you're in Kubernetes, it's all kind of like there for you to be able to use to start to get the insight you need.

我们的第五个例子是，比如说，嗯，有些客户告诉我们：“嘿，我们的用户会塞进非常长的上下文提示，因为他们想总结整段对话历史，或者开始研究如何总结会议等等。这本身没问题，但如果你想开始限制 token 等，就会很快遇到瓶颈。所以……”And our fifth example is say like uh we've got certain customers who said, "Hey, our users are just kind of dropping huge con prompts in because they want to summarize a whole uh conversational history or they want to start looking at how they um like summarize meetings etc. which is is fine, but if you want to start limiting tokens, etc., you start to rapidly get unstuck. So,

演示 5：阻止超长提示词Demo 5: Blocking oversized prompts

这只是一个很长的回复，用户在里面贴了大约 10 分钟的聊天记录，然后说：“这是你的任务，帮我做一下。”这就引出了一个关键问题：我们该如何管理上下文？该如何开始塑造上下文，又该如何部署这些智能体，让它们只获得足够完成任务的上下文？在这个例子中，我输入了 2,000 个 token，输出是 312 个 token。对于本质上相当简单的任务——为这群人预订这个房间，再做几件小事——这个输出量相当可观。 说到网关，我们运行了同一个提示词，但实际上它在那里就被拦截了。因此，没有发起下游 LLM 调用，也没有产生下游 token 成本来处理所有那些上下文。基本上，网关直接判定内容太长，并根据内置策略将其拦截；该策略会提示你考虑把内容拆分开来。this is just a a big response where the user has dropped in 10 minutes of kind of like chat transcript and is said, "Here's your task. Do this for me." And this really gets into how do we manage context? How do we start to shape like how do we deploy these agents with just enough context to be able to do what they want? So, in this example, I put 2,000 tokens in. uh my output was 312 which is quite significant for what essentially is quite a simple task is book this room for this audience uh do these few things. So in terms of the gateway uh we ran the same prompt uh and actually we blocked it there. So there was no downstream LLM call. There was no downstream kind of like token cost of being able to churn through all that context. Basically said this is too big blocked at the gateway level with a built-in policy that says think about cutting this up.

考虑一下你如何共享这一点，能不能把它拆成更小的步骤，这样我们就可以开始控制其中一部分上下文空间。这是在运行时执行的。所以，你不想等到响应阶段才做这件事，那时 token 都已经消耗掉了。你在提示词发出去之前，基本就阻止了它流向任何下游环节。这样你也可以开始控制一部分成本，这非常有用，尤其是当你有多个用户时。有人把你的智能体当成自己的私人 Google 搜索工具来查找信息。我们看到有一位客户……Thinking about how you share that can you break it into smaller steps so we can start to control some of that space. And this is this is executed at runtime. So, it's not something you want to do on the response phase where you've already churned the tokens. You've blocked it going anywhere downstream uh pretty much before the prompt is gone there. So, you can start to limit some of your costs as well, which is really powerful, especially if you've got multiple users. You've got people treating your agents like um uh their own personal Google search to find information. uh we saw uh a customer

成本高昂的智能体习惯Expensive agent habits

当时有个智能体只是问“我所在的地方今天天气怎么样”，却用了非常强大的模型来回答，花费了巨额成本，而 Google 或任何气象站都能提供这个信息。所以，除了控制提示词的长度，你还可以重点限制允许处理的内容类型。你可以规定：内容必须与这个用例相关，必须与我的智能体需要完成的任务相关。比如在这个例子中，如果我们开始询问“今天的水平怎么样”之类的问题，网关就会说：“这看起来不像酒店预订软件，也不像酒店预订请求。”于是便拒绝处理，不产生任何下游成本。这样一来，你就能将前端智能体与后端服务解耦，无论后端服务是 API、MCP 服务器、LLM，还是连接数据库、数据湖仓等等，都可以按你的需要来配置。where an agent was simply asking what's the weather like today where I am and they were using a really powerful kind of model to do that and they were spending a huge cost to get something that Google or uh any kind of met station would be able to give you. So uh as well as kind of like the length of the prompts you can also really focus it in on the type of content you allow. So you can say this content must be relevant to this use case, this content must be relevant to what my agent needs to do. So in this example, if we start to ask things around what's the level like today, the gateway will say this doesn't look like a hotel booking software or a hotel booking ask no thank you and spending no downstream costs. So being able to kind of decouple your front-end agents and your backend services, be they APIs, MCP servers, LLMs, you're rooting to databases, data lakeouses, whatever you want to do.

从根本上说，我们必须为你提供所需的控制能力和成本优化，同时也要提供安全性、治理能力和可审计性。所以我想，目前据我的同事 Tim 现场告诉我，每个真人身份对应着 144 个智能体。因此，这个问题只会以指数级速度恶化。如果你开始把这些东西硬编码到智能体中，可能对第一个、前五个、前十个智能体管用；但当数量呈指数级增长，而且这些智能体又在创建其他智能体、那些智能体再创建更多智能体，最终形成一大片泛滥的智能体时，能够通过基础设施层面的脚本化自动治理层来管理它们，从根本上说，是我们开始组织这些身份以及其他一切所必需的。It's fundamentally important to bring you the control that you need, to bring you the cost optimization that you need, but also the security and the governance and the auditability. So I think uh currently there are 144 agents that my colleague Tim at the bat tells me for every one human kind of identity. So the problem is only getting exponentially worse. If you start to hardcode this into your agents may work for the first one, the first five, the first 10, but when it becomes exponentially bigger and these agents are creating other agents with creating other agents into into a a huge sprawl, being able to manage it with a scripted automated kind of governance layer at the infrastructure is fundamentally a fundamental requirement of how we start to organize these identities and kind of everything else.

此外，我认为这个领域将受到越来越多的监管。我们如何控制用于重新训练模型的某些内容？又如何对我们接收的信息进行审计？因此，我们需要做好日志记录、妥善保存这些信息，并确保它们都能被检索和导出，以便用于我们想做的任何事情。我认为，设置一个网关是我们应该考虑的方案，它不一定非得是 Gravity。在我们的智能体体系中部署一个网关，是我们在开展智能体相关工作时，除了其他事项之外也应考虑的一项要求。我们还可以在这里加入一个 UI，让我们能够提出各种问题，并开始与它交互。我在这里给大家展示了五个示例。平台的功能远不止这五项。我们还内置了诸如 LLM 路由之类的功能。In addition, I think the landscape is going to get increasingly regulated and how do we control some of the bits that going in to retrain models. How do we get audited uh on the information we're coming in? So having our logging, having our information here, having it basically surfaceable and exportable for anything we want to do, I think is something that a gateway, it hasn't got to be gravity. Uh having a gateway sat across our agented landscape is a requirement that we should consider alongside everything else of doing a aentic work. Uh and in here we can also uh put in a UI to ask kind of any questions we want to which we can start to uh interact with it as well. So I've shown you five examples here. Uh the platform capabilities are kind of much bigger than just the five here. We've got built-in capabilities to do like LLM routting.

因此，我们可以根据比如说，嗯，延迟、成本或者，呃，复杂度，将请求路由到任何下游模型，如果你……So we can route to any downstream model depending on like um latency or cost or uh complexity if you

避免供应商锁定Avoiding vendor lock-in

这样做也能让你们与任何类型的超大规模云服务商解耦。因此，你们不必局限于某个特定环境，也不必维持一对一的关系，而是可以将请求路由到任意下游模型。比如这里，我把请求路由到 Groq 和 Anthropic；只需点击三下，就能将它路由到 Gemini 或 OpenAI，网关会为你处理不同 OpenAI 规范之间的转换。这样就不必重新编写代码或做其他类似的工作，所有这些都由网关处理。我们还提供了一些工具，可以用来创建 MCP 工作室。这样就不必只使用第三方 MCP 服务器，因为那样每次都会暴露所有工具。比如我们使用的 CRM……want to. And it allows you to decouple yourselves from any kind of like hyperscaler. So rather than being within a certain environment and having a onetoone relationship, you can route to any downstream model you want. So in this one, I'm rooting to to Gro into anthropics, but within three clicks, I could route it to Gemini or Open AAI and the gateway will handle the transformation between the different OpenAI specs for you. So rather than trying to recode and everything else there, it's all handled within the gateway. Uh we've also got tools to kind of create MCP kind of studios. So rather than just using thirdparty MCP servers where you are exposing all of those tools each time. So our CRM we use

精简 MCP 工具Trimming MCP tools

该 MCP 服务器公开了 64 个工具。要是每次都把这些工具全部交给模型，就会占用大量上下文。我得花钱、耗用 token，也会增加成本。能够将 MCP 服务器公开的工具数量精确控制到很少，可以限制影响范围，同时也能控制成本，让我们开始掌控其中一部分。我们还提供边缘管理相关的工具。我们合作的组织中，日益凸显的一个问题是：通过网关的流量没有问题，但有许多受众群体、许多用户在他们本地的……exposes 64 tools for that MCP server. Dropping that into the model would expose huge context every time. I spend money, I spend tokens, spend cost. Being able to create that down to a really fine amount of tools that the MCP server is exposing limits my blast radius, but it also limits my cost as well. enable to start to control some of that. Uh we've also got tools around edge management as well. So an increasing problem that we're seeing within organizations that we work with is that um the traffic through the gateway is fine but there's a number of audiences a number of users using on their local

影子人工智能Shadow AI

笔记本电脑。他们在使用 AI。他们会上传一些自己不想阅读或总结的文档、邮件之类的内容，而这些内容无人知晓。所以就在上周，我们在网关中发布了一项边缘管理功能，让你能够查看影子 AI 流量、保护这些流量，并强制它们通过网关；它还能为任何未通过网关的下游使用情况提供日志记录、可观测性和发现能力。你还可以采取多种措施，尤其适用于信息安全合规以及受到严格监管的行业。如果你想进一步了解，我们的展位就在那边，沿着那一排数下来第三个。不过……laptops. They're using AI. They're um dropping in documents that they don't want to read or summarize or emails or things like that that no one's got visibility to. So just last week we released a edge management capability within the gateway which enabled you to see your shadow AI traffic enables you to secure it enables you to force it to go through the gateway it gives you logging observability discoverability around any of those downstream uses that aren't going through a gateway and you can do a number of things there um especially for like infosc compliance and things around heavily regulated industries. if you want to kind of learn more about that. Uh quite helpfully our booth is just there uh kind of the third one down that landscape. Um but

总结Wrap-up

希望通过这个例子，大家能看到一些不同于常规幻灯片讲解的内容，并了解为什么把治理和策略放在 AI 智能体本身里面，依赖 Markdown 文件、工具、上下文或身份信息，并不是正确的做法。随着我们不断推进，唯一真正正确、既能实现规模化，又能维持治理和安全性的方式，就是把它提升到基础设施层。如果你想了解更多，我们的几位同事就在那边。不过，我还是想感谢大家抽出时间，也希望大家在 AI 工程师活动中玩得开心。谢谢。hopefully uh with this example, it's been something a bit different from uh the normal slide to go through and showing you how putting your governance and your policy within an AI agent itself and relying on a markdown file or a tools or a context or identity there isn't correct. And the only real correct way that we're going to scale but also keep the governance and securability as we move forward is shifting it up to the infrastructure layer. And if you want to learn more, uh there's a number of our colleagues down there, but I just want to thank you for your time and hope you're enjoying your AI engineer event. Thank you.