The Pulse：科技公司转向开放 AI 模型

你好，我是 Gergely。本期是《务实工程师》通讯的额外免费特刊。在每一期中，我都会从资深工程师和工程领导者的视角，介绍大型科技公司和初创公司的相关动态。今天，我们将介绍上周《脉搏》一期中的五个主题之一。完整订阅用户已于七天前收到下面这篇文章。如果这封邮件是他人转发给你的，可以在这里订阅。Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of five topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

五月，我曾介绍过一种新兴趋势：企业希望削减 AI 支出，并从工程部门开始。人们正在尝试不同的方法：In May, I covered an emerging trend of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:

• 尝试在推理服务提供商上运行更便宜的开放模型
  Experimentation with running cheaper, open models on inference providers
• 加大对模型路由的投入，将更简单的请求路由到成本更低的模型。
  More investment in model routing to route simpler requests to cheaper models
• 关于如何高效利用 AI 模型的知识分享会
  Knowledge-sharing sessions on how to use AI models cost-effectively
• 设置每位开发者的每月 AI 使用限额
  Setting per-developer monthly AI usage limits

几个月后，据我接触过的消息人士透露，似乎已有几家公司成功实现了这一点。A few months later, it seems that several companies have managed to achieve this, according to sources I’ve spoken with.

Uber：AI 成本下降 50%Uber: AI costs down 50%

Uber 在今年头三个月就花光了全年 AI 预算，因此，5 月份听到 Uber 首席运营官表示，在看不到顶尖模型带来收益的情况下，越来越难以证明在 Claude Code 等工具上的投入是合理的，也就不足为奇了。没过多久，这家网约车巨头的工程团队便开始着手研究如何优化 AI 支出，而他们的努力并没有白费。Uber managed to blow through its annual AI budget in the first three months of this year, and it wasn’t a surprise to hear, in May, Uber’s COO say that it was getting harder to justify spending on tools like Claude Code without seeing benefits from the leading models. It wasn’t long until engineering teams at the ridesharing giant set to work on how to optimize AI spend, and their efforts weren’t in vain.

Uber 将每次 AI 请求的成本降低了 34%，将每次 AI 会话的成本降低了 52%：Uber cut the cost per AI request by 34%, and the cost per AI session by 52%:

当然，Uber 持续使用更多 AI 令牌并启动更多 AI 会话，但得益于优化，尽管使用量大幅增加，成本自 3 月以来一直持平：Of course, Uber keeps using more AI tokens and starting more AI sessions, but thanks to optimizations the cost has been flat since March, despite significantly more usage:

他们是如何在 Uber 做到这一点的？他们采用了多种不同方法的组合：How did they pull it off at Uber? It was via a combination of different approaches:

• 开放权重模型，通过推理运行：在推理服务上运行开放权重模型，其成本显著低于前沿模型。
  Open weight models, run using inference: run open weight models on inference services, which are significantly cheaper than frontier ones.
• 优化模型选择：对所有可用的前沿模型和开源模型进行基准测试，以准确了解它们当前的能力
  Optimized model selection: benchmark all available frontier and open models, to build an accurate picture of their present capabilities
• 持续进行基准测试：根据实际工作每周运行基准测试，并对其进行更新
  Ongoing benchmarking: run benchmarks every week based on real work, and update them
• 更便宜的子代理模型：子代理负责不需要使用最昂贵模型的小型任务
  Cheaper subagent models: subagents do smaller tasks not requiring the most expensive models
• 降低模型工作量：Uber 发现，对于高级模型，将默认工作量设为“中等”可获得最佳的成本产出比
  Reduce model effort: Uber found that defaulting to Medium effort gives the best cost-to-output ratio with advanced models
• 优化请求：当令牌数超过 400K 时触发自动压缩，即使对于拥有 1M 上下文窗口的模型也是如此
  Optimize requests: trigger automatic compaction above 400K tokens, even for models with 1M context windows
• 缓存提示词：在使用 Uber 自有的 Minions 工具时缓存提示词，以节省成本
  Cache prompts: cache prompts to save money when using Uber’s own harness, Minions
• ……以及更多内容：Uber 撰写了一篇工程博客文章，详细介绍了数十项优化措施，在不明显改变代理生成代码质量的情况下，降低了 token 成本。
  … and more: Uber wrote an engineering blog post detailing the dozens of optimizations taken to reduce token cost without noticeable change in the quality of code generated by agents

从外部来看，最大的一项收益似乎是 Uber 转而在某些任务中使用开放模型。与前沿模型相比，开放模型的成本低 2 到 20 倍：From the outside, the single biggest win seems to be Uber’s transition to using open models for certain tasks. Open models cost 2-20x less, compared to frontier ones:

Pinterest 通过弃用前沿模型节省了 90% 以上的成本Pinterest makes 90%+ cost savings by dropping frontier models

上个月 Pinterest 财报电话会议中的有趣细节揭示了这家社交媒体平台通过运行开放模型节省了多少成本。以下是 CEO William Ready 所说的话（着重部分为我所加）：Interesting details from Pinterest’s earnings call last month reveal how much the social media platform saves by running open models. Here’s what CEO William Ready said (emphasis mine):

我们对模型部署的方式包括：针对 Pinterest 特定使用场景打造的自有紧凑型专用模型，以及在安全的云基础设施中、于我们自有环境内完成后训练的适用开源模型。当我们采用开源模型（例如 Pinterest Assistant）时，与封闭式第三方模型相比，我们的使用场景取得了更优异的性能，因为我们能够利用高度独特的数据对开源模型进行后训练。借助开源模型，我们将每笔交易的成本控制在可比封闭式专有模型成本的 8% 以下。这为我们提供了充足的空间，能够随着时间推移进一步深化和拓展这些能力，以实现差异化、高效且具备成本效益的成果。

“Our approach to model deployment includes our own compact fit-for-purpose models built for Pinterest-specific use cases and suitable open source models post-trained in our own environment within our secure cloud infrastructure. When we leverage open source models, such as with Pinterest Assistant, we are seeing superior performance for our use cases when compared to closed third-party models because we are able to post-train open models on our highly unique data.

With open models, we are achieving cost per transaction at less than 8% of the cost of comparable closed proprietary models. This gives us substantial headroom to deepen and extend these capabilities over time in a way that is differentiated, highly effective, and cost efficient.”

基本上，过去 Pinterest 在封闭的前沿模型上运行某项任务需要花费 100 美元，而现在通过使用自有或租用推理基础设施上的开源模型，只需花费 8 美元！Basically, what used to cost Pinterest $100 to run on a closed, frontier model, they now spend $8 on by using open models on owned or rented inference!

AT&T：用开放模型替换 Claude，可节省 56% 亚洲欧美AT&T: 56% savings by swapping Claude for open models

AT&T 拥有 10 万名员工，是 AI 领域的大手笔投入者。这家电信巨头将工作负载转移到开放模型后，在 AI 输出质量仅下降 2% 的情况下，将 AI 账单削减了 56%。据《The Information》报道：With 100,000 employees, AT&T is a big spender on AI. The telco giant cut its AI bill by 56% while measuring a 2% decrease in the quality of AI’s output, after they moved workloads over to open models. From The Information:

奥斯汀表示，他发现开源模型“与 Anthropic 和 OpenAI 等公司出售的旧款模型一样出色，甚至更好”。例如，他说，AT&T 的软件开发人员仍然依赖尖端模型来完成生成代码等复杂任务，但对于生成已提交代码的摘要等不太繁重的任务，则可以使用成本更低的开源模型。他还表示，在公司开始使用路由服务提供商 LiteLLM 后，一些高级 AI 任务（例如编码）的成本最高下降了 56%，而 AI 的表现质量仅下降了 2%。

“Austin said he’s found that open source models are “just as good or better” than older models sold by the likes of Anthropic and OpenAI. For instance, AT&T’s software developers still rely on cutting-edge models for complex tasks like generating code, but can use cheaper open source models for less intense tasks like generating summaries of previously submitted code, he said.

After the company began using router provider LiteLLM, the costs of some advanced AI tasks such as coding fell by as much as 56% while the quality of the AI’s performance fell just 2%, Austin said.”

与市场上的其他产品相比，Anthropic 的定价是否过高？Anthropic overpriced compared to the rest of the market?

就在几个月前，Anthropic 还是工程师们首选的模型（Claude）和工具框架（Claude Code）提供商。但 Anthropic 的模型价格正在大幅上涨，而与此同时，开放权重模型——以及 OpenAI 的模型——正变得便宜得多。与此同时，Opus 5 的价格比 GPT-5.6 Luna xhigh 和 DeepSeek 等模型高出 100 倍（！！）。对一些科技公司来说，这可能实在高得无法忽视：Only a few months ago, Anthropic was the preferred model (Claude) and harness (Claude Code) among engineers. But Anthropic’s models are becoming steeply more expensive at a time when open weight models – and also OpenAI – are getting much cheaper. Meanwhile, Opus 5 is 100x more expensive (!!) than models like GPT-5.6 Luna xhigh and DeepSeek. That may be simply too much to ignore for some tech companies:

看到这些数据，我并不惊讶越来越多的科技公司希望在推理服务商的平台上运行开放权重模型提供商的模型，因为与 Anthropic 的模型能力相当的模型可以带来显著的成本节省。Seeing this data, I’m not surprised that more tech companies are looking to run open weight providers on inference providers, due to the significant savings available from a model that’s similarly capable as one from Anthropic.

Stripe、Coinbase、Uber 和 Ramp 采用的有效做法What worked for Stripe, Coinbase, Uber & Ramp

Databricks 的工程团队采访了 Stripe、Coinbase、Uber 和 Ramp 的工程师，并收集了不同方法如何帮助他们节省成本的信息。总结如下：The engineering team at Databricks interviewed engineers at Stripe, Coinbase, Uber, and Ramp, and collected how different approaches helped save costs for them. The summary:

要回答本报告标题中提出的问题，显然，使用开放模型确实是节省成本最多的方法，其次是智能模型路由。成本控制和上下文优化也能降低开支，但在效果上远不及前两种技术。To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results.

这篇文章发布一周后，Ramp 的 Ara Krahzian 确认，根据 Ramp 的数据，排名前 1% 的企业在 8 月的 AI 支出确实下降了 10%：A week after publishing this article, Ara Krahzian at Ramp has confirmed that AI spend in August, has, indeed, declined at the top 1% of businesses by 10%, based on Ramp data:

我敢打赌，那些公司并没有减少 token 的使用量，而是在按照上文所述的方式优化成本。I’d wager those companies are not spending fewer tokens, but they are optimizing cost, in ways outlined above.

阅读上周完整一期的《The Pulse》，或查看本周的《The Pulse》。本周刊涵盖：Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. CPU 短缺的新趋势：在 AI 公司引发 GPU 短缺和内存短缺之后，如今我们又正在经历 CPU 短缺，这要归因于 AI 智能体在使用工具时消耗了更多 CPU。如果你未来需要更多算力：趁现在还能获得，尽快 확보起来（即使这样做成本很高）。
   New trend of CPU shortages: after a GPU shortage and memory shortage driven by AI companies, we’re now experienceding a CPU shortage, thanks to AI agents using a lot more CPU with tool usage. If you will need more compute in the future: secure it now, while you can (even if it’s expensive to do so).
2. 更多新冠疫情时期独角兽的增长梦宣告破灭：Miro 以 13.5 亿美元的价格将自己出售给 Bending Spoons，而它在 2022 年的估值曾高达 170 亿美元。Airtable 上个月也遭遇了类似的估值下调，而如今看来，一批曾获风险投资支持、估值过高的公司正不顾一切地寻求出售。
   Growth dream ends for more COVID-era unicorns: Miro sold itself to Bending Spoons for $1.35B, after it was valued at $17B in 2022. Airtable saw a similar valuation cut last month, and it seems a batch of now-overvalued, VC-funded companies are desperate to sell.
3. 行业动态：Google 加班让 Borg 在 SpaceX 的数据中心运行起来；SpaceX 将 Claude Code 的令牌用量削减了 90%；OpenAI 发布 Astra；Meta 揭晓 Muse（马克·扎克伯格还在此次发布中提交了生产代码）；——马克·扎克伯格亲自为其作出了贡献；OpenAI 的智能体再次失控。
   Industry Pulse: Overtime at Google to get Borg working on SpaceX’s data centers; SpaceX cuts Claude Code tokens by 90%; OpenAI launches Astra; Meta unveils Muse (and Mark Zuckerberg pushed production code in this release); – to which Mark Zuckerberg made personal contributions; OpenAI’s agents go rogue, again.
4. 当 AI 处理事故时，工程师会不会逐渐失去应对能力？在航空业，飞行员每六个月会接触一次紧急情况，以保持敏锐的关键问题解决技能。在科技行业，我们可能也需要类似的机制，尤其是在 AI 将负责处理较简单的事故时。
   Do engineers lose touch when AI handles incidents? In the aviation industry, pilots are exposed to emergency situations every six months, to keep their critical problem solving skills sharp. In the tech industry, we might need something similar, especially if AI would take on handling of the simpler incidents.

订阅我的每周通讯，即可在收件箱中收到此类文章。内容相当不错，也是 Substack 上排名第一的软件工程通讯。Subscribe to my weekly newsletter to get articles like this in your inbox. It's a pretty good read - and the #1 software engineering newsletter on Substack.