Token 成本只是小头：Jellyfish 的 Steve Jamison 谈 AI 编程投入

简要描述Brief Description

Token 才是便宜的部分——史蒂夫·贾米森，JellyfishTokens Are the Cheap Part — Steve Jamison， Jellyfish

目录Table of Contents

• AI 支出与合并 PR 的吞吐量同步变化
  AI spend and merged PR throughput move together
• 将开发人员薪资纳入成本模型
  Add developer salaries to the cost model
• 每个已合并拉取请求的分摊成本
  Allocated cost per merged pull request
• 比较消费较低与较高的群体
  Comparing lighter and heavier spending groups
• 将开发者与他们之前的几周进行比较
  Comparing developers with their own prior weeks
• 回滚可作为有限的质量代理指标
  Reverts as a limited quality proxy
• 工作量与代码行数
  Work size and lines of code
• 限制代币支出前需要询问的问题
  Questions to ask before capping token spend

AI 支出与合并 PR 的吞吐量同步变化AI spend and merged PR throughput move together

大家好，我是 Steve Jameson，Jellyfish 的 AI 研究与工程负责人。今天我想向大家说明，为什么 token 才是便宜的那部分。我接下来分享的研究基于 Jellyfish 平台的数据——这是业内规模最大的工程工作流数据集，并与 AI 遥测数据相结合。AI 成本不断攀升，已经不是什么秘密了。与去年同期相比，成本上涨了 60 倍。应用场景广泛多样，模型和运行框架也在不断进步，效果越来越好，其带来的影响令人振奋。但谈到预算和支出，话题就没那么有趣了。这些讨论涉及同行、高层领导和董事会成员。尤其是在缺少可靠数据的情况下，他们往往不是最容易沟通的对象。而且如今这可是真金白银。产出也在增长：在同样的一年时间里，工程师人均合并 PR 数量增长了 43%。Hi, I'm Steve Jameson, AI research and engineering leader at Jellyfish. I'm here to show you why tokens are the cheap part. The research that I'll present is drawn from the Jellyfish platform. The industry's largest set of engineering workflow data joined AI telemetry data. It's no secret that AI costs are climbing. They're up 60x from this time last year. The use cases are wide and varied. The models and harnesses are getting much better. The implications are exciting. But discussions around budget and spend are not as much fun. These conversations involve your peers and senior leadership and the board. Not always the easiest audiences, especially when you don't have the right data. And it's real money these days. Output is up, too. Merged PRs per engineer are up 43% over the same one-year horizon.

这简直不可思议。我们从未见过这样的生产力投入，尤其是在这么短的时间尺度上。大量研究和实际观察数据都表明，AI 和 AI 驱动的编码正在带来这些吞吐量提升。甚至我们的图表也显示了这种相关性。看看拐点出现在什么位置：2026 年初，AI 支出开始增加，吞吐量提升也随之开始持续增长。 不过，这两个数字之间存在一种矛盾。该如何理解它们？支出增长了 60 倍，吞吐量也提升了 43%。这算好还是坏？43% 是个很大的数字，但 60 倍也同样惊人。怎么知道投入是否足够？是否还有更多生产力提升，只待我们掏钱解锁？我们该把支出增加 50%？再翻一倍？又或者，怎么知道自己花得太多了？我相信你们中很多人都感受过这种两难。This is phenomenal. We've not seen productivity inputs like this, especially on this time scale. There's plenty of research and anecdotal data to know that AI and AI-driven coding are driving these throughput gains. Heck, even our charts show the correlation. Notice where the inflection points are. AI spend increased in early 2026 when throughput gains start a steady increase of their own. There's a tension with these two numbers, though. How do you reconcile them? Spend is up 60x. Throughput is also up 43%. Is that good? Bad? 43% is a big number. But so is 60x. How do you know if you're spending enough? Is there more productivity gain locked behind a checkbook? Should we increase spend 50%? 2x more? Or how do you know if you spent too much? I'm sure many of you have felt the tension here.

CFO 或董事会在要求提供这笔支出的凭据。你能给他们看看吗？合并的 PR 数量增长了 43%，这能让他们满意吗？另外，如果你限制用量和支出，会发生什么？你是在阻止大家把 token 白白烧掉，还是在阻碍效率进一步提升？你可以问问其他公司的同行，他们花了多少钱。不过，对许多公司来说，这仍然是个不断变化的目标。但你要如何将这些情况与自己的组织相提并论？显然，只比较他们的账单和你的账单是不够的。我们听说过……我们听说过有些公司在 AI 上花费 10 万、50 万，甚至数百万。他们设定 AI 预算，随后又随着现实逐渐明朗而重新调整预算。The CFO or board are asking for the receipts of the spend. Can you show them? Well, merge PRs are up 43% satisfy them? Also, what happens if you cap use and spend? Are you stopping from setting tokens on fire? Or are you stopping further efficiency gains? You can ask peers at other companies what they're paying. For many, it's still a moving target though. But, how do you equate that to your org? Surely, just comparing their bill to yours is inadequate. We've heard that reports of We've heard the reports of companies spending 100k, 500k, even millions on AI. Setting then resetting their AI budgets as reality sets in.

将开发人员薪资纳入成本模型。Add developer salaries to the cost model

归根结底，这两个数字讲不清全貌。即便把它们放在一起，也遗漏了成本中最大的一项。这两个数字都没有包含最大的成本：开发者薪资。AI 正以前所未有的规模提升开发者的生产力。它不只是另一种工具，而是智能。这种智能极大地改变了每位开发者的产出能力。因此，我们需要以不同的方式看待它的成本：不能把它当作一项额外开销，而应视为产出每个 PR 所需成本的一部分。我们把这称为 PR 的智能成本：你支付给工程师的薪酬，加上你为 AI 支付的费用。把它们合并到同一个成本项中。AI 约占产出另一个单位已合并代码成本的 2.5%。薪资占其余的 97.5%。这与其他公布的数据相符，而这些数据来自 2026 年 8 月。Ultimately, these two numbers can't tell the story. And even together, they miss the biggest line on the cost side. These two numbers leave out the biggest cost of all, developer salary. AI is making developers more productive on a scale we haven't seen before. It isn't just another tool, it's intelligence. This intelligence dramatically changes how much each dev can produce. That's why we need to think about its cost differently. Not off to the side as overhead, but it's part of what it takes to produce each PR. We're going to call this intelligence costs of a PR. What you pay your engineers plus what you pay for AI. Put them together in one cost bucket. AI is about 2 and 1/2% of the cost to produce another unit of merged code. Salary is the other 97 and 1/2%. This is in line with other reported figures, and this data is from August 2026.

在这里，令牌的成本很低。不过，这说的是当前的相对成本，并非美元金额。我们来看看过去一年合并一个 PR 的成本发生了什么变化。Here, tokens are the cheap part. But, that's current relative costs, not dollars. Let's look at what happened to the cost of a merged PR over the last year.

每个已合并拉取请求的分摊成本Allocated cost per merged pull request

主持人：[清了清嗓子]Host: [clears throat]

嘉宾：过去一年，每个已合并 PR 的成本下降了 26%。说的不是创建的 PR，而是合并的 PR。这里计入了审查时间，而且这些 PR 都通过了一定程度的质量门槛。这意味着什么？同期，每个已合并 PR 成本的下降，包含了 AI 成本增长 60 倍的影响。这种变化也几乎普遍存在：我们研究的公司中，83% 的每个 PR 成本都有所下降。这说明了什么？额外的吞吐量收益正在分摊到更多 PR 上，或者说，[清嗓] 薪资成本正被摊薄到更多 PR 上，而且这种情况发生的速度快于 AI 成本的增长。Guest: The dollar cost of a merged PR fell 26% over the last year. Not PRs created, but PRs merged. Review time is factored in, and some level of quality gates has been cleared. What That decline in cost per merged PR includes a 60x increase in AI costs over the same time frame. The change is nearly universal, too. 83% of the companies we studied showed a drop in cost per PR. What does this tell us? The additional throughput gains are being spread or are [clears throat] spreading salary thinner across PRs, and this is happening faster than the AI costs are increasing.

比较消费较少和消费较多的群体Comparing lighter and heavier spending groups

代币支出范围有多大？我们按代币使用量对公司进行排名，然后分成五分位组，也就是五等份。这里的范围相当大。支出最少的一组，AI 支出约占 PR 成本的 0.3%。这相当于每位开发者每月约 6000 万个代币。我们把 AI 人均支出最高的 20% 称为高支出组。在这一组中，AI 支出占 PR 成本的 8.5%。这相当于每位开发者每月约 25 亿个代币，比支出最少的一组高出约两个数量级。接下来的几张幻灯片中，我们会使用这些分组，看看代币支出如何影响其他指标。首先看 PR 吞吐量，以每位开发者合并的 PR 数量衡量，并按照上一张幻灯片中的相同 AI 支出五分位组进行划分。支出排名后五分之四的群体，PR 吞吐量都有所提升，增幅约为 40%，上下浮动。而高支出组的提升更大，增幅达到 67%。What's the range in token spend? We ranked companies by token usage, then grouped them into quintiles or fifths. The range here is quite large. The lightest group, about 0.3% of the cost of a PR is is AI. This represents about 60 million tokens per dev per month. The top 20% of AI spend per dev, we're going to call the heaviest spend group. There, 8 and 1/2% of the PR costs is AI. This represents about 2.5 billion tokens per dev per month. About two orders of magnitude more than the lightest group. We'll use these groups on the next slides to see how token spend influences other metrics. First, PR throughput measured as merged PRs per dev. Split by the same AI spend quintiles on the previous slide. The bottom 4/5 of spenders saw PR throughput gains of around 40% give or take. But the heavy spend group improved more at a 67% increase.

AI 支出越多，吞吐量提升就越大。现在我们来看看每个 PR 的成本。各方面再次都出现了积极结果。所有 AI 使用群体的每个 PR 成本都在下降。不过，和之前一样，支出最高的五分之一企业改善幅度最大：过去一年中，它们每个 PR 的成本下降了 32%。其他四组每个 PR 的成本则下降了约 25%。与投入较少的同行相比，加大 AI 投入带来了更明显的每个 PR 成本改善。 因此我们看到，和总体平均值的情况一样，每个 PR 成本的变化主要由 PR 吞吐量驱动，而非 AI 支出。重度支出者的 PR 吞吐量提升了 67%，而其他同行约为 40%。把相同的薪资成本分摊到当月更多的 PR 上，产生的影响更大，因为 token 才是成本低的部分。More AI spend led to more throughput gains. Now let's look at the cost per PR. Positive results are seen again across the board. Cost per PR is dropping for all AI usage groups. However, again, the heaviest fifth so the the companies that spent the most improved the most. They saw a cost per PR decrease of 32% over the last year. The other four groups saw about a 25% decrease in cost per PR. Investing in spending more on AI fueled more of an improvement in cost per PR compared to peers who leaned in less. So we saw that the pop- same same as we saw in the population average, cost per PR changes are driven largely by PR throughput, not by AI spend. The heavy spenders experienced PR throughput gains of 67% compared to about 40-ish percent for their peers. Spreading the same salary across more PRs in the month has more of an effect than the rising AI costs because tokens are the cheap part.

这并不是说组织应该把 token 白白烧掉、肆意花钱。这里我们看的是横轴上的 token 使用量，以及纵轴上的每个 PR 成本。起初曲线陡然下降：相对较少的 AI 支出一开始就能带来显著收益。随着 token 使用量增加，每个 PR 成本的降幅会逐渐减小。这就是边际收益递减。不过我想强调，边际收益递减并不等于没有收益。我们看到，排名前 10% 的组织使用的 token 是中位数水平的 8 倍，每个 PR 的成本低 27 25%。总有一天，token 不再是成本低廉的部分；但总的来说，对现实中大多数公司而言，我们离那个临界点还很远。更多 token 会带来更多 PR，进而降低交付成本。This is not to say that org should light tokens on fire and spend with abandon. Here we're looking at tokens used on the horizontal axis and cost per PR on the vertical axis. There's a steep drop at the beginning where relative small AI spend results in large gains initially. As token usage increases, the cost per PR improvement decreases. This is diminishing returns. I want to stress though, diminishing returns is not the same as no returns. We're seeing that the top 10% are spending 8x the tokens compared to the median and achieving a 27 25% lower cost per PR. There will be a time when tokens aren't the cheap part, but by and large for most companies out in the wild we're not close to that threshold. More tokens leads to more PRs, which leads to a lower cost of delivery.

将开发者与他们自己之前的几周进行比较Comparing developers with their own prior weeks

到目前为止，我们谈的都是公司。接下来我想岔开一下，谈谈我们对开发者开展的一项研究。我们比较了开发者大量使用 token 的那些周，与同一批开发者 token 使用量较低的周。So far we've talked about companies. Next I'd like to take a tangent and talk about a study we did with developers. Here we looked at developer weeks with heavy token use compared to the same developer weeks with lower token use.

主持人：[咳嗽] [清了清嗓子]Host: [cough] [clears throat]

嘉宾：在这里，我们看到同样的现象，只不过是在个人层面。比较每位开发者大量使用智能体技能的那些周与他们不使用技能的那些周。他们在 token 上的花费大约是原来的两倍，合并的 PR 多了 27%；把薪资和 token 成本合并计算后，每个 PR 的成本则降低了约 18%。谈到这项工作时，我常会遇到两个合理的疑问。第一，工作速度更快会不会导致质量变差？第二，我们看到的 PR 数量增加和吞吐量提升，是否是由更小的工作单元所推动的？Guest: Here we saw the same phenomenon just at the individual level. Compare each developers heavy agent skill weeks with their own weeks without skills. They spend about twice as much on tokens, merged 27% more PRs, and reach PR and each PR costs about 18% less once salary and tokens are counted together. There are two fair curiosities I get when I talk about this work. First, does faster work result in worse work? And second, are the more PRs the throughput gains that we're seeing, are they being driven by smaller chunks of work?

回滚作为有限的质量代理指标Reverts as a limited quality proxy

首先令人好奇的是：工作变得更便宜、更快，质量会变差吗？如果 AI 让 PR 更快、成本更低，我们是不是只是在提交更多错误？衡量这一点的一个指标是 river reverts，也就是撤销此前 PR 的已合并 PR。使用量最高的五分之一群体，在 token 用量大幅增长之前起点最低，而且所有群体的撤销率都略有下降。不过，这些变化幅度都略低于数据中的噪声水平，因此不能据此证明这里发生了改善。更重要的是，我们没有看到任何令人担忧的迹象，表明 token 用量更高、PR 交付速度更快会导致普遍或严重的质量问题。First curiosity is cheaper, faster work, worse work? If AI makes PRs faster and cheaper, are we just shipping more mistakes? One proxy for this is river reverts. Merged PRs that undo an earlier PR. The heaviest fifth started lowest before most of the token growth, and revert rates drop slightly for every group. Though these deltas are just slightly below the noise in the data. It's not evidence of a improvement here. However, more importantly, we're not seeing any worrying signs of rampant or severe quality issues with higher token spends and faster PR throughput.

工作量和代码行数Work size and lines of code

我的第二个疑问是：吞吐量的提升是否是由更小的 PR 带来的？事实证明，并非如此。PR 并没有变小。实际上，AI 正在让每个 PR 生成的代码行数增加。这对在场的开发者来说并不意外。如果你曾经对冗长、没必要的行内 AI 注释，或者满是大多无关测试的文件翻过白眼，请举手。生成式 AI 就喜欢生成代码。另外，开发者创建代码的门槛很低，而且大家更倾向于在每个 PR 里多做一些，而不是再开一个工单。只需一两个提示词，我就能处理这个边缘情况，或者把这个和那个元素合并起来。这样需要协调的 PR 就少了。不过，这和大家如今承受的 PR 审查压力并不相符。还有……My second curiosity is throughput driven by smaller PRs? And this is demonstrably no. Not smaller PRs. In fact, AI is driving an increase in line level lines of code generated per PR. This is no surprise to the devs in the room. Raise your hand if you've rolled your eyes at verbose, unnecessary inline AI comments, or a file full of mostly irrelevant tests. Gen AI Gen AI likes to generate code. Also, the bar to create is low for devs, and there's a pull to do more per PR instead of spinning up another ticket. It's just one, maybe two prompts, and I'll handle this edge case, or I can combine this and that element. Fewer PRs to wrangle. This doesn't mesh well with the PR review stress folks are feeling these days, though. There's

主持人：[清了清嗓子]Host: [clears throat]

嘉宾：此外，AI 正变得越来越强大，所以我们的目标也越来越宏大。深入探讨究竟是什么推动了代码行数变化的程度，超出了本次谈话的范围。不过，PR 吞吐量的提升并不是因为 PR 变小了。大多数人对此会觉得理所当然，但验证一下仍然很有好处。回到开头的问题：多花 60% 的钱，换来多 43% 的产出，值得吗？单独看这些数字，你无法判断。你应该同时跟踪两者。它们是重要的数据，但只是辅助指标。每个 PR 的 AI 花费，体现的是你购买的智能资源，用来帮助开发人员加快工作速度；每位工程师提交的 PR 数量，则体现了这笔花费带来的产出。不过，只看这两个指标会掩盖其中的权衡。Guest: Also, AI is just becoming more capable, so we're getting more ambitious. It is beyond this talk to dig into what's actually driving those level of lines of code changes. However, the increase in PR throughput is not coming from smaller PRs. This is intuitive for most, but it's still good to verify. Back to the opening question. Is 60% more spend worth 43% more output? You can't tell in isolation. You should track both. They are important numbers, but they're supporting metrics. AI spend per PR is the intelligence that you're buying as a resource to help make your devs go faster. PRs per engineer is what you're getting with that spend. However, looking at those hides the trade-off.

主持人：[清了清嗓子]Host: [clears throat]

嘉宾：不过，把薪资也纳入计算，再算一下成本与吞吐量的比值，答案就出来了。同一时期内，每个 PR 的成本下降了 26%。显然，尽管账单在增加，AI 仍在提高成本效率。而且，相对而言，为实现这些效率提升所需的 AI 支出很少。AI 在每个 PR 的成本中仍只占很小一部分，平均为 2.5%。对于高支出群体，这一比例也不到 10%。Token 才是便宜的那部分。Guest: At but put salary into the equation and take the ratio of cost to throughput, and there you get your answer. Cost per PR is down 26% over that same time period. It's clear AI is driving cost efficiencies despite the increasing bills. And the AI the AI spend needed to get those efficiencies is small, relatively speaking. AI is still a sliver of what a PR costs. Two and a half percent on average. Under 10% for the heavy spender group. Tokens are the cheap part.

主持人：[清了清嗓子]Host: [clears throat]

嘉宾：在我们观察到的范围内，增加支出持续带来了回报。重度使用令牌的用户，PR 吞吐量提升了 67%，而支出较低的同行提升了 40%。这使每个 PR 的成本下降了 32%，而支出较低的同行下降了约 25%。从 PR 成本与令牌用量的比值曲线来看，按令牌用量排名前 10% 的公司月度数据，令牌用量是中位数的 10.8 倍，但每个 PR 的成本低了 25%。AI 正在让开发者的生产力提升到前所未有的水平。它不只是又一个工具，而是智能能力，改变了每位开发者的产出。因此，我们需要以不同的方式看待它的成本：不要把它当作额外的管理开销，而要视为产出每个 PR 所需成本的一部分。你需要把 AI 支出和人工支出，也就是员工薪资成本，加在一起，才能算出每个 PR 的真实成本。Guest: In the range that we see, more spend kept paying. Heavier token users saw 67% PR throughput gain compared to 40% for their lower spend peers. This drove cost per PR down 32% compared to about 25% for the lower spend peers. Looking at the PR cost to token ratio curve, the top 10% of company months by token usage resulted in 10 8x the tokens, but 25% lower cost per PR compared to the median. AI is making developers more productive on a scale we haven't seen before. It isn't just another tool, it is intelligence, which changes how much each dev can produce. That's why we need to think about its cost differently, not off to the side as overhead, but part of what it takes to produce each PR. You need to add the AI spend and the human spend the human salary spend together to calculate your real costs per PR.

在限制令牌支出前要提出的问题Questions to ask before capping token spend

这并不是说可以挥霍无度。当然，你应该留意失控的任务，也应该跟踪 AI 支出。我不是说不要关注每个 PR 中投入了多少 AI、AI 支出有多少。我是说，别让它成为你的核心衡量指标。换一种思路，或许能引导你重新考虑其他投资该如何、何时进行，比如集成模型路由器、对自己的模型进行后训练、设置 token 限额等等。要提高效率，控制 AI 成本更有效，还是继续投资于能提升吞吐量的事情（比如进一步推动 AI 的应用）更有效？无论如何，每个月都要跟踪每个 PR 的智能总成本，也就是人力加上人工智能的成本，并将其与 AI 账单和 PR 吞吐量放在一起看。趋势如何？吞吐量是否在持续提升？成本效率呢——交付成本又如何？This isn't to say spend frivolously. Sure, you should watch for runaway jobs. You should track your AI spend. I'm not saying don't look at how much AI is going how much AI spend is going in each PR. I'm saying don't let that be your driving metric. A new framing might steer how and when you think about other investments, like integrating model routers, post-training your own model, implementing token limits, etc. Will you get more efficiencies out of trying to control AI costs or by continuing to invest in things that increase throughput, like more AI enablement? Either way, track the total intelligence costs, human plus artificial, per PR every month. Next to that AI bill and next to the PR throughput. How are you trending? Is throughput continuing to improve? What about those cost efficiencies, the cost to deliver?

同时关注回滚情况和质量指标，作为制衡。我们不需要 AI 垃圾代码扰乱整个流程。因此，进入预算季时，把每个 PR 的成本纳入讨论。不能只看 token 账单，也不能只看吞吐量的提升。账单告诉你花了多少钱；每个 PR 的成本则告诉你这些投入是否有效。而眼下，对大多数公司来说，这个指标表明应该继续投资。[清了清嗓子] 要提升效率，tokens 是成本中最便宜的部分。做得好，你已经读到了最后。以下是获取我们最新研究动态的方式。这些研究同样基于业内规模最大的工程工作流与 AI 遥测数据集。你可以在 LinkedIn 上关注我、我的同事或 Jellyfish。订阅我们的 Substack，也可以访问我们网站上的 Jellyfish 博客。如果你想了解如何跟踪组织的工程工作流和 AI 成效，包括每个合并 PR 的成本，请访问我们的网站预约演示，或联系我，我会帮你对接合适的同事。Watch reverts and quality metrics beside all of this as a counterweight. We don't need AI slop mucking up the gears. So, heading into budget season, bring cost per PR to the table. Not just your token bill, not just the throughput bump. The bill tells you what you spent. Cost per PR tells you whether it worked. And right now, for most companies, it says keep investing. [clears throat] Tokens are the cheap part to get efficiency gains. Well done. You made it to the end. Here's how you can stay up to date with our latest research, which again draws from the largest engineering workflow and AI telemetry data set in the industry. You can follow me on LinkedIn, my colleagues, or Jellyfish. Subscribe to our Substack, or you can check out our Jellyfish blog on the website. If you want help tracking your org's engineering workflow and AI effectiveness, including cost per merge PR, go to our website and book a demo, or reach out to me and I'll get you connected to the right folks.

我再介绍一下，我是 Jellyfish 的 AI 研究与工程负责人 Steve Jameson。感谢收看。再见。Again, I'm Steve Jameson, AI research and engineering leader at Jellyfish. Thanks for watching. Cheers.