读见

您的下一个 DSL 作者是语言模型

原题Your Next DSL Author Is a Language Model

约 18 分钟约 4,310 字原文英文已翻译

推荐理由冷门 DSL 写砸,根因常是语料频率,不是不懂领域。Typed Domain Grounding 把领域嵌进 Kotlin/TS 等宿主语言,让编译器当预言机,用生成-编译-修复拦住幻觉语法。50 题基准上更严类型换来更高保真。做内部语言或工具链,这是模型外最硬的尺子。

读前速览

5 个要点

Typed Domain Grounding 将 DSL 嵌入到训练数据丰富的宿主语言中,使编译器能够捕获语法错误,从而减少 LLM 幻觉。

  1. LLM 在 DSL 中的幻觉源于低训练数据频率,而非知识缺失。

    模型生成流畅但语法错误的 DSL 代码是因为它们在语料库中从相似结构进行插值;检索增强只能纠正事实错误,无法修正记法错误。

  2. Typed Domain Grounding 通过在高频宿主语言中嵌入类型化内部 DSL,让编译器充当预言来解决记法差距。

    通过使用命名参数、枚举、构建器作用域等手段将领域错误暴露为编译时类型错误,模型可以根据编译器反馈进行修复。

  3. TDG 的五个构建块是:内嵌 DSL、由训练数据亲和力决定的宿主语言、编译器作为预言、Generate‑Compile‑Repair 循环以及按需教学通道。

    这些部分共同确保语法正确性、复用现有工具链、提供修复提示并在生成过程中提供相关示例。

  4. 通过 kUML 和思考实验中的类型化 Kotlin 基础设施 DSL 的示例,该方法能够将幻觉的属性、错误类型、缺失引用和无效枚举作为普通编译错误捕获。

    这些例子表明编译器反馈循环在实践中有效,并且可以从建模领域推广到其他领域。

  5. 虽然 TDG 提高了语法忠实度,但它不解决语义理解差距,牺牲了记法自由,并要求对宿主语言作出承诺;设计者应将其视为新 DSL 的默认考虑,而不是强制重写已有 DSL。

    该方法以可靠性换取灵活性,带来学习曲线和生态限制的实际成本。

完整摘要

文章认为, LLM 在生成 特定领域语言 ( DSL ) 代码时产生的幻觉主要是 训练数据频率问题,而非 知识缺口。 模型能够流畅地生成语法错误的 DSL 语句,因为它们在语料库中从相似的结构进行插值;检索增强生成(RAG)可以修正事实错误,但无法解决记法问题。 为了弥补这一差距,作者提出了 Typed Domain Grounding(TDG):将 DSL 作为一个类型化的内部 DSL 嵌入到在 LLM 训练数据中出现频率高的宿主语言中(例如 Kotlin、TypeScript、Python)。 通过设计 API 使领域错误表现为编译时类型错误——使用命名参数、枚举、构建器作用域和封闭层次结构——编译器成为一个能够立即标记幻觉语法的预言。 TDG 包含五个构建块:(1)一个内嵌而非外部的 DSL,它继承宿主语言的工具链;(2)根据语料亲和力选择宿主语言;(3)通过严格的 API 设计让编译器充当预言;(4)Generate‑Compile‑Repair(GCR)循环,利用编译器诊断作为修复提示;(5)按需教学通道,为所需结构提供 curated、可编译的示例。 通过 kUML(一个 Kotlin 内嵌的 UML/SysML v2/C4 DSL)以及一个思考实验中的类型化 Kotlin 基础设施 DSL 的说明表明,该方法能够将幻觉的属性、错误的类型、缺失的引用和无效的枚举作为普通的编译错误捕获出来。 文章将 TDG 与先前的工作如 TypeChat、Grammar Prompting、ThingTalk 和 AI‑oriented 语法进行对比,强调 TDG 避免发明新语法,而是利用已有的高频宿主语言。 最后,文章指出 TDG 的局限:它不会填补语义理解的空白,牺牲了记法自由,并且需要对宿主语言作出承诺;因此,它应被视为新 DSL 的默认考虑因素,而不是强制要求重写已有的 DSL。

目录 · 24 节
查看原文

关键要点Key Takeaways

  • 大型语言模型(LLM)在特定领域语言上的幻觉,从根本上说是训练数据频率问题,而不是知识问题。模型能够可靠地编写主流语言,是因为这些语言在其训练语料库中占据很大比重;而对于任何数据稀疏或新近设计的语言,模型则会凭空编造语法。
    Large language model (LLM) hallucination on a domain-specific language is fundamentally a training data-frequency problem, not a knowledge problem. Models write mainstream languages reliably because those languages are dense in their training corpus and invent syntax for anything sparse or newly designed.
  • 检索增强生成(RAG)为模型响应中的事实提供依据,但对用于表达这些事实的记法无能为力,因为模型即使拥有完全准确的知识,仍可能生成语法上自创的领域特定语言(DSL)语句。
    Retrieval-augmented generation (RAG) grounds the facts in a model's response, but does nothing for the notation used to express them, because a model can have perfectly accurate knowledge and still emit a syntactically invented domain-specific language (DSL) statement.
  • 类型化领域落地(TDG)通过将领域嵌入训练数据丰富的宿主语言中,作为一种类型化内部 DSL,并塑造其 API,使领域错误以编译器类型错误的形式显现,而不是悄无声息地失败,从而弥合了这一差距。
    Typed Domain Grounding (TDG) closes that gap by embedding the domain as a typed internal DSL inside a training-data-rich host language and shaping its API so that domain errors surface as compiler type errors instead of silent failures.
  • 在包含 50 项任务的基准测试中,使用 Claude Sonnet 5 时,尽管该方法的首次编译成功率较低,但其结构保真度高于两个宽松的外部 DSL,幻觉率也更低。对于 GPT-4o,同样的模式也成立,但并非对所有测试模型都如此。证据表明,严格且会拒绝无效结果的判定器可以收敛到比宽容型判定器更可靠的输出,而不是表明它总是如此。
    In a fifty-task benchmark with Claude Sonnet 5, this approach reached higher Structural Fidelity and a lower hallucination rate than two lenient external DSLs, even though it had a lower first-try compile rate. The same pattern holds for GPT-4o, but not for every model tested. Evidence shows that a strict, rejecting oracle can converge to more reliable output than a forgiving one, not that it always does.
  • 这种方法确实存在实际的局限性和成本。它能够弥合语法上的差距,却无法弥合理解上的差距。它还牺牲了记法上的自由,并要求对宿主语言作出承诺,因此,语言设计者应将其视为设计新 DSL 时的默认考量,而不是强制改写那些已经正常运行的 DSL。
    The approach has real limits and real costs. It closes syntax gaps but not understanding gaps. It also trades away notational freedom and demands a host-language commitment, so language designers should treat it as a default consideration for new DSLs rather than a mandate to rewrite ones that already work.

向前沿语言模型请求 Python,你得到的就是 Python。向它请求你们团队自研的建模记法,你得到的却只是某种貌似该记法的东西:臆造的关键字、猜测的参数名,以及规范中从未出现过的结构,而且这一切都以十足的自信呈现出来。惯常的补救办法是增加提示词、延长上下文并改进检索。真正持久的解决办法,是在模型看到你的语言之前就做出的设计决策。首先不要使用外来的语法。这个决策有一个名称:类型化领域 grounding(Typed Domain Grounding)。利用你现有的类型系统,今天就可以实现它。Ask a frontier language model for Python and you get Python. Ask it for your team's home-grown modeling notation and you get something that merely resembles that notation: invented keywords, guessed parameter names, and constructs that were never in the spec, all delivered with perfect confidence. The reflexive fix is more prompting, longer context, and better retrieval. The durable fix is a design decision made before the model ever sees your language. Don’t use a foreign syntax in the first place. That decision has a name: Typed Domain Grounding. It is implementable today with the type system you already have.

流利源于频繁练习Fluency Is Frequency

模型之所以能很好地编写 Kotlin、TypeScript、Python 和 SQL,主要是因为这些语言在其训练数据中出现了数百万次。粗略来说,语法能力是训练数据频率的函数;对代码大语言模型基准测试的研究直接记录了这种关系:这些基准按语言的出现频率和流行度进行分类,并发现对于低资源语言,模型性能会相应下降 [8]。语料库密集的地方,生成结果就可靠;语料库稀疏的地方,生成结果就会退化为拼贴,语调和表面上的自信却没有变化。这种缺失使得失败极易被忽视。A model writes Kotlin, TypeScript, Python, and SQL well for one dominant reason. Those languages appear millions of times in its training data. Syntactic competence is, to a first approximation, a function of training-data frequency, a relationship documented directly in code-LLM benchmarks that classify languages by frequency and popularity and find performance degrading accordingly for low-resource languages [8]. Where the corpus is dense, generation is reliable. Where it is sparse, generation degrades into pastiche, without change in tone or apparent confidence. This absence makes the failure so easy to miss.

从模型的角度来看,每一种新近设计的外部 DSL,其确切语法在训练语料中的出现频率都是零。尽管如此,模型通常仍会保留从其见过的、在语法上相近的语法体系、API 和记法中迁移而来的先验知识。这种迁移并不能挽救问题。相反,它正是下文所述插值失败背后的机制。Every freshly designed external DSL is, from the model's point of view, a language with a training-corpus frequency of zero for its exact syntax. That said, the model typically still carries transferable priors from syntactically adjacent grammars, APIs, and notations it has seen. That transfer is not a rescue. Rather, it is the mechanism behind the interpolation failure described next.

面对不熟悉的记法时,模型不会大声报错;它会进行插值。它会偷偷带入 Mermaid 或 PlantUML 中的箭头语法,调用你的 API 按理说可能会有、但实际上并不存在的 addRelation 或 withLabel;还会因为某个语法近亲接受三个参数,就向一个只接受两个参数的构造传入三个参数。每个错误都是语料库中最可能的续写,被应用到了语料库从未包含过的一种语言上。输出流畅、缩进规范,却是错误的。A model confronted with an unfamiliar notation does not fail loudly; it interpolates. It smuggles in arrow syntax from Mermaid or PlantUML, calls the addRelation or withLabel your API would plausibly have but doesn't and hands three arguments to a construct that takes two because a syntactic cousin takes three. Each mistake is the corpus's most probable completion, applied to a language the corpus never contained. The output is fluent, well-indented, and wrong.

传统 DSL 的一个优点反而让这个问题变得更糟:宽容性。图表工具和配置解析器经过精心设计,会通过跳过无法理解的行、渲染出一个最佳猜测结果,并且从不因拼写错误而报错来宽容对待用户。对人类来说,这是友好待客;对机器作者来说,这却是一扇陷阱门。A traditional DSL virtue makes this problem worse: leniency. Diagram tools and config parsers are engineered to forgive by skipping the line you don't understand, rendering a best guess, and never punishing a typo. For humans, that is hospitality. For machine authors, it is a trap door.

一个模型生成了一个包含十个类别的图表,其中使用了一个自创的关系关键字。解析器跳过了那一行。最终进入设计文档的是一张看起来很专业、但恰好缺少一个关联关系的图表。A model produces a ten-class diagram with one invented relationship keyword. The parser skips that line. What lands in the design document is a professional-looking diagram missing exactly one association.

如果是人打错了字,应该会注意到,因为这种关联是人有意建立的。模型没有任何意图,而下游工程师在看到九个正确的类别和整洁的排版后,没有理由怀疑这张图片是不完整的。A human who mistyped would have noticed, because a human intended the association. The model intended nothing and the engineer downstream, primed by nine correct classes and clean typography, has no reason to suspect the picture is incomplete.

让语言立足,而不仅仅是事实Grounding the Language, Not Just the Facts

在日常的大语言模型工程中,“grounding”(基于依据)几乎已经成为 RAG 的同义词。获取相关文档,将其放入上下文中,模型的陈述就能始终锚定于真实来源。这是数据空间中的 grounding。它约束了模型对世界的描述,却无法解决我们的问题:即使上下文中的事实完全无误,模型仍然可能生成一条语法凭空捏造的 DSL 语句。In everyday LLM engineering, "grounding" has become nearly synonymous with RAG. Fetch the relevant documents, put them in context, and the model's claims stay anchored to real sources. That is grounding in the data space. It disciplines what the model says about the world. It does nothing for our problem: A model with flawless facts in its context can still emit a DSL statement with invented grammar.

当被要求在没有检索机制的情况下绘制支付系统图时,模型可能会展示支付服务直接写入账本,而实际上两者之间隔着一个消息队列;其语法无可挑剔,但内容是错误的。检索可以解决这一问题。然而,在文档已纳入上下文的情况下,同一个模型可能会用一种它只半知半解的表示法来表达正确的队列,凭空捏造出该语言并不存在的队列组件关键字。在这种情况下,内容是正确的,但表示法出了问题。再多的检索也无济于事,因为失败源于模型对目标语言的掌握能力,而不是知识。Asked to diagram a payment system with no retrieval in place, a model may show the payment service writing directly to the ledger when a message queue actually sits between them with impeccable syntax, but incorrect content. Retrieval fixes that. But with the documentation in context, the same model can express the correct queue in a notation it half-knows, inventing a queue-component keyword the language does not have. In this situation, the content is correct, but the notation is broken. No amount of additional retrieval helps, because the failure lives in the model's competence with the target language, not its knowledge.

类型化领域落地为另一条轴提供了基础。它不是将答案的内容锚定在检索到的数据上,而是将表示法本身锚定在模型已经知道如何编写的内容上,落地于能力空间。两者彼此正交,可以很好地结合:通过 RAG 检索事实,再通过类型化且经编译器检查的 DSL 表达这些事实。Typed Domain Grounding grounds this other axis. Instead of anchoring an answer's content in retrieved data, it anchors the notation itself in what the model already knows how to write, grounding in the competence space. The two are orthogonal and combine cleanly, retrieving facts via RAG and expressing them through a typed, compiler-checked DSL.

图 1。两种接地空间。(来源:作者使用 Google 的 Nano Banana 创建。)Figure 1. Two spaces of grounding. (Source: created by author using Google’s Nano Banana). 

在图 1 中,RAG 将响应内容锚定在数据空间中。TDG 则将语言本身锚定在由相互正交且可组合的管线构成的能力空间中,而不是竞争者空间。In Figure 1, RAG grounds a response's content in the data space. TDG grounds the language itself in the competence space of orthogonal, combinable pipelines,  not competitors. 

这一原则可以用一句话概括:“绝不要臆造 LLM 从未见过的语法:将该领域嵌入它已经掌握的语言中,并让编译器充当裁判。”The principle fits in one sentence. "Never invent syntax an LLM has never seen: Embed the domain in a language it already knows and let the compiler be the oracle".

五个构建模块Five Building Blocks

TDG 是一组精简的设计承诺。TDG is a small stack of design commitments.

嵌入式,而非外部的Embedded, Not External 

该领域在主流宿主语言中成为一种内部 DSL,无需新增语法,也无需定制解析器。读起来像一门专用语言的内容,实际上是普通的宿主语言代码,因此可以在第一天就继承现有的完整工具链(例如语法高亮、代码补全、跳转到定义、安全重命名、调试器,以及审阅者熟悉的差异对比);否则,这些功能中的每一项都可能需要多年时间来开发。The domain becomes an internal DSL inside a mainstream host language with no new grammar and no bespoke parser. What reads like a dedicated language is ordinary host-language code, so the full existing toolchain (e.g., highlighting, completion, jump-to-definition, safe rename, debugger, and reviewer-familiar diffs) is inherited on day zero, where each piece would otherwise be a multi-year project.

由训练数据亲和性选择的宿主语言Host Language Chosen by Training Data Affinity

经典的嵌入式 DSL 文献(例如 Paul Hudak 和 Martin Fowler 的作品)会权衡表达能力、类型系统的强大程度以及工具支持。TDG 增加了一个在 LLM 出现之前不可能存在的标准:语料库覆盖度。Kotlin、TypeScript 或 Python 自带大量既有的先验熟悉度。没有哪家前沿实验室会公布其确切的训练数据构成,但这一论点只需要关注两者之间的差距。任何主流语言在公开代码中的代表性,都远远超过一个全新的 DSL 所可能达到的程度。优雅但罕见的语言几乎无法享有这些优势。这些标准通常会指向同一个方向,但当它们出现分歧时,语料库也应当拥有发言权。Classic embedded-DSL literature (such as works by Paul Hudak and Martin Fowler) weighs expressiveness, type-system strength, and tooling. TDG adds a criterion that couldn't have existed before LLMs: corpus presence. Kotlin, TypeScript, or Python bring substantial prior fluency for free. No frontier lab publishes its exact training mix, but the argument only needs the gap. Any mainstream language is far better represented in public code than a new DSL could ever be. An elegant but rare language brings almost none of that. The criteria usually pull the same direction, but where they diverge, the corpus gets a vote.

编译器即预言机The Compiler As Oracle 

设计 API,使领域错误以类型错误的形式暴露出来:使用带名称的参数,而非位置参数;使用枚举,而非魔法字符串;并让构建器作用域只暴露当前上下文中有效的构造。密封层次结构可以让 API 表达:关系末端恰好是四种事物之一。凭空捏造的第五种变体没有任何类型可以归属。接收者类型决定了代码块内部究竟存在哪些函数。在类体之外使用 attribute(...) 会立即导致名称解析失败。非法嵌套会变成该语言无法表达的一句话。这里的严格性是实现机制,而非吹毛求疵。Shape the API so domain errors surface as type errors with named parameters over positional ones, enums over magic strings, and builder scopes exposing only constructs valid in the current context. Sealed hierarchies let the API say a relationship end is exactly one of four things. A hallucinated fifth variant has no type to inhabit. Receiver types control which functions even exist inside a block. An attribute(...) outside a class body fails name resolution immediately. Illegal nesting becomes a sentence the language cannot say. Strictness here is the mechanism, not pedantry.

生成-编译-修复(GCR)循环The Generate-Compile-Repair (GCR) Loop 

模型生成,编译器检查,诊断信息再作为修复提示返回。编译器反馈修复循环在自调试文献中已有充分论述 [1]——GCR 本身并不是 TDG 的创新之处。TDG 的新增之处在于,同时将这一熟悉的循环用于两个目的:作为修复通道,纠正有问题的输出;作为测量工具,揭示某个给定 DSL 究竟多么频繁地需要修复,以及最初为何需要修复。修复提示中包含失败的代码片段、逐字保留的诊断信息和源代码位置,从而能够进行有针对性的编辑,而不是盲目重新生成;后者只会从产生该错误的分布中再次采样。在 kUML 的设置中,该循环被限制为三次尝试。修复一旦成功,通常很早就会成功。模型在经过三轮带有位置信息的反馈后仍然失败,说明它缺少的是某个概念,而不是一个逗号。The model generates, the compiler checks, and diagnostics flow back as a repair prompt. Compiler-feedback repair loops are already well documented in the self-debugging literature [1] — GCR itself is not the novel part of TDG. What TDG adds is putting that familiar loop to two uses at once: a repair channel that fixes broken output, and a measurement instrument that reveals how often, and why, a given DSL needs fixing in the first place. The repair prompt carries the failed snippet, the verbatim diagnostic, and the source location, enabling a targeted edit rather than blind regeneration, which merely resamples from the distribution that produced the mistake. The loop is bounded with three attempts in kUML's setup. Repairs that succeed tend to succeed early. A model still failing after three rounds of located feedback is missing a concept, not a comma.

按需教学频道An On-Demand Teaching Channel 

嵌入解决了语法问题。但词汇问题依然存在:有哪些构造,它们分别叫什么?把完整的 API 文档塞进每个提示词,只会让五个相关构造淹没在九十五个无关构造之中。TDG 的答案是一种模型可以在生成过程中途查询的可调用工具——例如,可以将其实现为一个模型上下文协议服务器——它只针对模型当下所需的确切构造,返回少量经过精心筛选、已知可用且已经编译通过的示例,而不是倾泻一大堆不相关的文档。其效果大小并不隐晦:对于 kUML,kuml.examples 工具将 SysML v2 图表的首次编译成功率从 25% 提高到了 87.5%。Embedding solves syntax. A vocabulary question remains: Which constructs exist and what are they called? Stuffing full API docs into every prompt buries the five relevant constructs under ninety-five irrelevant ones. TDG's answer is a callable tool the model can query mid-generation — implementable, for instance, as a Model Context Protocol server — that returns a small set of curated, known-good, already-compiling examples for exactly the construct the model needs at that moment, instead of a firehose of unrelated documentation. The effect size is not subtle: for kUML, a kuml.examples tool raised the first-compile rate for SysML v2 diagrams from twenty-five to 87.5 percent.

模块 1-3 是使设计成为 TDG 的部分,而模块 4-5 则是使 TDG 实例在实践中表现良好的部分。这些是互补的组成部分,而非定义性要素。与其他理念相比,TDG 并不是检索增强生成(见上文)。它为语言提供依据,而不是为事实提供依据。此外,TDG 不像一些面向 LLM 的语法提案那样设计新的语法。相反,它选择一种训练数据丰富的现有语法,并让该语言自身的编译器负责检查。下面的两个示例展示了这种方法在实践中的样子。随后将对相关工作进行完整比较。Blocks 1-3 are what makes a design TDG, while blocks 4-5 are what makes a TDG instance perform well in practice. These are complementary pieces, rather than definitional. In relation to other ideas, TDG is not Retrieval-Augmented Generation (see above). It grounds the language, not the facts. Also, TDG does not design a new syntax the way some LLM-oriented grammar proposals do. Instead, it selects an existing, training-data-rich syntax and lets that language's own compiler do the checking. The two illustrations below show what this approach looks like in practice. A full comparison against related work follows them.

示例一:kUML——Oracle 可量化之处Illustration One: kUML, Where the Oracle Is Measurable

kUML 是一种嵌入 Kotlin 的建模语言,用于创建 UML、SysML v2 和 C4 图。目前为止,它是经过最全面测量的 TDG 实例。kUML is a Kotlin-embedded modeling language for UML, SysML v2, and C4 diagrams. It is to date the most thoroughly measured TDG instance.

代码示例 1(Kotlin):Code Example 1 (Kotlin):

classDiagram(name = "Order") {
    val customer = classOf(name = "Customer") {
        attribute(name = "customerId", type = "String")
    }
    val order = classOf(name = "Order") {
        attribute(name = "date", type = "LocalDate")
        operation(name = "cancel") { returns(typeName = "Boolean") }
    }
    association(source = customer, target = order) {
        source { multiplicity(spec = "1") }
        target { multiplicity(spec = "0..*") }
    }
}

对语言模型而言,这只是普通的 Kotlin,其中包含接收者 Lambda、命名参数和普通变量引用——这些构造都是它从训练数据中的所有公开 Kotlin 代码库里吸收的。该建模领域依托于模型已经掌握的语法。To a language model this is ordinary Kotlin  with receiver lambdas, named arguments, and plain variable references as constructs it has absorbed from every public Kotlin codebase in its training data. The modeling domain rides on syntax the model already commands.

即使它仍然产生幻觉,当一个虚构的聚合(例如 source = customer 且 target = order)在顶层(任何关联块之外)发起调用时,Kotlin 编译器也会给出精确且定位明确的错误。将同样的幻觉写成 PlantUML 或 Mermaid,宽松的渲染器很可能仍会生成图表,却悄无声息地遗漏一个元素。When it hallucinates anyway, when an invented aggregation (e.g., source = customer and target = order) makes a call at the top level, outside any association block, the Kotlin compiler answers with an exact, located error. Write the equivalent hallucination in PlantUML or Mermaid and the lenient renderer will most likely produce a diagram anyway, quietly missing an element.

图 2。IDE 截图。(来源:作者制作。)Figure 2. IDE screenshot. (Source: created by author.) 

虚构的 aggregation(...) 调用会在 IntelliJ IDEA 编辑器中直接被标记出来,而实时预览面板会即时报告“无法解析的引用 ‘aggregation’”——编译器充当预言机,此时作者仍在输入。The invented aggregation(...) call is flagged directly in the IntelliJ IDEA editor, and the live-preview panel reports "Unresolved reference 'aggregation'" in real time - the compiler acting as oracle while the author is still typing.

这个幻觉没有变成一张细微出错的图表,而是变成了一条红色下划线。The hallucination did not become a subtly wrong diagram. It became a red underline.

幻影引用也会以同样的方式消亡。假设模型编写了关联(例如,source = order、target = invoice),却从未声明过 invoice 分类器。在 kUML 中,图表元素是普通的 Kotlin 值,因此“哪些模型元素存在”属于变量解析问题。编译器会在确切的行号和列号处回答:未解析的引用(invoice)。修复提示词也就自然而然地生成了。将同一个幻影交给一种宽松的、基于名称的表示法,结果会发生比沉默更糟糕的事情:许多图表工具只要发现某条边引用了一个节点,就会贴心地立即创建该节点。这个臆造出来的 invoice 不会被丢弃,而是会被提升为一个真实但空白的类;它从未出现在任何规范中,却看起来与相邻元素一样完全合法。一条工具链会把幻影转换为一个有明确位置的错误;另一条则会把它转换成图表中的既有陈设。Phantom references die the same way. Suppose the model writes association (e.g., source = order and target = invoice) without ever declaring an invoice classifier. In kUML, diagram elements are ordinary Kotlin values So "which model elements exist" is variable resolution. The compiler answers Unresolved reference (invoice) at an exact line and column. The repair prompt writes itself. Run the same phantom through a lenient name-based notation and something worse than silence happens: Many diagram tools helpfully create a node the moment an edge references it. The hallucinated invoice isn't dropped, it's promoted to a real, empty class that was never in any specification, looking exactly as legitimate as its neighbors. One toolchain converts the phantom into a located error; the other converts it into furniture.

插图二:基础设施即代码中的思想实验Illustration Two: A Thought Experiment in Infrastructure as Code

为了说明 TDG 是一种原则,而不是 kUML 的特性。要测试该模式能否推广到一个已经实现并测量过它的领域之外,需要定义基础设施。这一节是一个思想实验,而不是第二个测量结果。目标是架构上的通用性,而不是增加额外的实证证据。Terraform 的 HCL 是一种外部 DSL,但它的支持程度异常充足。模型很少会弄错自身的块语法。问题位于更底层。各个资源类型的属性名称和结构属于提供商模式,而不是语言本身。To show TDG is a principle and not a kUML feature. Testing whether the pattern generalizes beyond a domain where it has already been implemented and measured requires infrastructure definitions. This section is a thought experiment, not a second measured result. The goal is architectural generalizability, not additional empirical evidence. Terraform's HCL is an external DSL, but an unusually well-fed one. Models rarely botch their own block syntax. The trouble sits one level down. Attribute names and shapes of individual resource types are provider schema, not language.

模型在编写 `aws_s3_bucket` 块时,会在那些只依稀记得、且因版本漂移而变化的属性之间猜测;但公平地说,Terraform 确实会在本地捕获这类问题。`terraform validate` 命令和 Terraform 语言服务器(`terraform-ls`)会根据从已安装插件获取的提供商 schema,检查属性名称、类型和引用,而无需调用云 API。这个检查机制本身就说明了问题所在:它配备了 schema 分发机制、专门的验证阶段以及语言服务器,旨在为外部 DSL 弥补嵌入式 DSL 的宿主编译器从第一天起就能提供的能力。Terraform 的 Cloud Development Kit(CDK)已经能用五种语言生成带类型的提供商绑定 [2],而 TDG 将宿主语言引入基础设施的做法也是如此。无论使用 HCL 还是带类型的 DSL,本地检查都无法触及 schema 之下的那一层。AMI 是否存在、你是否拥有相应权限,或配额是否充足,都是如此。只有当某个组件与真实的云 API 通信时,这些问题才会显现出来。A model writing an aws_s3_bucket block guesses among half-remembered, version-drifted attributes, but to its credit, Terraform does catch this locally, The terraform validate command and the Terraform language server (terraform-ls) check attribute names, types, and references against the provider schema fetched from the installed plugin, without involving a cloud API. That check is itself the tell: It is an entire apparatus with schema distribution, a dedicated validate phase, and a language server, built to recover for an external DSL what an embedded DSL's host compiler hands over on day zero. Cloud Development Kit (CDK) for Terraform already generates typed provider bindings in five languages [2], TDG's host-language move applied to infrastructure. What no local check reaches, in HCL or a typed DSL alike, is the tier below schema. This is true whether the AMI exists, you have permission, or the quota holds. It is visible only once something talks to the real cloud API.

现在,设想将同样的基础设施表示为一个类型化的 Kotlin DSL。这是一个思想实验,并非现有的库;其理念与 Pulumi、AWS CDK 和 CDK for Terraform 已经通过其他语言中的类型化绑定所实现的做法相同 [2],不过这些工具目前都没有提供官方的 Kotlin SDK。Now imagine the same infrastructure as a typed Kotlin DSL. This is a thought experiment, not an existing library and is in the spirit of what Pulumi, AWS CDK, and CDK for Terraform already do with typed bindings in other languages [2], none of which currently ships an official Kotlin SDK.

代码示例 2(Kotlin):Code Example 2 (Kotlin):

// Thought experiment – no existing library, illustrating the TDG principle
// in a domain beyond modeling.
val assets = s3Bucket(name = "assets") {
    versioning = Versioning.ENABLED
    encryption = ServerSideEncryption.AES256
}

lambdaFunction(name = "thumbnailer") {
    runtime = Runtime.JAVA_21
    memoryMb = 512
    trigger = s3Trigger(bucket = assets, event = S3Event.OBJECT_CREATED)
}

The four classic hallucination classes against this design:

- Invented attribute (i.e., encryptionMode instead of encryption) becomes a compile error naming the properties actually available.
- Wrong value type (i.e., memoryMb = "512MB") becomes a type error before any tool runs.
- Reference to a never-declared bucket becomes an unresolved variable, caught at the exact line.
- A trigger event S3 does not have fails against the S3Event enum instead of being accepted as a string.

图 3。对比图。(来源:作者绘制。)Figure 3. Comparison graphic. (Source: created by author.)

图 3 展示了相同的四类幻觉。Terraform 也能在本地捕获它们,通过 `terraform validate`/`terraform-ls` 以及文中所述的 provider-schema 机制。类型化 Kotlin DSL 则以同样的方式免费捕获它们,依靠的是标准编译器。在真实的 Terraform 中,这四类问题也都会在本地通过同一套 schema 机制被拦截。在类型化变体中,它们会作为普通的编译器错误被拦截,例如不需要协议、不需要单独的验证阶段,也不需要构建或维护语言服务器。还剩下一个问题:哪项检查能够与真实、实时的 API 保持同步?Figure 3 shows the same four hallucination classes. Terraform catches them locally too, via terraform validate/terraform-ls and the provider-schema apparatus described in the text. The typed Kotlin DSL catches them the same way, for free, via the stock compiler. 
In real Terraform, all four also die locally, through that same schema apparatus. In the typed variant, they die as ordinary compiler errors, such as no protocol, no separate validate phase, and no language server to build or maintain. One question remains: which check stays current with the real, live API?

这里的两种方法是对称的,谁也没有胜出:terraform validate 会根据锁定文件中固定的 provider 版本进行检查,而类型化 DSL 会根据构建文件中固定的 SDK 版本进行检查——并且这两种固定版本都可能在不知不觉中落后于真实、在线的 API。两者之下剩下的,是 HCL 和 Kotlin 同样无法看见的实时云状态(即资源是否存在、权限和配额),除非有东西调用 API。嵌入式 DSL 的优势并不是比 Terraform 的机制实现更多验证;而是以几乎为零的增量成本实现同样的验证,并采用修复循环最容易消费的格式(编译器诊断信息)。这一模式可以推广到任何同时具备类型化内部 DSL、支持快速批处理的编译器,以及训练数据丰富的宿主语言的领域:Gradle 的 Kotlin DSL、Compose 风格的 UI 构建器,以及用类型化配置 DSL 替代无模式 YAML 的方案,都符合这一条件。Here the two approaches are symmetric, and neither wins: terraform validate checks against the provider version pinned in the lock file, the typed DSL checks against the SDK version pinned in the build file - and either pin can silently lag the real, live API. What is left below both is live cloud state (i.e., existence, permissions, and quota) equally invisible to HCL and Kotlin until something calls the API. The embedded DSL's advantage isn't more validation than Terraform's apparatus achieves; it's the same validation at near-zero incremental cost, in the format (compiler diagnostics) a repair loop consumes best. The pattern generalizes to any domain with a typed internal DSL, a fast batch-capable compiler, and a training-data-rich host language: Gradle's Kotlin DSL, compose-style UI builders, and typed configuration DSLs replacing schema-less YAML all qualify.

TDG 相对于先前工作的定位与非定位What TDG Is and Isn't Relative to Prior Work

相关工作 / 方法 / TDG 的不同之处:Prior work / Approach / How TDG differs:

微软 TypeChat [3]Microsoft TypeChat [3] 

将 TypeScript 类型作为响应模式,而将编译器诊断类型作为修复提示。另一方面,TDG 会验证数据结构(即响应模式)。TDG 嵌入了一套具有语义和工件生成能力的完整领域语言。TypeScript types as a response schema, while compiler diagnostics type as a repair prompt. TDG, on the other hand, validates data structures (i.e., response schemas).TDG embeds a complete domain language with semantics and artifact generation.

语法提示 [4]Grammar Prompting [4] 

将目标 DSL 的 BNF 语法输入上下文。TDG 通过缓解外来语法问题来解决这一问题,以构造性的方式将其消解。无需教授任何外来语法。Feed the target DSL's BNF grammar into the context. TDG differs by mitigating the foreign-syntax problem, dissolving it constructively. There is no foreign grammar to teach.

ThingTalk [5]ThingTalk [5] 

将类型化 DSL 作为语义解析的目标语言。TDG 作为一种具有自身语法的外部 DSL,与 TDG 恰恰要避免的这种构造有所不同。Typed DSL as the target language for semantic parsing. TDG differs as an external DSL with its own syntax, which is exactly the construction TDG avoids.

“面向 AI 的语法”[6]"AI-Oriented Grammar" [6] 

为 LLM 设计一种全新的、更节省 token 的语法(例如 SimPy 是 Python 的衍生语法,而不是同名的仿真库)。TDG 的不同之处在于它不设计新语法,而是选择一种现有且训练数据丰富的语法。Design a new, more token-efficient syntax for LLMs (e.g., SimPy as a Python derivative, not the identically-named simulation library). TDG differs by not designing a new syntax. Instead, it selects an existing, training-data-rich one.

RAG 依据支撑(例如 AGREE [7])RAG Grounding (e.g., AGREE [7]) 

在回答中锚定事实(数据空间)。TDG 的不同之处在于它锚定于语言空间(例如正交且可组合)。Anchor facts in responses (data space). TDG differs by anchoring in language space (e.g., orthogonal and combinable).

编译修复文献 [1]Compile-Repair Literature [1] 

借助编译器反馈修复循环(即自我调试和自我修复)。TDG 的不同之处在于,它将修复循环视为一种辅助性构建模块和测量工具,而不是其定义性创新。Repair loops with compiler feedback (i.e., self-debugging and self-repair). TDG differs by treating the repair loop as a supporting building block and measurement instrument, not as its defining novelty.

与 TDG 最接近的是 TypeChat [3]。二者都使用主流类型系统来检查权限,并将诊断信息反馈给模型,但 TypeChat 验证的是答案是否符合某个接口、情感结果是否为正面、负面或中性,以及提取出的对象是否包含正确的字段。TDG 则将领域本身转化为一种具有语义和下游制品生成能力的语言,检查响应模式根本无法发现的问题:关联是否连接了实际存在的分类器,转换是否位于状态机内部。TypeChat 问的是“这个答案的结构是否正确?”,而 TDG 问的是“这是否是该领域中的有效句子?”,后者是一个更丰富的问题,并由同样的编译器机制进行检查。The closest relative to TDG is TypeChat [3]. Both use a mainstream type system to check authority and feed diagnostics back to the model, but TypeChat validates that an answer conforms to an interface, that a sentiment result is positive, negative, or neutral, and that an extracted object has the right fields. TDG turns the domain itself into a language with semantics and downstream artifact generation, checking things a response schema never sees: whether an association connects classifiers that actually exist, whether a transition sits inside a state machine. TypeChat asks "Is this answer shaped correctly?", whereas TDG asks "Is this a valid sentence in the domain?", which is a richer question, checked by the same compiler machinery.

语法提示 [4] 和面向 AI 的语法路线 [6] 都接受了 TDG 所否定的前提,即必须存在一种新语法。前者在运行时将语法加载到上下文中。后者设计出一种令牌效率更高的语法,但其语料库频率仍为零,因此能力差距依然存在,必须通过教学来弥合。TDG 提出的是“如何教授一种外来语法?”之前的问题:为什么一定要有这种语法?ThingTalk [5] 则是一个警示性案例:它具有清晰的类型定义,却仍然处于外部,仍然面临训练数据问题。Grammar Prompting [4] and the AI-oriented-grammar line [6] both accept the premise TDG rejects, that a new syntax has to exist. One ships the grammar into context at runtime. The other designs a more token-efficient grammar with a corpus frequency still at zero, so the competence gap remains and must be bridged by teaching. TDG asks the questions that come before "How do you teach a foreign syntax? Why have one at all? ThingTalk [5] is the cautionary data point that is cleanly typed, still external, and still carrying the training-data problem.

数据与一个诚实的限定八码The Numbers and an Honest Caveat

在一项包含五十个任务、对比 kUML、PlantUML 和 Mermaid 的基准测试中,运行 GCR 循环的嵌入式、经编译器检查的语言在 Claude Sonnet 5 上取得了最高的结构保真度(45.3%,相比之下为 37.9% 和 37.4%)以及最低的幻觉率(15.4%,相比之下为 16.5% 和 21.9%)。这些任务涵盖 UML、SysML v2 和 C4 图类型,每项任务都以一份源规范为依据。输出结果根据结构保真度(即与该规范的节点集合和边集合相似度,而非更深层的语义检查)和幻觉率进行评分。许多内容都是规范从未要求的。两个指标必须结合起来看。工具可以通过罗列所有看似合理的内容来伪造较高的保真度,而幻觉率正好能识别出这种策略。In a fifty-task benchmark pitting kUML against PlantUML and Mermaid, the embedded, compiler-checked language running the GCR loop reached the highest Structural Fidelity (45.3 percent vs. 37.9 percent and 37.4 percent) and the lowest hallucination rate (15.4 percent vs. 16.5 percent and 21.9 percent) with Claude Sonnet 5. The tasks spanned UML, SysML v2, and C4 diagram types, each anchored to a source specification. Outputs were scored on Structural Fidelity (i.e., node-set and edge-set similarity to that specification, not a deeper semantic check) and hallucination rate. Much appeared that the specification never asked for. The axes matter jointly. A tool could fake high fidelity by spraying out everything plausible. The hallucination rate catches exactly that strategy.

公共展示页现在已在共享的 GCR 测试框架下涵盖九次模型运行。整体情况比标题中的数字所暗示的更依赖具体模型。完整的优势——更高的结构保真度和更低的幻觉率——在 Claude Sonnet 5 和 GPT-4o 上成立,但在其他模型上并不普遍成立。Gemini 2.5 Flash 和 Gemini 2.5 Pro 在结构保真度方面都将 Mermaid 或 PlantUML 排在前面,Qwen3-Coder:30b 也呈现出相同模式。Grok 4 给出了整个研究中 kUML 最小的结构保真度优势(36.8% 对 34.6% 和 36.4%),但其幻觉率却是最高的(28.1% 对 22.4% 和 23.0%),这意味着它只是严格意义上的 oracle,却没有相应的收益。The public gallery now covers nine model runs under a shared GCR harness.The picture is more model-dependent than the headline number suggests. The full win, higher Structural Fidelity and lower hallucination rate, holds for Claude Sonnet 5 and GPT-4o. It does not hold uniformly elsewhere. Gemini 2.5 Flash and Gemini 2.5 Pro both put Mermaid or PlantUML ahead on Structural Fidelity and Qwen3-Coder:30b shows the same pattern. Grok 4 gives kUML its narrowest Structural Fidelity win in the whole study (36.8 percent vs. 34.6 percent and 36.4 percent), but its highest hallucination rate (28.1 percent vs. 22.4 percent and 23.0 percent), which is a strict oracle without a matching payoff. 

这种效果确实存在,并且在多个模型系列中反复出现;对于本文深入讨论的两个模型而言,这一效果最为明显,但它并非所有受测模型都具备的特性。严格来说,本文标题中的数字仅适用于 Claude Sonnet 5。全文评分完全由算法执行,而非由 LLM 或人工评审。每个单元格反映的是一次生成运行,最多进行三次 GCR 修复尝试,并非多次采样后的平均值。单次运行无法提供方差估计,因此上述差异属于方向性证据:当差距较大且方向在多个模型系列中保持一致时,证据最为有力。全部九次运行的逐模型详细结果已发布在公开基准测试图库中(例如 kuml.dev/benchmark-gallery)。The effect is real and repeats across several model families, strongest for the two discussed in depth here, but it is not a property of every model tested. The honest scope of this article's headline numbers is Claude Sonnet 5. Scoring throughout is fully algorithmic, rather than an LLM or human judge. Each cell reflects a single generation run capped at three GCR repair attempts, not an average over repeated samples. Single runs carry no variance estimate, so the deltas above are directional evidence, strongest where margins are wide and the direction holds across model families. The full per-model breakdown for all nine runs is on the public benchmark gallery (e.g., kuml.dev/benchmark-gallery).

该基准测试并未显示 Claude Sonnet 5 具有更高的首次编译率。相比之下,kUML 的比率实际上低于宽松工具的首次尝试成功率。这种不对称性正是关键所在:严格的预言机在一开始就拒绝更多结果,而最终结果与更高的结构保真度和下游更低的幻觉率相关。这种相关性是该基准测试直接展示出来的,尽管它并不能证明造成差距的就是“拒绝并修复”本身,而不是三种表示法之间的其他差异。这是最简约的解读,而不是一次孤立的因果检验。教学通道进一步凸显了这一点:在提供 kuml.examples 后,经过修复循环,SysML v2 的编译率从百分之二十五上升到百分之八十七点五,也就是任务数从八个中的两个增加到八个中的七个;仅使用 Claude Sonnet 5,每个单元格只运行一次。这是编译结果,而不是保真度结果。完整的细分数据见公开的方法论页面。The benchmark did not show a higher first-shot compile rate for Claude Sonnet 5.  In comparison, kUML's rate was actually lower than the lenient tools' first-try success. That asymmetry is the point: The strict oracle rejects more up front and the result correlates with higher Structural Fidelity and lower hallucination downstream. This correlation demonstrates the benchmark directly, though the explanation that rejection-and-repair itself drives the gap (rather than some other difference between the three notations). This is the most parsimonious reading, not an isolated causal test. The teaching channel sharpens this further: With kuml.examples available, the SysML v2 compile rate after the repair loop rose from twenty-five percent to 87.5 percent, which amounted to two of eight versus seven of eight tasks, using Claude Sonnet 5 only, with a single run per cell. That is a compilation result, not a fidelity result The full breakdown is on the public methodology page.

坦率地说,TDG 弥合的是语法上的缺口,而不是理解上的缺口。模型可以严谨地生成能够编译的模型,但这些模型在实质上可能是错误的。在同一项基准测试中,尽管编译器没有发现任何问题,有个模型在 C4 架构图方面的语义仍然很浅薄。每个构造都合法,但架构本身却很单薄。判定器检查的是格式是否良好,而不是是否明智。TDG 能把优秀模型变成可靠的作者,但不能把能力较弱的模型变成优秀的架构师。Stating the caveat plainly, TDG closes gaps in syntax, not understanding. A model can cleanly produce compiling models that are substantively wrong. In the same benchmark, one model remained semantically shallow on C4 architecture diagrams even though the compiler found nothing to object to. Every construct was legal; the architecture itself was thin. The oracle checks well-formedness, not wisdom. TDG turns a good model into a reliable author, rather than turning a weak model into a good architect.

给语言设计者的六条建议Six Recommendations for Language Designers

  • 先提出“外部语法”这个问题,并不是因为嵌入式 DSL 总是更优(见下文“这会给你带来什么代价”),而是因为这个问题值得得到明确回答,而不是出于惯性被跳过:为什么这不是一个 Kotlin/TypeScript/Python 构建器 API?
    Ask the foreign-syntax question first not because embedded DSLs are always superior (see "What This Costs You" below), but because the question deserves an explicit answer rather than being skipped by habit: Why is this not a Kotlin/TypeScript/Python builder API?
  • 在选择宿主语言时,应将训练数据亲和性视为与类型系统强度和生态系统成熟度同等重要的选择标准。
    Treat training-data affinity as a selection criterion on equal footing with type-system strength and ecosystem maturity when choosing a host language.
  • 通过命名参数、有意优先使用枚举而非字符串,以及由各自决定哪些幻觉类别应成为编译错误、哪些应保持不可见的上下文作用域构建器,来有意识地设计编译器的严格程度。
    Design the compiler's strictness deliberately with named parameters, enums over strings, and context-scoped builders each deciding which hallucination class becomes a compile error versus staying invisible.
  • 将诊断视为一种接口:它使用精确的源代码位置、命名的预期值以及可解析的输出,来确定修复循环的收敛速度。
    Treat diagnostics as an interface that uses exact source locations, named expected values, and parseable output to determine how fast a repair loop converges.
  • 构建一个教学频道,提供经过精选、CI 验证且可调用的示例,其效果优于静态填充到提示词中的文档。
    Build a teaching channel with curated, CI-verified, callable examples outperform documentation stuffed statically into the prompt.
  • 取消宽容处理。要认识到,默默“修复”格式错误的输入是预言机的信息泄露,而不是一种体贴,因此默认应严格且明确地失败。
    Abolish leniency. Recognize that silently "fixing" malformed input is a leak in the oracle, not a courtesy, so fail strictly and loudly by default.

这会让你付出什么代价What This Costs You

TDG 要付出三项实际代价。记法自由度:内部 DSL 看起来像宿主语言。例如,classDiagram(name = "Order") { ... } 读起来比专门设计的紧凑记法更慢,不过,一旦图表渲染出来,DSL 源码通常就不再是阅读界面。已经投入多年、构建出可用外部 DSL 的既有投资,并不能成为将其彻底推倒重来的理由;TDG 关注的是下一种语言或模式,而不是本季度的迁移。 宿主语言承诺:用户需要使用宿主工具链;对人类用户而言,还需要具备熟练阅读该语言的能力。对于已经身处这一生态系统的团队来说,这种承诺不会带来额外成本——但对于从未见过 lambda 的领域专家受众而言,这确实是一项需要首次承担的成本。TDG costs three real things. 
Notational Freedom: 
An internal DSL looks like the host language. As an example, classDiagram(name = "Order") { ... } reads more slowly than a purpose-built compact notation, though the DSL source is rarely the reading surface once diagrams render. 
Existing Investment 
Years sunk into a working external DSL aren't an argument to burn it down; TDG is about the next language or schema, not this quarter's migration. 
A Host-Language Commitment
Users need the host toolchain and, for humans, reading fluency in it. That commitment costs nothing extra for teams already living in that ecosystem — but it is a real, first-time cost for a domain-expert audience that has never seen a lambda.

作为交换,你将获得一整套编译器、IDE、重构工具链和幻觉预言机;这些东西无需你亲自构建,而你最可能的未来作者早已对它们了如指掌。对于越来越多的领域而言,这笔交易明显有利于 TDG,但它终究是一笔交易。In exchange you get a full compiler, IDE, refactoring toolchain, and hallucination oracle, none of which you build, all of which your most probable future author already knows. For a growing class of domains that trade is lopsided in TDG's favor, but it is a trade.

第二个问题The Second Question

DSL 设计一直在追问:一种语言能够在多大程度上贴近其领域和人类读者。如今,另一个问题也作为一个独立的设计维度与之并列,而不是取代前者,也不一定在每种情境下都与前者同等重要:这门语言与其最有可能的未来作者已经能够完成的事情有多接近?它的工具链又能免费为你提供多少验证?对于主要作者仍是人类领域专家的 DSL,经典的可读性问题仍然可能占据主导地位;而对于越来越多由模型生成或供模型读取的 DSL,这第二个问题的重要性则会按比例提升。认真对待这两个问题的设计者,最终会构建出这样的语言:幻觉不再是输出中无声的幽灵,而会变成编辑器里的红色下划线。这里没有任何事情需要等待未来的研究。宿主语言、编译器以及教学通道协议如今都已经存在。DSL design has always asked how close a language can come to the domain and the human reader. A second question now stands beside it as a design dimension in its own right, not as a replacement and not necessarily as an equal one in every context: How close does the language already sit to what its most probable future author can already do and how much verification does its toolchain hand you for free? For DSLs whose primary authors remain human domain experts, the classic readability question can still dominate; for DSLs increasingly generated or read by models, this second question gains proportionally more weight. Designers who take both seriously end up building languages where hallucinations stop being silent phantoms in the output and start being red underlines in the editor. Nothing here waits on future research. The host languages, compilers, and teaching-channel protocol all exist today.

参考文献References

  1. X. Chen, M. Lin, N. Schärli, D. Zhou: "Teaching Large Language Models to Self-Debug." arXiv:2304.05128, 2023; and T. X. Olausson, J. P. Inala, C. Wang, J. Gao, A. Solar-Lezama: "Demystifying GPT Self-Repair for Code Generation." arXiv:2306.09896, 2023
  2. Pulumi(pulumi.com)、AWS CDK(aws.amazon.com/cdk)和 CDK for Terraform(developer.hashicorp.com/terraform/cdktf)使用带类型的提供商 SDK,通过通用编程语言定义基础设施。目前这些工具都没有正式发布 Kotlin SDK;本文中的 Kotlin 示例是一个思想实验,并非现有库。
    Pulumi (pulumi.com), AWS CDK (aws.amazon.com/cdk), CDK for Terraform (developer.hashicorp.com/terraform/cdktf) infrastructure definition in general-purpose languages with typed provider SDKs. None currently ships an official Kotlin SDK; the Kotlin example in this article is a thought experiment, not an existing library.
  3. Microsoft: TypeChat. GitHub repository, 2023. https://github.com/microsoft/TypeChat
  4. B. Wang、Z. Wang、X. Wang、Y. Cao、R. A. Saurous、Y. Kim:“使用大型语言模型进行领域特定语言生成的语法提示。” NeurIPS 2023。arXiv:2305.19234
    B. Wang, Z. Wang, X. Wang, Y. Cao, R. A. Saurous, Y. Kim: "Grammar Prompting for Domain-Specific Language Generation with Large Language Models." NeurIPS 2023. arXiv:2305.19234
  5. G. Campagna 等人:《Genie:虚拟助手命令自然语言语义解析器生成器》。PLDI 2019(ThingTalk 作为类型化目标语言;另请参阅 arXiv:2203.12751)
    G. Campagna et al.: "Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands." PLDI 2019 (ThingTalk as the typed target language; see also arXiv:2203.12751)
  6. Z. Sun、X. Du、Z. Yang、L. Li、D. Lo:“AI 程序员就在我们身边:重新思考面向高效代码生成的编程语言语法。”ISSTA 2024。arXiv:2404.16333
    Z. Sun, X. Du, Z. Yang, L. Li, D. Lo: "AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation." ISSTA 2024. arXiv:2404.16333
  7. X. Ye, R. Sun, S. Ö. Arık, T. Pfister: "Effective Large Language Model Adaptation for Improved Grounding" (AGREE). Google Research, NAACL 2024. arXiv:2311.09533
  8. F. Cassano 等人:《MultiPL-E:一种用于神经代码生成基准测试的可扩展多语言方法》。IEEE《软件工程汇刊》,2023 年。arXiv:2208.08227
    F. Cassano et al.: "MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation." IEEE Transactions on Software Engineering, 2023. arXiv:2208.08227

值得记下

模型写 Kotlin、TypeScript、Python 和 SQL 表现良好的主要原因是这些语言在其训练数据中出现次数极多。
不要为 LLM 从未见过的语法发明新语法:将领域嵌入到 LLM 已知的语言中,让编译器成为预言。
模型生成,编译器检查,诊断信息反馈作为修复提示。
Typed Domain Grounding 将这一其他轴进行 grounding。
编译器反馈修复循环在自调试文献中已有详细记载[1] —— GCR 本身并不是 TDG 的新颖部分。