An OpenAI-compatible endpoint that picks the model for you — send model: "auto", pay for the lane that actually answered一个 OpenAI 兼容的接口，替你选模型——发 model: "auto"，按真正回答的那条腿计费
Status. Live and serving. Four difficulty lanes plus two media lanes, driven by a real LLM classifier. What is not wired yet: the effort dial (model choice only for now) and multi-turn context is the caller's responsibility — see §07 and §09.状态。已上线可用。四条难度腿 + 两条媒体腿，由真实的 LLM 分类器驱动。尚未接入的：effort 档位（当前只选模型），以及多轮上下文由调用方负责——见 §07 与 §09。
One endpoint, one API key, and a model name of auto. Everything else is the OpenAI Chat Completions API you already know.一个接口、一把 API key、模型名填 auto。除此之外就是你已经熟悉的 OpenAI Chat Completions API。
Most turns in a real workload are easy, and paying frontier prices for all of them is the largest avoidable cost in any LLM product. The Starchild API spends one cheap classifier call per request to decide which model should answer, then forwards the request to that model. You are billed for the model that actually ran, not for the most expensive one you might have needed.真实业务里大多数请求都很简单，为所有请求支付顶级模型价格，是任何 LLM 产品中最大的一笔可避免成本。Starchild API 每次请求花一次廉价的分类器调用来决定谁来回答，然后把请求转发给那个模型。你按实际执行的模型计费，而不是按你「可能需要」的最贵那个。
It is a drop-in replacement: point any OpenAI-compatible SDK at the base URL and change the model name. No new client library, no proprietary request shape.它是可直接替换的：把任何 OpenAI 兼容 SDK 的 base URL 指过来，改一下模型名即可。不需要新的客户端库，也没有私有的请求格式。
curl https://sc-router.fly.dev/v1/chat/completions \
-H "Authorization: Bearer $SC_ROUTER_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "auto",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'from openai import OpenAI
client = OpenAI(
base_url="https://sc-router.fly.dev/v1",
api_key=os.environ["SC_ROUTER_KEY"],
)
resp = client.chat.completions.create(
model="auto", # <- the only change
messages=[{"role": "user", "content": "..."}],
)
print(resp.choices[0].message.content)
print(resp.model) # which model actually answeredimport OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://sc-router.fly.dev/v1",
apiKey: process.env.SC_ROUTER_KEY,
});
const r = await client.chat.completions.create({
model: "auto",
messages: [{ role: "user", content: "..." }],
});Getting a key. Keys are issued per person with a spend cap attached — ask for one rather than sharing an existing key, so spend stays attributable. Streaming (stream: true), temperature, max_tokens, tools and every other standard parameter pass straight through to the chosen model.拿 key。key 按人签发并带独立额度上限——请单独申请，不要共用别人的 key，这样花销才归属得清楚。流式（stream: true）、temperature、max_tokens、tools 以及其它所有标准参数都会原样透传给选中的模型。
Both come from the same fact: some lanes run reasoning models, which emit thinking tokens before the answer. Neither is a bug, but both will look like one if you are not expecting them.两件事同源：部分腿使用推理模型，它们会在正文之前先产出思考 token。都不是 bug，但如果没预期到，看起来都像 bug。
1 · Give max_tokens real room. Thinking tokens and the answer share the same max_tokens budget. Set it too low and the budget is spent before the answer starts — you get content: null with finish_reason: "length", which reads like a broken endpoint. A short question survives on 256; anything that produces code or a structured answer needs 800 or more. When in doubt, leave max_tokens unset and let the model's own default apply.1 · max_tokens 要给够。思考 token 和正文共用同一份 max_tokens 预算。给太小，预算在正文开始前就用完了——你会拿到 content: null 加 finish_reason: "length"，看起来就像接口坏了。短问题 256 够用；要出代码或结构化答案的，给 800 以上。拿不准就不要设 max_tokens，让模型用自己的默认值。
2 · When streaming, expect empty delta.content first. While the model is thinking, the text arrives in delta.reasoning_content and delta.content is an empty string. A client that only watches delta.content will appear frozen for the first stretch of the stream. Either render reasoning_content as a thinking indicator, or simply ignore it and wait — the answer does arrive on delta.content.2 · 流式下 delta.content 一开始是空的。模型思考期间，文本走 delta.reasoning_content，而 delta.content 是空字符串。只监听 delta.content 的客户端在流的前一段会显得卡住。要么把 reasoning_content 渲染成「思考中」提示，要么直接忽略它等着——正文最终一定会走 delta.content 过来。
The classifier is a separate, isolated step in front of the answer — the same shape as the main agent flow, with the model-picking pulled out into its own fast pass.分类器是回答之前一个独立隔离的步骤——和主流程同构，只是把「选模型」这一步单独拎出来跑一趟快的。
Two calls, one answer. The classifier reply is a single JSON object — it never reaches the user.两次调用，一个回答。分类器的输出只是一个 JSON 对象，永远不会送到用户面前。
Concretely, per request:具体到每个请求：
{"lane": "...", "reason": "..."}. Temperature is 0.分类器一趟。一个小而快的模型拿到路由规则作为 system prompt，加上裁剪过的对话记录，回复 {"lane": "...", "reason": "..."}。temperature 为 0。auto for the chosen lane's real model and forwards the untouched request.改写并转发。网关把 auto 换成选中腿的真实模型，然后原样转发请求。The classifier can never fail your request. No key, network timeout, malformed JSON, an invented lane name — every failure path resolves to the normal lane and the request proceeds. A routing problem degrades routing quality; it never turns into an error the caller sees. The fallback is deliberately normal rather than cheap: a misroute should land on the safe side.分类器永远不会让你的请求失败。没有 key、网络超时、JSON 解析不了、编了个不存在的腿名——所有失败路径都落到 normal 腿并继续执行。路由出问题只会降低路由质量，绝不会变成调用方看到的错误。兜底刻意选 normal 而非 cheap：判错要落在安全的一侧。
Four difficulty lanes and two media lanes. You never name a lane — the classifier does — but knowing them makes the response headers readable.四条难度腿 + 两条媒体腿。你不需要指定腿名（分类器会选），但知道它们能让你读懂响应头。
| Lane腿 | Model模型 | When it is picked什么时候会选它 |
|---|---|---|
| cheap | deepseek-v4-flash | The whole turn is one self-contained pass: a lookup, a translation, one calculation, a single fact, an isolated snippet.整轮是一次自足的处理：查询、翻译、一次计算、单个事实、孤立的代码片段。 |
| normal | gpt-5.6-luna | The default working lane. Dependent steps, fetched data, deliverables, tool-chaining, or a strict output contract — but routine thinking.默认主力腿。有依赖步骤、需要取数、要交付物、工具串联，或有严格输出契约——但思考本身是常规的。 |
| strong | gemini-3.7-flash | Genuine reasoning difficulty: tricky algorithms, architecture trade-offs, subtle multi-cause debugging — where a plausible-but-wrong answer is expensive.真正的推理难度：棘手算法、架构权衡、多因交织的隐蔽 bug——「看似合理但错了」代价很高的场合。 |
| escalation | glm-5.3 | A recovery lane, not a difficulty tier. The user explicitly asks for a stronger model, or an earlier lane already failed this same task.兜底重试腿，不是难度档。用户明确要更强的模型，或前一条腿在同一任务上已经失败过。 |
| vision | gpt-5.6-luna | Any turn carrying an image. Decided without a classifier call.任何带图片的请求。不经分类器直接判定。 |
| video | gemini-3.7-flash | Any turn carrying a video. Shares a model with strong — that is deliberate, not a bug.任何带视频的请求。与 strong 共用模型是刻意设计，不是 bug。 |
Difficulty, not length. The single most common wrong intuition is that a long or technical-sounding request should get a big model. It should not. A three-paragraph request to reformat some text is mechanical and lands on cheap; a two-line question about a race condition is hard and lands on strong. The rules explicitly forbid judging by length, topic, or vocabulary.看难度，不看长度。最常见的错误直觉是：长的、听起来技术的请求应该给大模型。不对。三段长的「把这段文字重排一下」是机械活，落 cheap；两行的「并发下的竞态」是难题，落 strong。规则里明令禁止按长度、主题或用词判断。
Two surfaces carry the same data: a structured conductor object in the JSON body, and a flat projection of it in HTTP headers. They are generated from one source, so they can never disagree.两个表面承载同一份数据：JSON body 里的结构化 conductor 对象，以及它在 HTTP 头上的扁平投影。二者同源生成，不可能对不上。
conductor blockbody 里的 conductor 块Read this if you want structured data. OpenAI-compatible SDKs ignore unknown top-level fields, so its presence is safe for existing clients.需要结构化数据就读它。OpenAI 兼容 SDK 会忽略不认识的顶层字段，所以它的存在对现有客户端是安全的。
{
"version": "3.3",
"request_id": "9180e63a-…",
"mode": "auto", // auto | pinned | passthrough
"session": { "id": "demo-1", "source": "header" },
"routing": {
"lane": "cheap-lane",
"requested": "auto",
"model": "deepseek/deepseek-v4-flash-0731",
"reason": "single-fact, self-contained question",
"fallback": false, // true = 分类器失败，走了兜底腿
"classifier": {
"model": "google/gemini-3.1-flash-lite",
"latency_ms": 878.0,
"cost": 0.00007713,
"billed": false, // 分类成本是否计入你的账单
"prompt_tokens": 1539,
"completion_tokens": 33,
"cached_tokens": 1525 // prompt cache 命中量
},
"effort": null, // 预留：思考档位，尚未接入
"candidates": [], // 预留：候选腿及排序
"policy": {} // 预留：allowed / excluded lanes
},
"cost": {
"currency": "USD",
"upstream": 0.00001736, // 回答本身的成本
"routing": 0.00007713, // 分类器那一次的成本
"markup_rate": 0,
"billed": 0.00001736, // 实际记到你账上的金额
"source": "upstream"
},
"usage": { "prompt_tokens": 11, "completion_tokens": 40, "total_tokens": 51 },
"timing": { "classify_ms": 878.0, "total_ms": 1992.4 },
"upstream": {
"provider": "openrouter",
"id": "4fa84f8a-…",
"finish_reason": "stop",
"is_byok": false
}
}Field stability. Fields are only ever added, never removed or repurposed. A breaking change bumps version. The three null/empty fields under routing are reserved shape — they will be populated later without changing the contract, so code against them defensively rather than assuming they stay empty.字段稳定性。字段只增不删、不改语义。破坏性变更会提 version。routing 下那三个 null／空字段是预留的结构位——将来会被填上而不改契约，所以按「可能有值」来写代码，别假设它们永远为空。
The same values, flattened — convenient for logging and for streamed responses. Nineteen headers are returned on every request; the ones worth logging are marked.同样的值，扁平化——方便打日志，也方便流式响应取用。每个请求都会返回 19 个头，值得记录的已标出。
| Header响应头 | Meaning含义 |
|---|---|
| -version | Contract version, currently 3.3.契约版本，当前 3.3。 |
| -request-id | log this Our id for the turn. Quote it when reporting anything.建议记录 我们这边这一轮的 id。反馈问题时带上它。 |
| -mode | auto (classifier chose) · pinned (you named a lane) · passthrough (you named an upstream model).auto（分类器选的）· pinned（你点了腿名）· passthrough（你点了上游模型）。 |
| -session-id | Echo of your x-session-id request header, if you sent one.回显你发的 x-session-id 请求头（如果发了）。 |
| -lane | log this Which lane served the turn.建议记录 这一轮由哪条腿服务。 |
| -requested | What you originally sent as model.你最初发的 model 值。 |
| -model | log this The real upstream model that answered.建议记录 真正回答的上游模型。 |
| -reason | log this The classifier's one-line justification. First place to look when a route seems wrong.建议记录 分类器的一行理由。路由看起来不对时，先看这个。 |
| -fallback | true means classification failed and the failsafe lane took over. Worth alerting on if it stops being rare.true 表示分类失败、由兜底腿接手。如果它不再罕见，值得报警。 |
| -classifier-model | Which model did the classification.做分类的是哪个模型。 |
| -classify-ms | Classifier latency. Typically 0.2–0.9s.分类器耗时。通常 0.2–0.9 秒。 |
| -classifier-cached-tokens | Prompt-cache hits on the rules text. A high number here is why routing is cheap.规则文本的 prompt cache 命中量。这个数高，路由才便宜。 |
| -routing-cost | What the classifier call cost, in USD.分类器那次调用的成本（美元）。 |
| -routing-billed | false today: routing is on us, you are not charged for it.当前为 false：路由成本我们自己承担，不向你收取。 |
| -upstream-cost | Cost of the answer itself.回答本身的成本。 |
| -markup-rate | Markup applied. 0 = at cost.加价率。0 表示原价。 |
| -billed | log this What actually hit your account. Equals (upstream + routing×billed) × (1 + markup).建议记录 真正记到你账上的金额。等于 (upstream + routing×是否计费) × (1 + 加价率)。 |
| -cost-source | Where the cost figure came from — upstream means the provider reported it directly.成本数字的来源——upstream 表示供应商直接实报。 |
| -total-ms | End-to-end time, classification included.端到端耗时，含分类。 |
All header names carry the x-conductor- prefix, elided above for width. A live example:所有头名都带 x-conductor- 前缀，上表为宽度省略。一个真实样本：
x-conductor-version: 3.3
x-conductor-mode: auto
x-conductor-lane: cheap-lane
x-conductor-model: deepseek/deepseek-v4-flash-0731
x-conductor-reason: The request is a single-fact, self-contained question.
x-conductor-fallback: false
x-conductor-classifier-model: google/gemini-3.1-flash-lite
x-conductor-classify-ms: 878.0
x-conductor-classifier-cached-tokens: 1525
x-conductor-routing-cost: 0.0000771250
x-conductor-routing-billed: false
x-conductor-upstream-cost: 0.0000173600
x-conductor-billed: 0.0000173600
x-conductor-total-ms: 1992.4The standard model field in the JSON body also reflects the model that actually ran — not auto — so an SDK that logs resp.model gets the truth for free.JSON body 里标准的 model 字段同样反映实际执行的模型（不是 auto），所以只要 SDK 记录了 resp.model，你白拿一份真相。
Auto-routing is opt-in per request. Any other model name is passed through untouched, so the same key works for both styles:自动路由是按请求选用的。其它任何模型名都会原样透传，因此同一把 key 两种用法都能用：
cheap-lane, normal-lane, strong-lane or escalation-lane as the model name to skip the classifier and go straight to that tier. Useful for A/B tests and for workloads you have already characterised.钉住某条腿。把模型名写成 cheap-lane、normal-lane、strong-lane 或 escalation-lane，跳过分类器直达该档。适合做 A/B 对比，或用于你已经摸清楚的固定负载。openrouter/ — e.g. openrouter/anthropic/claude-opus-4.5 — and the gateway acts as a plain proxy. There is no model whitelist; your spend cap is the guardrail.点名任意上游模型。加 openrouter/ 前缀——例如 openrouter/anthropic/claude-opus-4.5——网关就退化成普通代理。没有模型白名单，你的额度上限就是护栏。GET /v1/models with your key lists every routable name.查有哪些可用。带上你的 key 请求 GET /v1/models，会列出所有可路由的名字。Legacy names changed meaning. An earlier version shipped three lanes where strong-lane meant luna and escalation-lane meant gemini. Under V3.3 luna is normal and gemini is strong. The old names still resolve — as legacy-strong-lane and legacy-escalation-lane — so nothing breaks, but if you pinned a lane by name before this release, re-read the table in §04: strong-lane now points somewhere else.旧腿名的语义变了。早期版本是三档，其中 strong-lane 指 luna、escalation-lane 指 gemini。V3.3 里 luna 是 normal、gemini 是 strong。旧名仍然可用（对应 legacy-strong-lane 与 legacy-escalation-lane），不会 404，但如果你在本次发布前按名字钉过腿，请重读 §04 的表：strong-lane 现在指向的是另一个模型。
The API is stateless, exactly like the OpenAI API. It keeps no conversation for you — and that has one routing consequence worth understanding.这个 API 是无状态的，和 OpenAI API 完全一样。它不替你保存对话——而这一点对路由有一个值得理解的影响。
Send the full messages array every turn, as you already do with any OpenAI-compatible endpoint. The gateway reads the tail of it to classify, so the array is not just history for the model — it is the evidence the API judges on.每一轮都把完整的 messages 数组发过来，就像你在任何 OpenAI 兼容接口上做的那样。网关会读取它的尾部来做分类，所以这个数组不只是给模型看的历史——它同时是路由判断的依据。
A message like “go on”, “redo it”, or a pasted stack trace has no complexity of its own. Judged alone it looks trivial and would route to cheap — which is wrong if it is the next step of a hard debugging session. The classifier therefore inherits the lane from the task in flight, but it can only do that if it can see the task. Send only the latest message and every follow-up looks trivial.「继续」「重做」或者贴一段报错，这类消息本身没有复杂度。单独看它们像琐事，会被路由到 cheap——如果它其实是一场艰难 debug 的下一步，那就错了。分类器因此会从在途任务继承档位，但前提是它看得见那个任务。只发最新一条消息，每个追问看起来都像琐事。
x-conductor-reason before filing anything — it usually says exactly what the classifier thought it saw.如果某次定腿看起来不对，先读 x-conductor-reason 再反馈——它通常会直说分类器以为自己看到了什么。Each key carries its own spend cap and rate limit; spend is recorded per call against the key that made it. Two lines of cost make up a routed request:每把 key 有自己的额度上限和速率限制；每次调用的花销都记在发起它的那把 key 上。一次被路由的请求包含两笔成本：
x-conductor-routing-cost.路由，即分类器那次调用，单独通过 x-conductor-routing-cost 暴露。Costs are read from what the upstream actually reports per call rather than from a maintained price table, so they do not drift when a provider changes pricing.成本取自上游逐笔实报，而不是我们自己维护的价目表，因此供应商改价时不会产生偏差。
Stated plainly, because you will hit these before we fix them.直说，因为在我们修好之前你就会撞上。
| Edge边界 | What it means for you对你意味着什么 |
|---|---|
| Routing cost路由成本 | The classifier can cost more per call than a cheap-lane answer, because the full rules text goes in as its system prompt. Prompt caching is confirmed working — around 1525 of 1539 prompt tokens hit cache — which keeps routing at roughly $0.00008 per request. You are not billed for it either way (routing-billed: false), but it does mean the margin on trivial turns is thinner than it looks.分类器每次调用的成本可能高于 cheap 腿回答一次，因为完整规则文本要作为 system prompt 送进去。提示词缓存已确认生效——1539 个 prompt token 中约 1525 个命中缓存——路由成本因此稳定在每请求约 $0.00008。无论如何这笔钱不向你收取（routing-billed: false），但它意味着琐碎请求上的实际利润比看起来薄。 |
| Reasoning lanes推理腿 | The cheap lane runs a reasoning model, so thinking tokens share the max_tokens budget and stream on delta.reasoning_content. See the two callouts in §02 — this is the most common first-call surprise.cheap 腿使用推理模型，因此思考 token 与正文共用 max_tokens 预算，且在流式下走 delta.reasoning_content。见 §02 的两条提示——这是第一次调用最常见的意外。 |
| Model scope可用模型范围 | Passthrough resolves against the OpenRouter catalogue only. Self-hosted endpoints, vLLM deployments and provider-side fine-tunes (ft:…) are not reachable today. Custom per-key model mappings are not supported yet either — everyone runs the official routing rules.透传只解析 OpenRouter 目录内的模型。自托管 endpoint、vLLM 部署、以及供应商侧微调模型（ft:…）目前接不进来。按 key 的自定义模型映射同样尚未支持——所有人跑的都是官方路由规则。 |
| Effort dialeffort 档位 | Routing picks the model only. Per-turn thinking-effort selection is designed but not wired; each model uses its own default tier.路由目前只选模型。逐轮思考档位的选择已设计但未接入；各模型使用自身默认档。 |
| Streaming metadata流式元数据 | Streaming works. Verify header behaviour on your own client before depending on lane headers in a streamed response.流式可用。若你打算依赖流式响应里的腿信息头，请先在自己的客户端上验证其行为。 |
| Cold start冷启动 | After a long idle period the first request can take ~20s while the machine wakes. Steady-state latency is the classifier (~0.2–0.6s) plus the chosen model.长时间闲置后第一个请求可能耗时约 20 秒（机器唤醒）。稳态延迟 = 分类器（约 0.2–0.6 秒）+ 选中模型。 |
Reporting a bad route. Send the request, the value of x-conductor-reason, and which lane you expected. The routing rules are a plain Markdown file, not code — a misroute is usually fixed by editing prose, and that is a fast turnaround.反馈路由错误。把请求内容、x-conductor-reason 的值、以及你期望的腿一起发来。路由规则是一个纯 Markdown 文件而不是代码——判错通常改几句话就能修好，周转很快。
Measured 2026-08-22. Every number below is from a real run, not a projection.2026-08-22 实测。以下每个数字都来自真实运行，不是推算。
Fly edge (sc-router.fly.dev, iad)
├── machine 1 ┐
├── machine 2 │ performance-2x: 2 dedicated cores / 4GB
├── machine 3 │ identical container, three processes each
└── machine 4 ┘
└── console :4200 ← only port exposed, 2 workers
litellm :4000 ← loopback only, 2 workers
account :4100 ← loopback only, 1 process
└── Postgres (Fly managed, iad, max_connections=100)All four machines are peers — no leader, no sticky sessions. Any machine can serve any request.四台机器完全对等——无主从，无会话粘连。任意机器都能处理任意请求。
Two vantage points, because they answer different questions. In-container measures the service; over-the-internet measures what a client actually experiences.两个观测点，回答的是不同问题。容器内测的是服务本身，公网测的是客户端的真实体验。
| Concurrency并发 | In-container, 1 worker (before)容器内，1 worker（扩容前） | In-container, 2 workers (after)容器内，2 worker（扩容后） | Over internet, 4 machines公网，4 台机器 |
|---|---|---|---|
| 10–20 | 633 req/s · p99 53ms | 629 req/s · p99 64ms | 171 req/s · p99 352ms |
| 50 | 178 req/s · p99 1101ms | 173 req/s · p99 1147ms | — |
| 100 | 112 req/s · p99 4998ms | 136 req/s · p99 4935ms | 62 req/s · p99 6369ms |
| 200 | 192 req/s · p99 5609ms | 144 req/s · p99 6908ms | 76 req/s · p99 10827ms |
The worker increase did not show a measurable gain. Doubling workers on both layers produced curves within noise of the single-worker run. This is a limitation of the measurement, not proof the change was useless: the load generator is a single-process asyncio client sharing the same two cores as the service it is testing, and /health returns from memory — so the test saturates the client and the event-loop scheduler long before it saturates CPU. A meaningful verification needs a load generator on separate hardware, driving requests that actually hold a connection. Until that runs, treat the post-scale capacity figure as unverified.worker 翻倍没有测出可见提升。两层都翻倍后，曲线与单 worker 时的差异在噪声范围内。这是测量方法的局限，不能反过来证明改动无效：压测客户端是单进程 asyncio，且与被测服务共用同样的两个核，而 /health 是纯内存返回——测试在压满 CPU 之前，先压满了客户端和事件循环调度器。要真正验证，需要在独立硬件上发压，且请求要真的占住连接。在那之前，扩容后的容量数字应视为未经验证。
What the numbers do establish: zero 5xx at every level, all the way to 200 concurrent. The degradation is queueing, not failure — throughput flattens and tail latency stretches, but nothing is dropped.这些数字确实证明了的：所有档位、直到 200 并发，零 5xx。劣化表现为排队而非失败——吞吐走平、尾延迟拉长，但没有请求被丢弃。
A health check returns instantly; an LLM call holds its connection for 2–30 seconds waiting on the upstream. So the meaningful unit is concurrent connections held:健康检查瞬时返回，而 LLM 调用要占住连接 2–30 秒等待上游。因此有意义的单位是并发持有连接数：
cluster concurrency ≈ 400 (8 event loops × ~50 each)
average hold ≈ 8s
→ steady state ≈ 50 req/s
→ with 4:1 peak-to-trough ≈ 1M requests/day sustainableAdding machines multiplies every process in the container — including the two background loops that charge usage and reconcile Stripe payments. Left alone, four machines with two workers each would run eight copies of both.加机器会让容器内每个进程都翻倍——包括扣费同步和 Stripe 对账这两个后台循环。放任不管，4 台机器 × 2 worker 就是各跑 8 份。
Two independent defences, verified live after the scale-out:两道独立防线，扩容后已实测验证：
| Layer层 | Mechanism机制 | Verified验证结果 |
|---|---|---|
| Leader election选主 | Postgres advisory lock per loop; whoever wins runs it. Session-scoped, so a dead process releases the lock automatically and the next one takes over.每个循环一把 Postgres advisory lock，谁抢到谁跑。锁是会话级的，进程死了自动释放，下一个接管。 | 1 holder each across 8 processes8 个进程中各只有 1 个持有者 |
| Ledger idempotency账本幂等 | request_id unique index with ON CONFLICT DO NOTHING. Rows that conflict never reach RETURNING, so the balance deduction cannot fire twice for one call.request_id 唯一索引配 ON CONFLICT DO NOTHING。冲突行不进 RETURNING，因此同一次调用的余额扣减不可能触发两次。 | Structural — holds even if leader election fails entirely结构性保证——即使选主完全失效也成立 |
The second layer is the one that matters. Leader election is an optimisation that avoids N× redundant table scans and N× Stripe API calls; the unique index is what actually makes double-charging impossible.真正兜底的是第二层。选主只是优化，避免 N 倍的重复全表扫描和 N 倍的 Stripe API 调用；让重复计费成为不可能的，是那个唯一索引。
Where the money goes, and the one number that caps how far this can scale.钱花在哪，以及决定扩容天花板的那个数字。
Four performance-2x machines at roughly $30.74/month each ≈ $123/month, plus managed Postgres (not yet costed). The per-machine figure comes from a third-party pricing compilation, not Fly’s official page — treat it as an estimate until reconciled against an actual invoice.4 台 performance-2x，每台约 $30.74/月，合计约 $123/月，外加托管 Postgres（尚未核价）。单台价格来自第三方价格汇编而非 Fly 官方页面——在与真实账单核对前应视为估算。
This is the larger half, and it is where routing earns its keep. Current per-token pricing by lane:这是更大的那一半，也是路由价值所在。当前各腿单价：
| Lane腿 | Input输入 | Output输出 |
|---|---|---|
| cheap | $0.14/M | $0.28/M |
| normal | $0.20/M | $1.20/M |
| strong | $0.375/M | $1.875/M |
| escalation | $1.40/M | $4.40/M |
A 1.4× to 4× spread between lanes. saved_spend records exactly this difference — the cost of the request as routed, against what it would have cost on the most expensive lane.各腿之间有 1.4 到 4 倍的价差。saved_spend 记录的正是这个差额——实际路由的成本，对比若走最贵腿的成本。
Postgres allows 100 connections, 3 of them reserved. Current draw after scaling to four machines: 42 of 100. Each machine costs roughly 10 connections, with litellm’s prisma pool the largest consumer.Postgres 允许 100 个连接，其中 3 个保留。扩到 4 台后当前占用：42/100。每台机器约占 10 个连接，其中 litellm 的 prisma 池是大头。
| Machines机器数 | Connections预计连接 | Headroom水位 | Verdict结论 |
|---|---|---|---|
| 4 (now)4（当前） | 42 | 42% | Comfortable宽裕 |
| 6 | ~62 | 62% | Still safe仍安全 |
| 8 | ~82 | 82% | Peak jitter will hit the ceiling峰值抖动会撞顶 |
| 10+ | 100+ | — | Requires pgbouncer first必须先上 pgbouncer |
Hitting the ceiling produces too many connections — a site-wide outage, not a graceful degradation. Six machines is the practical limit before a connection pooler becomes mandatory.撞顶的后果是 too many connections——全站级故障，不是优雅降级。在上连接池代理之前，6 台是实际上限。
Scaling runbook. min_machines_running in fly.toml is a floor declaration, not an instruction — it will not create machines. Use flyctl scale count N, then update fly.toml to match or the next deploy shrinks the fleet back. Worker counts are environment variables (CONSOLE_WORKERS, LITELLM_WORKERS) and need no code change.扩容操作手册。fly.toml 里的 min_machines_running 是下限声明而非指令——它不会创建机器。要用 flyctl scale count N，然后同步改 fly.toml，否则下次部署会缩回去。worker 数是环境变量（CONSOLE_WORKERS、LITELLM_WORKERS），无需改代码。
For the design thinking behind the classifier — why structure beats surface, how the rules file stays the single source of truth — see Conductor Mode.关于分类器背后的设计思路——为什么看结构不看表面、规则文件如何保持唯一事实源——见 Conductor Mode。