← Starchild Wiki

Jev: Fast Judgements in the Agent LoopJev:agent 循环里的快速判断

What we hand to a small classifier, what we deliberately keep away from it, and how we keep it honest哪些判断交给小型分类模型、哪些刻意不交给它,以及怎么持续校验

TypeSafe System One · jev-latest · updated 2026-09-26

01What Jev isJev 是什么

Jev does not write text. Give it a short state and a few typed questions; in about 200 ms it returns a choice, or the probability of a yes.Jev 不生成文字。给它一段简短的状态和几个限定答案类型的问题,大约 200 毫秒后返回一个选项,或一个“是”的概率。

An agent makes dozens of small decisions every turn — which model to use, whether a message corrects the current task or starts a new one, whether a reply needs quick-action buttons. Each used to be a full LLM call costing about a second. Jev makes them nearly free, as long as the question is the right shape.agent 每一轮都要做很多小判断:用哪个模型、这条消息是在纠正当前任务还是开新任务、这条回复要不要出快捷按钮。以前每个判断都是一次完整的 LLM 调用,要一秒左右。只要问题的形状合适,交给 Jev 几乎不花钱。

The rule: Jev gets single-pass judgements over short input with a small, fixed set of answers. Code computes the facts before Jev sees them, and code decides what to do with the answer.准则:Jev 只做单次判断,输入要短、候选答案少而固定。交给 Jev 之前由代码先算好事实,拿到答案后也由代码决定怎么做。

Code computes facts first Jev choice / p, ~200 ms Code threshold → action act LLM fallback unsure / Jev down

Division of labour. Jev says what something is; code decides what happens next. Every site fails open to the previous behaviour.分工。Jev 判断"这是什么",代码决定"接下来做什么"。每一处出错时都退回原来的做法(fail-open)。

02Where Jev runs today目前用 Jev 的地方

Ten decision points. Every one fails open: if Jev errors or is unsure, the previous behaviour takes over.共十处。每一处都 fail-open:Jev 出错或拿不准时,退回原来的做法。

Site环节Question Jev answersJev 回答的问题What it buys带来什么
Conductor routingConductor 路由Which difficulty lane fits this turn?这一轮该走哪个难度档?Classifies ~96% of traffic at ~90% lower cost than the LLM classifier承担约 96% 的分类流量,成本比 LLM 分类器低约 90%
Browser step浏览器每步决策Which element or action next?下一步点哪个元素、做什么?One cheap decision per page step每一步页面操作只花一次低成本判断
Compaction prune压缩剪枝Is this old tool output obsolete?这条旧工具输出已经过时了吗?Frees context without dropping unseen or evidence-bearing output腾出上下文,但没看过的、带证据的输出一律保留
USER.md gateUSER.md 门禁User preference, or implementation record?这是用户偏好,还是实现记录?Keeps PR numbers and fix logs out of the user profile挡住 PR 号、修复记录写进用户画像
Review gate复盘预判Does this turn contain anything durable?这一轮有值得沉淀的内容吗?Skips empty background reviews跳过没有内容的后台复盘
Memory dedup记忆去重Duplicate or update of an existing entry?和已有条目重复,还是在更新它?Fewer near-duplicate memory entries减少近似重复的记忆条目
Inject classification插话分类Supplement, correction, urgent or independent?补充、纠正、紧急,还是独立新任务?~200 ms instead of ~1 s on the real-time path实时路径上从约 1 秒降到约 200 毫秒
Quick-action buttons快捷按钮Final, proposal, question, options or follow-up?最终结论、提议、提问、选项还是追问?Buttons match the reply; replies over 4000 chars go to the LLM按钮和回复类型对得上;超过 4000 字的回复交给 LLM
Turn extraction每轮结论提取Worth extracting a conclusion?这一轮值得提取结论吗?Skips extraction on turns with nothing durable没有可沉淀内容的轮次直接跳过
Cancel intent取消意图Is the user stopping or redirecting?用户是在叫停或改方向吗?Keeps the agent from resuming dropped work (keyword prefilter first)防止 agent 继续已被叫停的任务(先过关键词预筛)

03Where Jev doesn't fit不适合交给 Jev 的判断

If a decision needs reasoning across several steps or reading a long document, Jev is the wrong tool — however cheap it looks.凡是需要跨多步推理、或要读完一篇长文档才能下的判断,都不该交给 Jev,哪怕它看起来很省钱。

Kind of judgement判断类型Why Jev struggles为什么不行Hand it to交给谁
Weighing evidence across a run — “did the agent really finish?”跨整轮权衡证据——“agent 真的做完了吗?”The answer depends on which earlier tool results support which claim; probabilities cluster near 0.5 even with the evidence in view答案取决于前面哪些工具结果支撑哪句话,证据就在眼前时概率也挤在 0.5 附近Code checks the artifact (file exists, test passed); LLM for the rest代码直接核对产物(文件在不在、测试过没过),其余交给 LLM
Understanding long text — “does this summary still keep the request?”理解长文本——“这份摘要还保留着用户的请求吗?”Multi-thousand-character input blurs “kept” and “lost”; no threshold separates them几千字的输入里“保留”和“丢失”分不开,找不到能用的阈值LLM, or a structured field the summary must fillLLM,或让摘要填一个结构化字段
Judging progress over a sequence — “is the agent looping?”判断一串动作有没有进展——“agent 在原地打转吗?”Polling and sectioned reads look like repetition; telling them apart is multi-step轮询、分段读文件看起来都像重复,区分它们本身就是多步判断Code counters with explicit rules (same call + same result N times)代码计数加明确规则(同样的调用、同样的结果连续 N 次)
Open-ended generation or parameter filling开放式生成、填参数Jev only picks from answers you list; it cannot write or composeJev 只能从给定答案里选,不能写,也不能组合LLMLLM
Confirming an outcome确认结果A confident “done” is not proof the file was written高置信的“完成”不能证明文件真的写了Code代码

04Keeping it honest持续校验

Every successful Jev call is logged with its full input, so any threshold or wording change can be re-tested on real traffic before it ships.每次 Jev 调用成功都会记下完整输入,改阈值或改问题描述之前,可以先在真实流量上重测。

The replay log stores state, questions, answers, model and latency on the machine itself and rotates at 20 MB. The replay script reads it two ways: offline, it reports answer distributions and how many decisions cross a given threshold; live, it re-asks Jev — optionally with new question wording — and reports agreement with the original answers.回放日志把状态、问题、答案、模型和耗时存在本机,超过 20 MB 轮换。回放脚本有两种用法:离线统计答案分布、某个阈值会越过多少条;在线把记录重新发给 Jev(可以换一套问题描述),对比和原答案的一致率。

Three habits make the numbers trustworthy:三个习惯让这些数字可信:

1. Hold out a slice. The script always reports the most recent 30% of records separately; a threshold that only works on the samples it was tuned on shows up immediately.
2. Shadow first. A new site logs its answers without acting on them until the distribution is known.
3. Keep the log clean. Tests write to a sandbox, never to the machine’s real decision log, and billing rows from replay are persisted before a script exits.
1. 留一组样本不参与调参。脚本固定把最近 30% 的记录单独报告,只在调参样本上好看的阈值会马上暴露。
2. 先旁路运行。新环节先只记录、不拦截,看清分布再定阈值。
3. 保持日志干净。测试只写沙箱,不写机器上真实的判断日志;回放产生的计费记录在脚本退出前落盘。

05Before adding a new Jev site新增 Jev 环节前的检查

1. Compute anything countable in code first — lengths, truncation, test results — and give Jev the computed facts.
2. Keep the input short and the answers fixed; ask several questions about one state in a single call.
3. Run it in shadow mode and set thresholds from the logged distribution, confirmed on a holdout slice.
4. Fail open to the previous behaviour, and verify outcomes in code.
5. If it needs reasoning across steps or reading a long document, it belongs in section 03, not here.
1. 能用代码算的先算好——长度、是否截断、测试结果——只把算好的事实交给 Jev。
2. 输入要短、答案固定;同一段状态的多个问题一次问完。
3. 先旁路运行,用记录下来的分布定阈值,再在留出样本上确认。
4. 出错时退回原来的做法,结果用代码独立核对。
5. 需要跨步推理或阅读长文档的,归到第 03 节,不在这里。