A voice you can think out loud to — that manages your threads, your tasks and your running work, and leaves everything exactly where the web page expects it一个可以边想边说的声音——它替你管理线程、任务和正在跑的工作，并把一切都留在网页原本期待的位置上
Not a voice skin on a chatbot. Live is the spoken front door to the same agent, the same threads and the same tasks you already have — with one extra job: it keeps the overview while you talk.不是给聊天机器人套一层语音皮。Live 是通往同一个 agent、同一批线程、同一批任务的语音正门——外加一项职责：你说话的时候，它替你握着全局。
The requirement in one sentence: I can talk to Live like a chief of staff — "where are we on the routing benchmark, what's blocked, start the compaction PR review" — and when I sit back down at the web page, every thread is exactly where Live left it, nothing is duplicated, nothing is lost, and I can carry on typing. Everything below is in service of that sentence.需求一句话：我可以像对幕僚长一样跟 Live 说话——「路由基准跑到哪了、什么卡着、把压缩那个 PR 的审查启动一下」——然后回到网页时，每条线程都停在 Live 留下的位置，没有重复、没有丢失，我接着打字就行。下面所有内容都为这句话服务。
| Live mustLive 必须 | Which means这意味着 |
|---|---|
| Know what we're doing知道我们在干什么 | Open a session already holding the thread's gist, open tasks and running jobs — no "remind me what this is about".开口时就已持有线程梗概、未完成事项和正在跑的任务——不需要「先提醒我一下这是什么」。 |
| Leave a trace where I'll look把痕迹留在我会看的地方 | Every spoken turn that matters lands in the thread as a message; the web page shows it like anything else.每一轮有意义的语音都作为消息落进线程；网页把它当普通消息显示。 |
| Hear what happens without me听见没有我在场时发生的事 | A background job finishing, a PR check turning green, a message from another thread — Live learns it silently and mentions it when relevant.后台任务完成、PR 检查变绿、另一条线程来了消息——Live 静默获知，在相关时提起。 |
| Drive, not just answer推进，而不只是回答 | Start, cancel, queue and hand off work across threads — using the runtime's own primitives, never a private task list of its own.跨线程启动、取消、排队、交接工作——用 runtime 自己的原语，绝不维护一份私有任务表。 |
| Stay out of the way不添乱 | Small talk, clarification and progress questions are answered locally in milliseconds; only real work is delegated.闲聊、澄清、问进度在本地毫秒级答掉；只有真正的工作才委托出去。 |
The thread is the single source of truth. Live has no memory, no task list and no tools of its own. It is a second door into the room described in Sessions & Threads.线程是唯一事实源。Live 没有自己的记忆、任务表或工具。它只是通往会话与线程里那间房的第二扇门。
Fig 1.图 1. Web and Live both read from and write into the same thread. Neither holds state the other cannot see.Web 与 Live 都从同一条线程读、往同一条线程写。任何一方都不持有另一方看不见的状态。
This is also what OpenAI's own delegation guide prescribes for GPT-Live. In client delegation mode the model has no tools: a session.delegation.created event carries only metadata, the application assembles the real request from the transcript and its own state, and results come back through three text channels — thinking.append (silent context), commentary.append (say this), instructions.append (behave differently). Their words: "Give it a short prompt for conversation style and when to delegate. Keep detailed instructions, business rules, and tool workflows in the backend." Our backend is Starchild; the five demo "tools" (ask / check / cancel / list / memory) were a Realtime-era habit and are removed.这也正是 OpenAI 自己的委托指南对 GPT-Live 的规定。客户端委托模式下模型没有工具：session.delegation.created 事件只带元数据，应用自己用转写和自身状态组装真正的请求，结果通过三条文本通道回传——thinking.append（静默上下文）、commentary.append（说出来）、instructions.append（改变行为）。原话：「给它一段短 prompt 管对话风格和何时委托。详细指令、业务规则和工具流程留在后端。」我们的后端就是 Starchild；demo 里那五个「工具」（ask / check / cancel / list / memory）是 Realtime 时代的习惯，已移除。
The browser does WebRTC. The bridge does protocol. A binding does the one thing that differs per scenario. The runtime does everything else — it already does.浏览器管 WebRTC，bridge 管协议，binding 管每个场景唯一不同的那件事，runtime 管其余一切——它本来就在管。
Fig 2.图 2. Dependencies point down only. Adding a scenario = adding one binding file and one URL parameter.依赖只向下。加一个场景 = 加一个 binding 文件加一个 URL 参数。
starchild-live/ core/ live-session.mjs seed(text) · think(text, id) · say(text, id) · instruct(text) — slicing, throttling, merging live here transcripts.mjs DataChannel events → completed user / live turns delegator.mjs delegation.created → binding.handle() → think/say(id); barge-in → binding.interrupt(id) bindings/ binding.d.ts the interface below thread.mjs bind to an existing thread onboarding.mjs fresh thread + onboarding script + user state orchestrator.mjs main thread + fan-in of every other thread and job adapters/ starchild-runtime.mjs snapshot · chat · cancel · events · append — the only file that knows :8000 server.mjs POST /api/session · Sideband attach + reconnect · SSE mirror · journal · /status /health index.html WebRTC audio + display only — no control-plane logic
| Binding.* | Contract约定 |
|---|---|
| seed() | Text placed in session.input at connect: what we are doing, what is open, what is running. Comes from the thread's gist and task board, not from a model call.建连时放进 session.input 的文本：在做什么、什么未完成、什么在跑。来自线程梗概与任务板，不额外调模型。 |
| handle(text, ctx) | One delegated user turn in, a stream of {kind:'think'|'say'|'done', text} out. The delegator forwards each chunk with the delegation id — progress, first sentence, result.进一个被委托的用户轮次，出一串 {kind:'think'|'say'|'done', text}。delegator 带着 delegation id 逐块转发——进度、首句、结果。 |
| onUserTurn / onLiveTurn | Turns Live handled by itself, so they can be written into the thread. Nothing spoken is allowed to exist only in audio.Live 自己处理掉的轮次，用于写回线程。任何说出口的话都不允许只存在于音频里。 |
| events() | Things that happened without the user: a message typed on the web, a job finishing, another thread reporting in. Forwarded as thinking.append(null).用户不在场时发生的事：网页上打了一句、任务完成、另一条线程汇报。以 thinking.append(null) 转发。 |
| interrupt(id) | User barged in while a delegation was running. Returns whether it actually stopped — Live is told "stopped" only when true.委托进行中用户插话。返回是否真的停了——只有为真时才告诉 Live「已停」。 |
OpenAI provides exactly four things: an audio front end, a seed slot, one "I need help" signal, and three text channels back. Every named piece of this framework is built on top of those four.OpenAI 恰好只提供四样东西：一个音频前端、一个开场灌入槽、一个「我需要帮忙」的信号、三条文本回传通道。框架里所有带名字的部件都是搭在这四样东西之上的。
| Layer层 | OpenAI GPT-Live API (official)OpenAI GPT-Live API（官方） | Starchild Live (ours)Starchild Live（我们的） |
|---|---|---|
| Transport传输 | Primary connection: WebRTC or WebSocket · wss://…/v1/live/sessions/{id}/attach sideband for server-side control (same bearer key; act on each event once)主连接：WebRTC 或 WebSocket · wss://…/v1/live/sessions/{id}/attach Sideband 供服务端控制（同一把 key；每个事件只处理一次） | Browser WebRTC carries audio only; the bridge attaches the sideband and owns the control plane. Browser-forwarded events are ignored while the sideband is open; falls back to browser forwarding if attach fails浏览器 WebRTC 只传音频；bridge 附着 Sideband 接管控制面。Sideband 打开期间忽略浏览器转发；附着失败回落浏览器路径 |
| Session会话 | POST /v1/live/sessions · WebRTC · delegation.mode = client · short instructions | /api/session {binding, …} picks a binding and forwards选定 binding 后转发 |
| Seed灌入 | session.input — a slot for text history at connect——建连时的文本历史槽位 | Binding.seed() — decides what goes in (gist · tasks · jobs)——决定放什么（梗概 · 任务 · 在跑作业） |
| Speech语音 | Full duplex, VAD, turn-taking, transcript delta events only — no turn-completed event; turns are cut by silence gaps (900 ms user / 1200 ms Live)全双工、VAD、话轮、转写 delta / completed 事件 | core/transcripts assembles completed turns; browser passes audio through拼出完整轮次；浏览器只透传音频 |
| Delegation委托 | session.delegation.created {delegation_id} — metadata only, no task text——只有元数据，没有任务文本 | core/delegator → Binding.handle(text) — where the work goes and how it streams back——工作去哪、结果怎么流回 |
| Return回传 | thinking.append · commentary.append · instructions.append — string, ≤500 tok, id or null——字符串，≤500 tok，带 id 或 null | core/live-session think / say / instruct — slicing, throttling, merging——切片、节流、合并 |
| Write-back写回 | — (no concept)——（无此概念） | Binding.onUserTurn / onLiveTurn → runtime R2runtime R2 |
| Inflow回流 | "You may push UI context via thinking.append(null)"「可以用 thinking.append(null) 推 UI 上下文」 | Binding.events() ← runtime R1; speak-or-stay-silent policyruntime R1；说或不说策略 |
| Interrupt打断 | "Barge-in does not cancel backend work; confirm before saying stopped"「插话不取消后端工作；确认后再说已停」 | Binding.interrupt(id) → boolean · runtime R3runtime R3 |
| Backend后端 | Responses delegation (managed) — not usedResponses 委托（托管）——不使用 | Starchild runtime via adapters/starchild-runtime; thread = single source of truth经 adapters/starchild-runtime 接 Starchild runtime；线程即唯一事实源 |
| Scenarios场景 | — | thread · onboarding · orchestrator bindings; cross-thread via main threadthread · onboarding · orchestrator 三种 binding；跨线程经主线程 |
| Temporary过渡件 | — | voice-log buffer (until R2) · session-prefix probingvoice-log 缓冲（等 R2）· 事件轮询（等 R1）· 会话前缀探测 |
Four connections, each doing one job. The browser is no longer on the control path: a locked phone, a background tab or a bridge restart no longer silences the agent.四条连接，各司其职。浏览器不再处在控制链路上：手机锁屏、标签页切后台、bridge 重启，都不会再让 Agent 失声。
Fig 2b.图 2b. Orange = the control plane, now server-held. Grey = audio and runtime calls.橙色 = 控制面，现由服务端持有；灰色 = 音频与 runtime 调用。
| # | Link连接 | Protocol协议 | Carries传什么 | Why this one为什么用它 |
|---|---|---|---|---|
| ① | Browser ⇄ OpenAI浏览器 ⇄ OpenAI | WebRTC (UDP) | Two-way real-time audio双向实时音频 | Millisecond latency; drops a packet rather than stalling毫秒级延迟；宁丢包不卡顿 |
| ② | Bridge ⇄ OpenAIBridge ⇄ OpenAI | Sideband WebSocket (TCP) | Transcript deltas, delegation.created, usage → in; thinking / commentary / instructions.append → out转写 delta、delegation.created、用量 → 入；thinking / commentary / instructions.append → 出 | Reliable, ordered, and independent of the browser's lifecycle可靠有序，且不依赖浏览器生命周期 |
| ③ | Bridge → BrowserBridge → 浏览器 | SSE | Mirror of what the bridge sent, sideband status服务端发出内容的镜像、Sideband 状态 | Read-only UI; the browser never injects into the DataChannel any more只读 UI；浏览器不再向 DataChannel 注入 |
| ④ | Bridge ⇄ StarchildBridge ⇄ Starchild | HTTP/SSE | Delegated turns, streamed agent replies, thread events委托轮次、Agent 流式回复、线程事件 | Same contract the runtime already exposes与 runtime 现有契约一致 |
| Situation场景 | v0.3 — browser relays controlv0.3——浏览器中转控制 | v0.4 — server-held sidebandv0.4——服务端持 Sideband |
|---|---|---|
| Phone locked / tab in background手机锁屏 / 标签页切后台 | DataChannel pauses, agent hears nothingDataChannel 暂停，Agent 收不到话 | Bridge receives transcripts directly; the task keeps runningbridge 直接收转写，任务照常跑 |
| Bridge restartbridge 重启 | Browser still connected to OpenAI; bridge no longer knows the session — events dropped silently浏览器仍连着 OpenAI，bridge 不认识该会话——事件被静默丢弃 | Bridge holds session_id and re-attaches the same sessionbridge 持有 session_id，重新 attach 同一会话 |
| Injecting results注入结果 | One hop through the browser经浏览器转发一跳 | Sent directly; works even with the browser gone服务端直发；浏览器断了也能注入 |
| Barge-in / speculative prefetch打断 / 投机预取 | Depends on the browser forwarding speech_started依赖前端转发 speech_started | Seen server-side in real time; guardrails possible服务端实时可见，可加 guardrail |
| Resilience容错 | — | Exponential-backoff reconnect (≤6); session.closed stops it; attach failure falls back to browser relay指数退避重连（≤6 次）；session.closed 不重连；附着失败回落浏览器中转 |
| Observability可观测 | Only the browser sees events只有前端能看见 | journal.jsonl (in / sb / out) · /api/bridge/:id/status · /api/healthjournal.jsonl（in / sb / out）· /api/bridge/:id/status · /api/health |
Community frameworks (LiveKit Agents DuplexModel, Twilio GPTLiveProvider, Pipecat) also hold the control plane server-side, but route audio through the server too. We keep audio browser-direct — one hop less — because our target is web voice plus multi-thread agent orchestration, not telephony.社区框架（LiveKit Agents DuplexModel、Twilio GPTLiveProvider、Pipecat）同样把控制面放在服务端，但音频也经服务端转发。我们让音频浏览器直连、少一跳，因为目标是网页语音 + 多线程 Agent 调度，而不是电话。
Seed, delegate, write back, flow in. Every acceptance test maps to one of them.灌入、委托、写回、回流。每条验收项都对应其中一条。
Fig 3.图 3. ① seed · ② delegate · ③ write back · ④ flow in. Orange = the three GPT-Live channels; grey = runtime calls.① 灌入 · ② 委托 · ③ 写回 · ④ 回流。橙色是 GPT-Live 的三条通道，灰色是 runtime 调用。
| Flow流 | Today (v0.4, live)现状（v0.4，已上线） | Target目标 |
|---|---|---|
| ① Seed① 灌入 | Last 10 turns of the thread, 400 chars each线程最近 10 轮，每条 400 字 | Runtime's own compaction summary + gist + open task_plan items + running jobs — so Live and web share one answer to "what are we doing"runtime 自己的压缩摘要 + 梗概 + 未完成 task_plan 项 + 正在跑的任务——Live 与网页对「我们在干什么」只有一个答案 |
| ② Delegate② 委托 | /chat/stream {thread_id}; final text spoken/chat/stream {thread_id}，最终文本被说出 | Reasoning/tool events → think(id) as progress; first sentence → say(id) within 3 s; run_id returned so interrupt can cancel推理/工具事件 → think(id) 作进度；首句 3 秒内 say(id)；返回 run_id 以便打断时取消 |
| ③ Write back③ 写回 | Voice-only turns buffered in the bridge, handed over on the next delegation语音专属轮暂存在 bridge，下次委托时补交 | Appended to the thread immediately with source: voice; web renders them with a small mic mark立即以 source: voice 追加进线程；网页用一个小麦克风标记渲染 |
| ④ Flow in④ 回流 | Subscribed to /push/events; payload fields guessed已订阅 /push/events，字段靠猜 | Typed events with a one-line human summary: message_added · run_done · job_done · cross_thread带类型和一句人话摘要的事件：message_added · run_done · job_done · cross_thread |
Live decides in the moment whether to answer or delegate — the same judgement a good assistant makes about whether to look something up.Live 在当下决定是自己答还是委托——和一个好助理判断「要不要去查一下」是同一种判断。
Fig 4.图 4. A delegated turn. Live acknowledges immediately, receives progress silently, speaks the first sentence as soon as it exists, and keeps the conversation open throughout.一次委托轮次。Live 立即应答，静默接收进度，首句一出现就说，全程对话不中断。
| Live answers itselfLive 自己答 | Live delegatesLive 委托 |
|---|---|
| "What were we doing?" · "Say that again" · "How's that task going?" · small talk · confirming what it heard「我们刚在做什么」·「再说一遍」·「那个任务怎么样了」· 闲聊 · 确认听到的内容 | Anything needing facts, files, tools, writing, a decision to record, or work to start / stop / hand off一切需要事实、文件、工具、写东西、记录决定、启动 / 停止 / 交接工作的事 |
| Written back via ③ so the web sees it通过 ③ 写回，网页能看到 | Lands in the thread by construction — it is a thread turn天然落在线程里——它本身就是一个线程轮次 |
This is where Live stops being a microphone and becomes a chief of staff. It does not get its own scheduler — it gets a live task board assembled from what the runtime already tracks, and the runtime's own verbs to act on it.在这里 Live 不再只是麦克风，而成为幕僚长。它不拿一套自己的调度器——它拿的是从 runtime 已有追踪拼出的实时任务板，以及 runtime 自己的动词来操作它。
Fig 5.图 5. Fan-in on the left, runtime verbs on the right. The orchestrator owns nothing; it observes and dispatches.左侧汇入，右侧是 runtime 动词。Orchestrator 不拥有任何东西，只观察与派发。
Three design choices make this safe to run all day:三个设计选择让它可以整天开着：
| Choice选择 | Why原因 |
|---|---|
| Task board is derived, never stored任务板是派生的，从不存储 | Built on demand from task_plan, scheduled_task list, background processes, and each thread's gist. There is no second list to drift out of sync — the answer to "what's running" is identical whether you ask Live or type it.按需从 task_plan、scheduled_task list、后台进程和各线程梗概拼出。没有第二份表会漂移——问 Live 和打字问「有什么在跑」，答案一致。 |
| Cross-thread actions go through the main thread跨线程动作经主线程走 | "Tell the PR thread to rebase" becomes a delegation to the main thread, whose agent uses sessions_message — the same handoff path with the same ownership and loop guards described in Sessions & Threads. Live never talks to a side thread directly.「让 PR 那条线程 rebase」变成对主线程的一次委托，由其 agent 用 sessions_message 交接——与会话与线程里描述的同一条路径、同样的所有权和防循环守卫。Live 从不直接对旁线程说话。 |
| Speak-or-stay-silent policy is explicit「说还是不说」的策略是显式的 | Events arrive as silent thinking by default. Only three classes are promoted to commentary: something the user asked to be told about, a blocker that needs a decision, and completion of a task started by voice in this session. Everything else waits to be asked.事件默认以静默 thinking 到达。只有三类被提升为 commentary：用户明确要求告知的、需要决策的阻塞、以及本次会话中由语音启动的任务完成。其余等被问。 |
A new scenario never touches core, the browser or the runtime. It implements the five methods.新场景永不碰 core、浏览器或 runtime。它只实现那五个方法。
| Binding | seed() | handle() | events() |
|---|---|---|---|
| thread | Gist + open tasks + running jobs of one thread一条线程的梗概 + 未完成事项 + 在跑任务 | Delegate to that thread委托到该线程 | That thread's events该线程的事件 |
| onboarding | User name / language / timezone + "steps done: …, next: …" from app state用户名 / 语言 / 时区 + 来自应用状态的「已完成…下一步…」 | Delegate to a fresh thread whose system prompt is the onboarding script (already exists as the user-onboarding skill)委托到一条新线程，其系统提示就是 onboarding 剧本（已有 user-onboarding skill） | Milestones: bot linked, first skill installed, first preview — each a one-line think(null); key ones promoted to speech里程碑：绑定 bot、装第一个 skill、第一次 preview——每个一句 think(null)；关键节点提升为语音 |
| orchestrator | Task board across all threads跨所有线程的任务板 | Delegate to the main thread; it fans out委托到主线程，由它分发 | Fan-in of every thread + jobs, filtered by the policy above所有线程 + 任务的汇入，按上文策略过滤 |
Selecting one is a URL: /live?binding=thread&thread_id=…, /live?binding=onboarding, /live?binding=orchestrator. The web page's "Live voice" entry (starchild-web #202) builds the first form; the onboarding flow would build the second at the step where a new user is invited to talk.选择哪个就是一个 URL：/live?binding=thread&thread_id=…、/live?binding=onboarding、/live?binding=orchestrator。网页的「Live voice」入口（starchild-web #202）拼第一种；onboarding 流程在邀请新用户开口的那一步拼第二种。
The test of "seamless" is not the voice session — it is the moment you close the tab and keep typing.「无缝」的检验不在语音会话里，而在你关掉标签页接着打字的那一刻。
Fig 6.图 6. A thread that was half spoken, half typed. The only visible difference is a small source mark.一半说、一半打出来的线程。唯一可见差别是一个小小的来源标记。
| Guarantee保证 | How it is kept如何做到 |
|---|---|
| No duplicates不重复 | A delegated turn is written by the runtime once, as a normal turn. A voice-only turn is appended once by the bridge. The two paths are disjoint by construction.被委托的轮次由 runtime 作为普通轮次写一次；语音专属轮由 bridge 追加一次。两条路径天然不相交。 |
| No lost work不丢工作 | Work started by voice is a run / sub-agent / job owned by the thread, not by the Live session. Closing the tab ends the audio, nothing else.语音启动的工作是归属线程的 run / 子代理 / 任务，不归 Live 会话。关标签页只结束音频，别的不受影响。 |
| No stale gist梗概不过期 | Spoken turns count as thread activity, so the background gist refresh described in Sessions & Threads and Compaction picks them up like any other message.语音轮次算线程活动，会话与线程和压缩里描述的后台梗概刷新会像对待其他消息一样吸收它们。 |
| No second brain没有第二个大脑 | Live's only per-session state is the audio connection. Memory, SOUL, skills, Conductor routing all apply because the work runs in the thread.Live 每会话唯一的状态是音频连接。记忆、SOUL、skills、Conductor 路由全部生效，因为工作跑在线程里。 |
| Busy thread is handled, not hidden线程忙时有处理，不隐瞒 | If the thread is mid-run when you speak, the runtime today merges your message into that run and returns nothing to the bridge; Live says so ("merged — the answer will land in the thread"). Target: queue_if_busy so a voice turn queues as a fresh turn instead.你说话时线程正在跑，runtime 目前会把消息并入当前 run、不返回任何东西给 bridge；Live 会说明（「已并入，答案会落在线程里」）。目标：queue_if_busy，让语音轮排成新一轮而非并入。 |
Three additive endpoints. None changes an existing path; each unlocks one flow.三个加法接口。不改任何现有路径，每个解锁一条流。
| # | Endpoint接口 | Unlocks解锁 |
|---|---|---|
| R1 | GET /push/events · typed payload: {type, thread_id, summary, run_id?} with type ∈ message_added · run_started · run_done · job_done · cross_thread类型化载荷：{type, thread_id, summary, run_id?}，type ∈ message_added · run_started · run_done · job_done · cross_thread | ④ flow in without guessing; orchestrator fan-in④ 回流不再靠猜；orchestrator 汇入 |
| R2 | POST /session/{id}/messages {role, content, meta:{source:'voice'}} | ③ immediate write-back; removes the bridge's voice-log buffer③ 即时写回；删掉 bridge 的 voice-log 缓冲 |
| R3 | POST /chat/stream · emit run_id first; accept queue_if_busy先发 run_id；接受 queue_if_busy | Real interrupt → cancel with confirmation; no silent merge on busy threads真正的打断 → 带确认的取消；线程忙时不再静默并入 |
Web-side: render meta.source === 'voice' with a mic mark (one line in the message component). The "Live voice" entry (#202) is already in review.网页侧：对 meta.source === 'voice' 渲染一个麦克风标记（消息组件里一行）。「Live voice」入口（#202）已在审查中。
| Phase阶段 | Deliverable交付 | Status状态 |
|---|---|---|
| P0 | Thread binding on the demo: seed · delegate · buffered write-back · event inflow (official-skills #174, 0.3.0)demo 上的线程绑定：灌入 · 委托 · 缓冲写回 · 事件回流（official-skills #174，0.3.0） | Live on this machine; A1/A2 pass, A3 depends on R1已在本机运行；A1/A2 通过，A3 依赖 R1 |
| P1 | Restructure into core / bindings / adapters; delete the five tools and legacy voice-history; ThreadBinding only. No behaviour change.重排为 core / bindings / adapters；删五工具与旧 voice-history；仅 ThreadBinding。行为不变。 | Next下一步 |
| P2 | Runtime R1 → R2 → R3; bridge switches to immediate write-back and confirmed cancel; web mic markruntime R1 → R2 → R3；bridge 切到即时写回与确认取消；网页麦克风标记 | — |
| P3 | OnboardingBinding + entry in the onboarding flowOnboardingBinding + onboarding 流程入口 | — |
| P4 | OrchestratorBinding: task board, fan-in, speak-or-stay-silent policyOrchestratorBinding：任务板、汇入、说或不说策略 | — |
| Acceptance test验收项 | Flow对应流 | |
|---|---|---|
| A1 | Open Live on a thread, ask "what are we doing?" — it answers from the gist without delegating在一条线程上打开 Live，问「我们在做什么」——它凭梗概直接答，不委托 | ① |
| A2 | Ask something needing facts — the turn and its answer appear in the web thread问一件需要事实的事——该轮和回答出现在网页线程里 | ② |
| A3 | Type a line on the web, then ask Live what you just typed在网页打一句，再问 Live 你刚打了什么 | ④ |
| A4 | Small talk stays local; facts get delegated — checked against the event log闲聊留在本地，事实类被委托——对照事件日志检查 | — |
| A5 | Close and reopen the tab; A1 still passes and nothing was duplicated in the thread关掉再打开标签页；A1 仍过，线程里无重复 | ①③ |
| A6 | First spoken sentence of a delegated answer within 3 s委托类回答的首句 3 秒内说出 | ② |
| A7 | Interrupt a running delegation; Live says "stopped" only after the run is actually cancelled打断进行中的委托；只有 run 真正取消后 Live 才说「已停」 | R3 |
| A8 | Start a job by voice, close Live, come back to the web — the result is in the thread用语音启动一个任务，关掉 Live，回到网页——结果在线程里 | §08§08 |
Checked against OpenAI's delegation guide (09-12), LiveKit Agents 1.8 GPT-Live plugin (09-11) and Pipecat 1.9 (09-11). All three ship the same shape we drew in §03. The interesting findings are the hard limits and the two places where voice genuinely cannot do what the web page does.对照了 OpenAI 委托指南（09-12）、LiveKit Agents 1.8 的 GPT-Live 插件（09-11）和 Pipecat 1.9（09-11）。三家交付的都是 §03 画的那个形态。真正有价值的发现是硬限制，以及语音确实做不到网页能做的两处。
| Our piece我们的部件 | LiveKit 1.8 | Pipecat 1.9 | Verdict结论 |
|---|---|---|---|
| core/live-session | GPTLiveModel (duplex model) + adapter（duplex 模型）+ 适配器 | OpenAILiveLLMService | Same layer, same three calls (append_thinking / append_commentary, same id = continue)同一层、同样三个调用（append_thinking / append_commentary，同 id = 续报） |
| Binding.handle() | delegation_created handler; "carries no task text"; @function_tool is ignored with a warning in client mode处理器；「不带任务文本」；client 模式下 @function_tool 被忽略并告警 | BackendLLMWorker — "runs any Pipecat LLM service as the backend"——「让任意 Pipecat LLM 服务当后端」 | Our Binding is exactly their worker. Removing the five tools is not a choice, it is what the API does.我们的 Binding 就是他们的 worker。删五工具不是选择，是 API 本来的行为。 |
| Binding.seed() | chat_ctx before start — max 128 messages / 8 192 rendered tokens, oldest dropped first; append-only after start启动前的 chat_ctx——上限 128 条 / 8 192 渲染 token，超出先丢最旧；启动后只可追加 | same service limit同一服务上限 | Answers our open question E8. A raw-turn seed will silently lose the oldest context; the gist-based seed in §04 is required, not nice-to-have.回答了悬而未决的 E8。按原始轮次灌入会静默丢最旧上下文；§04 的梗概式灌入是必需，不是锦上添花。 |
| Binding.events() | append context items after start启动后追加上下文项 | — | OpenAI: "send at start and on relevant change; skip unchanged; merge rapid changes". Our ≤1/1.5 s merge matches.OpenAI：「开始时和相关变化时发；未变不发；快速变化合并」。我们的 ≤1/1.5 s 合并一致。 |
| Mode模式 | Delegation mode is fixed at session creation; switching needs a new session. Audio only — images go to the backend, never to Live.委托模式在建会话时固定，切换要开新会话。只有音频——图片走后端，永不进 Live。 | One binding per session; a "switch to orchestrator" is a reconnect, which is fine.每会话一个 binding；「切到 orchestrator」就是重连一次，可接受。 | |
Live never calls scheduled_task, preview, wallet or any other tool. A delegated turn is a normal turn in the thread, so the agent uses its own tools exactly as it does when you type. "Start a preview for the deck" by voice and by keyboard run the same code path. The bridge only needs read access for the seed and the task board — and those endpoints already exist on the runtime today (verified on this machine):Live 从不调用 scheduled_task、preview、wallet 或任何工具。被委托的轮次就是线程里的普通轮次，agent 用自己的工具，和你打字时完全一样。语音说「给 deck 起个 preview」和键盘敲出来跑的是同一条代码路径。bridge 只需要读权限做灌入和任务板——而这些接口 runtime 今天就有（本机已验证）：
| You ask Live…你对 Live 说…… | Who does it谁来做 | Bridge reads (seed / board)bridge 读取（灌入 / 任务板） | Fit贴合度 |
|---|---|---|---|
| "Is the API monitor job still paused?"「API 监控那个任务还暂停着吗」 | Answered from seed if on the board; else agent via scheduled_task list在任务板上就凭灌入直接答；否则 agent 用 scheduled_task list | GET /scheduled-jobs (53 jobs, status/schedule/title)（53 个任务，含状态/计划/标题） | seamless |
| "Spin up a preview for the wiki"「把 wiki 起个 preview」 | Agent → preview serve; Live says "it's up, link is in the thread"agent → preview serve；Live 说「起好了，链接在线程里」 | GET /api/preview/list (id, status, port)（id、状态、端口） | seamless — link consumed on web——链接在网页消费 |
| "What's running right now?"「现在有什么在跑」 | Seed / orchestrator board灌入 / orchestrator 任务板 | GET /chat/runs?thread_id · GET /tasks · GET /scheduled-jobs | seamless |
| "Stop that" (mid-delegation)「停下」（委托进行中） | POST /chat/runs/cancel via Binding.interrupt经 Binding.interrupt | — | needs R3 (run_id not yet returned)（run_id 尚未返回） |
| "Send the report to Telegram"「把报告发到 Telegram」 | Agent asks to confirm (external action) → you say "yes" → next delegationagent 请求确认（外部动作）→ 你说「是」→ 下一次委托 | — | seamless — confirmation is just another turn——确认就是又一个轮次 |
| "Add my OpenAI key" · "Sign this tx" · "Connect Gmail"「加我的 OpenAI key」·「签这笔交易」·「连 Gmail」 | Agent fires request_env_input / user_wallet_tx / composio_connect — these are web UI cards (SSE)agent 触发 request_env_input / user_wallet_tx / composio_connect——都是网页 UI 卡片（SSE） | — | hand-off — Live says "I've put a secure prompt in the thread, open the page"; never speaks secrets——Live 说「安全输入框已放在线程里，打开页面」；绝不念出密钥 |
| "Look at this screenshot"「看看这张截图」 | Not by voice — image must reach the backend from the web page语音做不到——图片必须从网页送达后端 | — | web only |
So the compatibility question has a crisp answer: everything the agent can do, Live can trigger; everything that must be seen or clicked stays on the web, and Live points you there. That boundary is not a limitation of our design — it is where the web page and the voice genuinely differ, and the thread is what lets you cross it mid-sentence.于是兼容性问题有了清晰答案：agent 能做的一切，Live 都能触发；必须看或点的一切留在网页，Live 把你指过去。这条边界不是设计的局限——它就是网页与语音真正不同的地方，而线程是让你在句子中间跨过这条边界的东西。
| Grows well长得好 | Watch要盯着 |
|---|---|
| New scenario = new binding; core/browser/runtime untouched (matches LiveKit's "one handler per agent class").新场景 = 新 binding；core / 浏览器 / runtime 不动（对应 LiveKit「每个 Agent 类一个处理器」）。 | 8 192-token seed ceiling: orchestrator board must summarise, not list. Budget: gist 40% · open tasks 30% · running jobs 20% · voice-only tail 10%.8 192 token 灌入上限：orchestrator 任务板必须摘要而非罗列。预算：梗概 40% · 未完成 30% · 在跑 20% · 语音尾部 10%。 |
| New agent capability = zero Live work, by construction.新的 agent 能力 = Live 侧零工作，结构使然。 | Client mode has no tool channel — if OpenAI later adds one, we still route through the thread; do not let a "quick tool" creep back into Live.client 模式没有工具通道——即便 OpenAI 日后加了，仍走线程；别让「顺手加个小工具」溜回 Live。 |
| Swapping the speech vendor = replace core/live-session only (Pipecat proves the worker survives that).换语音厂商 = 只换 core/live-session（Pipecat 证明 worker 层能活过这种替换）。 | One delegation mode per session; if a managed-backend variant is ever wanted, it is a different binding and a reconnect, not a flag.每会话一种委托模式；若日后想要托管后端变体，那是另一个 binding 加一次重连，不是一个开关。 |
Sources: OpenAI "Delegation and tools in GPT-Live" (developers.openai.com, 2026-09-12); LiveKit "OpenAI GPT-Live plugin guide" (docs.livekit.io, 2026-09-11); Pipecat 1.9.0 release notes (2026-09-11); runtime endpoints probed on this machine 2026-09-14.来源：OpenAI《Delegation and tools in GPT-Live》（developers.openai.com，2026-09-12）；LiveKit《OpenAI GPT-Live plugin guide》（docs.livekit.io，2026-09-11）；Pipecat 1.9.0 发布说明（2026-09-11）；runtime 接口于 2026-09-14 在本机实测。
Sections 01–11 are the design. This section is what is actually deployed and verified today: the proxied session path, the Sideband gate, how your speech becomes thread messages, and what happens on hang-up, mute and time-out.01–11 是设计稿。本节是今天真正部署并验证过的东西：代理会话链路、Sideband 门控、你的语音如何变成线程消息，以及挂断、静音、超时时的行为。
Fig 9.图 9. The live path as deployed. Orange = control-plane frames the browser sees over SSE (never injected into the DataChannel); grey = runtime HTTP/DB. Both branches end with a bridge.persisted that fires only after the DB confirmed the write.已部署的链路。橙色 = 浏览器经 SSE 看到的控制面帧（绝不注入 DataChannel）；灰色 = runtime 的 HTTP / 数据库调用。两个分支都以 bridge.persisted 收尾，且只在数据库确认写入后才发出。
| Stage阶段 | Where位置 | Behaviour today当前行为 |
|---|---|---|
| Session create建会话 | sc-proxy | HTTP only. Injects the OpenAI key, audits, rate-limits. Upstream 4xx is returned with its original status + upstream_status header, never collapsed to 502.只走 HTTP。注入 OpenAI key、审计、限流。上游 4xx 原状态码透传并带 upstream_status 头，不折成 502。 |
| Sideband attachSideband 接入 | sc-ws-proxy :8081 | WebSocket only, X-Caller-Id for billing. Gate: the SDP answer is released to the browser only after a protocol success frame (session.started / session.updated); malformed or error frames fail the attach, the upstream session is closed and the browser gets 503 — no unbilled sessions. Both proxies must hold the same key: attach only works with the key that created the session.只走 WebSocket，X-Caller-Id 计费。门控：只有收到协议成功帧（session.started / session.updated）才把 SDP answer 放给浏览器；坏帧或 error 帧视为接入失败，关闭上游会话并给浏览器 503——不产生未计费会话。两个代理必须持同一把 key：只能用创建会话的那把 key 去 attach。 |
| Your speech → one message你的话 → 一条消息 | Delegator | Transcription closes a segment on every ~0.9 s pause. Segments are accumulated and flushed as ONE [Voice] message when Live starts answering, or handed whole to the delegation. Fallback flush after 6 s of nothing. Speaking again only extends the window.转写每停顿约 0.9 秒切一段。各段累积，直到 Live 开口回答才合成一条 [Voice] 落库，或整段交给委派。6 秒无任何动作才兜底落库。再开口只会延长窗口。 |
| Delegate委派 | ThreadBinding | One /chat/stream per delegation.created. Idle thread → new turn. Busy thread → the runtime merges the message into the running turn (inject relation), Live is told "merged, answer will appear in the conversation". Task count = number of delegations, not messages.每个 delegation.created 发一条 /chat/stream。线程空闲 → 新开一轮；线程忙 → runtime 把消息并入正在跑的那轮（inject relation），Live 被告知「已合并，结果稍后出现在对话里」。task 数 = 委派次数，不是消息数。 |
| Persisted signal落库信号 | Bridge → SSE | bridge.persisted means "confirmed in the DB": direct writes require insert count > 0; authed delegations fire only on persistence_complete, never at stream end (a timed-out write goes to background and relies on the existing agent:completed refresh). The web maps it to triggerMessageRefresh(thread_id).bridge.persisted 的含义是「数据库已确认」：直写要求 insert count > 0；鉴权委派只在 persistence_complete 时发，绝不在流结束时发（写入超时转后台，靠原有的 agent:completed 刷新）。网页收到后 triggerMessageRefresh(thread_id)。 |
| Hang-up挂断 | Bridge.close | Stops transcripts, delegator consumers and the Sideband. Does not call cancel_run: a running thread turn finishes and lands in the thread. Only a spoken cancel ("stop / never mind") goes through cancel_all.停止转写、委派消费者和 Sideband。不调 cancel_run：正在跑的线程回合会跑完并写进线程。只有口头取消（「停 / 别做了」）才走 cancel_all。 |
| Microphone麦克风 | liveVoice.ts | Never auto-picks iPhone/iPad Continuity mics; saved choice → first local input. Switching mid-call is replaceTrack, no reconnect. A browser/OS mic block ends the track permanently — the client re-acquires the same device and swaps it in. There is no in-bar mute yet.永不自动选 iPhone/iPad 连续互通麦克风；优先上次选择 → 第一个本机输入。通话中切换用 replaceTrack，不重连。浏览器/系统级禁用会永久结束音轨——客户端会重新获取同一设备并换回去。通话条尚无静音按钮。 |
| Duration时长 | OpenAI | No limit on our side beyond the 6 s transcript fallback and disconnect reaping. The ceiling is the Realtime session itself (documented max 60 min); on upstream close the bar shows the reason and offers Restart Live.我们这边没有时长上限，只有 6 秒转写兜底和断连回收。上限来自 Realtime 会话本身（文档口径最长 60 分钟）；上游关闭时通话条显示原因并提供 Restart Live。 |
| Labels标签 | Web | Call bar: You / Live / Agent rows, one row per exchange, collapsed to ~3 lines, expandable to 50 vh. Thread: plain Voice / Live pills; an untagged assistant message is a delegated agent reply.通话条：You / Live / Agent 分行，一轮一行，默认折叠约 3 行，可展开到半屏。线程内：素色 Voice / Live 标签；无标签的助手消息即委派后的 Agent 回复。 |
Verified on this machine 2026-09-18: proxied create → ws-proxy attach → session.started in 0.24 s; real write count=1 → one persisted signal, count=0 → none. Code: starchild-clawd core/live/* (PR #1696, #1707, #1723), sc-proxy PR #42, starchild-web PR #204.2026-09-18 于本机验证：代理建会话 → ws-proxy 接入 → 0.24 秒收到 session.started；真实写入 count=1 → 一次落库信号，count=0 → 无。代码：starchild-clawd core/live/*（PR #1696、#1707、#1723），sc-proxy PR #42，starchild-web PR #204。