vultr/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-05-06 18:10:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(gateway): add VoiceClaw realtime brain endpoint (#70938)

Browse Source 
Adds the VoiceClaw-compatible realtime brain WebSocket endpoint backed by Gemini Live, with owner-auth gating, async OpenClaw tool handoff, docs, and lifecycle tests.

Maintainer fixup: terminal upstream errors now send the error, emit session.ended while the client socket is still open, then close the client-facing socket.

Co-authored-by: Michael Yagudaev <1386966+yagudaev@users.noreply.github.com>
This commit is contained in:
Michael Yagudaev
2026-04-24 13:00:04 -07:00
committed by GitHub
parent 7536993397
commit c997a9f978
15 changed files with 2680 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/gateway/index.md
View File

@@ -176,6 +176,42 @@ OPENCLAW_CONFIG_PATH=~/.openclaw/b.json OPENCLAW_STATE_DIR=~/.openclaw-b opencla
Detailed setup: [/gateway/multiple-gateways](/gateway/multiple-gateways).
## VoiceClaw real-time brain endpoint
OpenClaw exposes a VoiceClaw-compatible real-time WebSocket endpoint at
`/voiceclaw/realtime`. Use it when a VoiceClaw desktop client should talk
directly to a real-time OpenClaw brain instead of going through a separate relay
process.
The endpoint uses Gemini Live for real-time audio and calls OpenClaw as the
brain by exposing OpenClaw tools directly to Gemini Live. Tool calls return an
immediate `working` result to keep the voice turn responsive, then OpenClaw
executes the actual tool asynchronously and injects the result back into the
live session. Set `GEMINI_API_KEY` in the gateway process environment. If
gateway auth is enabled, the desktop client sends the gateway token or password
in its first `session.config` message.
Real-time brain access runs owner-authorized OpenClaw agent commands. Keep
`gateway.auth.mode: "none"` limited to loopback-only test instances. Non-local
real-time brain connections require gateway auth.
For an isolated test gateway, run a separate instance with its own port, config,
and state:
```bash
OPENCLAW_CONFIG_PATH=/path/to/openclaw-realtime/openclaw.json \
OPENCLAW_STATE_DIR=/path/to/openclaw-realtime/state \
OPENCLAW_SKIP_CHANNELS=1 \
GEMINI_API_KEY=... \
openclaw gateway --port 19789
```
Then configure VoiceClaw to use:
```text
ws://127.0.0.1:19789/voiceclaw/realtime
```
## Remote access
Preferred: Tailscale/VPN.