forked from org1/hermes-voice-gateway
Stack: rudolf-T worker, x86_64, 14.5 GiB RAM. Replaces hermes-agent STT/TTS providers with direct faster-whisper + piper (hermes-agent's TTS providers only target file-based voice-bubble delivery, not streaming PCM for browser WebSocket). Bridge: browser <-> voice-gateway <-> hermes-proxy <-> Hermes. * gateway/server.py: aiohttp app with /ws, /healthz, / static * gateway/pipeline.py: STT (faster-whisper base), LLM (POST /v1/chat/completions streaming), TTS (piper ru_RU-denis-medium) * gateway/static/: browser mic capture + WS playback with VAD-based flush on silence * Dockerfile on nousresearch/hermes-agent:v2026.8.3 * compose.yaml with placement node.hostname==rudolf-T Wire protocol documented in server.py header and README. Half-duplex by design; barge-in and server-VAD deferred until proven needed.
2.8 KiB
2.8 KiB
hermes-voice-gateway
Local voice realtime bridge: browser ↔ voice-gateway (WebSocket+PCM) ↔ hermes-proxy (OpenAI-compatible) ↔ Hermes (minimax API).
Architecture
[Browser] [voice-gateway] [hermes-proxy] [Hermes]
mic/speaker ⇄ WS+PCM on :8765 ⇄ POST /v1/chat ⇄ minimax API
│
├─ STT: faster-whisper base (VAD-filtered)
└─ TTS: piper ru_RU-denis-medium
Half-duplex by design: client sends PCM chunks, server flushes STT on silence, calls LLM with streaming, splits reply into sentences, sends each sentence to TTS, streams PCM back.
Stack layout
gateway/server.py— aiohttp WS endpoint + HTTP staticgateway/pipeline.py— STT / LLM / TTS adapter classesgateway/config.py— env-driven configgateway/static/{index.html,app.js,styles.css}— browser clientDockerfile— image build on top ofnousresearch/hermes-agent:v2026.8.3compose.yaml— Swarm stack with placement onrudolf-T
Required env
| Var | Default | Notes |
|---|---|---|
HERMES_PROXY_URL |
http://hermes-proxy:8645 |
OpenAI-compatible upstream |
HERMES_PROXY_TOKEN |
empty | Bearer if hermes-proxy requires one |
PIPER_VOICE_PATH |
/opt/piper-voices/ru_RU-denis-medium.onnx |
bundled in voice_gateway_models volume |
LLM_MODEL |
MiniMax-M3 |
passed verbatim to /v1/chat/completions |
SAMPLE_RATE_IN |
16000 |
PCM in from browser |
SAMPLE_RATE_OUT |
22050 |
PCM out to browser |
MAX_SESSION_SECONDS |
300 |
hard timeout per WS session |
MAX_SILENCE_MS |
1500 |
browser-side VAD trigger |
Wire protocol
Client → server (WS):
- binary frame: PCM Int16 mono at
SAMPLE_RATE_IN - text JSON
{"type": "flush"}— force STT - text JSON
{"type": "reset"}— clear conversation history - text JSON
{"type": "ping"}— keep-alive
Server → client (WS):
- binary frame: PCM Int16 mono at
SAMPLE_RATE_OUT - text JSON
{"type": "ready"}— handshake complete - text JSON
{"type": "transcript", "text": "..."} - text JSON
{"type": "reply", "text": "..."}— full assistant text after TTS finishes - text JSON
{"type": "error", "message": "..."}
Running locally
docker compose build
docker compose up
open http://localhost:8765/
The voice-gateway expects hermes-proxy reachable at http://hermes-proxy:8645 on the same hermes_proxy overlay. In local dev, override HERMES_PROXY_URL=http://host.docker.internal:8645.
Dependencies
nousresearch/hermes-agent:v2026.8.3— base image, brings hermes-agent CLI (not strictly required by gateway code, but matches the rest of hermes_infra).faster-whisper— STT, modelbaseint8 quantized, downloaded on first use.piper-tts+onnxruntime— TTS, voice model must be mounted atPIPER_VOICE_PATH.aiohttp— HTTP/WS server.