hermes d05459476d feat: voice-gateway service (PR B + PR C in single PR)
Stack: rudolf-T worker, x86_64, 14.5 GiB RAM.
Replaces hermes-agent STT/TTS providers with direct faster-whisper +
piper (hermes-agent's TTS providers only target file-based voice-bubble
delivery, not streaming PCM for browser WebSocket).

Bridge: browser <-> voice-gateway <-> hermes-proxy <-> Hermes.

* gateway/server.py: aiohttp app with /ws, /healthz, / static
* gateway/pipeline.py: STT (faster-whisper base), LLM (POST
  /v1/chat/completions streaming), TTS (piper ru_RU-denis-medium)
* gateway/static/: browser mic capture + WS playback with VAD-based
  flush on silence
* Dockerfile on nousresearch/hermes-agent:v2026.8.3
* compose.yaml with placement node.hostname==rudolf-T

Wire protocol documented in server.py header and README. Half-duplex
by design; barge-in and server-VAD deferred until proven needed.
2026-09-12 21:20:12 +00:00
2026-09-12 21:18:36 +02:00

hermes-voice-gateway

Local voice realtime bridge: browser ↔ voice-gateway (WebSocket+PCM) ↔ hermes-proxy (OpenAI-compatible) ↔ Hermes (minimax API).

Architecture

[Browser]      [voice-gateway]    [hermes-proxy]    [Hermes]
 mic/speaker ⇄ WS+PCM on :8765 ⇄ POST /v1/chat  ⇄ minimax API
                │
                ├─ STT: faster-whisper base (VAD-filtered)
                └─ TTS: piper ru_RU-denis-medium

Half-duplex by design: client sends PCM chunks, server flushes STT on silence, calls LLM with streaming, splits reply into sentences, sends each sentence to TTS, streams PCM back.

Stack layout

  • gateway/server.py — aiohttp WS endpoint + HTTP static
  • gateway/pipeline.py — STT / LLM / TTS adapter classes
  • gateway/config.py — env-driven config
  • gateway/static/{index.html,app.js,styles.css} — browser client
  • Dockerfile — image build on top of nousresearch/hermes-agent:v2026.8.3
  • compose.yaml — Swarm stack with placement on rudolf-T

Required env

Var Default Notes
HERMES_PROXY_URL http://hermes-proxy:8645 OpenAI-compatible upstream
HERMES_PROXY_TOKEN empty Bearer if hermes-proxy requires one
PIPER_VOICE_PATH /opt/piper-voices/ru_RU-denis-medium.onnx bundled in voice_gateway_models volume
LLM_MODEL MiniMax-M3 passed verbatim to /v1/chat/completions
SAMPLE_RATE_IN 16000 PCM in from browser
SAMPLE_RATE_OUT 22050 PCM out to browser
MAX_SESSION_SECONDS 300 hard timeout per WS session
MAX_SILENCE_MS 1500 browser-side VAD trigger

Wire protocol

Client → server (WS):

  • binary frame: PCM Int16 mono at SAMPLE_RATE_IN
  • text JSON {"type": "flush"} — force STT
  • text JSON {"type": "reset"} — clear conversation history
  • text JSON {"type": "ping"} — keep-alive

Server → client (WS):

  • binary frame: PCM Int16 mono at SAMPLE_RATE_OUT
  • text JSON {"type": "ready"} — handshake complete
  • text JSON {"type": "transcript", "text": "..."}
  • text JSON {"type": "reply", "text": "..."} — full assistant text after TTS finishes
  • text JSON {"type": "error", "message": "..."}

Running locally

docker compose build
docker compose up
open http://localhost:8765/

The voice-gateway expects hermes-proxy reachable at http://hermes-proxy:8645 on the same hermes_proxy overlay. In local dev, override HERMES_PROXY_URL=http://host.docker.internal:8645.

Dependencies

  • nousresearch/hermes-agent:v2026.8.3 — base image, brings hermes-agent CLI (not strictly required by gateway code, but matches the rest of hermes_infra).
  • faster-whisper — STT, model base int8 quantized, downloaded on first use.
  • piper-tts + onnxruntime — TTS, voice model must be mounted at PIPER_VOICE_PATH.
  • aiohttp — HTTP/WS server.
S
Description
No description provided
Readme
60 KiB
Languages
Python 56.9%
JavaScript 30.8%
CSS 6.7%
HTML 4.2%
Dockerfile 1.4%