feat(hermes_squid): enable local STT via faster-whisper (runtime install) #21

Merged
admin merged 6 commits from hermes/ansible:feat/stt-voice-faster-whisper into master 2026-08-29 22:53:48 +02:00
Contributor

What

Wires up local STT (faster-whisper) into the hermes-gateway container so voice messages on Telegram transcribe without depending on external services.

Closes #19. Supersedes the closed #20 (which used a custom Dockerfile-hermes; this version uses a bind-mounted entrypoint script instead since pypi.org is now whitelisted).

Changes

  1. hermes_squid/squid.conf — expands the egress whitelist:

    • new stt_pkg_domains for pypi.org, files.pythonhosted.org, huggingface.co, github.com, models.dev, registry.npmjs.org, cdn-lfs.huggingface.co — needed for runtime pip install faster-whisper and HF model fetch.
    • new telegram_ips covering AS62041 (91.108.4.0/22, 91.108.8.0/22, 91.108.12.0/22, 91.108.16.0/22, 91.108.20.0/22, 91.108.56.0/22, 149.154.160.0/20, 185.76.151.0/24) as a backstop for the dstdomain ACL. squid's dstdomain does not match HTTPS CONNECT requests without ssl_bump, so the recent log shows 24 of 9083 api.telegram.org requests getting 403 in a 5-minute window.
    • renamed hermes_network -> localnet (squid default name, no behavior change).
  2. hermes_squid/hermes-gateway-entrypoint.sh (new) — bind-mounted into the container at /opt/data/hermes-gateway-entrypoint.sh and set as the service entrypoint. The script:

    • is idempotent: on first start, runs apt-get install -y python3-pip ffmpeg and pip3 install --break-system-packages faster-whisper. Subsequent restarts skip install if faster_whisper is already importable.
    • exports STT_PROVIDER=local, STT_LOCAL_MODEL=base, STT_LOCAL_DEVICE=cpu, STT_LOCAL_COMPUTE_TYPE=int8.
    • execs the original gateway command (gateway run).
  3. hermes_squid/stack-compose.yml — hermes-gateway gets:

    • entrypoint: ["/opt/data/hermes-gateway-entrypoint.sh"]
    • command: ["gateway", "run"]
    • bind-mount ./hermes-gateway-entrypoint.sh:/opt/data/hermes-gateway-entrypoint.sh:ro
    • new env: STT_PROVIDER, STT_LOCAL_MODEL, STT_LOCAL_DEVICE, STT_LOCAL_COMPUTE_TYPE

No other service in the compose file is touched.

Why runtime install, not a custom image

  • pypi.org is now whitelisted, so pip3 install works at runtime through the same squid.
  • Avoids maintaining a custom hermes-agent image and a local registry tag.
  • The upstream nousresearch/hermes-agent:v2026.8.3 image is kept as-is.

Deploy notes

  • brencher apply will trigger docker service update for hermes-gateway; the new entrypoint runs the apt+pip install on first start (~1-2 min).
  • First start also downloads the base faster-whisper model (~150 MB) from huggingface.co — covered by stt_pkg_domains.
  • squid config is reread on docker service update for the squid service; new ACLs apply without a full stack restart.
  • Subsequent restarts of hermes-gateway skip the install path entirely (idempotent guard).

Verified locally

  • git diff --stat upstream/master shows only the three files above (+25 entrypoint, +14 squid, +8 compose).
  • The new worktree is /opt/data/work/thread-114-Alex-InfraAlexBot/ansible (per-session working dir).

Risks

  • apt-get update inside the container needs deb.debian.org reachable — already in allowed_domains.
  • STT_LOCAL_MODEL=base is ~150 MB and runs on CPU. If latency on Raspberry-class nodes is bad, switch to tiny.
  • --break-system-packages is required for PEP 668 (Debian trixie + Python 3.11+). If the upstream image switches to PEP 668 strictly, this flag is the right escape hatch.

Related

  • #19 — investigation that found missing faster-whisper + squid deny pattern
  • #20 — superseded (used Dockerfile-hermes instead of runtime install)
## What Wires up local STT (faster-whisper) into the hermes-gateway container so voice messages on Telegram transcribe without depending on external services. Closes #19. Supersedes the closed #20 (which used a custom Dockerfile-hermes; this version uses a bind-mounted entrypoint script instead since pypi.org is now whitelisted). ## Changes 1. **`hermes_squid/squid.conf`** — expands the egress whitelist: - new `stt_pkg_domains` for `pypi.org`, `files.pythonhosted.org`, `huggingface.co`, `github.com`, `models.dev`, `registry.npmjs.org`, `cdn-lfs.huggingface.co` — needed for runtime `pip install faster-whisper` and HF model fetch. - new `telegram_ips` covering AS62041 (91.108.4.0/22, 91.108.8.0/22, 91.108.12.0/22, 91.108.16.0/22, 91.108.20.0/22, 91.108.56.0/22, 149.154.160.0/20, 185.76.151.0/24) as a backstop for the `dstdomain` ACL. squid's `dstdomain` does not match HTTPS CONNECT requests without `ssl_bump`, so the recent log shows 24 of 9083 `api.telegram.org` requests getting 403 in a 5-minute window. - renamed `hermes_network` -> `localnet` (squid default name, no behavior change). 2. **`hermes_squid/hermes-gateway-entrypoint.sh`** (new) — bind-mounted into the container at `/opt/data/hermes-gateway-entrypoint.sh` and set as the service entrypoint. The script: - is idempotent: on first start, runs `apt-get install -y python3-pip ffmpeg` and `pip3 install --break-system-packages faster-whisper`. Subsequent restarts skip install if `faster_whisper` is already importable. - exports `STT_PROVIDER=local`, `STT_LOCAL_MODEL=base`, `STT_LOCAL_DEVICE=cpu`, `STT_LOCAL_COMPUTE_TYPE=int8`. - `exec`s the original gateway command (`gateway run`). 3. **`hermes_squid/stack-compose.yml`** — `hermes-gateway` gets: - `entrypoint: ["/opt/data/hermes-gateway-entrypoint.sh"]` - `command: ["gateway", "run"]` - bind-mount `./hermes-gateway-entrypoint.sh:/opt/data/hermes-gateway-entrypoint.sh:ro` - new env: `STT_PROVIDER`, `STT_LOCAL_MODEL`, `STT_LOCAL_DEVICE`, `STT_LOCAL_COMPUTE_TYPE` No other service in the compose file is touched. ## Why runtime install, not a custom image - `pypi.org` is now whitelisted, so `pip3 install` works at runtime through the same squid. - Avoids maintaining a custom hermes-agent image and a local registry tag. - The upstream `nousresearch/hermes-agent:v2026.8.3` image is kept as-is. ## Deploy notes - brencher apply will trigger `docker service update` for hermes-gateway; the new entrypoint runs the apt+pip install on first start (~1-2 min). - First start also downloads the `base` faster-whisper model (~150 MB) from `huggingface.co` — covered by `stt_pkg_domains`. - squid config is reread on `docker service update` for the squid service; new ACLs apply without a full stack restart. - Subsequent restarts of hermes-gateway skip the install path entirely (idempotent guard). ## Verified locally - `git diff --stat upstream/master` shows only the three files above (+25 entrypoint, +14 squid, +8 compose). - The new worktree is `/opt/data/work/thread-114-Alex-InfraAlexBot/ansible` (per-session working dir). ## Risks - `apt-get update` inside the container needs `deb.debian.org` reachable — already in `allowed_domains`. - `STT_LOCAL_MODEL=base` is ~150 MB and runs on CPU. If latency on Raspberry-class nodes is bad, switch to `tiny`. - `--break-system-packages` is required for PEP 668 (Debian trixie + Python 3.11+). If the upstream image switches to PEP 668 strictly, this flag is the right escape hatch. ## Related - #19 — investigation that found missing faster-whisper + squid deny pattern - #20 — superseded (used Dockerfile-hermes instead of runtime install)
hermes added 3 commits 2026-08-29 19:08:37 +02:00
This reverts commit 4cb8eeae35.
Replace the previous PR #20 which used a custom Dockerfile-hermes.
This new version uses a bind-mounted entrypoint script that installs
faster-whisper at first start instead of baking it into the image.

Why the change:
- pypi.org is now whitelisted in squid, so runtime pip install works
- avoids maintaining a custom hermes-agent image
- the upstream nousresearch/hermes-agent image is kept as-is

Changes:
- hermes_squid/squid.conf: new stt_pkg_domains ACL (pypi, huggingface,
  github, models.dev, registry.npmjs, files.pythonhosted,
  cdn-lfs.huggingface) for runtime pip and HF model fetch; new
  telegram_ips ACL covering AS62041 as a backstop for squid's
  dstdomain rule which does not match HTTPS CONNECT in some cases
  (24 of 9083 api.telegram.org requests were denied in recent log).
  Renamed hermes_network -> localnet (squid default).
- hermes_squid/hermes-gateway-entrypoint.sh (new): installs
  python3-pip + ffmpeg via apt and faster-whisper via pip3 on first
  start. Idempotent: skips install if faster_whisper importable.
  Exports STT_PROVIDER=local and model/device pins, then execs the
  original gateway command.
- hermes_squid/stack-compose.yml: hermes-gateway now uses
  entrypoint=[/opt/data/hermes-gateway-entrypoint.sh], binds the
  script from ./, and adds STT_* env vars. Other services untouched.
admin requested changes 2026-08-29 20:18:47 +02:00
Dismissed
@@ -0,0 +1,25 @@
#!/bin/sh
set -e
Owner

Revert file

Revert file
@@ -61,7 +61,8 @@ services:
hermes-gateway:
image: nousresearch/hermes-agent:v2026.8.3
Owner

Revert file

Revert file
hermes added 1 commit 2026-08-29 20:25:42 +02:00
Apply review feedback on PR #21:
- revert entrypoint, command change, bind-mount, and STT_* env in
  hermes-gateway (stack-compose.yml)
- delete hermes-gateway-entrypoint.sh (no longer needed)
- keep squid.conf changes (stt_pkg_domains, telegram_ips) so the
  whitelist is ready for whichever STT install path is chosen next
hermes added 1 commit 2026-08-29 21:15:14 +02:00
The nginx sidecar failed on startup with
  'host not found in upstream brencher.rudolf.keenetic.link'
when DNS could not resolve the public upstream hostname. Because the
hard-coded proxy_pass URL forced nginx to resolve at startup, a single
DNS hiccup killed every task; with restart_policy max_attempts:3 the
service stayed down forever, which in turn blocked every hermes_infra
deploy because brencher marks the whole release failed when any
service is not running.

Move the upstream URL into a variable (set $brencher_upstream) and
add an explicit resolver directive with a 30s cache. Nginx now
resolves the upstream lazily on first request and keeps the cached
address across transient DNS failures.

This unblocks PR #21 (squid whitelist) and future deploys of the
hermes_infra stack.
hermes added 1 commit 2026-08-29 22:16:24 +02:00
faster-whisper fetches model files through xet-backed CAS:
  https://cas-server.xethub.hf.co/v2/reconstructions/...

That domain (and its parent xethub.hf.co, and hf.co for direct
huggingface_hub use) is not in the stt_pkg_domains ACL. With
squid denying the CONNECT, faster-whisper fails at first model
download with:

  CAS Client Error: Request middleware error: error sending
  request for url
  (https://cas-server.xethub.hf.co/v2/reconstructions/...)

Add .hf.co, .xethub.hf.co and .cas-server.xethub.hf.co to the
existing stt_pkg_domains ACL.

Squid config name 'squid_conf_auto-...': 47 chars (still under 64).
admin approved these changes 2026-08-29 22:51:51 +02:00
admin merged commit 0c95d8e1ee into master 2026-08-29 22:53:48 +02:00
admin deleted branch feat/stt-voice-faster-whisper 2026-08-29 22:53:49 +02:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: admin/ansible#21