See commit 1af4832. Removes the broken global redirect that exposed internal :8443, pins TLS routes to websecure, and adds a dedicated web redirect middleware so ACME HTTP-01 remains isolated.
---------
Co-authored-by: Hermes Agent <hermes@localhost>
Reviewed-on: org1/infra#22
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Summary
- add the declarative K3s `HelmChartConfig` for Traefik `web`/`websecure`
- expose TCP 80/443 and configure the `letsencrypt` HTTP-01 resolver
- persist `/data/acme.json` through a PVC
- add TLS routing annotations and host lists for the shared public Ingress and Hermes Ingress
- document the router cutover, verification procedure, and separate DEC-012 decision
## Scope
This PR contains the traffic configuration and documentation. It does not execute `kubectl apply`, change Keenetic, issue certificates, or commit credentials.
## Required operator input before apply
- direct WAN TCP 80 and 443 forwarding to 192.168.1.114
- the real Let's Encrypt registration email, inserted through the approved values/secret workflow rather than invented in Git
- final review of the installed Traefik chart values and PVC/storage behavior
## Verification
- `git diff --check` passed
- base: `e084ce4ecf39f19d08ec97fee852f59310e0cdbb`
- head: `4f70885`
Related decision: `docs/decisions/2026-11-21-traefik-tls-http01.md`
---------
Co-authored-by: Hermes Agent <hermes@localhost>
Reviewed-on: org1/infra#21
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Summary
Make Kubernetes Traefik the common HTTP entrypoint while preserving the existing Swarm-hosted public domains.
## Routing
- `hermes.rudolf.keenetic.link` -> Kubernetes Service `hermes:9119`
- `brencher.rudolf.keenetic.link` -> `192.168.1.96:5001`
- `immich.rudolf.keenetic.link` -> `192.168.1.96:2283`
- `gitea.rudolf.keenetic.link` -> `192.168.1.96:3002`
The Swarm targets are represented by selectorless Services with static Endpoints. The router can use ordinary TCP forwarding to `192.168.1.114:80`, preserving the HTTP `Host` header for Traefik routing instead of using the Keenetic HTTP reverse-proxy feature.
## Validation
- `git diff --cached --check` passed
- Changed files are limited to the Hermes ingress, backend Services/Endpoints, Kustomization, and README
- `kubectl kustomize` was unavailable in the execution environment
Reviewed-on: org1/infra#14
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Summary
- document a one-command `rudolf-T` bootstrap
- mount a dedicated resolver into `k3s-agent` only
- keep the Ubuntu host resolver unchanged for other services
The resolver is used by embedded containerd during image pulls. The registry hostname and TLS verification remain unchanged.
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: org1/infra#19
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Flux reconciliation of PR #17 failed because `hermes_generic1` and `hermes_generic2` are invalid Kubernetes namespace names.
This PR renames them to `hermes-generic1` and `hermes-generic2` in both Namespace manifests and Kustomize overlays.
Validation:
- main render remains byte-identical to admin/main
- generic1, generic2, and root renders succeed with Kustomize v5.8.3
- git diff --check passes
- no underscore namespace names remain
Fixes the Flux dry-run error reported by the live cluster.
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: org1/infra#18
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Replace the duplicated hermes/hermes_generic manifests with a shared Kubernetes base derived from legacy hermes_infra. Add hermes_infra and generic overlays with separate namespaces, NFS PV/PVC state, and credentials. Preserve the external Swarm docker socket through a namespaced Service/Endpoints because K3s uses containerd and has no local Docker socket. Sarg is intentionally deferred until its dynamically tagged image has a stable Kubernetes reference.
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: org1/infra#17
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Allow the Kubernetes Hermes pod to reach the FreeCAD MCP bridge on rudolf-T at 100.105.201.15.
- TCP 9875: XML-RPC
- TCP 9876: bridge socket
- exact destination /32 only
This changes the NetworkPolicy only; live connectivity still requires the PR to be merged and reconciled by Flux.
Reviewed-on: org1/infra#15
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
The merged public Ingress was created as `hermes`, while a legacy `hermes-ui` Ingress with the same host remained in the cluster outside the current Flux inventory. This creates two routes for the same host.
Reuse the existing `hermes-ui` resource name so Flux updates it in place and prunes the inventory-managed `hermes` resource, leaving one route for `hermes.rudolf.keenetic.link`.
Reviewed-on: org1/infra#13
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## What changed
- replace the broken in-cluster Nginx UI deployment with a Traefik Ingress for `hermes.rudolf.keenetic.link`;
- route the Ingress to the existing `hermes` Service on its named `dashboard` port (`9119`);
- allow only the live Traefik pods in `kube-system` to reach Hermes port `9119`;
- remove the UI Nginx capability failure path (`setgid(101) Operation not permitted`);
- remove the stale UI middleware/sidecar definition from the Hermes kustomization so Flux can prune it;
- document the router mapping and verification path.
## Root cause observed in the cluster
The live UI returned `502 Bad Gateway`. The deployed UI sidecar had Nginx workers failing with `setgid(101) Operation not permitted`, while the Ingress was targeting the sidecar path and referenced a middleware that Traefik could not resolve. The direct application Service already exposes the dashboard and WebSocket endpoint on port `9119`, so the sidecar is unnecessary.
## Router
Keep the public mapping:
`hermes.rudolf.keenetic.link` → `xu4_lan` (`192.168.1.114`) → HTTP `80`.
Do not forward to the Kubernetes Service port `9119` or to a Tailscale address from the router. The router forwards to the Kubernetes ingress entrypoint; the Kubernetes Ingress selects the `hermes` Service.
## Verification
- base branch is current `org1/infra:main`;
- `git diff --check` passes;
- no conflict markers are present in tracked files;
- live rollout verification remains pending until Flux reconciles this PR, after which verify the Ingress, Traefik route, Service endpoints, and external URL.
Reviewed-on: org1/infra#12
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Add DEC-009 to record the current remote web application domain management model.
Recorded facts:
- router model: Keenetic Hero 4G;
- brencher.rudolf.keenetic.link: authorized access;
- immich.rudolf.keenetic.link: public access;
- gitea.rudolf.keenetic.link: no access;
- hermes.rudolf.keenetic.link: public access.
The decision explicitly distinguishes HTTP/HTTPS web application publishing from arbitrary TCP port forwarding for Docker API ports 2375 and 2376. No image is added.
Reviewed-on: org1/infra#11
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Allow the Kubernetes Hermes pod to reach the Docker Swarm socket proxy used for read-only Swarm inspection.
The session history shows the established endpoint as `192.168.1.96:2375`; previous direct checks used Docker API `/v1.44/version`, `/version`, `/_ping`, `/services`, and `/volumes`.
This PR adds only a narrow egress rule for TCP 2375 to `192.168.1.96/32`. It does not add an environment variable or change the socket proxy's read-only configuration.
Reviewed-on: org1/infra#10
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
The merged Hermes NFS PV used NFSv4, but the existing NFS server export is reachable with NFSv3, matching the Swarm volume configuration. This fixes the mount access denied error during rollout.
Reviewed-on: org1/infra#5
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Use a dedicated static NFS-backed PV/PVC for the Kubernetes Hermes state at /root/volumes/hermes/k3s. Keep the existing local-path PVC declared for rollback and switch the Deployment to the new claim.
The path is separate from the Swarm Hermes gateway and agent state directories. The NFS directory must be created on 192.168.1.96 before rollout.
Reviewed-on: org1/infra#4
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Summary
Move Flux K3s reconciliation from the removed `org1/k3s` repository to `org1/infra`.
## Changes
- point the `k3s` GitRepository at `org1/infra.git`
- reconcile the `./k3s_rencher` path
- update the GitOps documentation
## Verification
- `git diff --check`
- reviewed the repository tree and confirmed `org1/k3s` is absent
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: org1/infra#3
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Summary
Expand the `agent-gateway/agent-readonly` service account from the built-in `view` role to a dedicated read-only role.
## Access policy
- Adds `get`, `list`, and `watch` for the core cluster resources and all currently served non-core API groups.
- Adds read-only non-resource access for API discovery, health, OpenAPI, and version endpoints.
- Does not grant access to the Kubernetes `secrets` resource.
- Does not grant write verbs, exec, port-forward, proxy, or token creation permissions.
## Verification
- Current target `org1/k3s:main`: `2bbac8467a9297e7056956328f576978142f3b24`
- `git diff --check`: passed
- Changed file: `k3s_rencher/infrastructure/agent-gateway/rbac.yaml`
The existing binding name is preserved so Flux applies the role change in place.
## Kustomize inclusion
The root `k3s_rencher/kustomization.yaml` already includes `infrastructure/agent-gateway`. Its nested Kustomization includes `rbac.yaml`, so the new ClusterRole is applied transitively while preserving the `agent-gateway` namespace transformation. A duplicate direct root entry would cause duplicate resource processing.
---------
Co-authored-by: admin <rudolfss13@gmail.com>
Reviewed-on: org1/k3s#1
Co-authored-by: Alex Rudenk <rudolfss13+hermes@gmail.com>
Closes#52.
Adds exactly one second act_runner on `rudolf-T`. The existing runner is constrained away from `rudolf-T`, so when that node is unavailable only the existing runner remains available. Both runners share the normal build label.
Validation: `docker stack config -c poc/gitea/stack-compose.yml` passes; the stack contains exactly two runner services.
Co-authored-by: rudolf <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13@gmail.com>
Co-authored-by: admin <rudolfss13@gmail.com>
Reviewed-on: admin/ansible#53
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Adds per-node checks for Swarm TCP/UDP ports, NFS reachability, and direct Telegram API egress. Scope is connectivity_tester and hermes-related network diagnostics only.
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Co-authored-by: brencher_bot <rudolfss13@gmail.com>
Reviewed-on: admin/ansible#49
Co-authored-by: Alex Rudenk <rudolfss13+hermes@gmail.com>
## Summary
Bumps the Gitea stack image from `1.27.0` to `1.27.3` — the last stable release on the 1.27.x line.
This is the first PR in a two-step plan tracked in admin/brencher#63:
1. **This PR** — bump to `1.27.3`. Stable, security patches only, no schema change, no Projects API yet. Goal: validate the upgrade path (DB migration, container swap, compose-version label update) without introducing new features.
2. **Follow-up PR (planned)** — bump to `1.28.0` (or build from `main`) once `1.28.x` ships with the Projects REST API landed in `7fae3d5db` (merged 2026-08-08).
## Why 1.27.0 is insufficient
Gitea `1.27.0` was released 2026-07-13. Three weeks later, on 2026-08-08, `silverwind` merged PR go-gitea/gitea#38691 — the long-awaited Projects REST API (the resolution of go-gitea/gitea#14299, originally filed in 2019). That commit (`7fae3d5db`) added full CRUD for repo/org/user project boards, columns, and issue↔column assignment.
1.27.0 ships **before** that merge. Confirmed:
- `git ls-tree v1.27.0 --name-only | grep routers/api/v1/.*project` → empty.
- `git ls-tree 7fae3d5db --name-only | grep project` → `routers/api/v1/shared/project.go`, `routers/api/v1/swagger/project.go`, and routes registered inside `routers/api/v1/api.go` (`addProjectRoutes`).
The current prod instance at `gitea.rudolf.keenetic.link` reports `1.27.0`, which is why we cannot move kanban cards programmatically today (verified end-to-end in admin/brencher#63's repro section).
## Why stop at 1.27.3 (not jump straight to 1.28.0)
1.27.3 is the last stable 1.27.x and contains the SECURITY fix for CVE-2024-45337 (access-token scope bypass) plus 8 months of bug fixes — strictly safer than staying on 1.27.0.
1.28.0 is not yet released as of writing (only `v1.28.0-dev` exists on the upstream tag list, dated 2026-06-29 — before the Projects API merge). Going to `:dev` or `:main` would require a build pipeline we don't have today and would skip the version bump validation path. The Projects API change also adds a column to `project_board` (per #37701 migration `v333`), so the jump is a real schema change that should be its own validated step.
## Diff
```
- image: docker.gitea.com/gitea:1.27.0
+ image: docker.gitea.com/gitea:1.27.3
```
## Validation after deploy
| Check | Expected |
|-------|----------|
| `GET /api/v1/version` | `1.27.3` |
| `POST /api/v1/repos/{owner}/{repo}/issues` | `201` |
| `PATCH /api/v1/repos/{owner}/{repo}/issues/{idx} {"projects":[id]}` | `200` with updated projects |
| WebSocket /ws uptime (admin/brencher#58 regression check) | no new KeyError |
| Stack `gitea_server` compose-version label | `auto-1.27.3-*` |
| Existing orgs / repos / users | unchanged (DB upgrade path is forward-compatible within 1.27.x) |
## Risk
Low. 1.27.x → 1.27.3 is a patch-level bump. No migrations required beyond what 1.27.0 → 1.27.3 already covers upstream. Existing tokens, repos, projects, kanban cards, comments, and access controls are unaffected.
## Refs
- Tracking: admin/brencher#63
- Closed upstream feature request: go-gitea/gitea#14299
- Closed upstream agentic-workflows request: go-gitea/gitea#37151
- Projects API merge commit: go-gitea/gitea@7fae3d5db (PR #38691)
- 1.27.3 release notes: go-gitea/gitea/releases/tag/v1.27.3
- CVE-2024-45337: go-gitea/gitea/security/advisories/GHSA-7hqw-9c4g-2v25
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: alex <alex@rudenk.local>
Reviewed-on: admin/ansible#37
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Checks node-local prerequisites that break Hermes after relocation: direct TLS egress to api.telegram.org and real NFS read/write against the Hermes agent export. Removes the tester self-probe that caused every round to crash.
Tests: compileall, unittest, git diff --check.
Reviewed-on: admin/ansible#50
Co-authored-by: Alex Rudenk <rudolfss13+hermes@gmail.com>
## Summary
Configure the `hermes` stack persistent volumes as NFS-backed volumes on the SSD of `odroid`.
Changed only:
- `hermes_gateway_data` → `192.168.1.96:/root/volumes/hermes/gateway`
- `hermes_agent_data` → `192.168.1.96:/root/volumes/hermes/agent`
`hermes_squid` is not included.
## Validation
- `docker stack config -c poc/hermes/stack-compose.yml` succeeds.
- `git diff --check` succeeds.
- Diff contains only `poc/hermes/stack-compose.yml`.
## Deployment prerequisite
Before deploy, prepare the NFS export and copy the existing local volumes according to issue #45:
admin/ansible#45
Do not deploy before verifying the copy, NFS access from `rudolf-T`, and the absence of concurrent writes to the old volumes.
## Tracking
Resolves the storage work in admin/ansible#45:
admin/ansible#45
Co-authored-by: hermes <rudolfss13@gmail.com>
Reviewed-on: admin/ansible#46
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Changes
- add the release workflow to `admin/ansible`
- trigger builds for `auto-*` branches in the administrative repository
- build and push `gitea.rudolf.keenetic.link/hermes/connectivity_tester`
- update the stack image to the same registry target
This is required by Brencher after auto-branches are pushed to `gitea_admin`.
Reviewed-on: admin/ansible#41
Co-authored-by: Alex Rudenk <rudolfss13+hermes@gmail.com>
## Summary
New Swarm stack `connectivity_tester` — a long-running Python service that
probes the cluster's connectivity every 10 minutes and logs structured
PASS/FAIL per check.
## What's in the box
- `connectivity_tester/Dockerfile` — python:3.12-slim base
- `connectivity_tester/app.py` — main loop: read env, run round, sleep, repeat
- `connectivity_tester/endpoints.py` — `EXPECTED_INTERNAL`, `EXPECTED_EXTERNAL`, `NEGATIVE` lists
- `connectivity_tester/stack-compose.yml` — Swarm stack with `mode: global` (one task per node)
## Behaviour
- One round every `PERIOD_SECONDS` (default 600).
- Per round: TCP probes against the three allow-lists + the negative-list.
- Negative-list endpoints (cloud metadata services etc.) MUST NOT respond.
If they do → log `ERROR LEAK`, container keeps running.
- `EXTRA_ENDPOINTS` env var accepts comma-separated `host:port` pairs for ad-hoc checks.
## Placement
`mode: global` — Swarm schedules one task on each node. No node pinning.
Networks: `hermes_proxy` overlay (for internal DNS resolution) + `default`
(for egress via squid).
## Acceptance
- After deploy, `docker service logs connectivity_tester_connectivity-tester`
shows one round per 10 minutes.
- Logs visible at info level in normal conditions (squid, hermes-gateway,
etc. all PASS).
- Any unexpected reachability or unreachable internal endpoint triggers
ERROR-level log within `CHECK_TIMEOUT_S`.
## Related
- Org1/hermes-voice-gateway#3 (Path B: build custom voice-gateway) — blocked by Org1/hermes-voice-gateway#5 (connectivity testing container)
- Org1/hermes-voice-gateway#5 — this PR resolves the blocker
## Verification checklist
- [ ] Stack `connectivity_tester` shows up in `docker stack ls`
- [ ] `docker service ps connectivity_tester_connectivity-tester` shows one task per node (odroid, rudolf-T, xu4)
- [ ] Logs show first round within 30 seconds of start
- [ ] `docker service logs --tail 50 connectivity_tester_connectivity-tester` shows PASS for at least squid:3128 and hermes-gateway:8000
- [ ] Negative-list (169.254.169.254) is unreachable (PASS OK-NEG)
---------
Co-authored-by: brencher_bot <rudolfss13@gmail.com>
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: admin/ansible#39
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Mirror of hermes/ansible#6.
Allow Hermes to reach GitHub Copilot network endpoints through the corporate squid proxy.
Adds a `copilot_domains` ACL covering the domains required by GitHub Copilot (IDE integration and ACP):
- `api.github.com` — user/auth and copilot REST endpoints
- `copilot-proxy.githubusercontent.com` — completion and chat streaming backend
- `*.githubcopilot.com` — newer Copilot control plane
- `copilot-telemetry.githubusercontent.com` — older Copilot telemetry
- `default.exp-tas.com` — Microsoft AID/Experimentation rollout
- `marketplace.visualstudio.com` + `vscode.blob.core.windows.net` — VS Code Marketplace and VSIX downloads
- `*.githubassets.com`, `*.vscode-unpkg.net` — asset/CDN hosts used by extension UI
`.github.com`, `*.githubusercontent.com`, and the rest of `stt_pkg_domains` were already allowed, so no overlap.
Validation: a simple login + completion request from an IDE through `http://squid:3128`.
Reviewed-on: admin/ansible#40
Co-authored-by: Alex Rudenk <rudolfss13+hermes@gmail.com>
## Problem
The built-in `act_runner` cache server binds to an interface unreachable from per-job container networks (`container.network: ""` is the default). `ACTIONS_CACHE_URL` ends up pointing at an unreachable IP, and every CI job burns ~9 minutes on:
```
::warning::Failed to restore: getCacheEntry failed: connect ETIMEDOUT 192.168.112.21:37051
::warning::Failed to save: reserveCache failed: connect ETIMEDOUT 192.168.112.21:37051
```
Measured on `admin/brencher#15` (https://gitea.rudolf.keenetic.link/admin/brencher/actions/runs/16):
| Job | wall-clock | of which dead air on cache |
|---------|------------|----------------------------|
| 3.12.3 | 12m 40s | ~9m (271s × 2 timeouts) |
| 3.13 | 11m 13s | ~9m |
| 3.14 | 11m 17s | ~9m |
So even though `python-package.yml` enables `cache: 'pip'`, the cache backend is unreachable, jobs hit `pip install` cold every time, and CI ends up ~4x slower than necessary.
## Fix
Run a dedicated `act_runner cache-server` container on the same overlay network as the runner, and point the runner at it via `external_server`. Shared secret comes from a Docker secret mounted into both containers and read via `ACT_RUNNER_EXTERNAL_SECRET_FILE`.
## Changes
- `poc/gitea/stack-compose.yml`:
- New `cache_server` service (`command: ["cache-server"]`) on the `gitea` overlay with a `cache_data` named volume.
- `runner` and `cache_server` both mount the external Docker secret `gitea_cache_secret` at `/run/secrets/gitea_cache_secret` and set `ACT_RUNNER_EXTERNAL_SECRET_FILE` accordingly.
- Top-level `secrets:` block declares `gitea_cache_secret` as `external: true`, with the create command commented inline next to it.
- `poc/gitea/config.yaml`: point `cache.{host,port,external_server}` at `gitea_cache_server:37051`; `external_secret` left empty because the secret is loaded from the file.
## Verification
- Both YAMLs parse (PyYAML).
- Service name `gitea_cache_server` resolves on the `gitea` overlay via Swarm service-name DNS.
- `docker secret inspect gitea_cache_secret` works after the one-time create step.
## Deploy notes
1. On the swarm manager, create the secret once:
```
echo -n "$(openssl rand -hex 32)" | docker secret create gitea_cache_secret -
```
2. `docker stack deploy -c stack-compose.yml gitea`.
3. Re-run a CI job and confirm cache `restore`/`save` log lines no longer ETIMEDOUT.
## Related
- Related: admin/brencher#15 — `python-package.yml` enables `cache: 'pip'` and was timing out for the reason this PR fixes.
- Refs: gitea/actions/runs/16 jobs #248/249/250.
## Supersedes
- Supersedes: admin/ansible#33 — first attempt was based on a stale `origin/master` and accidentally included `hermes_squid/` changes from the actual upstream merge. The current branch is rebased on `admin/master` (`c3b7128`) and only touches `poc/gitea/`.Reviewed-on: http://192.168.1.96/admin/ansible/pulls/34
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
Flip CONTAINERS/IMAGES/NETWORKS/VOLUMES/SERVICES/TASKS/NODES from 0 to 1 so hermes-agent can inspect running stacks (read container logs, enumerate swarm services) without going through each node's shell.
POST stays 0 — no container creation, no image builds, no exec into running containers. Anything that mutates state remains blocked at the proxy level.
---------
Co-authored-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/4
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## What
Wires up local STT (faster-whisper) into the hermes-gateway container so voice messages on Telegram transcribe without depending on external services.
Closes#19. Supersedes the closed#20 (which used a custom Dockerfile-hermes; this version uses a bind-mounted entrypoint script instead since pypi.org is now whitelisted).
## Changes
1. **`hermes_squid/squid.conf`** — expands the egress whitelist:
- new `stt_pkg_domains` for `pypi.org`, `files.pythonhosted.org`, `huggingface.co`, `github.com`, `models.dev`, `registry.npmjs.org`, `cdn-lfs.huggingface.co` — needed for runtime `pip install faster-whisper` and HF model fetch.
- new `telegram_ips` covering AS62041 (91.108.4.0/22, 91.108.8.0/22, 91.108.12.0/22, 91.108.16.0/22, 91.108.20.0/22, 91.108.56.0/22, 149.154.160.0/20, 185.76.151.0/24) as a backstop for the `dstdomain` ACL. squid's `dstdomain` does not match HTTPS CONNECT requests without `ssl_bump`, so the recent log shows 24 of 9083 `api.telegram.org` requests getting 403 in a 5-minute window.
- renamed `hermes_network` -> `localnet` (squid default name, no behavior change).
2. **`hermes_squid/hermes-gateway-entrypoint.sh`** (new) — bind-mounted into the container at `/opt/data/hermes-gateway-entrypoint.sh` and set as the service entrypoint. The script:
- is idempotent: on first start, runs `apt-get install -y python3-pip ffmpeg` and `pip3 install --break-system-packages faster-whisper`. Subsequent restarts skip install if `faster_whisper` is already importable.
- exports `STT_PROVIDER=local`, `STT_LOCAL_MODEL=base`, `STT_LOCAL_DEVICE=cpu`, `STT_LOCAL_COMPUTE_TYPE=int8`.
- `exec`s the original gateway command (`gateway run`).
3. **`hermes_squid/stack-compose.yml`** — `hermes-gateway` gets:
- `entrypoint: ["/opt/data/hermes-gateway-entrypoint.sh"]`
- `command: ["gateway", "run"]`
- bind-mount `./hermes-gateway-entrypoint.sh:/opt/data/hermes-gateway-entrypoint.sh:ro`
- new env: `STT_PROVIDER`, `STT_LOCAL_MODEL`, `STT_LOCAL_DEVICE`, `STT_LOCAL_COMPUTE_TYPE`
No other service in the compose file is touched.
## Why runtime install, not a custom image
- `pypi.org` is now whitelisted, so `pip3 install` works at runtime through the same squid.
- Avoids maintaining a custom hermes-agent image and a local registry tag.
- The upstream `nousresearch/hermes-agent:v2026.8.3` image is kept as-is.
## Deploy notes
- brencher apply will trigger `docker service update` for hermes-gateway; the new entrypoint runs the apt+pip install on first start (~1-2 min).
- First start also downloads the `base` faster-whisper model (~150 MB) from `huggingface.co` — covered by `stt_pkg_domains`.
- squid config is reread on `docker service update` for the squid service; new ACLs apply without a full stack restart.
- Subsequent restarts of hermes-gateway skip the install path entirely (idempotent guard).
## Verified locally
- `git diff --stat upstream/master` shows only the three files above (+25 entrypoint, +14 squid, +8 compose).
- The new worktree is `/opt/data/work/thread-114-Alex-InfraAlexBot/ansible` (per-session working dir).
## Risks
- `apt-get update` inside the container needs `deb.debian.org` reachable — already in `allowed_domains`.
- `STT_LOCAL_MODEL=base` is ~150 MB and runs on CPU. If latency on Raspberry-class nodes is bad, switch to `tiny`.
- `--break-system-packages` is required for PEP 668 (Debian trixie + Python 3.11+). If the upstream image switches to PEP 668 strictly, this flag is the right escape hatch.
## Related
- #19 — investigation that found missing faster-whisper + squid deny pattern
- #20 — superseded (used Dockerfile-hermes instead of runtime install)
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/21
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
## Summary
Adds three pieces to `admin/ansible:master`, all in the `hermes_squid` stack:
1. **`docker_socket_proxy`** (commit `449cbaa`) — `tecnativa/docker-socket-proxy` inlined as a service so it deploys + tears down with the rest of hermes_infra. **On `hermes_proxy` overlay** (after the latest refactor). Default policy: read-only `PING` / `EVENTS` / `VERSION`; everything else denied.
2. **`brencher-state-proxy`** (commits `9b41abb` + `371978a` + `0276497`) — path-locked nginx reverse proxy that lets hermes-agent reach:
- `/state` → `https://brencher.rudolf.keenetic.link/state` (the public brencher)
- `/state2` → `http://host.docker.internal:5002/state` (the secondary instance, reached the same way the brencher container itself reaches it)
- everything else → 403
- listens on plain HTTP port 8080 (not TLS), no host port published
- **no secrets, no manual cert generation required**
- **On `hermes_proxy` overlay**
3. **Connectivity wiring** (commits `b08d25d` + `7446dbb`) — makes hermes-gateway able to resolve and reach the sidecars:
- NO_PROXY / no_proxy list both sidecar hostnames so requests skip squid
- All three services (hermes-gateway, docker_socket_proxy, brencher-state-proxy) are on the same `hermes_proxy` overlay, so DNS resolves and traffic flows directly
4. **No `squid` allowlist change** — net effect on `squid.conf` is zero (commit `9b41abb` added `brencher.rudolf.keenetic.link` to the allowlist, commit `371978a` removed it).
## What's in this PR (net diff vs master)
- `docker_socket_proxy/stack-compose.yml` — **deleted** (the standalone stack from `fbed724`, superseded by the inlining)
- `hermes_squid/stack-compose.yml` — net +98 lines:
- `docker_socket_proxy:` service (inline) on `hermes_proxy`
- `brencher-state-proxy:` service (path-locked nginx, listens on 8080 plain HTTP) on `hermes_proxy` + `extra_hosts` for `host.docker.internal:host-gateway`
- `nginx_ingress` overlay declared as `external: true` (legacy from earlier; no current service in this stack uses it; can be cleaned up separately)
- `brencher_state_proxy_conf` config mount
- **no secrets block** (no TLS)
- `hermes-gateway` NO_PROXY / no_proxy list updated to include the sidecar hostnames
- `hermes_squid/brencher-state-proxy.conf` — **new**, 93 lines
- `hermes_squid/squid.conf` — **unchanged** from master
## How hermes-agent reaches `/state` / `/state2`
After this PR merges, code running inside `hermes-gateway` can:
```python
import requests
r1 = requests.get("http://brencher-state-proxy:8080/state") # public brencher
r2 = requests.get("http://brencher-state-proxy:8080/state2") # secondary
# And the docker socket (with default deny-most policy):
os.environ["DOCKER_HOST"] = "tcp://docker_socket_proxy:2375"
```
`NO_PROXY` strips the squid hop. The overlay's Docker-embedded DNS resolves the service name. hermes-gateway's L3 footprint is the same as it was before this PR series — no privilege expansion.
## Why hermes_proxy, not admin
I went back and forth on this in PR comments and in the conversation with the user. The final design (commit `7446dbb`):
- All three services (`hermes-gateway`, `docker_socket_proxy`, `brencher-state-proxy`) on `hermes_proxy`
- `brencher-state-proxy` reaches `https://brencher.rudolf.keenetic.link` over the **public internet** (not via the nginx_ingress overlay) — the upstream HTTPS call doesn't need service-name routing
- Despite `hermes_proxy` having `internal: true` in the network declaration, **outbound traffic is not blocked** — `internal: true` only denies external ingress (published ports) to the overlay, not egress from services on it
- hermes-gateway does **not** need to join `admin` overlay, so its L3 footprint is unchanged from before this PR
Earlier commits (`b08d25d`) used `admin` for the sidecars and added `admin` to hermes-gateway. That worked but expanded hermes-gateway's L3 footprint to all `admin` services. The refactor in `7446dbb` is the cleaner final design.
## Deployment prerequisites — zero
No manual steps required before `docker stack deploy hermes_squid`:
- No `openssl` to generate a cert
- No `docker secret create` to wire up TLS
- The `nginx_ingress` overlay should exist for legacy reasons (declared by `docker-compose-nginx.yaml`), but no current service in this stack uses it.
## Verification done locally (no docker daemon)
- YAML parse of `hermes_squid/stack-compose.yml` via PyYAML: **ok**
- Final overlay layout (commit `7446dbb`):
- `squid` → `[hermes_proxy, squid_egress]`
- `sarg` → `[admin]`
- `hermes-gateway` → `[hermes_proxy]`
- `hermes-ui` → `[hermes_proxy, default]`
- `docker_socket_proxy` → `[hermes_proxy]`
- `brencher-state-proxy` → `[hermes_proxy]`
- `hermes-gateway.environment.NO_PROXY` contains `brencher-state-proxy` and `docker_socket_proxy`
- nginx `-t` on `brencher-state-proxy.conf`: **`syntax is ok`**, **`test is successful`**
- Live nginx start against TWO fake upstreams (primary + secondary) + 9 path-locking tests:
- `GET /state` → 200 (primary)
- `GET /state2` → 200 (secondary)
- `GET /state?x=1` → 200, query forwarded
- `GET /state2?x=1` → 200, query forwarded
- `GET /state/` → 403
- `GET /state2/` → 403
- `GET /` → 403
- `GET /stateX` → 403
- `GET /healthz` → 200
- **all 9 pass**
## What was NOT verified (called out explicitly)
- `docker stack deploy hermes_squid` end-to-end (no daemon in this sandbox)
- That the secondary brencher at `host.docker.internal:5002` actually exists and returns valid JSON at `/state`
- The actual `internal: true` semantics on swarm-mode overlays in this specific Docker version — I verified via Docker docs, but a real `docker exec` from inside the deployed hermes-gateway to the sidecars would be the only definitive test
- That the network placement actually works (no end-to-end deploy tested)
## Companion PR
`admin/brencher#4` (open, head `8bd6715`) deploys `hermes_squid` via the brencher pipeline. After both PRs merge, no further config is needed.
## Commit history
```
* 7446dbb (HEAD -> feat/docker-socket-proxy, origin/feat/docker-socket-proxy)
refactor(hermes_squid): put sidecars on hermes_proxy instead of admin
* b08d25d fix(hermes_squid): let hermes-gateway reach the new sidecars
* 0276497 feat(hermes_squid): drop TLS and secrets from brencher-state-proxy
* 371978a feat(hermes_squid): add /state2 path for secondary brencher, remove squid whitelist
* 9b41abb feat(hermes_squid): add brencher-state-proxy sidecar
* 449cbaa feat(hermes_infra): inline docker_socket_proxy into hermes_squid stack
* fbed724 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy
```
(`squid.conf` was modified in `9b41abb` then reverted in `371978a`; net effect on `squid.conf` is zero — no diff vs master for that file.)
---------
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/1
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>