Commit Graph
131 Commits
Author SHA1 Message Date
hermes 6848d114a7 chore(hermes_squid): remove brencher-state-proxy
Drop the brencher-state-proxy sidecar from hermes_infra -- it is
no longer needed. The sidecar existed to relay /state from
brencher.rudolf.keenetic.link into the admin overlay; that relay is
unnecessary since brencher ingress handles the path itself.

Changes vs the brencher-state-proxy commit that was merged in #4:
- delete hermes_squid/brencher-state-proxy.conf
- remove the brencher-state-proxy: service block from
  hermes_squid/stack-compose.yml
- remove the brencher_state_proxy_conf: top-level config object
- drop brencher-state-proxy from the NO_PROXY / no_proxy lists
- update the surrounding comments
2026-08-29 22:47:36 +00:00
c3b7128006 hermes_squid: open docker_socket_proxy to full read-only scope (#4)
Flip CONTAINERS/IMAGES/NETWORKS/VOLUMES/SERVICES/TASKS/NODES from 0 to 1 so hermes-agent can inspect running stacks (read container logs, enumerate swarm services) without going through each node's shell.

POST stays 0 — no container creation, no image builds, no exec into running containers. Anything that mutates state remains blocked at the proxy level.

---------

Co-authored-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/4
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-08-30 00:09:56 +02:00
hermesandhermes 0c95d8e1ee feat(hermes_squid): enable local STT via faster-whisper (runtime install) (#21)
## What

Wires up local STT (faster-whisper) into the hermes-gateway container so voice messages on Telegram transcribe without depending on external services.

Closes #19. Supersedes the closed #20 (which used a custom Dockerfile-hermes; this version uses a bind-mounted entrypoint script instead since pypi.org is now whitelisted).

## Changes

1. **`hermes_squid/squid.conf`** — expands the egress whitelist:
   - new `stt_pkg_domains` for `pypi.org`, `files.pythonhosted.org`, `huggingface.co`, `github.com`, `models.dev`, `registry.npmjs.org`, `cdn-lfs.huggingface.co` — needed for runtime `pip install faster-whisper` and HF model fetch.
   - new `telegram_ips` covering AS62041 (91.108.4.0/22, 91.108.8.0/22, 91.108.12.0/22, 91.108.16.0/22, 91.108.20.0/22, 91.108.56.0/22, 149.154.160.0/20, 185.76.151.0/24) as a backstop for the `dstdomain` ACL. squid's `dstdomain` does not match HTTPS CONNECT requests without `ssl_bump`, so the recent log shows 24 of 9083 `api.telegram.org` requests getting 403 in a 5-minute window.
   - renamed `hermes_network` -> `localnet` (squid default name, no behavior change).

2. **`hermes_squid/hermes-gateway-entrypoint.sh`** (new) — bind-mounted into the container at `/opt/data/hermes-gateway-entrypoint.sh` and set as the service entrypoint. The script:
   - is idempotent: on first start, runs `apt-get install -y python3-pip ffmpeg` and `pip3 install --break-system-packages faster-whisper`. Subsequent restarts skip install if `faster_whisper` is already importable.
   - exports `STT_PROVIDER=local`, `STT_LOCAL_MODEL=base`, `STT_LOCAL_DEVICE=cpu`, `STT_LOCAL_COMPUTE_TYPE=int8`.
   - `exec`s the original gateway command (`gateway run`).

3. **`hermes_squid/stack-compose.yml`** — `hermes-gateway` gets:
   - `entrypoint: ["/opt/data/hermes-gateway-entrypoint.sh"]`
   - `command: ["gateway", "run"]`
   - bind-mount `./hermes-gateway-entrypoint.sh:/opt/data/hermes-gateway-entrypoint.sh:ro`
   - new env: `STT_PROVIDER`, `STT_LOCAL_MODEL`, `STT_LOCAL_DEVICE`, `STT_LOCAL_COMPUTE_TYPE`

No other service in the compose file is touched.

## Why runtime install, not a custom image

- `pypi.org` is now whitelisted, so `pip3 install` works at runtime through the same squid.
- Avoids maintaining a custom hermes-agent image and a local registry tag.
- The upstream `nousresearch/hermes-agent:v2026.8.3` image is kept as-is.

## Deploy notes

- brencher apply will trigger `docker service update` for hermes-gateway; the new entrypoint runs the apt+pip install on first start (~1-2 min).
- First start also downloads the `base` faster-whisper model (~150 MB) from `huggingface.co` — covered by `stt_pkg_domains`.
- squid config is reread on `docker service update` for the squid service; new ACLs apply without a full stack restart.
- Subsequent restarts of hermes-gateway skip the install path entirely (idempotent guard).

## Verified locally

- `git diff --stat upstream/master` shows only the three files above (+25 entrypoint, +14 squid, +8 compose).
- The new worktree is `/opt/data/work/thread-114-Alex-InfraAlexBot/ansible` (per-session working dir).

## Risks

- `apt-get update` inside the container needs `deb.debian.org` reachable — already in `allowed_domains`.
- `STT_LOCAL_MODEL=base` is ~150 MB and runs on CPU. If latency on Raspberry-class nodes is bad, switch to `tiny`.
- `--break-system-packages` is required for PEP 668 (Debian trixie + Python 3.11+). If the upstream image switches to PEP 668 strictly, this flag is the right escape hatch.

## Related

- #19 — investigation that found missing faster-whisper + squid deny pattern
- #20 — superseded (used Dockerfile-hermes instead of runtime install)

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/21
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-08-29 22:53:46 +02:00
hermesandhermes 4cb8eeae35 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy (#1)
## Summary
Adds three pieces to `admin/ansible:master`, all in the `hermes_squid` stack:

1. **`docker_socket_proxy`** (commit `449cbaa`) — `tecnativa/docker-socket-proxy` inlined as a service so it deploys + tears down with the rest of hermes_infra. **On `hermes_proxy` overlay** (after the latest refactor). Default policy: read-only `PING` / `EVENTS` / `VERSION`; everything else denied.

2. **`brencher-state-proxy`** (commits `9b41abb` + `371978a` + `0276497`) — path-locked nginx reverse proxy that lets hermes-agent reach:
   - `/state` → `https://brencher.rudolf.keenetic.link/state` (the public brencher)
   - `/state2` → `http://host.docker.internal:5002/state` (the secondary instance, reached the same way the brencher container itself reaches it)
   - everything else → 403
   - listens on plain HTTP port 8080 (not TLS), no host port published
   - **no secrets, no manual cert generation required**
   - **On `hermes_proxy` overlay**

3. **Connectivity wiring** (commits `b08d25d` + `7446dbb`) — makes hermes-gateway able to resolve and reach the sidecars:
   - NO_PROXY / no_proxy list both sidecar hostnames so requests skip squid
   - All three services (hermes-gateway, docker_socket_proxy, brencher-state-proxy) are on the same `hermes_proxy` overlay, so DNS resolves and traffic flows directly

4. **No `squid` allowlist change** — net effect on `squid.conf` is zero (commit `9b41abb` added `brencher.rudolf.keenetic.link` to the allowlist, commit `371978a` removed it).

## What's in this PR (net diff vs master)
- `docker_socket_proxy/stack-compose.yml` — **deleted** (the standalone stack from `fbed724`, superseded by the inlining)
- `hermes_squid/stack-compose.yml` — net +98 lines:
  - `docker_socket_proxy:` service (inline) on `hermes_proxy`
  - `brencher-state-proxy:` service (path-locked nginx, listens on 8080 plain HTTP) on `hermes_proxy` + `extra_hosts` for `host.docker.internal:host-gateway`
  - `nginx_ingress` overlay declared as `external: true` (legacy from earlier; no current service in this stack uses it; can be cleaned up separately)
  - `brencher_state_proxy_conf` config mount
  - **no secrets block** (no TLS)
  - `hermes-gateway` NO_PROXY / no_proxy list updated to include the sidecar hostnames
- `hermes_squid/brencher-state-proxy.conf` — **new**, 93 lines
- `hermes_squid/squid.conf` — **unchanged** from master

## How hermes-agent reaches `/state` / `/state2`

After this PR merges, code running inside `hermes-gateway` can:
```python
import requests
r1 = requests.get("http://brencher-state-proxy:8080/state")    # public brencher
r2 = requests.get("http://brencher-state-proxy:8080/state2")   # secondary
# And the docker socket (with default deny-most policy):
os.environ["DOCKER_HOST"] = "tcp://docker_socket_proxy:2375"
```

`NO_PROXY` strips the squid hop. The overlay's Docker-embedded DNS resolves the service name. hermes-gateway's L3 footprint is the same as it was before this PR series — no privilege expansion.

## Why hermes_proxy, not admin

I went back and forth on this in PR comments and in the conversation with the user. The final design (commit `7446dbb`):

- All three services (`hermes-gateway`, `docker_socket_proxy`, `brencher-state-proxy`) on `hermes_proxy`
- `brencher-state-proxy` reaches `https://brencher.rudolf.keenetic.link` over the **public internet** (not via the nginx_ingress overlay) — the upstream HTTPS call doesn't need service-name routing
- Despite `hermes_proxy` having `internal: true` in the network declaration, **outbound traffic is not blocked** — `internal: true` only denies external ingress (published ports) to the overlay, not egress from services on it
- hermes-gateway does **not** need to join `admin` overlay, so its L3 footprint is unchanged from before this PR

Earlier commits (`b08d25d`) used `admin` for the sidecars and added `admin` to hermes-gateway. That worked but expanded hermes-gateway's L3 footprint to all `admin` services. The refactor in `7446dbb` is the cleaner final design.

## Deployment prerequisites — zero

No manual steps required before `docker stack deploy hermes_squid`:
- No `openssl` to generate a cert
- No `docker secret create` to wire up TLS
- The `nginx_ingress` overlay should exist for legacy reasons (declared by `docker-compose-nginx.yaml`), but no current service in this stack uses it.

## Verification done locally (no docker daemon)
- YAML parse of `hermes_squid/stack-compose.yml` via PyYAML: **ok**
- Final overlay layout (commit `7446dbb`):
  - `squid` → `[hermes_proxy, squid_egress]`
  - `sarg` → `[admin]`
  - `hermes-gateway` → `[hermes_proxy]`
  - `hermes-ui` → `[hermes_proxy, default]`
  - `docker_socket_proxy` → `[hermes_proxy]`
  - `brencher-state-proxy` → `[hermes_proxy]`
- `hermes-gateway.environment.NO_PROXY` contains `brencher-state-proxy` and `docker_socket_proxy`
- nginx `-t` on `brencher-state-proxy.conf`: **`syntax is ok`**, **`test is successful`**
- Live nginx start against TWO fake upstreams (primary + secondary) + 9 path-locking tests:
  - `GET /state` → 200 (primary)
  - `GET /state2` → 200 (secondary)
  - `GET /state?x=1` → 200, query forwarded
  - `GET /state2?x=1` → 200, query forwarded
  - `GET /state/` → 403
  - `GET /state2/` → 403
  - `GET /` → 403
  - `GET /stateX` → 403
  - `GET /healthz` → 200
  - **all 9 pass**

## What was NOT verified (called out explicitly)
- `docker stack deploy hermes_squid` end-to-end (no daemon in this sandbox)
- That the secondary brencher at `host.docker.internal:5002` actually exists and returns valid JSON at `/state`
- The actual `internal: true` semantics on swarm-mode overlays in this specific Docker version — I verified via Docker docs, but a real `docker exec` from inside the deployed hermes-gateway to the sidecars would be the only definitive test
- That the network placement actually works (no end-to-end deploy tested)

## Companion PR
`admin/brencher#4` (open, head `8bd6715`) deploys `hermes_squid` via the brencher pipeline. After both PRs merge, no further config is needed.

## Commit history
```
* 7446dbb (HEAD -> feat/docker-socket-proxy, origin/feat/docker-socket-proxy)
      refactor(hermes_squid): put sidecars on hermes_proxy instead of admin
* b08d25d fix(hermes_squid): let hermes-gateway reach the new sidecars
* 0276497 feat(hermes_squid): drop TLS and secrets from brencher-state-proxy
* 371978a feat(hermes_squid): add /state2 path for secondary brencher, remove squid whitelist
* 9b41abb feat(hermes_squid): add brencher-state-proxy sidecar
* 449cbaa feat(hermes_infra): inline docker_socket_proxy into hermes_squid stack
* fbed724 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy
```
(`squid.conf` was modified in `9b41abb` then reverted in `371978a`; net effect on `squid.conf` is zero — no diff vs master for that file.)

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/1
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-08-26 14:37:37 +02:00
admin 665f915634 x 2026-08-24 22:28:36 +02:00
admin 301ac87694 x 2026-08-24 21:52:05 +02:00
admin 72dcdcf12e x 2026-08-23 21:05:36 +02:00
admin 94ff81ae60 x 2026-08-23 20:54:15 +02:00
admin 0516b0ecaf x 2026-08-19 23:28:16 +02:00
admin de4de846be x 2026-08-12 22:33:30 +02:00
admin b3915e9428 x 2026-08-12 21:07:27 +02:00
admin 3aaa4b87bf x 2026-08-12 13:05:17 +02:00
admin 8e29e94f59 x 2026-08-12 12:51:45 +02:00
admin a7add7e894 x 2026-08-12 12:44:27 +02:00
admin 78d2c260fd gitea 2026-08-11 21:45:35 +02:00
admin e6dbfa36ac x 2026-08-05 14:31:38 +02:00
admin f12e6f8d91 x 2026-08-05 13:48:03 +02:00
admin 2941c9129c x 2026-08-05 12:49:04 +02:00
admin acabdca074 x 2026-08-05 11:56:55 +02:00
admin b085d19bda x 2026-08-05 11:48:12 +02:00
admin eaad0125b1 x 2026-08-05 11:46:57 +02:00
admin fb6bcc42f2 x 2026-08-05 11:19:30 +02:00
admin faf83ddfd4 x 2026-08-05 11:10:16 +02:00
admin 84f9124ecc x 2026-08-05 00:51:16 +02:00
admin d7dafaff5f x 2026-08-05 00:28:06 +02:00
admin 4c54d04c45 x 2026-08-05 00:26:51 +02:00
admin 9ae08556ee x 2026-08-05 00:22:32 +02:00
admin 9c9917471d x 2026-08-05 00:13:31 +02:00
admin d4393b5797 x 2026-08-05 00:08:03 +02:00
admin e6c30f8381 x 2026-08-05 00:02:35 +02:00
Copilotandrudolf1 86ddb32236 Inline config files into stack-compose.yml and remove separate config files (#7)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-08-04 12:42:25 +02:00
admin 30c1019abb Update stack-compose.yml 2026-08-04 11:49:30 +02:00
admin ec1888ba44 Update stack-compose.yml 2026-08-04 11:43:22 +02:00
admin d865ed9dc4 Change node hostname constraint in stack-compose.yml 2026-08-04 11:36:50 +02:00
admin 7112ff4efe x 2026-08-01 23:20:30 +02:00
admin ea189f7cb9 hermes-squid 2026-08-01 23:18:20 +02:00
admin ea12d7e7f9 gitea 2026-07-26 20:46:33 +02:00
admin c6c1e636c8 hermes 2026-06-28 22:41:16 +02:00
7b1d8d8226 Ansible/copilot/update ansible version to 2 7 5 (#5)
* feat: update immich v2.5.6 → v2.7.5

Agent-Logs-Url: https://github.com/rudolf1/uber_backup/sessions/288ed0cf-5c32-4ba8-9df9-6fec2d899afc

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>

* Update Redis digest from immich docker-compose.prod.yml

Agent-Logs-Url: https://github.com/rudolf1/uber_backup/sessions/0fd33671-72d6-47fd-bbf2-a9ccbacdca5a

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>

* Disable healthcheck for server service

Agent-Logs-Url: https://github.com/rudolf1/uber_backup/sessions/dbff4537-9c0c-4d7a-972a-1f73b8fa38c6

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>

* Rollback Immich upload mount path change

Agent-Logs-Url: https://github.com/rudolf1/uber_backup/sessions/1b2c0587-8cc5-4ee6-a592-571ef0aedbd5

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>

* revert: restore immich server healthcheck settings

Agent-Logs-Url: https://github.com/rudolf1/uber_backup/sessions/c43105ab-1893-4c2a-bbdc-7621fe566a61

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-05-23 12:36:14 +02:00
admin 73642d7e71 Ansible/registry (#3)
* registry

* x

* x

* x

* x

* x

* x

* x
2026-03-21 16:48:02 +01:00
admin 320c1ba097 Ansible/immich up (#2)
* immich up

* up

* up

* up

* up

* up

* immich up

* up
2026-02-11 21:01:29 +01:00
admin 2327b76171 immich up 2026-02-08 14:17:10 +01:00
admin 96ba30718a immich readme 2025-12-23 12:21:12 +01:00
admin adf6a18dfb Ansible/immich up (#1)
* immich up

* immich 2.4.1
2025-12-20 15:17:36 +01:00
admin 99b148a07e immich up 2025-10-16 19:02:25 +02:00
admin e73c7d6e98 immich 2025-09-23 12:50:43 +02:00
admin 2b734aa078 immich 2025-09-16 22:14:02 +02:00
admin 91458d6b2c immich 2025-09-16 21:37:47 +02:00
admin 8d1651a561 improvements 2025-06-02 23:56:25 +02:00
admin 46eb915399 Nextcloud rebalance 2025-05-16 23:19:05 +02:00