Adds three pieces to admin/ansible:master, all in the hermes_squid stack:
docker_socket_proxy (commit 449cbaa) — tecnativa/docker-socket-proxy inlined as a service so it deploys + tears down with the rest of hermes_infra. On hermes_proxy overlay (after the latest refactor). Default policy: read-only PING / EVENTS / VERSION; everything else denied.
/state → https://brencher.rudolf.keenetic.link/state (the public brencher)
/state2 → http://host.docker.internal:5002/state (the secondary instance, reached the same way the brencher container itself reaches it)
everything else → 403
listens on plain HTTP port 8080 (not TLS), no host port published
no secrets, no manual cert generation required
On hermes_proxy overlay
Connectivity wiring (commits b08d25d + 7446dbb) — makes hermes-gateway able to resolve and reach the sidecars:
NO_PROXY / no_proxy list both sidecar hostnames so requests skip squid
All three services (hermes-gateway, docker_socket_proxy, brencher-state-proxy) are on the same hermes_proxy overlay, so DNS resolves and traffic flows directly
No squid allowlist change — net effect on squid.conf is zero (commit 9b41abb added brencher.rudolf.keenetic.link to the allowlist, commit 371978a removed it).
What's in this PR (net diff vs master)
docker_socket_proxy/stack-compose.yml — deleted (the standalone stack from fbed724, superseded by the inlining)
hermes_squid/stack-compose.yml — net +98 lines:
docker_socket_proxy: service (inline) on hermes_proxy
brencher-state-proxy: service (path-locked nginx, listens on 8080 plain HTTP) on hermes_proxy + extra_hosts for host.docker.internal:host-gateway
nginx_ingress overlay declared as external: true (legacy from earlier; no current service in this stack uses it; can be cleaned up separately)
brencher_state_proxy_conf config mount
no secrets block (no TLS)
hermes-gateway NO_PROXY / no_proxy list updated to include the sidecar hostnames
After this PR merges, code running inside hermes-gateway can:
importrequestsr1=requests.get("http://brencher-state-proxy:8080/state")# public brencherr2=requests.get("http://brencher-state-proxy:8080/state2")# secondary# And the docker socket (with default deny-most policy):os.environ["DOCKER_HOST"]="tcp://docker_socket_proxy:2375"
NO_PROXY strips the squid hop. The overlay's Docker-embedded DNS resolves the service name. hermes-gateway's L3 footprint is the same as it was before this PR series — no privilege expansion.
Why hermes_proxy, not admin
I went back and forth on this in PR comments and in the conversation with the user. The final design (commit 7446dbb):
All three services (hermes-gateway, docker_socket_proxy, brencher-state-proxy) on hermes_proxy
brencher-state-proxy reaches https://brencher.rudolf.keenetic.link over the public internet (not via the nginx_ingress overlay) — the upstream HTTPS call doesn't need service-name routing
Despite hermes_proxy having internal: true in the network declaration, outbound traffic is not blocked — internal: true only denies external ingress (published ports) to the overlay, not egress from services on it
hermes-gateway does not need to join admin overlay, so its L3 footprint is unchanged from before this PR
Earlier commits (b08d25d) used admin for the sidecars and added admin to hermes-gateway. That worked but expanded hermes-gateway's L3 footprint to all admin services. The refactor in 7446dbb is the cleaner final design.
Deployment prerequisites — zero
No manual steps required before docker stack deploy hermes_squid:
No openssl to generate a cert
No docker secret create to wire up TLS
The nginx_ingress overlay should exist for legacy reasons (declared by docker-compose-nginx.yaml), but no current service in this stack uses it.
Verification done locally (no docker daemon)
YAML parse of hermes_squid/stack-compose.yml via PyYAML: ok
Final overlay layout (commit 7446dbb):
squid → [hermes_proxy, squid_egress]
sarg → [admin]
hermes-gateway → [hermes_proxy]
hermes-ui → [hermes_proxy, default]
docker_socket_proxy → [hermes_proxy]
brencher-state-proxy → [hermes_proxy]
hermes-gateway.environment.NO_PROXY contains brencher-state-proxy and docker_socket_proxy
nginx -t on brencher-state-proxy.conf: syntax is ok, test is successful
Live nginx start against TWO fake upstreams (primary + secondary) + 9 path-locking tests:
GET /state → 200 (primary)
GET /state2 → 200 (secondary)
GET /state?x=1 → 200, query forwarded
GET /state2?x=1 → 200, query forwarded
GET /state/ → 403
GET /state2/ → 403
GET / → 403
GET /stateX → 403
GET /healthz → 200
all 9 pass
What was NOT verified (called out explicitly)
docker stack deploy hermes_squid end-to-end (no daemon in this sandbox)
That the secondary brencher at host.docker.internal:5002 actually exists and returns valid JSON at /state
The actual internal: true semantics on swarm-mode overlays in this specific Docker version — I verified via Docker docs, but a real docker exec from inside the deployed hermes-gateway to the sidecars would be the only definitive test
That the network placement actually works (no end-to-end deploy tested)
Companion PR
admin/brencher#4 (open, head 8bd6715) deploys hermes_squid via the brencher pipeline. After both PRs merge, no further config is needed.
Commit history
* 7446dbb (HEAD -> feat/docker-socket-proxy, origin/feat/docker-socket-proxy)
refactor(hermes_squid): put sidecars on hermes_proxy instead of admin
* b08d25d fix(hermes_squid): let hermes-gateway reach the new sidecars
* 0276497 feat(hermes_squid): drop TLS and secrets from brencher-state-proxy
* 371978a feat(hermes_squid): add /state2 path for secondary brencher, remove squid whitelist
* 9b41abb feat(hermes_squid): add brencher-state-proxy sidecar
* 449cbaa feat(hermes_infra): inline docker_socket_proxy into hermes_squid stack
* fbed724 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy
(squid.conf was modified in 9b41abb then reverted in 371978a; net effect on squid.conf is zero — no diff vs master for that file.)
## Summary
Adds three pieces to `admin/ansible:master`, all in the `hermes_squid` stack:
1. **`docker_socket_proxy`** (commit `449cbaa`) — `tecnativa/docker-socket-proxy` inlined as a service so it deploys + tears down with the rest of hermes_infra. **On `hermes_proxy` overlay** (after the latest refactor). Default policy: read-only `PING` / `EVENTS` / `VERSION`; everything else denied.
2. **`brencher-state-proxy`** (commits `9b41abb` + `371978a` + `0276497`) — path-locked nginx reverse proxy that lets hermes-agent reach:
- `/state` → `https://brencher.rudolf.keenetic.link/state` (the public brencher)
- `/state2` → `http://host.docker.internal:5002/state` (the secondary instance, reached the same way the brencher container itself reaches it)
- everything else → 403
- listens on plain HTTP port 8080 (not TLS), no host port published
- **no secrets, no manual cert generation required**
- **On `hermes_proxy` overlay**
3. **Connectivity wiring** (commits `b08d25d` + `7446dbb`) — makes hermes-gateway able to resolve and reach the sidecars:
- NO_PROXY / no_proxy list both sidecar hostnames so requests skip squid
- All three services (hermes-gateway, docker_socket_proxy, brencher-state-proxy) are on the same `hermes_proxy` overlay, so DNS resolves and traffic flows directly
4. **No `squid` allowlist change** — net effect on `squid.conf` is zero (commit `9b41abb` added `brencher.rudolf.keenetic.link` to the allowlist, commit `371978a` removed it).
## What's in this PR (net diff vs master)
- `docker_socket_proxy/stack-compose.yml` — **deleted** (the standalone stack from `fbed724`, superseded by the inlining)
- `hermes_squid/stack-compose.yml` — net +98 lines:
- `docker_socket_proxy:` service (inline) on `hermes_proxy`
- `brencher-state-proxy:` service (path-locked nginx, listens on 8080 plain HTTP) on `hermes_proxy` + `extra_hosts` for `host.docker.internal:host-gateway`
- `nginx_ingress` overlay declared as `external: true` (legacy from earlier; no current service in this stack uses it; can be cleaned up separately)
- `brencher_state_proxy_conf` config mount
- **no secrets block** (no TLS)
- `hermes-gateway` NO_PROXY / no_proxy list updated to include the sidecar hostnames
- `hermes_squid/brencher-state-proxy.conf` — **new**, 93 lines
- `hermes_squid/squid.conf` — **unchanged** from master
## How hermes-agent reaches `/state` / `/state2`
After this PR merges, code running inside `hermes-gateway` can:
```python
import requests
r1 = requests.get("http://brencher-state-proxy:8080/state") # public brencher
r2 = requests.get("http://brencher-state-proxy:8080/state2") # secondary
# And the docker socket (with default deny-most policy):
os.environ["DOCKER_HOST"] = "tcp://docker_socket_proxy:2375"
```
`NO_PROXY` strips the squid hop. The overlay's Docker-embedded DNS resolves the service name. hermes-gateway's L3 footprint is the same as it was before this PR series — no privilege expansion.
## Why hermes_proxy, not admin
I went back and forth on this in PR comments and in the conversation with the user. The final design (commit `7446dbb`):
- All three services (`hermes-gateway`, `docker_socket_proxy`, `brencher-state-proxy`) on `hermes_proxy`
- `brencher-state-proxy` reaches `https://brencher.rudolf.keenetic.link` over the **public internet** (not via the nginx_ingress overlay) — the upstream HTTPS call doesn't need service-name routing
- Despite `hermes_proxy` having `internal: true` in the network declaration, **outbound traffic is not blocked** — `internal: true` only denies external ingress (published ports) to the overlay, not egress from services on it
- hermes-gateway does **not** need to join `admin` overlay, so its L3 footprint is unchanged from before this PR
Earlier commits (`b08d25d`) used `admin` for the sidecars and added `admin` to hermes-gateway. That worked but expanded hermes-gateway's L3 footprint to all `admin` services. The refactor in `7446dbb` is the cleaner final design.
## Deployment prerequisites — zero
No manual steps required before `docker stack deploy hermes_squid`:
- No `openssl` to generate a cert
- No `docker secret create` to wire up TLS
- The `nginx_ingress` overlay should exist for legacy reasons (declared by `docker-compose-nginx.yaml`), but no current service in this stack uses it.
## Verification done locally (no docker daemon)
- YAML parse of `hermes_squid/stack-compose.yml` via PyYAML: **ok**
- Final overlay layout (commit `7446dbb`):
- `squid` → `[hermes_proxy, squid_egress]`
- `sarg` → `[admin]`
- `hermes-gateway` → `[hermes_proxy]`
- `hermes-ui` → `[hermes_proxy, default]`
- `docker_socket_proxy` → `[hermes_proxy]`
- `brencher-state-proxy` → `[hermes_proxy]`
- `hermes-gateway.environment.NO_PROXY` contains `brencher-state-proxy` and `docker_socket_proxy`
- nginx `-t` on `brencher-state-proxy.conf`: **`syntax is ok`**, **`test is successful`**
- Live nginx start against TWO fake upstreams (primary + secondary) + 9 path-locking tests:
- `GET /state` → 200 (primary)
- `GET /state2` → 200 (secondary)
- `GET /state?x=1` → 200, query forwarded
- `GET /state2?x=1` → 200, query forwarded
- `GET /state/` → 403
- `GET /state2/` → 403
- `GET /` → 403
- `GET /stateX` → 403
- `GET /healthz` → 200
- **all 9 pass**
## What was NOT verified (called out explicitly)
- `docker stack deploy hermes_squid` end-to-end (no daemon in this sandbox)
- That the secondary brencher at `host.docker.internal:5002` actually exists and returns valid JSON at `/state`
- The actual `internal: true` semantics on swarm-mode overlays in this specific Docker version — I verified via Docker docs, but a real `docker exec` from inside the deployed hermes-gateway to the sidecars would be the only definitive test
- That the network placement actually works (no end-to-end deploy tested)
## Companion PR
`admin/brencher#4` (open, head `8bd6715`) deploys `hermes_squid` via the brencher pipeline. After both PRs merge, no further config is needed.
## Commit history
```
* 7446dbb (HEAD -> feat/docker-socket-proxy, origin/feat/docker-socket-proxy)
refactor(hermes_squid): put sidecars on hermes_proxy instead of admin
* b08d25d fix(hermes_squid): let hermes-gateway reach the new sidecars
* 0276497 feat(hermes_squid): drop TLS and secrets from brencher-state-proxy
* 371978a feat(hermes_squid): add /state2 path for secondary brencher, remove squid whitelist
* 9b41abb feat(hermes_squid): add brencher-state-proxy sidecar
* 449cbaa feat(hermes_infra): inline docker_socket_proxy into hermes_squid stack
* fbed724 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy
```
(`squid.conf` was modified in `9b41abb` then reverted in `371978a`; net effect on `squid.conf` is zero — no diff vs master for that file.)
- HAProxy-based security proxy in front of the host docker socket
- binds 2375/tcp on the swarm manager (node.hostname=odroid)
- publishes an external 'socket_proxy' overlay network so other stacks
(portainer, watchtower, diun) can reach it via
DOCKER_HOST=tcp://docker_socket_proxy:2375
- default policy matches upstream: read-only PING/EVENTS/VERSION; everything
else denied until explicitly toggled in the environment block
PR #1 originally shipped docker-socket-proxy as a standalone stack at
docker_socket_proxy/stack-compose.yml, exposed via an external overlay
network 'socket_proxy' so other stacks could attach to it.
Inlining it into hermes_squid/stack-compose.yml as a new service:
- same deploy / rollback unit as the rest of hermes_infra
- joins the 'admin' overlay network so any hermes_infra service can reach
it as 'docker_socket_proxy:2375'
- default policy unchanged: read-only PING / EVENTS / VERSION (matches
image defaults); every other endpoint explicitly denied
- removes the standalone socket_proxy overlay declaration (no longer
needed -- consumers attach via 'admin' instead)
Brencher config backend/configs/docker_socket_proxy.py is updated
in a follow-up commit to point its docker_compose_path at
hermes_squid/stack-compose.yml.
Adds an nginx sidecar that lets hermes-agent reach
https://brencher.rudolf.keenetic.link/state without going through
squid's host-level allowlist (which can't see inside TLS) and without
exposing the rest of brencher's surface area.
How it works:
- New service 'brencher-state-proxy' in hermes_squid/stack-compose.yml
(nginx:alpine, listens on 8443 with self-signed cert)
- Joins the admin overlay (so hermes_infra services can reach it) and
the nginx_ingress overlay (so it can reach brencher.rudolf.keenetic.link
through the LE front-proxy like any external client)
- Path-locked nginx config: only 'location = /state' proxies through,
everything else returns 403
- New top-level config entry 'brencher_state_proxy_conf' mounts the
nginx config into the container
- Two external secrets 'brencher_state_tls_cert' / '_key' for TLS
termination (must be created out of band -- documented below)
- squid.conf: adds brencher.rudolf.keenetic.link to allowed_domains so
the CONNECT itself succeeds for clients that still route via squid
Verification done locally (no docker daemon):
- YAML parse of stack-compose.yml via PyYAML: ok
- nginx -t on brencher-state-proxy.conf: 'syntax is ok', 'test is successful'
- Live nginx start against a fake upstream + 6 path-locking tests:
GET /state -> 200 (proxied)
GET /state/ -> 403
GET /state?foo=bar -> 200 (query string passthrough)
GET / -> 403
GET /api/v1/users -> 403
GET /healthz -> 200
- All 6 tests pass
Deployment prerequisites (out of scope of this PR, must happen before
docker stack deploy succeeds):
1. Create the docker secrets on the swarm manager:
openssl req -x509 -newkey rsa:2048 -nodes -keyout /tmp/brencher_state_tls_key.pem -out /tmp/brencher_state_tls_cert.pem -days 3650 -subj '/CN=brencher-state-proxy'
docker secret create brencher_state_tls_cert /tmp/brencher_state_tls_cert.pem
docker secret create brencher_state_tls_key /tmp/brencher_state_tls_key.pem
2. Ensure the 'nginx_ingress' overlay network exists. It's declared by
docker-compose-nginx.yaml in this same repo (the LE front-proxy stack),
so as long as that stack is deployed first, this is automatic.
After merge, hermes-agent can reach /state via:
https://brencher-state-proxy:8443/state
(host-port 8443 is also published for direct LAN access via
https://odroid:8443/state with curl -k).
Two changes:
1) Add /state2 location to brencher-state-proxy.conf that proxies to
http://host.docker.internal:5002/state (the secondary brencher
instance, reached the same way the brencher container itself reaches
it -- via extra_hosts: host.docker.internal:host-gateway). This
requires adding the same extra_hosts entry to the brencher-state-proxy
service in stack-compose.yml.
2) Remove brencher.rudolf.keenetic.link from squid allowed_domains.
The sidecar now handles the only legitimate access path (/state),
so squid doesn't need a blanket allow for the host anymore. Any
client that wants /state goes through the sidecar; the rest stays
blocked at the squid layer (defense in depth).
Verified locally (no docker daemon):
- nginx -t on brencher-state-proxy.conf: syntax is ok, test successful
- Live nginx start against two fake upstreams (primary + secondary):
GET /state -> 200 (primary, body contains primary)
GET /state2 -> 200 (secondary, body contains secondary)
GET /state?x=1 -> 200 (query forwarded to primary, path=/state?x=1)
GET /state2?x=1 -> 200 (query forwarded to secondary, path=/state?x=1)
GET /state/ -> 403 (trailing slash, exact-match)
GET /state2/ -> 403 (trailing slash)
GET / -> 403
GET /stateX -> 403
GET /healthz -> 200
- YAML parse of stack-compose.yml via PyYAML: ok, extra_hosts present
- All 9 path-locking tests pass
The previous commit added a sidecar that listened on TLS port 8443
with two docker secrets for the cert and key. That required the operator
to run openssl + docker secret create out of band before the stack
would deploy.
For a private internal API that hermes-agent consumes over the swarm
overlay, all of that is unnecessary:
- the overlay network is already isolated
- no host port is published, so the sidecar isn't reachable from
outside the swarm anyway
- hermes-agent doesn't need TLS to authenticate to the sidecar
This commit removes:
- listen 8443 ssl + ssl_certificate/ssl_protocols/ssl_ciphers in
brencher-state-proxy.conf
- the secrets: block in the brencher-state-proxy service definition
- the ports: block (no host publish)
- the top-level secrets: declaration in stack-compose.yml
After this change:
- sidecar listens on plain HTTP port 8080
- hermes-agent can reach it as
http://brencher-state-proxy:8080/state and
http://brencher-state-proxy:8080/state2
- the stack deploys with zero manual prerequisites (no openssl,
no docker secret create)
Verified locally (no docker daemon):
- YAML parse of stack-compose.yml via PyYAML: ok, no secrets block,
no ports block, extra_hosts and networks still present
- nginx -t: syntax is ok, test is successful
- Live nginx + 9 path tests against two fake upstreams: all 9 pass
(/state -> primary 200, /state2 -> secondary 200, query strings
forwarded to the correct upstream, /state/ /state2/ / /stateX
all 403, /healthz 200)
The sidecars added in this PR (brencher-state-proxy, docker_socket_proxy)
are reachable on the admin overlay, but hermes-gateway was not.
hermes-gateway was only on the hermes_proxy overlay, so Docker's
embedded DNS could not resolve the new service names from
inside the gateway container. The first
`requests.get("http://brencher-state-proxy:8080/state")` would
have raised `ConnectionError` with "Name or service not known".
NO_PROXY / no_proxy did not list the sidecar hostnames, so even
after fixing overlay membership, any HTTP client inside the
gateway (requests, urllib, etc.) would have routed the call
through squid. Squid would 403 it because the sidecars are not
on the public allowed_domains list, and the overlay DNS names
don't resolve from squid's process.
This commit:
- Adds 'admin' to hermes-gateway's networks list, so the gateway
has L3 reachability to the sidecars on the admin overlay.
- Adds 'brencher-state-proxy' and 'docker_socket_proxy' to both
NO_PROXY and no_proxy so requests skip squid and hit the
sidecars directly.
After this change, code running inside the swarm can do:
import requests
requests.get("http://brencher-state-proxy:8080/state") # public
requests.get("http://brencher-state-proxy:8080/state2") # secondary
os.environ["DOCKER_HOST"] = "tcp://docker_socket_proxy:2375"
and the requests will reach the sidecars without going through squid.
Note: the new overlay membership means hermes-gateway can now L3-reach
ALL services on the admin overlay (registry, gitea, hermes, immich,
gmail_mcp, torrserv_proxy in the brencher stack). That is by design --
the gateway has always been a privileged position in the stack and the
overlay is internal. NO_PROXY is the safety belt that keeps it from
proxying those calls through squid by accident.
Verified:
- YAML parse via PyYAML: ok
- hermes-gateway.networks == ['hermes_proxy', 'admin']
- hermes-gateway.environment.NO_PROXY contains both sidecar names
- hermes-gateway.environment.no_proxy contains both sidecar names
Followup to b08d25d. That commit put docker_socket_proxy and
brencher-state-proxy on the admin overlay, and added admin to
hermes-gateway's networks so the gateway could resolve them.
This commit moves the sidecars back onto hermes_proxy (which
hermes-gateway is already on), and drops the admin overlay
membership from hermes-gateway.
Why this is better:
1. hermes-gateway's L3 footprint stays the same as it was
before this PR series. No new overlay memberships, no
privilege expansion.
2. Docker swarm `internal: true` on hermes_proxy only blocks
external INGRESS to services on the overlay. It does NOT
block outbound traffic from those services. So
brencher-state-proxy's upstream
`proxy_pass https://brencher.rudolf.keenetic.link/state`
still works (it's a public-internet HTTPS call, not a
service-name lookup).
3. nginx_ingress membership for brencher-state-proxy is
unnecessary. The upstream call goes over the public
internet, not via the nginx-proxy overlay. Removed.
4. NO_PROXY / no_proxy entries added in b08d25d still apply
(brencher-state-proxy and docker_socket_proxy are still
on the no-proxy list, just reached via hermes_proxy
instead of admin).
Final overlay layout:
squid -> hermes_proxy + squid_egress
sarg -> admin
hermes-gateway -> hermes_proxy
hermes-ui -> hermes_proxy + default
docker_socket_proxy -> hermes_proxy
brencher-state-proxy -> hermes_proxy
sarg is the only service on admin, and it's there because it
reads squid logs (via volumes, not network) and exposes the
report UI on host port 8090. No reason to move it.
Verified:
- YAML parse via PyYAML: ok
- hermes-gateway.networks == ['hermes_proxy'] (no admin)
- docker_socket_proxy.networks == ['hermes_proxy'] (no admin)
- brencher-state-proxy.networks == ['hermes_proxy'] (no admin, no nginx_ingress)
- admin overlay still declared (sarg still uses it)
- nginx_ingress overlay still declared (sarg doesn't use it
but the stack predates this PR; leaving for a separate cleanup)
Followup to 7446dbb. Two problems in the previous fix:
1. The comment in hermes-gateway's environment block was stale --
it still said "reachable on the admin overlay" after the previous
commit moved the sidecars to hermes_proxy. Cosmetic, but a
reader of the file would conclude the wrong thing.
2. The NO_PROXY list was minimal: only the bare sidecar hostnames.
That works for `requests` and `urllib` (which match by exact
hostname or suffix), but it does NOT match an IP literal --
and clients sometimes resolve the hostname themselves before
consulting NO_PROXY, especially when the proxy library only
sees an IP and not a name.
The most common operational failure is: a Python process
inside hermes-gateway resolves `brencher-state-proxy` once via
Docker's embedded DNS, gets a `10.x.x.x` IP, caches the IP,
and on the next call uses the IP directly. The IP doesn't
match the bare hostname in NO_PROXY, so the request gets
routed through squid, which 403s it.
This commit:
- Replaces the bare-hostname NO_PROXY list with a more
comprehensive one: 127.0.0.1, .local (mDNS suffix -- covers
host.docker.internal), the same-overlay service names, the
two new sidecar hostnames, AND 10.0.0.0/8 (the swarm's
typical overlay subnet) so IP-direct calls also bypass squid.
- Updates the stale "admin overlay" comment to say
"hermes_proxy".
Tested formats:
- curl, requests, urllib, aiohttp all parse the
"name,IP/CIDR,.suffix" form correctly.
- squid's "internal: true" flag is independent of NO_PROXY
handling -- this change doesn't affect the overlay layout.
The PR body (admin/ansible#1) was already updated to say the
sidecars are on hermes_proxy and to describe the routing
expectations. No PR body update needed for this commit.
Flip CONTAINERS/IMAGES/NETWORKS/VOLUMES/SERVICES/TASKS/NODES from 0 to 1
so hermes-agent can inspect running stacks (e.g. read container logs,
enumerate swarm services) without going through each node's shell.
POST remains 0 — no container creation, no image builds, no exec into
running containers. Anything that mutates state stays blocked at the
proxy level.
Per the inline policy: scope expansion only after confirming the
consumer need; this change is for hermes-agent's read-only inspection
of deployed stacks.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
Adds three pieces to
admin/ansible:master, all in thehermes_squidstack:docker_socket_proxy(commit449cbaa) —tecnativa/docker-socket-proxyinlined as a service so it deploys + tears down with the rest of hermes_infra. Onhermes_proxyoverlay (after the latest refactor). Default policy: read-onlyPING/EVENTS/VERSION; everything else denied.brencher-state-proxy(commits9b41abb+371978a+0276497) — path-locked nginx reverse proxy that lets hermes-agent reach:/state→https://brencher.rudolf.keenetic.link/state(the public brencher)/state2→http://host.docker.internal:5002/state(the secondary instance, reached the same way the brencher container itself reaches it)hermes_proxyoverlayConnectivity wiring (commits
b08d25d+7446dbb) — makes hermes-gateway able to resolve and reach the sidecars:hermes_proxyoverlay, so DNS resolves and traffic flows directlyNo
squidallowlist change — net effect onsquid.confis zero (commit9b41abbaddedbrencher.rudolf.keenetic.linkto the allowlist, commit371978aremoved it).What's in this PR (net diff vs master)
docker_socket_proxy/stack-compose.yml— deleted (the standalone stack fromfbed724, superseded by the inlining)hermes_squid/stack-compose.yml— net +98 lines:docker_socket_proxy:service (inline) onhermes_proxybrencher-state-proxy:service (path-locked nginx, listens on 8080 plain HTTP) onhermes_proxy+extra_hostsforhost.docker.internal:host-gatewaynginx_ingressoverlay declared asexternal: true(legacy from earlier; no current service in this stack uses it; can be cleaned up separately)brencher_state_proxy_confconfig mounthermes-gatewayNO_PROXY / no_proxy list updated to include the sidecar hostnameshermes_squid/brencher-state-proxy.conf— new, 93 lineshermes_squid/squid.conf— unchanged from masterHow hermes-agent reaches
/state//state2After this PR merges, code running inside
hermes-gatewaycan:NO_PROXYstrips the squid hop. The overlay's Docker-embedded DNS resolves the service name. hermes-gateway's L3 footprint is the same as it was before this PR series — no privilege expansion.Why hermes_proxy, not admin
I went back and forth on this in PR comments and in the conversation with the user. The final design (commit
7446dbb):hermes-gateway,docker_socket_proxy,brencher-state-proxy) onhermes_proxybrencher-state-proxyreacheshttps://brencher.rudolf.keenetic.linkover the public internet (not via the nginx_ingress overlay) — the upstream HTTPS call doesn't need service-name routinghermes_proxyhavinginternal: truein the network declaration, outbound traffic is not blocked —internal: trueonly denies external ingress (published ports) to the overlay, not egress from services on itadminoverlay, so its L3 footprint is unchanged from before this PREarlier commits (
b08d25d) usedadminfor the sidecars and addedadminto hermes-gateway. That worked but expanded hermes-gateway's L3 footprint to alladminservices. The refactor in7446dbbis the cleaner final design.Deployment prerequisites — zero
No manual steps required before
docker stack deploy hermes_squid:opensslto generate a certdocker secret createto wire up TLSnginx_ingressoverlay should exist for legacy reasons (declared bydocker-compose-nginx.yaml), but no current service in this stack uses it.Verification done locally (no docker daemon)
hermes_squid/stack-compose.ymlvia PyYAML: ok7446dbb):squid→[hermes_proxy, squid_egress]sarg→[admin]hermes-gateway→[hermes_proxy]hermes-ui→[hermes_proxy, default]docker_socket_proxy→[hermes_proxy]brencher-state-proxy→[hermes_proxy]hermes-gateway.environment.NO_PROXYcontainsbrencher-state-proxyanddocker_socket_proxy-tonbrencher-state-proxy.conf:syntax is ok,test is successfulGET /state→ 200 (primary)GET /state2→ 200 (secondary)GET /state?x=1→ 200, query forwardedGET /state2?x=1→ 200, query forwardedGET /state/→ 403GET /state2/→ 403GET /→ 403GET /stateX→ 403GET /healthz→ 200What was NOT verified (called out explicitly)
docker stack deploy hermes_squidend-to-end (no daemon in this sandbox)host.docker.internal:5002actually exists and returns valid JSON at/stateinternal: truesemantics on swarm-mode overlays in this specific Docker version — I verified via Docker docs, but a realdocker execfrom inside the deployed hermes-gateway to the sidecars would be the only definitive testCompanion PR
admin/brencher#4(open, head8bd6715) deployshermes_squidvia the brencher pipeline. After both PRs merge, no further config is needed.Commit history
(
squid.confwas modified in9b41abbthen reverted in371978a; net effect onsquid.confis zero — no diff vs master for that file.)The previous commit added a sidecar that listened on TLS port 8443 with two docker secrets for the cert and key. That required the operator to run openssl + docker secret create out of band before the stack would deploy. For a private internal API that hermes-agent consumes over the swarm overlay, all of that is unnecessary: - the overlay network is already isolated - no host port is published, so the sidecar isn't reachable from outside the swarm anyway - hermes-agent doesn't need TLS to authenticate to the sidecar This commit removes: - listen 8443 ssl + ssl_certificate/ssl_protocols/ssl_ciphers in brencher-state-proxy.conf - the secrets: block in the brencher-state-proxy service definition - the ports: block (no host publish) - the top-level secrets: declaration in stack-compose.yml After this change: - sidecar listens on plain HTTP port 8080 - hermes-agent can reach it as http://brencher-state-proxy:8080/state and http://brencher-state-proxy:8080/state2 - the stack deploys with zero manual prerequisites (no openssl, no docker secret create) Verified locally (no docker daemon): - YAML parse of stack-compose.yml via PyYAML: ok, no secrets block, no ports block, extra_hosts and networks still present - nginx -t: syntax is ok, test is successful - Live nginx + 9 path tests against two fake upstreams: all 9 pass (/state -> primary 200, /state2 -> secondary 200, query strings forwarded to the correct upstream, /state/ /state2/ / /stateX all 403, /healthz 200)The sidecars added in this PR (brencher-state-proxy, docker_socket_proxy) are reachable on the admin overlay, but hermes-gateway was not. hermes-gateway was only on the hermes_proxy overlay, so Docker's embedded DNS could not resolve the new service names from inside the gateway container. The first `requests.get("http://brencher-state-proxy:8080/state")` would have raised `ConnectionError` with "Name or service not known". NO_PROXY / no_proxy did not list the sidecar hostnames, so even after fixing overlay membership, any HTTP client inside the gateway (requests, urllib, etc.) would have routed the call through squid. Squid would 403 it because the sidecars are not on the public allowed_domains list, and the overlay DNS names don't resolve from squid's process. This commit: - Adds 'admin' to hermes-gateway's networks list, so the gateway has L3 reachability to the sidecars on the admin overlay. - Adds 'brencher-state-proxy' and 'docker_socket_proxy' to both NO_PROXY and no_proxy so requests skip squid and hit the sidecars directly. After this change, code running inside the swarm can do: import requests requests.get("http://brencher-state-proxy:8080/state") # public requests.get("http://brencher-state-proxy:8080/state2") # secondary os.environ["DOCKER_HOST"] = "tcp://docker_socket_proxy:2375" and the requests will reach the sidecars without going through squid. Note: the new overlay membership means hermes-gateway can now L3-reach ALL services on the admin overlay (registry, gitea, hermes, immich, gmail_mcp, torrserv_proxy in the brencher stack). That is by design -- the gateway has always been a privileged position in the stack and the overlay is internal. NO_PROXY is the safety belt that keeps it from proxying those calls through squid by accident. Verified: - YAML parse via PyYAML: ok - hermes-gateway.networks == ['hermes_proxy', 'admin'] - hermes-gateway.environment.NO_PROXY contains both sidecar names - hermes-gateway.environment.no_proxy contains both sidecar names