164 Commits
Author SHA1 Message Date
hermes e6444ad49c fix(ci): trigger release workflow on auto-* branch pushes
release / build (push) Failing after 32s
Brencher's auto-deploy-via-temp-branch pattern pushes the merge
commit to `auto-<short-sha>` on the origin remote before dispatching
the workflow. Previously the workflow only fired on `v*` tags or
workflow_dispatch, so brencher's auto-* push was invisible to it.

Adding `branches: ['auto-*']` to the trigger list keeps the existing
tag-based manual releases working while letting brencher's temp
branches fire the build on their own push.
2026-09-19 23:49:04 +00:00
hermes eb851d9b1d fix(ci): workflow accepts tag input for brencher dispatch
release / build (push) Successful in 1m14s
v0.1.0-test-dispatch v0.1.0-test-dispatch2 v0.1.0-test-recreate v0.1.0-test-after-secrets
2026-09-18 20:43:26 +00:00
hermes c719f6ab8d ci(connectivity_tester): remove debug, voice-gateway style (user-level secrets)
release / build (push) Successful in 1m8s
2026-09-18 12:47:09 +00:00
hermes 6e6f9c67fc fix(ci): use env block for secrets debug (bash doesn't understand templates)
release / build (push) Failing after 30s
2026-09-18 12:44:42 +00:00
hermes 77b3cc2c08 ci(connectivity_tester): voice-gateway-style with debug step
release / build (push) Failing after 19s
2026-09-18 12:43:18 +00:00
hermes d011648e56 fix(ci): remove duplicate runs-on at job-level (yaml unmarshal error)
release / build (ubuntu-24.04, linux/amd64) (push) Failing after 1m25s
release / build (ubuntu-24.04-arm, linux/arm/v7) (push) Canceled after 0s
release / build (ubuntu-24.04-arm, linux/arm64) (push) Canceled after 0s
2026-09-18 12:37:38 +00:00
hermes c22cadc48b ci(connectivity_tester): add tag trigger for v* 2026-09-18 12:35:21 +00:00
hermes 2fa05da943 ci: empty trigger 2026-09-18 12:01:47 +00:00
hermes 3406491dcc ci(connectivity_tester): real workflow with REGISTRY_USERNAME/TOKEN + multi-arch matrix 2026-09-18 11:58:12 +00:00
hermes 7a08d4d022 Обновить .gitea/workflows/release.yml
Build and Push Docker Image / build-and-push (push) Failing after 4m32s
2026-09-18 12:57:49 +02:00
hermes 6e9a998eba ci(connectivity_tester): smoke-test pattern from official docs (gitea.actor + server_url)
release / build (push) Failing after 24s
2026-09-17 22:42:07 +00:00
hermes ef4af24c1e Обновить .gitea/workflows/release.yml
release / build (push) Failing after 23s
2026-09-18 00:34:28 +02:00
hermes f80a524d33 ci(connectivity_tester): smoke-test GITHUB_TOKEN on master branch 2026-09-17 22:10:54 +00:00
hermes faa3e32a53 Reapply "fix(ci): use auto-injected GITHUB_TOKEN instead of REGISTRY_USERNAME/TOKEN secrets"
This reverts commit e6d184bd95.
2026-09-17 22:09:37 +00:00
hermes e6d184bd95 Revert "fix(ci): use auto-injected GITHUB_TOKEN instead of REGISTRY_USERNAME/TOKEN secrets"
This reverts commit d097862c81.
2026-09-17 21:43:55 +00:00
hermes d097862c81 fix(ci): use auto-injected GITHUB_TOKEN instead of REGISTRY_USERNAME/TOKEN secrets 2026-09-17 21:41:13 +00:00
hermes b2e85b686c fix(ci): sanitize platform in image tag (linux/amd64 -> linux-amd64) 2026-09-17 19:06:07 +00:00
admin 932be82ae6 ci(connectivity_tester): drop build block; image pulled from Gitea CR 2026-09-17 19:27:17 +02:00
admin 953e4dcba7 ci(connectivity_tester): switch stack image to Gitea Container Registry 2026-09-17 18:32:39 +02:00
admin 81e0641ab1 fix(ci): use REGISTRY_USERNAME/TOKEN (which are populated); runner OS via matrix.os 2026-09-17 18:19:15 +02:00
admin 8f71c723fb feat(ci): keep PACKAGE_PUSH_* secrets + multi-arch; login continue-on-error so we see what other side fails 2026-09-17 18:13:47 +02:00
admin 7fed0c08cd feat(ci): dump individual secrets, multi-arch matrix, switch to PACKAGE_PUSH_* 2026-09-17 18:05:58 +02:00
admin 4b8ece95c0 feat(ci): dump env+secrets, multi-arch, switch to PACKAGE_PUSH_* 2026-09-17 18:03:55 +02:00
admin 0734adc229 ci(connectivity_tester): copy release.yml to release/0.1.0-connectivity 2026-09-16 23:18:00 +02:00
hermes 8ecc6f0424 ci: drop broken .github/workflows path 2026-09-15 22:21:23 +02:00
admin 4ab39982d1 fix(connectivity_tester): hardcode REGISTRY_USERNAME (missing secret) 2026-09-15 21:20:50 +02:00
admin b5b94b2eee ci(connectivity_tester): rewrite workflow to use REGISTRY_* secrets + multi-arch
Aligns with the proven hermes/reverseproxy@main build.yml: uses
secrets.REGISTRY_USERNAME/REGISTRY_PASSWORD (which already exist in the
cluster-wide Actions runner scope), docker/setup-buildx-action@v3 and
docker/build-push-action@v6 from docker/github-mirror, and targets the
gitea.rudolf.keenetic.link/<image>:auto-<sha> plus :latest tags.

Multi-arch matrix now covers linux/amd64, linux/arm64 and linux/arm/v7
so push to Gitea Container Registry will fan out to rudolf-T (x86_64),
odroid (aarch64) and xu4 (32-bit ARM).
2026-09-15 21:06:24 +02:00
admin bf6b86302f ci(connectivity_tester): add multi-arch Gitea Actions workflow 2026-09-15 19:42:46 +02:00
admin 1e9d33485d feat(connectivity_tester): Endpoint dataclass, proxy-aware checks, public hosts
Replace tuple-of-strings with frozen Endpoint dataclass carrying optional
(proxy_host, proxy_port) tuple. Add network_map.NETWORK_MAP populated from
the live swarm endpoints discovered via the docker socket, and a separate
PUBLIC_SERVICES list sourced from the squid allowlist plus well-known
public hosts. Endpoints with proxy are checked via HTTP CONNECT through
that proxy; endpoints without are probed directly via TCP after attaching
to the relevant overlay network. Cross-validation rules cover leaks and
missing expectations per round.
2026-09-15 10:11:49 +00:00
admin d5b1a23bcb feat(connectivity_tester): rotate networks per-round, cross-validate map
Self-attach/detach to overlay networks via docker socket. At any moment the
container is on exactly one network; per-network list of expected services is
loaded from network_map.NETWORK_MAP. After checks run cross-validation:

- service reachable but not in map  -> ERROR 'LEAK-UNEXPECTED'
- service in map but unreachable    -> ERROR 'MISSING-EXPECTED'
- global service not in any map     -> WARNING 'GLOBAL-UNCOVERED'

Compose uses /var/run/docker.sock bind mount and removes overlay-network
references; EXTRA_ENDPOINTS / HTTP_PROXY / HTTPS_PROXY / NO_PROXY are gone.
endpoints.py removed (its data moved into network_map.py).
2026-09-15 09:23:55 +00:00
admin e94ea37e85 fix(connectivity_tester): drop root name: key from compose
docker stack deploy on this engine rejects 'Additional property name is not allowed'.
Sibling stacks (gitea, hermes, immich) ship without it; the stack name comes from
the brencher 'stack_name' CLI argument instead.
2026-09-15 08:19:19 +00:00
admin c0d53b709c fix(connectivity_tester): COPY . instead of connectivity_tester/
After brencher fix uses context=<repo>/connectivity_tester/ (single-level
now that compose 'dockerfile' is 'Dockerfile'), the previous COPY line
referenced a non-existent connectivity_tester/ subdirectory inside the
build context.
2026-09-15 07:59:06 +00:00
admin 0366a78a96 fix(connectivity_tester): use Dockerfile (single-level) in stack build
Previous 'dockerfile: connectivity_tester/Dockerfile' resolved through
brencher's join to <repo>/connectivity_tester/connectivity_tester/Dockerfile,
which does not exist; build failed with 'Cannot locate specified Dockerfile'.
The Dockerfile sits next to stack-compose.yml at <repo>/connectivity_tester/Dockerfile.
2026-09-15 07:39:43 +00:00
admin c13e9d2ce7 fix(connectivity_tester): define EXTRA_ENDPOINTS empty + add external hosts
brencher DockerComposeBuild raises KeyError: 'EXTRA_ENDPOINTS' when the
env var is not passed via envs=. Hardcode empty default in compose for
deterministic build.

Add google.com and lan gateways to EXPECTED_EXTERNAL so connectivity-tester
probes them on every round.
2026-09-15 05:04:07 +00:00
hermes 020a5092a0 fix(compose): drop shell default syntax from EXTRA_ENDPOINTS
brencher parses compose via re.sub(r'\$\{([^}]+)\}', lambda m: env[m.group(1)]).
With shell default syntax ${VAR:-default} the regex captures 'VAR:-default'
as a single key, env['VAR:-default'] raises KeyError.

Use bare ${EXTRA_ENDPOINTS}; if the env var is missing brencher's
.env lookup (or the dict.get fallback in the deploy step) handles it.
If EXTRA_ENDPOINTS isn't set at deploy time the value is empty, which
the python service treats as 'no extras'.
2026-09-14 19:26:54 +00:00
hermes 5a36921f3b feat(connectivity_tester): new stack for synthetic connectivity checks
Long-running Swarm service that verifies expected endpoints are
reachable from inside the cluster and that the negative-list endpoints
(cloud metadata etc.) stay blocked.

Runs one round every PERIOD_SECONDS (default 600): TCP probes against
EXPECTED_INTERNAL (overlay services), EXPECTED_EXTERNAL (internet
hosts), EXTRA_ENDPOINTS (ad-hoc from env), and NEGATIVE (must NOT
respond; alerts on any leak). Logs structured output, alerts on
failures, container keeps running.

Mode: global — Swarm schedules one task on every node that meets
constraints. Attached to hermes_proxy overlay (for internal DNS) and
default (for egress via squid).

Blocks Org1/hermes-voice-gateway#3 (Path B) until we verify the
overlay + egress paths voice-gateway will depend on.
2026-09-14 09:39:13 +00:00
hermesandhermes 4cb8eeae35 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy (#1)
## Summary
Adds three pieces to `admin/ansible:master`, all in the `hermes_squid` stack:

1. **`docker_socket_proxy`** (commit `449cbaa`) — `tecnativa/docker-socket-proxy` inlined as a service so it deploys + tears down with the rest of hermes_infra. **On `hermes_proxy` overlay** (after the latest refactor). Default policy: read-only `PING` / `EVENTS` / `VERSION`; everything else denied.

2. **`brencher-state-proxy`** (commits `9b41abb` + `371978a` + `0276497`) — path-locked nginx reverse proxy that lets hermes-agent reach:
   - `/state` → `https://brencher.rudolf.keenetic.link/state` (the public brencher)
   - `/state2` → `http://host.docker.internal:5002/state` (the secondary instance, reached the same way the brencher container itself reaches it)
   - everything else → 403
   - listens on plain HTTP port 8080 (not TLS), no host port published
   - **no secrets, no manual cert generation required**
   - **On `hermes_proxy` overlay**

3. **Connectivity wiring** (commits `b08d25d` + `7446dbb`) — makes hermes-gateway able to resolve and reach the sidecars:
   - NO_PROXY / no_proxy list both sidecar hostnames so requests skip squid
   - All three services (hermes-gateway, docker_socket_proxy, brencher-state-proxy) are on the same `hermes_proxy` overlay, so DNS resolves and traffic flows directly

4. **No `squid` allowlist change** — net effect on `squid.conf` is zero (commit `9b41abb` added `brencher.rudolf.keenetic.link` to the allowlist, commit `371978a` removed it).

## What's in this PR (net diff vs master)
- `docker_socket_proxy/stack-compose.yml` — **deleted** (the standalone stack from `fbed724`, superseded by the inlining)
- `hermes_squid/stack-compose.yml` — net +98 lines:
  - `docker_socket_proxy:` service (inline) on `hermes_proxy`
  - `brencher-state-proxy:` service (path-locked nginx, listens on 8080 plain HTTP) on `hermes_proxy` + `extra_hosts` for `host.docker.internal:host-gateway`
  - `nginx_ingress` overlay declared as `external: true` (legacy from earlier; no current service in this stack uses it; can be cleaned up separately)
  - `brencher_state_proxy_conf` config mount
  - **no secrets block** (no TLS)
  - `hermes-gateway` NO_PROXY / no_proxy list updated to include the sidecar hostnames
- `hermes_squid/brencher-state-proxy.conf` — **new**, 93 lines
- `hermes_squid/squid.conf` — **unchanged** from master

## How hermes-agent reaches `/state` / `/state2`

After this PR merges, code running inside `hermes-gateway` can:
```python
import requests
r1 = requests.get("http://brencher-state-proxy:8080/state")    # public brencher
r2 = requests.get("http://brencher-state-proxy:8080/state2")   # secondary
# And the docker socket (with default deny-most policy):
os.environ["DOCKER_HOST"] = "tcp://docker_socket_proxy:2375"
```

`NO_PROXY` strips the squid hop. The overlay's Docker-embedded DNS resolves the service name. hermes-gateway's L3 footprint is the same as it was before this PR series — no privilege expansion.

## Why hermes_proxy, not admin

I went back and forth on this in PR comments and in the conversation with the user. The final design (commit `7446dbb`):

- All three services (`hermes-gateway`, `docker_socket_proxy`, `brencher-state-proxy`) on `hermes_proxy`
- `brencher-state-proxy` reaches `https://brencher.rudolf.keenetic.link` over the **public internet** (not via the nginx_ingress overlay) — the upstream HTTPS call doesn't need service-name routing
- Despite `hermes_proxy` having `internal: true` in the network declaration, **outbound traffic is not blocked** — `internal: true` only denies external ingress (published ports) to the overlay, not egress from services on it
- hermes-gateway does **not** need to join `admin` overlay, so its L3 footprint is unchanged from before this PR

Earlier commits (`b08d25d`) used `admin` for the sidecars and added `admin` to hermes-gateway. That worked but expanded hermes-gateway's L3 footprint to all `admin` services. The refactor in `7446dbb` is the cleaner final design.

## Deployment prerequisites — zero

No manual steps required before `docker stack deploy hermes_squid`:
- No `openssl` to generate a cert
- No `docker secret create` to wire up TLS
- The `nginx_ingress` overlay should exist for legacy reasons (declared by `docker-compose-nginx.yaml`), but no current service in this stack uses it.

## Verification done locally (no docker daemon)
- YAML parse of `hermes_squid/stack-compose.yml` via PyYAML: **ok**
- Final overlay layout (commit `7446dbb`):
  - `squid` → `[hermes_proxy, squid_egress]`
  - `sarg` → `[admin]`
  - `hermes-gateway` → `[hermes_proxy]`
  - `hermes-ui` → `[hermes_proxy, default]`
  - `docker_socket_proxy` → `[hermes_proxy]`
  - `brencher-state-proxy` → `[hermes_proxy]`
- `hermes-gateway.environment.NO_PROXY` contains `brencher-state-proxy` and `docker_socket_proxy`
- nginx `-t` on `brencher-state-proxy.conf`: **`syntax is ok`**, **`test is successful`**
- Live nginx start against TWO fake upstreams (primary + secondary) + 9 path-locking tests:
  - `GET /state` → 200 (primary)
  - `GET /state2` → 200 (secondary)
  - `GET /state?x=1` → 200, query forwarded
  - `GET /state2?x=1` → 200, query forwarded
  - `GET /state/` → 403
  - `GET /state2/` → 403
  - `GET /` → 403
  - `GET /stateX` → 403
  - `GET /healthz` → 200
  - **all 9 pass**

## What was NOT verified (called out explicitly)
- `docker stack deploy hermes_squid` end-to-end (no daemon in this sandbox)
- That the secondary brencher at `host.docker.internal:5002` actually exists and returns valid JSON at `/state`
- The actual `internal: true` semantics on swarm-mode overlays in this specific Docker version — I verified via Docker docs, but a real `docker exec` from inside the deployed hermes-gateway to the sidecars would be the only definitive test
- That the network placement actually works (no end-to-end deploy tested)

## Companion PR
`admin/brencher#4` (open, head `8bd6715`) deploys `hermes_squid` via the brencher pipeline. After both PRs merge, no further config is needed.

## Commit history
```
* 7446dbb (HEAD -> feat/docker-socket-proxy, origin/feat/docker-socket-proxy)
      refactor(hermes_squid): put sidecars on hermes_proxy instead of admin
* b08d25d fix(hermes_squid): let hermes-gateway reach the new sidecars
* 0276497 feat(hermes_squid): drop TLS and secrets from brencher-state-proxy
* 371978a feat(hermes_squid): add /state2 path for secondary brencher, remove squid whitelist
* 9b41abb feat(hermes_squid): add brencher-state-proxy sidecar
* 449cbaa feat(hermes_infra): inline docker_socket_proxy into hermes_squid stack
* fbed724 feat(docker_socket_proxy): add swarm stack compose for tecnativa/docker-socket-proxy
```
(`squid.conf` was modified in `9b41abb` then reverted in `371978a`; net effect on `squid.conf` is zero — no diff vs master for that file.)

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/1
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-08-26 14:37:37 +02:00
admin 665f915634 x 2026-08-24 22:28:36 +02:00
admin 301ac87694 x 2026-08-24 21:52:05 +02:00
admin 72dcdcf12e x 2026-08-23 21:05:36 +02:00
admin 94ff81ae60 x 2026-08-23 20:54:15 +02:00
admin 0516b0ecaf x 2026-08-19 23:28:16 +02:00
admin de4de846be x 2026-08-12 22:33:30 +02:00
admin b3915e9428 x 2026-08-12 21:07:27 +02:00
admin 3aaa4b87bf x 2026-08-12 13:05:17 +02:00
admin 8e29e94f59 x 2026-08-12 12:51:45 +02:00
admin a7add7e894 x 2026-08-12 12:44:27 +02:00
admin 78d2c260fd gitea 2026-08-11 21:45:35 +02:00
admin e6dbfa36ac x 2026-08-05 14:31:38 +02:00
admin f12e6f8d91 x 2026-08-05 13:48:03 +02:00