fix(deploy): pass registry auth to all swarm nodes #78

Merged
admin merged 27 commits from hermes/brencher:feat/connectivity-tester-config into main 2026-09-29 23:13:38 +02:00
Contributor

Connectivity tester is global but fails on rudolf-T with No such image because deploy does not pass registry credentials to Swarm workers. This branch includes the connectivity_tester stack work plus --with-registry-auth in docker stack deploy.

Observed before merge: 3-node Swarm, 2 running tasks; rudolf-T rejects the pull. Verify 3/3 after deployment.

Connectivity tester is global but fails on rudolf-T with `No such image` because deploy does not pass registry credentials to Swarm workers. This branch includes the connectivity_tester stack work plus `--with-registry-auth` in `docker stack deploy`. Observed before merge: 3-node Swarm, 2 running tasks; rudolf-T rejects the pull. Verify 3/3 after deployment.
hermes added 26 commits 2026-09-29 19:24:34 +02:00
feat(configs): add connectivity_tester stack
Python package / build (3.12.3) (pull_request) Canceled after 1m30s
Python package / build (3.13) (pull_request) Canceled after 0s
Python package / build (3.14) (pull_request) Canceled after 0s
1f7ca6609d
Long-running Swarm service that runs TCP probes against an allow-list of
internal+external endpoints and a negative-list (cloud metadata etc.)
every 10 minutes. Logs structured PASS/FAIL per check.

Stack lives in admin/ansible under connectivity_tester/. One task per
node via mode: global. Resolves Org1/hermes-voice-gateway#5
(connectivity testing blocker for #3).
feat(configs): wire DockerComposeBuild into connectivity_tester pipeline
Python package / build (3.12.3) (pull_request) Successful in 2m20s
Python package / build (3.13) (pull_request) Successful in 2m17s
Python package / build (3.14) (pull_request) Successful in 2m19s
8b8bbbb230
admin/ansible#39 has a build section in stack-compose.yml, so the
image must be built locally — buildDocker=None would fail with
'No such image' on deploy. Added a proper DockerComposeBuild step
between checkout and deploy. Image tagged auto-<merge-sha> locally;
no registry push (publish=False).
feat(configs): relocate connectivity_tester stack into brencher repo
Python package / build (3.12.3) (pull_request) Failing after 2m18s
Python package / build (3.13) (pull_request) Failing after 2m7s
Python package / build (3.14) (pull_request) Failing after 2m6s
45861362c2
Move compose + Python service + Dockerfile into backend/configs/connectivity_tester/
so the stack can be deployed from a single repo (no cross-repo compose
lookup at deploy time). The brencher config now points at
admin/brencher instead of admin/ansible, and at the new in-repo
compose path.
chore(compose): add connectivity_tester to PROFILES env
Python package / build (3.12.3) (pull_request) Successful in 2m25s
Python package / build (3.13) (pull_request) Successful in 2m55s
Python package / build (3.14) (pull_request) Successful in 2m47s
2d51cd71f3
Without this, brencher's _discover_environments() does not see the
connectivity_tester.py module on next restart. Adding to the comma-
separated PROFILES env wires it in.

Also undo the misplaced relocation from 4586136: clone remote and
docker_compose_path point back at admin/ansible (the compose lives in
admin/ansible#39, not in brencher).
fix(configs): use unmerge=None for connectivity_tester
Python package / build (3.12.3) (pull_request) Failing after 2m23s
Python package / build (3.13) (pull_request) Failing after 2m23s
Python package / build (3.14) (pull_request) Failing after 2m24s
175d1f933e
Without an entry in the brencher UI state.branches for connectivity_tester,
GitUnmerge.progress() returns an empty set and the pipeline raises
'Expected exactly one version, got: set()'. This stalls the brencher
loop, blocking other stacks (including brencher self-deploy) from being
processed.

Same workaround as hermes_infra.py (which uses unmerge=None because
its stacks are managed via direct pushes, not via UI).
fix(configs): drop unmerge step from pipeline
Python package / build (3.12.3) (pull_request) Successful in 2m45s
Python package / build (3.13) (pull_request) Successful in 2m52s
Python package / build (3.14) (pull_request) Successful in 2m50s
ce2f5f6210
Same fix as hermes_infra.py — pass unmerge=None to SharedStateHolderInMemory
and skip the unmerge step entirely. Having unmerge=None in the pipeline
violates mypy's type signature (Pipeline expects AbstractStep, None
is not assignable), causing the brencher image build to fail at the
mypy step.

This is the same workaround as hermes_infra (PR #83): the stack is
managed via direct pushes, not via UI state, so unmerge is not needed.
fix(configs/connectivity_tester): drop UrlCheck; DNS not configured yet
Python package / build (3.12.3) (pull_request) Successful in 2m29s
Python package / build (3.13) (pull_request) Successful in 2m23s
Python package / build (3.14) (pull_request) Successful in 2m33s
db9b3b4cc4
Tracked in admin/brencher#68.
fix(configs/connectivity_tester): wire unmerge for self-deploy
Python package / build (3.12.3) (pull_request) Successful in 2m12s
Python package / build (3.13) (pull_request) Successful in 2m14s
Python package / build (3.14) (pull_request) Successful in 2m51s
055b7e2015
Replace state=SharedStateHolderInMemory(unmerge=None) with a GitUnmerge(clone, dockerSwarmCheck)-backed state holder. Without this, desired_branches is always empty, and brencher's polling loop falls through to 'Empty branches set' on every cycle, so the stack only deploys when an operator PATCHes the environment through the UI. With the unmerge in place the swarm service's org.brencher.version label drives state.branches and the stack self-deploys on each polling cycle.
docs(brencher): add AGENTS.md checklist for new stacks
Python package / build (3.12.3) (pull_request) Successful in 2m1s
Python package / build (3.13) (pull_request) Successful in 2m17s
Python package / build (3.14) (pull_request) Successful in 2m13s
5eef68a1e0
chore(configs/connectivity_tester): try pushing to Gitea Container Registry
Python package / build (3.12.3) (pull_request) Successful in 2m2s
Python package / build (3.13) (pull_request) Successful in 2m7s
Python package / build (3.14) (pull_request) Successful in 2m24s
555b46b746
publish=True + Gitea registry URL. push will fail with HTTP 401 until GITEA_REGISTRY_USERNAME/GITEA_REGISTRY_PASSWORD are wired in as brencher container env — that's the diagnostic to find out whether the bot token is actually allowed to push to gitea.rudolf.keenetic.link/hermes.
feat(configs): use PACKAGE_PUSH_USERNAME/TOKEN for Gitea CR; drop legacy registry
Python package / build (3.12.3) (pull_request) Successful in 2m9s
Python package / build (3.13) (pull_request) Successful in 2m14s
Python package / build (3.14) (pull_request) Successful in 2m12s
07911759d2
feat(steps): DockerComposePull for pulling prebuilt image
Python package / build (3.12.3) (pull_request) Canceled after 0s
Python package / build (3.13) (pull_request) Canceled after 0s
Python package / build (3.14) (pull_request) Canceled after 0s
b187e19a6e
feat(configs): brencher dispatches Actions workflow instead of build+push; deploy pulls image from Gitea CR
Python package / build (3.12.3) (pull_request) Canceled after 36s
Python package / build (3.13) (pull_request) Canceled after 0s
Python package / build (3.14) (pull_request) Canceled after 0s
0196bcda30
docs(AGENTS): document Actions build delegation pattern
Python package / build (3.12.3) (pull_request) Failing after 2m15s
Python package / build (3.13) (pull_request) Failing after 2m1s
Python package / build (3.14) (pull_request) Failing after 1m59s
2229acc7f2
fix(steps/actions): accept callable inputs (resolve at dispatch time)
Python package / build (3.12.3) (pull_request) Successful in 2m12s
Python package / build (3.13) (pull_request) Successful in 2m5s
Python package / build (3.14) (pull_request) Successful in 2m14s
0bb3d53c7a
fix(configs): WORKFLOW_REF=master (release/* doesn't trigger push)
Python package / build (3.12.3) (pull_request) Successful in 2m15s
Python package / build (3.13) (pull_request) Successful in 2m11s
Python package / build (3.14) (pull_request) Successful in 2m16s
e067b8c035
fix(steps/actions): clearer error when POST /dispatches 500
Python package / build (3.12.3) (pull_request) Successful in 2m49s
Python package / build (3.13) (pull_request) Successful in 2m16s
Python package / build (3.14) (pull_request) Successful in 2m8s
4fd29c8ea1
Refines the trigger_build failure mode: instead of opaque urllib.error.HTTPError,
raises a RuntimeError explaining the gitea-side bug + the tag-push workaround
(documented in the runtime message).
fix(steps/git): use force-fetch refspec so force-pushed branches are picked up
Python package / build (3.12.3) (pull_request) Successful in 2m43s
Python package / build (3.13) (pull_request) Successful in 2m30s
Python package / build (3.14) (pull_request) Successful in 2m37s
53b8796319
Default git fetch refspec refuses to update refs whose upstream SHA
diverges from the local copy. When a branch is force-pushed (e.g. our
smoke-test pushes of master, our re-applies of GITHUB_TOKEN fix), the
local remote-tracking ref points at the old SHA and git refuses to
overwrite it. This shows up as 'cannot lock ref ... is at <old> but
expected <new>' and fails the entire clone step.

Adding '+refs/heads/*:refs/remotes/<remote>/*' to each remote's refspec
lets the fetch force-update the remote-tracking refs. Each remote's
existing refspec is preserved; the force-refspec is appended only if
not already present.
This reverts commit 53b8796319.
fix(steps/git): use config_writer to set force-fetch refspec
Python package / build (3.12.3) (pull_request) Successful in 2m35s
Python package / build (3.13) (pull_request) Successful in 2m47s
Python package / build (3.14) (pull_request) Canceled after 35s
3dd0657733
GitPython Remote does not expose a set_refspec method. Use
repo.config_writer() to set remote.<name>.fetch '+refs/heads/*:refs/remotes/<name>/*'.
This bypasses the default safety against updating remote-tracking refs
that diverge from their upstream (force-pushed branches).

Sets the refspec for every remote, not just 'origin', so multi-remote
configs like connectivity_tester (hermes/ansible + admin/ansible) pick
up force-pushed branches on either side.
Revert "fix(steps/git): use config_writer to set force-fetch refspec"
Python package / build (3.12.3) (pull_request) Canceled after 42s
Python package / build (3.13) (pull_request) Canceled after 0s
Python package / build (3.14) (pull_request) Canceled after 0s
962b34e8ee
This reverts commit 3dd0657733.
fix(steps/git): use config_writer with force-fetch refspec
Python package / build (3.12.3) (pull_request) Successful in 2m30s
Python package / build (3.13) (pull_request) Successful in 2m43s
Python package / build (3.14) (pull_request) Successful in 2m37s
2b5c8952b3
Default git fetch refspec refuses to update refs whose upstream SHA
diverges from the local copy. When a branch is force-pushed (e.g. our
smoke-test pushes of master, our re-applies of GITHUB_TOKEN fix), the
local remote-tracking ref points at the old SHA and git refuses to
overwrite it. This shows up as 'cannot lock ref ... is at <old> but
expected <new>' and fails the entire clone step.

Adding '+refs/heads/*:refs/remotes/<remote>/*' to each remote via
config.set_value(section, option, value) lets the fetch force-update
the remote-tracking refs. Each remote gets its own refspec; the
existing 'origin' branchNamePrefix refspec is preserved.
docs(readme): add GITEA_TOKEN + Actions dispatch knobs to docker secret template
Python package / build (3.12.3) (pull_request) Successful in 2m29s
Python package / build (3.13) (pull_request) Successful in 2m31s
Python package / build (3.14) (pull_request) Successful in 2m31s
ac935ca8ca
The brencher stack build-delegation pattern (see AGENTS.md) needs four
env vars to fire `GiteaActionsTrigger`:

- GITEA_TOKEN: Gitea PAT used to POST /dispatches and poll /runs
- WORKFLOW_REPO: owner/repo that hosts the workflow file (default: hermes/ansible)
- WORKFLOW_FILE: workflow filename (default: release.yml)
- WORKFLOW_REF: branch or tag the workflow is triggered on (default: master)

Without GITEA_TOKEN the trigger step gets HTTP 401 from
/api/v1/repos/{owner}/{repo}/actions/runs; the pipeline halts at
trigger_build and the stack never deploys. Added the missing keys to
the secret-create `printf` block in README.instruction.md so the next
person wiring up a brencher stack doesn't have to discover this.

Defaults shown match the connectivity_tester stack wiring
(admin/brencher#65).
fix(configs): inline WORKFLOW_* in GiteaActions* ctor; keep only GITEA_TOKEN in secret
Python package / build (3.12.3) (pull_request) Successful in 2m47s
Python package / build (3.13) (pull_request) Successful in 2m41s
Python package / build (3.14) (pull_request) Successful in 2m31s
ea7c00a02f
The previous split between env vars and constructor args created two
sources of truth for the workflow target. Moving WORKFLOW_OWNER /
WORKFLOW_REPO / WORKFLOW_FILE / WORKFLOW_REF into the constructor
arguments makes the dispatch target obvious at the call site and
keeps the docker secret template minimal (only credentials it must
contain).

README.instruction.md now lists only GIT_USERNAME / GIT_PASSWORD /
GITEA_TOKEN in the `printf` block; WORKFLOW_* are documented in
AGENTS.md (already there) as stack-specific config knobs.
fix(deploy): forward registry auth to swarm nodes
Python package / build (3.12.3) (pull_request) Successful in 3m13s
Python package / build (3.13) (pull_request) Successful in 2m44s
Python package / build (3.14) (pull_request) Successful in 2m42s
bf4533c0fe
hermes added 1 commit 2026-09-29 19:57:45 +02:00
merge main: resolve connectivity tester conflicts
Python package / build (3.12.3) (pull_request) Successful in 2m43s
Python package / build (3.13) (pull_request) Successful in 3m16s
Python package / build (3.14) (pull_request) Successful in 2m10s
272a06a51c
admin merged commit 638c2a7e80 into main 2026-09-29 23:13:38 +02:00
admin deleted branch feat/connectivity-tester-config 2026-09-29 23:13:39 +02:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: admin/brencher#78