feat(configs): add connectivity_tester stack #65

Merged
admin merged 43 commits from hermes/brencher:feat/connectivity-tester-config into main 2026-09-24 23:01:23 +02:00
43 Commits
Author SHA1 Message Date
hermes 390f1fe8ed refactor(connectivity_tester): deploy without compose pull
Python package / build (3.12.3) (pull_request) Successful in 2m8s
Python package / build (3.13) (pull_request) Successful in 2m2s
Python package / build (3.14) (pull_request) Successful in 2m6s
2026-09-21 17:17:03 +00:00
hermes ebd23c63c4 fix(connectivity_tester): remove default branch selection
Python package / build (3.12.3) (pull_request) Canceled after 1m4s
Python package / build (3.13) (pull_request) Canceled after 0s
Python package / build (3.14) (pull_request) Canceled after 0s
2026-09-21 17:15:57 +00:00
hermes 0bdec9f1ec fix(docker): use writable config for registry login
Python package / build (3.12.3) (pull_request) Successful in 2m9s
Python package / build (3.13) (pull_request) Successful in 2m21s
Python package / build (3.14) (pull_request) Successful in 2m43s
2026-09-21 06:41:19 +00:00
hermes 7ba38ff72b fix(docker): keep registry owner out of compose
Python package / build (3.12.3) (pull_request) Successful in 2m44s
Python package / build (3.13) (pull_request) Successful in 2m6s
Python package / build (3.14) (pull_request) Successful in 2m24s
2026-09-20 19:14:51 +00:00
hermes 86aba66756 fix(docker): reuse gitea credentials for registry pull
Python package / build (3.12.3) (pull_request) Canceled after 1m21s
Python package / build (3.13) (pull_request) Canceled after 0s
Python package / build (3.14) (pull_request) Canceled after 0s
2026-09-20 19:12:42 +00:00
hermes 9194e89197 fix(docker): login before private registry pull
Python package / build (3.12.3) (pull_request) Successful in 3m10s
Python package / build (3.13) (pull_request) Successful in 2m8s
Python package / build (3.14) (pull_request) Successful in 2m12s
2026-09-20 19:03:57 +00:00
hermes d34ed2d15d fix(connectivity_tester): persist configured branch selection
Python package / build (3.12.3) (pull_request) Successful in 2m46s
Python package / build (3.13) (pull_request) Successful in 2m34s
Python package / build (3.14) (pull_request) Successful in 2m8s
2026-09-20 18:46:22 +00:00
hermes 80d0a2c282 fix(connectivity_tester): pass registry owner to compose
Python package / build (3.12.3) (pull_request) Successful in 2m49s
Python package / build (3.13) (pull_request) Successful in 2m17s
Python package / build (3.14) (pull_request) Successful in 2m13s
2026-09-20 18:04:26 +00:00
hermes 575da88c05 debug(web): log UI environment selections
Python package / build (3.12.3) (pull_request) Successful in 2m48s
Python package / build (3.13) (pull_request) Successful in 2m38s
Python package / build (3.14) (pull_request) Successful in 2m9s
2026-09-20 17:45:07 +00:00
hermes dd5ef3d27b fix(connectivity_tester): trigger admin workflow 2026-09-20 17:44:33 +00:00
hermes 6479a3e604 fix(configs): push auto branch to admin remote 2026-09-20 17:44:33 +00:00
hermes 01465fa1c5 fix(git): configure push remote for auto branches 2026-09-20 17:44:33 +00:00
hermes 41dc2f8485 fix(configs/connectivity_tester): use GITEA_* env for git auth
CheckoutMerged's auto-* push needs a token with write:repository
on hermes/ansible (and admin/ansible). The brencher container reads
creds via load_dotenv('/run/secrets/brencher-secrets') using
GIT_USERNAME/GIT_PASSWORD by default. The PAT Alex provisioned
(permissions: write:package, write:repository) is deployed into
GITEA_TOKEN/GITEA_USERNAME, not GIT_TOKEN, so the push uses the
wrong env vars and fails with 'Failed to authenticate user'.

Passing credEnvPrefix='GITEA' to the GitClone for connectivity_tester
makes brencher read GITEA_USERNAME + GITEA_PASSWORD for the auto
push, leaving the existing GIT_* defaults intact for every other
stack (so brencher.py / hermes_infra / brencher2 etc. keep
working with whatever's currently in brencher-secrets).
2026-09-20 17:44:33 +00:00
hermes d9e87a7d26 fix(steps/git): force-push auto-branch on every CheckoutMerged.progress()
The 'remote_branch_name is None' guard made auto-branch push
opt-in, not opt-out. When the merge commit already matched an
existing remote-tracking ref (e.g. a feat/* branch the workflow
was built on), brencher would skip the push and trigger the
workflow on whatever branch happened to point at the merge commit.
That branch is rarely the one the workflow is configured to watch,
so the workflow would 404 or stay silent.

Make auto-branch push unconditional when push=True: every cycle
where brencher detects a new HEAD in its watched remotes gets a
fresh 'auto-<short-sha>' branch on the origin remote, which the
workflow trigger (on push to 'auto-*') picks up. The earlier
'remote_branch_name' lookup is kept so the returned value is
useful for diagnostics, but no longer gates the push.
2026-09-20 17:44:33 +00:00
hermes e34ab51ad4 fix(steps/actions): accept Callable for ref to defer eval until progress()
Previous attempt to dispatch on the auto-* branch failed at module
load time: passing `ref=checkoutMerged.progress().remote_branch_name`
runs the whole CheckoutMerged pipeline before brencher has populated
self.env.id, raising BaseException('Environment not set for GitClone')
and preventing brencher from even importing connectivity_tester.

Add RefSource = Union[str, Callable[[], Optional[str]], None] mirroring
the existing InputsSource pattern. The Callable is only invoked from
within progress(), by which point the env is fully populated. The str
form is preserved for existing callers (e.g. WORKFLOW_REF='master').
2026-09-20 17:44:33 +00:00
hermes 0a53b64968 Revert "fix(configs): enable temporary branch push + dispatch on it"
This reverts commit 74229c167c.
2026-09-20 17:44:33 +00:00
hermes d3a2a4e02a fix(configs): enable temporary branch push + dispatch on it
Brencher now mirrors the auto-deploy-via-temp-branch pattern Alex
described: instead of triggering the workflow on the default branch,
brencher pushes the merge commit to `auto-<sha>` via CheckoutMerged's
existing push=True logic, then dispatches the workflow on that new
branch. The branch acts as a review point between build and deploy:
after trigger_build and await_build complete, the deploy step still
runs on the same local merge, but the workflow's provenance lives
on a remote branch Alex can inspect or revert before merging back to
default.

- checkoutMerged.push = True (was False). CheckoutMerged already
  builds the branch name as `auto-<commit1>-<commit2>...` from the
  sorted merges and force-pushes it to `origin` (see backend/steps/git.py
  ~line 311).
- trigger_build ref pulls `checkoutMerged.progress().remote_branch_name`
  instead of a hard-coded "master". This makes the workflow run on
  the auto-branch brencher just pushed.
2026-09-20 17:44:33 +00:00
hermes 878ade00ef fix(steps/actions): tolerate 204 No Content responses in _http
POST /api/v1/repos/{owner}/{repo}/actions/workflows/{file}/dispatches
returns HTTP 204 No Content on success (an empty body). The previous
implementation unconditionally called json.loads on the response,
which raised json.decoder.JSONDecodeError on the empty body. The
stack hit this with 'Expecting value: line 1 column 1 (char 0)' and
never advanced past GiteaActionsTrigger.

Return {} instead for empty / 204 responses so the trigger step
succeeds and the rest of the pipeline can run.
2026-09-20 17:44:33 +00:00
hermes f9eb7a1ffb fix(configs): inline WORKFLOW_* in GiteaActions* ctor; keep only GITEA_TOKEN in secret
The previous split between env vars and constructor args created two
sources of truth for the workflow target. Moving WORKFLOW_OWNER /
WORKFLOW_REPO / WORKFLOW_FILE / WORKFLOW_REF into the constructor
arguments makes the dispatch target obvious at the call site and
keeps the docker secret template minimal (only credentials it must
contain).

README.instruction.md now lists only GIT_USERNAME / GIT_PASSWORD /
GITEA_TOKEN in the `printf` block; WORKFLOW_* are documented in
AGENTS.md (already there) as stack-specific config knobs.
2026-09-20 17:44:33 +00:00
hermes 2b2f66afd0 docs(readme): add GITEA_TOKEN + Actions dispatch knobs to docker secret template
The brencher stack build-delegation pattern (see AGENTS.md) needs four
env vars to fire `GiteaActionsTrigger`:

- GITEA_TOKEN: Gitea PAT used to POST /dispatches and poll /runs
- WORKFLOW_REPO: owner/repo that hosts the workflow file (default: hermes/ansible)
- WORKFLOW_FILE: workflow filename (default: release.yml)
- WORKFLOW_REF: branch or tag the workflow is triggered on (default: master)

Without GITEA_TOKEN the trigger step gets HTTP 401 from
/api/v1/repos/{owner}/{repo}/actions/runs; the pipeline halts at
trigger_build and the stack never deploys. Added the missing keys to
the secret-create `printf` block in README.instruction.md so the next
person wiring up a brencher stack doesn't have to discover this.

Defaults shown match the connectivity_tester stack wiring
(admin/brencher#65).
2026-09-20 17:44:33 +00:00
hermes 9b82c917bc fix(steps/git): use config_writer with force-fetch refspec
Default git fetch refspec refuses to update refs whose upstream SHA
diverges from the local copy. When a branch is force-pushed (e.g. our
smoke-test pushes of master, our re-applies of GITHUB_TOKEN fix), the
local remote-tracking ref points at the old SHA and git refuses to
overwrite it. This shows up as 'cannot lock ref ... is at <old> but
expected <new>' and fails the entire clone step.

Adding '+refs/heads/*:refs/remotes/<remote>/*' to each remote via
config.set_value(section, option, value) lets the fetch force-update
the remote-tracking refs. Each remote gets its own refspec; the
existing 'origin' branchNamePrefix refspec is preserved.
2026-09-20 17:44:33 +00:00
hermes 10eb0a5d49 Revert "fix(steps/git): use config_writer to set force-fetch refspec"
This reverts commit 3dd0657733.
2026-09-20 17:44:32 +00:00
hermes fb5e9d1430 fix(steps/git): use config_writer to set force-fetch refspec
GitPython Remote does not expose a set_refspec method. Use
repo.config_writer() to set remote.<name>.fetch '+refs/heads/*:refs/remotes/<name>/*'.
This bypasses the default safety against updating remote-tracking refs
that diverge from their upstream (force-pushed branches).

Sets the refspec for every remote, not just 'origin', so multi-remote
configs like connectivity_tester (hermes/ansible + admin/ansible) pick
up force-pushed branches on either side.
2026-09-20 17:44:32 +00:00
hermes d3f09e40f3 Revert "fix(steps/git): use force-fetch refspec so force-pushed branches are picked up"
This reverts commit 53b8796319.
2026-09-20 17:44:32 +00:00
hermes c1c1108b70 fix(steps/git): use force-fetch refspec so force-pushed branches are picked up
Default git fetch refspec refuses to update refs whose upstream SHA
diverges from the local copy. When a branch is force-pushed (e.g. our
smoke-test pushes of master, our re-applies of GITHUB_TOKEN fix), the
local remote-tracking ref points at the old SHA and git refuses to
overwrite it. This shows up as 'cannot lock ref ... is at <old> but
expected <new>' and fails the entire clone step.

Adding '+refs/heads/*:refs/remotes/<remote>/*' to each remote's refspec
lets the fetch force-update the remote-tracking refs. Each remote's
existing refspec is preserved; the force-refspec is appended only if
not already present.
2026-09-20 17:44:32 +00:00
hermes c38622e138 fix(steps/actions): clearer error when POST /dispatches 500
Refines the trigger_build failure mode: instead of opaque urllib.error.HTTPError,
raises a RuntimeError explaining the gitea-side bug + the tag-push workaround
(documented in the runtime message).
2026-09-20 17:44:32 +00:00
hermes a1bac89a96 fix(configs): WORKFLOW_REF=master (release/* doesn't trigger push) 2026-09-20 17:44:32 +00:00
hermes 07ef686273 fix(steps/actions): accept callable inputs (resolve at dispatch time) 2026-09-20 17:44:32 +00:00
admin 4705755243 docs(AGENTS): document Actions build delegation pattern 2026-09-20 17:44:32 +00:00
admin 27413e4b9a feat(configs): brencher dispatches Actions workflow instead of build+push; deploy pulls image from Gitea CR 2026-09-20 17:44:32 +00:00
admin abdecd0736 feat(steps): DockerComposePull for pulling prebuilt image 2026-09-20 17:44:32 +00:00
admin 65c855960d feat(steps): GiteaActionsTrigger and GiteaActionsAwait for build dispatch 2026-09-20 17:44:32 +00:00
admin f7ff922606 feat(configs): use PACKAGE_PUSH_USERNAME/TOKEN for Gitea CR; drop legacy registry 2026-09-20 17:44:32 +00:00
admin f86aab5a1a chore(configs/connectivity_tester): try pushing to Gitea Container Registry
publish=True + Gitea registry URL. push will fail with HTTP 401 until GITEA_REGISTRY_USERNAME/GITEA_REGISTRY_PASSWORD are wired in as brencher container env — that's the diagnostic to find out whether the bot token is actually allowed to push to gitea.rudolf.keenetic.link/hermes.
2026-09-20 17:44:32 +00:00
admin 0766ef6bdc docs(brencher): add AGENTS.md checklist for new stacks 2026-09-20 17:44:32 +00:00
admin eabd348d92 fix(configs/connectivity_tester): wire unmerge for self-deploy
Replace state=SharedStateHolderInMemory(unmerge=None) with a GitUnmerge(clone, dockerSwarmCheck)-backed state holder. Without this, desired_branches is always empty, and brencher's polling loop falls through to 'Empty branches set' on every cycle, so the stack only deploys when an operator PATCHes the environment through the UI. With the unmerge in place the swarm service's org.brencher.version label drives state.branches and the stack self-deploys on each polling cycle.
2026-09-20 17:44:32 +00:00
admin 90555ca4ba fix(configs/connectivity_tester): drop UrlCheck; DNS not configured yet
Tracked in admin/brencher#68.
2026-09-20 17:44:32 +00:00
hermes 65b2ef5325 fix(configs): drop unmerge step from pipeline
Same fix as hermes_infra.py — pass unmerge=None to SharedStateHolderInMemory
and skip the unmerge step entirely. Having unmerge=None in the pipeline
violates mypy's type signature (Pipeline expects AbstractStep, None
is not assignable), causing the brencher image build to fail at the
mypy step.

This is the same workaround as hermes_infra (PR #83): the stack is
managed via direct pushes, not via UI state, so unmerge is not needed.
2026-09-20 17:44:32 +00:00
hermes eccbfb8065 fix(configs): use unmerge=None for connectivity_tester
Without an entry in the brencher UI state.branches for connectivity_tester,
GitUnmerge.progress() returns an empty set and the pipeline raises
'Expected exactly one version, got: set()'. This stalls the brencher
loop, blocking other stacks (including brencher self-deploy) from being
processed.

Same workaround as hermes_infra.py (which uses unmerge=None because
its stacks are managed via direct pushes, not via UI).
2026-09-20 17:44:32 +00:00
hermes 3365a1496d chore(compose): add connectivity_tester to PROFILES env
Without this, brencher's _discover_environments() does not see the
connectivity_tester.py module on next restart. Adding to the comma-
separated PROFILES env wires it in.

Also undo the misplaced relocation from 4586136: clone remote and
docker_compose_path point back at admin/ansible (the compose lives in
admin/ansible#39, not in brencher).
2026-09-20 17:44:32 +00:00
hermes be1e80a3b8 feat(configs): relocate connectivity_tester stack into brencher repo
Move compose + Python service + Dockerfile into backend/configs/connectivity_tester/
so the stack can be deployed from a single repo (no cross-repo compose
lookup at deploy time). The brencher config now points at
admin/brencher instead of admin/ansible, and at the new in-repo
compose path.
2026-09-20 17:44:32 +00:00
hermes a3b53ce177 feat(configs): wire DockerComposeBuild into connectivity_tester pipeline
admin/ansible#39 has a build section in stack-compose.yml, so the
image must be built locally — buildDocker=None would fail with
'No such image' on deploy. Added a proper DockerComposeBuild step
between checkout and deploy. Image tagged auto-<merge-sha> locally;
no registry push (publish=False).
2026-09-20 17:44:32 +00:00
hermes d95d97be68 feat(configs): add connectivity_tester stack
Long-running Swarm service that runs TCP probes against an allow-list of
internal+external endpoints and a negative-list (cloud metadata etc.)
every 10 minutes. Logs structured PASS/FAIL per check.

Stack lives in admin/ansible under connectivity_tester/. One task per
node via mode: global. Resolves Org1/hermes-voice-gateway#5
(connectivity testing blocker for #3).
2026-09-20 17:44:32 +00:00