Compare commits

...
Author SHA1 Message Date
Gitea Actions d5a7bc1e4e Merge remote-tracking branch 'origin/main' into main_corrupted 2026-09-14 16:43:38 +00:00
hermesandhermes 2e48d11b77 feat(ui): pre-fill branch filter from ?filter=<text> GET-param (#67)
## Summary

Lets the brencher UI pre-fill its branch filter from a `?filter=<text>` query
parameter on initial page load.

## Use case

Deep-link filtered views from external tooling (runbooks, Gitea issues,
dashboards). For example:

- `https://brencher.rudolf.keenetic.link/?filter=connectivity_tester` — go straight to the new stack after #65 lands.
- `https://brencher.rudolf.keenetic.link/?filter=hermes_infra%20v0.4.0` — multiple words (current filter does substring match on all).

## Changes

`frontend/app.js`: IIFE on initial load reads `URLSearchParams`, sets
`branchFilter.value`, calls `filterBranches()`. Empty / missing param
leaves the filter empty (no pre-fill, no apply) — same as before.

## Acceptance

- `?filter=foo` pre-fills the input AND applies the filter on initial render.
- Empty `?filter=` is a no-op (param present but empty string).
- No param at all is a no-op (no pre-fill, no apply).
- Existing inline typing behaviour unchanged.

## Resolves

- admin/brencher#66 (brencher UI: support ?filter=<text> GET-param that auto-fills the filter input)

## Related

- admin/brencher#65 (PR feat(configs): add connectivity_tester stack) — once #65 merges, the link `/<brencher-host>/?filter=connectivity_tester` becomes a one-click entry point to the new stack.

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: admin/brencher#67
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-14 18:43:01 +02:00
Gitea Actions 8219b743cf Merge remote-tracking branch 'origin/main' into main_corrupted 2026-09-11 22:16:32 +00:00
hermes 7938f4f0f0 fix(configs/hermes_infra): wire unmerge into state holder for restart recovery (#61)
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/61
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-12 00:15:38 +02:00
Gitea Actions 71d1e1c1af Merge remote-tracking branch 'origin/main' into main_corrupted 2026-09-11 15:08:42 +00:00
52692a75b8 fix(web): use setdefault in WebSocket broadcast to avoid KeyError race (#58)
## Symptom
Production brencher UI reports constant WebSocket reconnects (see
user observation 2026-09-05, fact #5). Prod logs from
`brencher_brencher-backend` (image `brencher:auto-74bf5aef-…`) show
`WebSocket /ws [accepted]` followed within ~3–10 s by
`KeyError: <starlette.websockets.WebSocket object …>` raised at
`backend/web.py:175`, then `connection closed`. Same UI reconnecting
after the 5 s backoff. The companion `brencher2` instance (one
stack, no `SECONDARY_BRENCHER`) shows 0 errors over the same window,
confirming that the same code on a single-stack single-source path
is race-free.

## Root cause
`ws_connections: Dict[WebSocket, Dict[str, Any]]` is mutated from
two coroutines that race on the asyncio event loop:
- `broadcast()` — invoked via `asyncio.run_coroutine_threadsafe`
  from the processing thread on every `onupdate()`. With 7 profiles
  on the new brencher this fires ≥10 times per ~100 s cycle, and
  the SecondaryManager’s `on_update=lambda: self.broadcast_all()`
  layers additional dispatches on top.
- `websocket_endpoint()` — the FastAPI handler running on the
  uvicorn loop. Between `await websocket.accept()` and
  `self.ws_connections[websocket] = {}`, the handler yields. A
  concurrent broadcast that was already iterating
  `self.ws_connections.keys()` may include the freshly accepted
  socket in `disconnected` (because its own `send_text` raised
  earlier and was logged), then `pop` removes it before
  `websocket_endpoint` writes back. The handler then hits
  `self.ws_connections[websocket]["branches"] = …` and raises
  `KeyError`. Starlette’s default error path closes the WS.

## Fix (this PR — variant `setdefault`)
Treat `ws_connections` as a sparse dict: rely on
`dict.setdefault` for atomic insert-or-get, and hold a local
`state` binding throughout the handler so every subsequent write
operates on the same dict object even if a concurrent broadcast
removes and re-adds the entry.

Diff is 12 lines, no API or protocol change:
- `broadcast()`: iterate `ws_connections.items()`; write to the
  local `state[event] = message` instead of
  `self.ws_connections[websocket][event] = message`.
- `websocket_endpoint()`: replace `ws_connections[websocket] = {}`
  with `state = ws_connections.setdefault(websocket, {})`; replace
  the two follow-up writes with `state["branches"] = …` /
  `state["environments"] = …`.

Properties:
- A broadcast scheduled before `setdefault` runs is a no-op for
  the freshly accepted client (its first frame is the handler’s
  initial `send_text`, so losing one in-flight broadcast is
  benign; the next periodic broadcast at `~10 s` re-sends).
- Stale `state` returned by `setdefault` is overwritten in place
  — no lock needed because CPython dict operations are atomic
  under the GIL, and we only ever touch the dict’s value binding,
  never its key existence.
- Code reads cleaner: there’s one `state = ws_connections[…]`
  binding per WS, no `self.ws_connections[websocket]["x"]`
  chains.

## Companion PRs (variants 3 and 4 — same race, different shape)
- `fix/ws-asyncio-lock` — explicit `asyncio.Lock` around the
  dict; serializes mutation but not `send_text`.
- `fix/ws-writer-loop` — single-writer task consuming an
  `asyncio.Queue`; broadcast becomes inherently sequential.

All three solve the same root cause with different trade-offs. The
recommended merge order is this one (smallest diff, lowest review
surface). Promote to `Lock` or `Queue` only if benchmarking shows
setdefault is insufficient under heavier connection concurrency.

## Validation
- `python3 -m py_compile backend/web.py` → OK.
- Diff: 12 lines in one file.

Manual verification on production is the user’s call. The prod
logs already show the pre-fix error pattern (`brencher_brencher-backend`
1.80 WS errors per WS_OPEN), so post-deploy the same log extraction
script (in `prod-ws-error-count` skill) should drop to 0.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: alex <alex@rudenk.local>
Co-authored-by: brencher_bot <brencher_bot@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/58
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-11 17:07:44 +02:00
admin 74bf5aefa2 fix(tests): add type ignore for non-string prefix in GitClone tests 2026-09-06 16:46:14 +02:00
admin 60b5af79fc Merge remote-tracking branch 'gitea/main' into main_corrupted 2026-09-06 16:36:52 +02:00
hermesandadmin ddce27f1af recover(auto-6fd59898): preserve 6fd598983a (PR #26 head, gmail_mcp) (#53)
Auto-recovery snapshot of shard auto-6fd59898 from image auto-04d3624a-34caac15-6fd59898-920249ec-afc2de34. Fossil branch. This commit was recovered from refs/pull/26/head (Gitea retains PR-head refs even after merge). PR #26 was merged; mergeable will likely be False.

---------

Co-authored-by: brencher_bot <rudolfss13@gmail.com>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/53
Co-authored-by: brencher_bot (hermes) <rudolfss13+hermes@gmail.com>
2026-09-06 16:28:18 +02:00
hermesandhermes 33caa4cb9f recover(auto-34caac15): preserve 34caac15ce (PR #12 commit, immich) (#52)
Auto-recovery snapshot of shard auto-34caac15 from image auto-04d3624a-34caac15-6fd59898-920249ec-afc2de34. Fossil branch. PR #12 (point immich at gitea admin/hermes ansible repos) was merged into admin/brencher:main; mergeable will likely be False because the content is already in main.

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/52
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-06 16:27:57 +02:00
hermesandadmin a9de490ebe recover(auto-04d3624a): preserve 04d3624afd (PR #28 commit, hermes v2) (#51)
Auto-recovery snapshot of shard auto-04d3624a from image auto-04d3624a-34caac15-6fd59898-920249ec-afc2de34. Fossil branch. The commit's content already landed in admin/brencher:main via PR #28 (closed) and is part of PR #11 family. Opening this PR is for archival/reference only; mergeable will likely be False because the content is already in main.

---------

Co-authored-by: admin <rudolfss13@gmail.com>
Co-authored-by: brencher_bot <rudolfss13@gmail.com>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/51
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-06 16:27:10 +02:00
hermesandadmin 99f8b73654 feat(git): per-remote branch prefix filtering for multi-remote GitClone (#31)
## Summary

`GitClone`'s top-level `branchNamePrefix` argument was silently ignored when constructed with the explicit `remotes=[...]` form (verified in `backend/steps/git.py:106`):

```python
if remote_name == "origin" and self.branchNamePrefix != "":
    # apply refspec
```

The prefix only fires when a remote is named `"origin"`. The `remotes=` form passes names through verbatim, so configs like the open PR #28 set the prefix but never apply it — all branches from all remotes are fetched.

This PR extends the API: each remote entry now optionally accepts a `prefix` field. When set, the fetch refspec for that remote becomes scoped to `<prefix>/*` branches. Backwards-compatible — remotes without `prefix` keep the existing behavior, and `url=` shorthand with `branchNamePrefix` still works.

## API

```python
clone = GitClone(remotes=[
    {"name": "github",       "url": "...", "prefix": "ansible"},
    {"name": "gitea_hermes", "url": "...", "prefix": "ansible"},
    {"name": "gitea_admin",  "url": "...", "prefix": "ansible"},
])
```

Each remote is independent — some can have `prefix`, others not.

## Changes

### `backend/steps/git.py`

- `__init__` (`remotes=` branch): validate `prefix` (string if set, reject non-string with `ValueError`), persist into the normalized entry. Empty string / None → no prefix applied (existing behavior).
- `progress()` fetch loop: replace the single `if remote_name == "origin"` check with per-entry logic. Effective prefix = `entry["prefix"] or (self.branchNamePrefix if remote_name == "origin" else "")`. Apply refspec per remote when effective prefix is non-empty.

### `tests/test_git_integration.py`

3 new tests in `TestGitIntegration`:

- `test_git_clone_remotes_per_remote_prefix` — both remotes prefixed; only `ansible/*` branches visible, `master` is filtered out.
- `test_git_clone_remotes_mixed_prefix` — one remote prefixed, one not; each behaves independently.
- `test_git_clone_remotes_prefix_validation` — `prefix=123` / `prefix=["ansible"]` raise `ValueError`; `prefix=""` and missing-`prefix` are accepted.

Existing `test_git_clone_named_remotes` continues to pass (verified — no behavior change for unprefixed remotes).

## Test results

```
29 passed in ~13s
```

(18 in `test_git_integration.py`, 11 in `test_dummy.py`/`test_caching_step.py`/`test_cached_env.py`.)

## Migration path

After merge:

- Open migration PRs (#26 gmail_mcp, #28 hermes, planned #19 gitea.py) — switch from `branchNamePrefix="ansible"` to `"prefix": "ansible"` on each remote.
- `hermes_infra.py` after PR #8 already drops `branchNamePrefix`, no change needed.
- Existing `url=` configs (immich, gmail_mcp-old, hermes-old, registry, gitea-old) — unaffected.

## Unblocks

PR #28 — once this lands, hermes.py can drop the (currently misleading) top-level `branchNamePrefix="ansible"` and add `"prefix": "ansible"` per remote.

Refs #30
Blocks #28
Part of #18 (parent migration ticket)

---------

Co-authored-by: admin <rudolfss13@gmail.com>
Co-authored-by: brencher_bot <rudolfss13@gmail.com>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/31
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-05 19:20:44 +02:00
hermesandadmin 272504eab3 feat(configs): point gmail_mcp at gitea admin/hermes mcps repos (#26)
## Summary

Updates `backend/configs/gmail_mcp.py` to point at the new `mcps` stack mirrors on gitea instead of the legacy `uber_backup` repo.

Remotes now:

- `gitea_hermes` → `https://gitea.rudolf.keenetic.link/hermes/mcps`
- `gitea_admin`  → `https://gitea.rudolf.keenetic.link/admin/mcps`

Uses the `remotes=[{name, url}, …]` form introduced by #3.

## What changed vs the original #26 body

- **Repo switched**: `uber_backup` → `mcps` (the gmail + telegram MCP server repo, now mirrored on gitea under both `admin/` and `hermes/`).
- **GitHub remote dropped**: matches the pattern in `hermes_infra.py` after #8 and `docker_socket_proxy.py` after #23. The brencher container cannot reach `github.com` (squid returns HTTP 403 on CONNECT tunnel), so keeping a `github` remote would hard-fail the clone and trap brencher in the same retry loop that blocked #23 on `admin/reverseproxy` before its mirror was created.
- **`branchNamePrefix="immoscout"` dropped**: the prefix is only applied to remotes named `origin` (see `backend/steps/git.py` ~L104); neither remote qualifies, so the field is a silent no-op. Matches the cleanup done in `hermes_infra.py` / `brencher.py`.

## Diff (vs `main`)

```diff
-clone = GitClone(url="https://github.com/rudolf1/uber_backup.git", branchNamePrefix="immoscout")
+clone = GitClone(remotes=[
+    {"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/mcps"},
+    {"name": "gitea_admin",  "url": "https://gitea.rudolf.keenetic.link/admin/mcps"},
+])
```

## Prerequisites (already done)

Both target mirrors exist on gitea:

- `admin/mcps` — already present (primary, not a fork).
- `hermes/mcps` — forked in this session (id=10, fork of `admin/mcps`).

## Test plan

1. From the brencher state file for `gmail_mcp`, register `gitea_hermes/feat/migrate-gmail-mcp-to-multi-remote` as the desired branch.
2. Watch brencher logs: `Cloning repositories [...]` should succeed for both `gitea_hermes` and `gitea_admin`.
3. Confirm `gmail_mcp` stack redeploys against the new repo.

Refs #20
Part of #18

---------

Co-authored-by: brencher_bot <rudolfss13@gmail.com>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/26
Co-authored-by: brencher_bot (hermes) <rudolfss13+hermes@gmail.com>
2026-09-05 19:20:16 +02:00
hermesandhermes fa9f8791bd feat(configs): point immich at gitea admin/hermes ansible repos (#12)
## Summary
Migrates `backend/configs/immich.py` from the legacy `url=` form (pointing at the unreachable `github.com/rudolf1/uber_backup.git`) to the new `remotes=[{name, url}, ...]` form introduced by PR #3 (named-remotes), pointing at the actual gitea repos where the ansible stack lives.

## What's in this PR
- `backend/configs/immich.py` — single file, +22/-1
  - `GitClone(url="https://github.com/rudolf1/uber_backup.git"[, branchNamePrefix=...])`
    →
    `GitClone(remotes=[{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/ansible"}, {"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/ansible"}])`
  - `branchNamePrefix` is dropped (it would be a silent no-op: `GitClone` only applies its prefix refspec to a remote named `origin` per `backend/steps/git.py` ~L104, and neither remote here is named `origin`)

## Why migrate
- The old URL never resolved (`uber_backup.git` doesn't exist on gitea anyway) — so the deploy path was broken even though the file looked correct.
- `github.com` returns HTTP 403 on CONNECT tunnel from the brencher container's egress proxy — verified in `admin/brencher#4` (commit `653aee9`).

## Scope decision: only `immich.py`
The other open configs (`registry.py`, `gitea.py`, `hermes.py`, `immich.py`, `gmail_mcp.py`, `torrserv_proxy.py`) are being migrated in **separate PRs**, one per file. Each PR has its own branch + verification.

## Verification (sandbox, no docker daemon)
- `python3 -m py_compile backend/configs/immich.py` — OK
- Real import via `from configs import immich`:
  - `clone.remotes` is a 2-entry list `[gitea_hermes, gitea_admin]`, `clone.urls` derived from it
  - no `github.com` substring anywhere in the constructed config
  - `clone.branchNamePrefix == ''`
- pytest: **26/26** passed on the no-docker / no-playwright suite
- mypy `--no-incremental --follow-imports=silent`: `Success: no issues found in 1 source file`
- **End-to-end smoke test**: built a parallel `GitClone(remotes=[...])` with both URLs pointed at `file:///tmp/ansible-upstream`, ran `clone.progress()`:
  - both named remotes created in the temp clone
  - 4 branches discovered (`master` + `feat/docker-socket-proxy` × 2 remotes)
  - `feat/docker-socket-proxy` visible on both `gitea_admin` and `gitea_hermes`

## What was NOT verified in the sandbox
- HTTPS clone from `gitea.rudolf.keenetic.link` on the live brencher-bot — requires the brencher container's git credentials to have push rights on `admin/ansible`.
- end-to-end `docker stack deploy` — needs a real swarm.

## After merge
After this PR merges into `admin/brencher:main`, the next brencher deploy of this environment will clone from gitea instead of the unreachable github URL.

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/12
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-05 19:19:47 +02:00
hermesandhermes 694c28109d feat(configs): point hermes at gitea admin/hermes ansible repos (#11)
## Summary
Migrates `backend/configs/hermes.py` from the legacy `url=` form (pointing at the unreachable `github.com/rudolf1/uber_backup.git`) to the new `remotes=[{name, url}, ...]` form introduced by PR #3 (named-remotes), pointing at the actual gitea repos where the ansible stack lives.

## What's in this PR
- `backend/configs/hermes.py` — single file, +22/-1
  - `GitClone(url="https://github.com/rudolf1/uber_backup.git"[, branchNamePrefix=...])`
    →
    `GitClone(remotes=[{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/ansible"}, {"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/ansible"}])`
  - `branchNamePrefix` is dropped (it would be a silent no-op: `GitClone` only applies its prefix refspec to a remote named `origin` per `backend/steps/git.py` ~L104, and neither remote here is named `origin`)

## Why migrate
- The old URL never resolved (`uber_backup.git` doesn't exist on gitea anyway) — so the deploy path was broken even though the file looked correct.
- `github.com` returns HTTP 403 on CONNECT tunnel from the brencher container's egress proxy — verified in `admin/brencher#4` (commit `653aee9`).

## Scope decision: only `hermes.py`
The other open configs (`registry.py`, `gitea.py`, `hermes.py`, `immich.py`, `gmail_mcp.py`, `torrserv_proxy.py`) are being migrated in **separate PRs**, one per file. Each PR has its own branch + verification.

## Verification (sandbox, no docker daemon)
- `python3 -m py_compile backend/configs/hermes.py` — OK
- Real import via `from configs import hermes`:
  - `clone.remotes` is a 2-entry list `[gitea_hermes, gitea_admin]`, `clone.urls` derived from it
  - no `github.com` substring anywhere in the constructed config
  - `clone.branchNamePrefix == ''`
- pytest: **26/26** passed on the no-docker / no-playwright suite
- mypy `--no-incremental --follow-imports=silent`: `Success: no issues found in 1 source file`
- **End-to-end smoke test**: built a parallel `GitClone(remotes=[...])` with both URLs pointed at `file:///tmp/ansible-upstream`, ran `clone.progress()`:
  - both named remotes created in the temp clone
  - 4 branches discovered (`master` + `feat/docker-socket-proxy` × 2 remotes)
  - `feat/docker-socket-proxy` visible on both `gitea_admin` and `gitea_hermes`

## What was NOT verified in the sandbox
- HTTPS clone from `gitea.rudolf.keenetic.link` on the live brencher-bot — requires the brencher container's git credentials to have push rights on `admin/ansible`.
- end-to-end `docker stack deploy` — needs a real swarm.

## After merge
After this PR merges into `admin/brencher:main`, the next brencher deploy of this environment will clone from gitea instead of the unreachable github URL.

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/11
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-05 19:19:22 +02:00
hermesandhermes 920249ec30 feat(configs): point gitea-config at gitea admin/hermes ansible repos (#10)
## Summary
Migrates `backend/configs/gitea.py` from the legacy `url=` form (pointing at the unreachable `github.com/rudolf1/uber_backup.git`) to the new `remotes=[{name, url}, ...]` form introduced by PR #3 (named-remotes), pointing at the actual gitea repos where the ansible stack lives.

## What's in this PR
- `backend/configs/gitea.py` — single file, +22/-1
  - `GitClone(url="https://github.com/rudolf1/uber_backup.git"[, branchNamePrefix=...])`
    →
    `GitClone(remotes=[{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/ansible"}, {"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/ansible"}])`
  - `branchNamePrefix` is dropped (it would be a silent no-op: `GitClone` only applies its prefix refspec to a remote named `origin` per `backend/steps/git.py` ~L104, and neither remote here is named `origin`)

## Why migrate
- The old URL never resolved (`uber_backup.git` doesn't exist on gitea anyway) — so the deploy path was broken even though the file looked correct.
- `github.com` returns HTTP 403 on CONNECT tunnel from the brencher container's egress proxy — verified in `admin/brencher#4` (commit `653aee9`).

## Scope decision: only `gitea.py`
The other open configs (`registry.py`, `gitea.py`, `hermes.py`, `immich.py`, `gmail_mcp.py`, `torrserv_proxy.py`) are being migrated in **separate PRs**, one per file. Each PR has its own branch + verification.

## Verification (sandbox, no docker daemon)
- `python3 -m py_compile backend/configs/gitea.py` — OK
- Real import via `from configs import gitea`:
  - `clone.remotes` is a 2-entry list `[gitea_hermes, gitea_admin]`, `clone.urls` derived from it
  - no `github.com` substring anywhere in the constructed config
  - `clone.branchNamePrefix == ''`
- pytest: **26/26** passed on the no-docker / no-playwright suite
- mypy `--no-incremental --follow-imports=silent`: `Success: no issues found in 1 source file`
- **End-to-end smoke test**: built a parallel `GitClone(remotes=[...])` with both URLs pointed at `file:///tmp/ansible-upstream`, ran `clone.progress()`:
  - both named remotes created in the temp clone
  - 4 branches discovered (`master` + `feat/docker-socket-proxy` × 2 remotes)
  - `feat/docker-socket-proxy` visible on both `gitea_admin` and `gitea_hermes`

## What was NOT verified in the sandbox
- HTTPS clone from `gitea.rudolf.keenetic.link` on the live brencher-bot — requires the brencher container's git credentials to have push rights on `admin/ansible`.
- end-to-end `docker stack deploy` — needs a real swarm.

## After merge
After this PR merges into `admin/brencher:main`, the next brencher deploy of this environment will clone from gitea instead of the unreachable github URL.

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/10
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-05 13:36:22 +02:00
hermes 75126cb2fd ci(merge_main): hardcode gitea.rudolf.keenetic.link (GITEA_SERVER_URL empty) (#47)
## Problem

Run #67 of `merge_main.yaml` failed with:

```
fatal: unable to access 'https:///admin/brencher.git/': URL rejected: No host part in the URL
```

The `PUSH_TOKEN` secret is now wired up correctly, but `${GITEA_SERVER_URL}` is empty in this Gitea Actions runtime (the variable the env-prefixed runner is supposed to set), so the substituted URL drops the host.

## Fix

Hardcode `gitea.rudolf.keenetic.link` in both `merge_main.yaml` and `merge_main_and_squash.yaml`. When the runner exposes `GITEA_SERVER_URL` in a future version, this can be replaced back with `${GITEA_SERVER_URL#https://}`.

## Supersedes

This is a follow-up to admin/brencher#46 which already wired `secrets.PUSH_TOKEN` but still had the empty env var.Reviewed-on: http://192.168.1.96/admin/brencher/pulls/47

Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-04 16:26:09 +02:00
admin 0e83458044 Обновить REST.instruction.md (#48)
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/48
2026-09-04 13:26:37 +02:00
admin 7dc36d7487 Merge pull request 'ci(merge_main): fix auth for self-hosted Gitea Actions' (#46) from hermes/brencher:fix/gitea-actions-merge-workflows into main
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/46
2026-09-04 13:08:06 +02:00
hermes b0e32f835a ci(merge_main): use PUSH_TOKEN instead of GITEA_PUSH_TOKEN
Match the secret name already created in admin/brencher.
2026-09-04 11:01:55 +00:00
hermes 1c0580750d ci(merge_main): fix auth for self-hosted Gitea Actions
Both workflows were written for github.com and relied on the implicit
secrets.GITHUB_TOKEN + github.repository + a hardcoded github.com URL.
On this self-hosted Gitea instance (ghauth.rudolf.keenetic.link) they fail
with "Invalid username or token. Password authentication is not supported
for Git operations." against https://github.com/admin/brencher.git.

The fix:
- Read the push token from a new repo-level secret GITEA_PUSH_TOKEN
  (admin/brencher). Same value is injected into the checkout and into the
  rewritten remote URL so push can succeed.
- Build the origin URL from Gitea-injected GITHUB_REPOSITORY and
  GITEA_SERVER_URL instead of hardcoding github.com.
- Update git user.email/name for the Gitea Actions identity.

To enable: create secret GITEA_PUSH_TOKEN in admin/brencher (a PAT with
write access to the repo) via the Gitea UI before merging this PR.
2026-09-04 08:03:46 +00:00
8059cc74ea ci: speed up python-package.yml from ~13min to ~3-4min (#15)
## Summary
Optimizes `.github/workflows/python-package.yml` so the build job runs in ~3-4 minutes instead of ~13 minutes (which is what the recent run on PR #5 took — see https://gitea.rudolf.keenetic.link/admin/brencher/actions/runs/14, started 23:52, completed 00:05, **13m 19s wall-clock**).

## Why it was slow
The current matrix has **3 Python versions × 5 checkers = 15 separate CI jobs**, each one:
1. Checking out the repo (~10s)
2. Setting up Python via `actions/setup-python@v3` with **no cache enabled** (~15-30s)
3. Running `pip install -r requirements.txt -r requirements-dev.txt` **from scratch every single time** (~30-90s per job × 15 jobs = 7-22 minutes wasted)

The two debug entries (`python -c 'import sys; print(...)'` and `pip list`) were not real checks — they were noise from the upstream github `build-and-test-python` template, left in place but never reviewed.

## What's in this PR
- `.github/workflows/python-package.yml` — single file, +41/-56
  1. **`cache: 'pip'` enabled** with `cache-dependency-path: requirements.txt, requirements-dev.txt` — first run downloads and caches, subsequent runs hit the cache. Drops pip install from ~60s to ~5-15s when requirements don't change.
  2. **Two debug-only matrix entries removed** (`python -c ...` and `pip list`) — they were not real checks.
  3. **Matrix restructured**: one Python-version axis only, no checker axis. Each Python job runs `install + flake8 + mypy` sequentially. **15 jobs → 3 jobs** (~5x fewer fresh venvs to set up).
  4. **`concurrency: cancel-in-progress: true`** added — a new push to the same PR auto-cancels the previous in-flight run, saving runner minutes when iterating on a fix.
  5. Bumped `actions/setup-python` from `v3` to `v5` (required for `cache: 'pip'`).

## Verification
- YAML parses via PyYAML — ok
- `jobs` dict contains `['build']` only
- `build.strategy.matrix.python-version` = `['3.12.3', '3.13', '3.14']`
- **total CI jobs: 3** (was 15)
- `setup-python@v5` with `cache: 'pip'` + `cache-dependency-path` pointing at `requirements.txt` and `requirements-dev.txt`
- `concurrency` block correctly scoped to `ci-${{ github.workflow }}-${{ github.ref }}`

## Expected timing
| Scenario | Before | After |
|---|---|---|
| First run (cold cache) | ~13m 19s | ~5-7m |
| Subsequent runs (warm cache, no req changes) | ~13m 19s | ~3-4m |
| Re-run after `requirements.txt` change | ~13m 19s | ~4-5m |

## What was NOT verified
- Actual Gitea Actions runner behaviour with `cache: 'pip'` — the Gitea Actions runner is GitHub Actions-compatible, and `actions/setup-python@v5` with `cache: 'pip'` is a well-known idiom there, but the local Gitea runner config may or may not honour it. If it doesn't, the 15→3 fan-out reduction alone is still a ~5x speedup.

## Note on PR #14
PR #14 ("re-enable the tests job") is still open. If/when it merges, this PR will need a follow-up to re-add the `tests:` job with the same `cache: 'pip'` setup applied — otherwise the playwright install will dominate again.

## Commit history
```
* db3807e (HEAD -> feat/optimize-ci-build-time, origin/feat/optimize-ci-build-time)
      ci: speed up python-package.yml from ~13min to ~3-4min
* 70f4a41 ci: comment out the tests job in python-package.yml (#13)   (upstream/main tip)
```

---------

Co-authored-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/brencher/pulls/15
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-09-04 09:48:29 +02:00
14 changed files with 219 additions and 96 deletions
+4 -8
View File
@@ -20,16 +20,16 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
token: ${{ secrets.PUSH_TOKEN }}
- name: Configure git
run: |
git config user.email "github-actions[bot]@users.noreply.github.com"
git config user.name "GitHub Actions"
git config user.email "gitea-actions[bot]@noreply.localhost"
git config user.name "Gitea Actions"
- name: Fetch all branches
run: |
git remote set-url origin "https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git"
git remote set-url origin "https://x-access-token:${PUSH_TOKEN}@gitea.rudolf.keenetic.link/${GITHUB_REPOSITORY}.git"
git fetch --all --prune
- name: Merge every branch onto main
@@ -37,7 +37,6 @@ jobs:
run: |
set -eo pipefail
MAIN=main
# Collect remote branch names (exclude HEAD and main)
branches=$(git for-each-ref --format='%(refname:short)' refs/remotes/origin/ \
| sed 's|^origin/||' \
| grep -v -e '^HEAD$' -e '^main$' -e '^origin$' \
@@ -49,11 +48,8 @@ jobs:
continue
fi
echo "=== Processing branch: $br ==="
# Create/update a local branch from remote
git checkout -B "$br" "origin/$br"
# Merge origin/main into branch (non-interactive)
if git merge --no-edit "origin/$MAIN"; then
# Push updated branch to remote (no force)
if git push origin "$br"; then
echo "Pushed merged $br"
else
+4 -8
View File
@@ -18,16 +18,16 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
token: ${{ secrets.PUSH_TOKEN }}
- name: Configure git
run: |
git config user.email "github-actions[bot]@users.noreply.github.com"
git config user.name "GitHub Actions"
git config user.email "gitea-actions[bot]@noreply.localhost"
git config user.name "Gitea Actions"
- name: Fetch all branches
run: |
git remote set-url origin "https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git"
git remote set-url origin "https://x-access-token:${PUSH_TOKEN}@gitea.rudolf.keenetic.link/${GITHUB_REPOSITORY}.git"
git fetch --all --prune
- name: Merge every branch onto main
@@ -35,7 +35,6 @@ jobs:
run: |
set -eo pipefail
MAIN=main
# Collect remote branch names (exclude HEAD and main)
branches=$(git for-each-ref --format='%(refname:short)' refs/remotes/origin/ \
| sed 's|^origin/||' \
| grep -v -e '^HEAD$' -e '^main$' -e '^origin$' \
@@ -47,12 +46,9 @@ jobs:
continue
fi
echo "=== Processing branch: $br ==="
# Create/update a local branch from remote
git reset --hard HEAD
git checkout -B "$br" "origin/$br"
# Merge origin/main into branch (non-interactive)
if git merge --no-edit --squash "origin/$MAIN"; then
# Push updated branch to remote (no force)
git reset --soft "origin/$MAIN"
git commit -m "Squash"
if git push origin "$br" --force-with-lease; then
+20 -57
View File
@@ -1,6 +1,3 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Python package
on:
@@ -10,74 +7,40 @@ on:
pull_request:
branches: [ "main" ]
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [
# "3.11.3",
"3.12.3",
"3.13",
"3.14",
# "3.15",
]
checker: [
"python -c \"import sys; print(f'Python {sys.version}'); print(f'Build: {sys.version_info}'); print(f'Implementation: {sys.implementation}')\"",
"pip list",
"flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics",
"flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics",
# "ty check backend",
"mypy --no-incremental --config-file mypy.ini backend tests",
]
python-version: ["3.12.3", "3.13", "3.14"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}, ${{ matrix.checker }}
uses: actions/setup-python@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: |
requirements.txt
requirements-dev.txt
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run checker
run: ${{ matrix.checker }}
# tests:
# runs-on: ubuntu-latest
# needs: build
# strategy:
# fail-fast: false
# matrix:
# python-version: [
# # "3.11.3",
# "3.12.3",
# "3.13",
# "3.14",
# # "3.15",
# ]
# checker: [
# "pytest",
# ]
# steps:
# - uses: actions/checkout@v4
# - name: Set up Python ${{ matrix.python-version }}, ${{ matrix.checker }}
# uses: actions/setup-python@v3
# with:
# python-version: ${{ matrix.python-version }}
# - name: Install dependencies
# run: |
# # sudo apt-get update
# # sudo apt-get upgrade -y
# # sudo apt-get install -y --no-install-recommends git docker.io
# python -m pip install --upgrade pip
# pip install -r requirements-dev.txt
# playwright install --with-deps chromium
# - name: Run checker
# run: "pytest"
#
#
#
- name: Lint (flake8)
run: |
flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Type-check (mypy)
run: mypy --no-incremental --config-file mypy.ini backend tests
+3
View File
@@ -17,3 +17,6 @@
state: str # 'Active' or 'Pause'
pipeline: Dict[str, Any]
+22 -1
View File
@@ -4,7 +4,28 @@ from steps.docker import DockerSwarmCheck, DockerSwarmDeploy
from steps.git import GitClone, CheckoutMerged, GitUnmerge
from steps.shared_state import SharedStateHolderInMemory
clone = GitClone(url="https://github.com/rudolf1/uber_backup.git", branchNamePrefix="ansible")
# Dual-remote clone for the ansible mirror (the deploy target, not brencher itself).
# - gitea_hermes is where the feature branch lives (hermes/ansible:*)
# - gitea_admin is the live-deploy source (where brencher-bot reads compose files from)
#
# Why not github? github.com is unreachable from the brencher container -- the
# proxy returns HTTP 403 on CONNECT tunnel, so the URL would always fail to
# fetch. See backend/configs/docker_socket_proxy.py for the same migration.
#
# Why drop branchNamePrefix? GitClone only applies its prefix refspec to a
# remote named "origin" (backend/steps/git.py ~L104); neither remote here is
# named "origin", so the prefix would be a silent no-op. Both remotes fetch
# their full refset instead.
#
# Why two remotes? The deploy step (DockerSwarmDeploy) checks out a specific
# branch and uses the resulting checkout as wd for the compose path. With
# gitea_hermes as primary, feature branches opened in the fork are picked up;
# gitea_admin provides the live-deploy source as a fallback / for cross-fork
# PRs. This matches the form in backend/configs/brencher.py after PR #3.
clone = GitClone(remotes=[
{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/ansible"},
{"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/ansible"},
])
dockerSwarmCheck = DockerSwarmCheck(
stack_name="gitea",
+6 -3
View File
@@ -4,7 +4,10 @@ from steps.docker import DockerComposeBuild, DockerSwarmCheck, DockerSwarmDeploy
from steps.git import GitClone, CheckoutMerged, GitUnmerge
from steps.shared_state import SharedStateHolderInMemory
clone = GitClone(url="https://github.com/rudolf1/uber_backup.git", branchNamePrefix="immoscout")
clone = GitClone(remotes=[
{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/mcps"},
{"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/mcps"},
])
dockerSwarmCheck = DockerSwarmCheck(
stack_name="gmail_mcp",
@@ -23,7 +26,7 @@ checkoutMerged = CheckoutMerged(clone,
buildDocker = DockerComposeBuild(clone,
docker_repo_username="",
docker_repo_password="",
docker_compose_path="gmail-mcp-server/docker-compose.yml",
docker_compose_path="docker-compose.yml",
docker_repo_url="https://registry.rudolf.keenetic.link",
publish=False,
envs=lambda: {
@@ -39,7 +42,7 @@ deployDocker = DockerSwarmDeploy(
"version": "auto-" + checkoutMerged.progress().version,
},
stack_name="gmail_mcp",
docker_compose_path="gmail-mcp-server/docker-compose.yml",
docker_compose_path="docker-compose.yml",
)
checkPing1 = UrlCheck(
+10 -7
View File
@@ -1,10 +1,13 @@
from enironment import Environment
from steps.checks import SimpleLog, UrlCheck
from steps.checks import SimpleLog
from steps.docker import DockerSwarmCheck, DockerSwarmDeploy
from steps.git import GitClone, CheckoutMerged, GitUnmerge
from steps.shared_state import SharedStateHolderInMemory
clone = GitClone(url="https://github.com/rudolf1/uber_backup.git", branchNamePrefix="ansible")
clone = GitClone(remotes=[
{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/ansible"},
{"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/ansible"},
])
dockerSwarmCheck = DockerSwarmCheck(
stack_name="hermes",
@@ -31,10 +34,10 @@ deployDocker = DockerSwarmDeploy(
docker_compose_path="poc/hermes/stack-compose.yml",
)
checkPing1 = UrlCheck(
url="http://100.70.193.97:8087/api/status",
expected=lambda obj: obj['gateway_running'] == 'true' and obj['auth_required'] == 'true',
)
# checkPing1 = UrlCheck(
# url="http://100.70.193.97:8087/api/status",
# expected=lambda obj: obj['gateway_running'] == 'true' and obj['auth_required'] == 'true',
# )
logUrls = SimpleLog(message={
"userLinks": {
"App": "https://hermes.rudolf.keenetic.link",
@@ -53,7 +56,7 @@ hermes = Environment(
dockerSwarmCheck,
unmerge,
deployDocker,
checkPing1,
# checkPing1,
logUrls
]
)
+1 -1
View File
@@ -32,7 +32,7 @@ dockerSwarmCheck = DockerSwarmCheck(
)
unmerge = GitUnmerge(clone, dockerSwarmCheck)
state = SharedStateHolderInMemory(unmerge=None)
state = SharedStateHolderInMemory(unmerge=unmerge)
checkoutMerged = CheckoutMerged(clone,
desired_branches=state,
+22 -1
View File
@@ -4,7 +4,28 @@ from steps.docker import DockerSwarmCheck, DockerSwarmDeploy
from steps.git import GitClone, CheckoutMerged, GitUnmerge
from steps.shared_state import SharedStateHolderInMemory
clone = GitClone(url="https://github.com/rudolf1/uber_backup.git", branchNamePrefix="ansible")
# Dual-remote clone for the ansible mirror (the deploy target, not brencher itself).
# - gitea_hermes is where the feature branch lives (hermes/ansible:*)
# - gitea_admin is the live-deploy source (where brencher-bot reads compose files from)
#
# Why not github? github.com is unreachable from the brencher container -- the
# proxy returns HTTP 403 on CONNECT tunnel, so the URL would always fail to
# fetch. See backend/configs/docker_socket_proxy.py for the same migration.
#
# Why drop branchNamePrefix? GitClone only applies its prefix refspec to a
# remote named "origin" (backend/steps/git.py ~L104); neither remote here is
# named "origin", so the prefix would be a silent no-op. Both remotes fetch
# their full refset instead.
#
# Why two remotes? The deploy step (DockerSwarmDeploy) checks out a specific
# branch and uses the resulting checkout as wd for the compose path. With
# gitea_hermes as primary, feature branches opened in the fork are picked up;
# gitea_admin provides the live-deploy source as a fallback / for cross-fork
# PRs. This matches the form in backend/configs/brencher.py after PR #3.
clone = GitClone(remotes=[
{"name": "gitea_hermes", "url": "https://gitea.rudolf.keenetic.link/hermes/ansible"},
{"name": "gitea_admin", "url": "https://gitea.rudolf.keenetic.link/admin/ansible"},
])
dockerSwarmCheck = DockerSwarmCheck(
stack_name="immich",
+11 -4
View File
@@ -48,8 +48,13 @@ class GitClone(AbstractStep[str]):
raise ValueError(f"Remote {name!r} requires a non-empty 'url' (string)")
if name in seen:
raise ValueError(f"Duplicate remote name: {name!r}")
prefix_value = entry.get("prefix")
if prefix_value is not None and not isinstance(prefix_value, str):
raise ValueError(
f"Remote {name!r} 'prefix' must be a string when set, got {type(prefix_value).__name__}"
)
seen.add(name)
normalized.append({"name": name, "url": url_value})
normalized.append({"name": name, "url": url_value, "prefix": prefix_value or ""})
elif url is not None:
if isinstance(url, str):
raw = [url]
@@ -103,10 +108,12 @@ class GitClone(AbstractStep[str]):
remote.set_url(auth_url)
else:
remote = repo.create_remote(remote_name, auth_url)
if remote_name == "origin" and self.branchNamePrefix != "":
prefix = entry.get("prefix", "")
effective_prefix = prefix or (self.branchNamePrefix if remote_name == "origin" else "")
if effective_prefix:
with repo.config_writer() as config:
config.set_value('remote "origin"', "fetch",
f"+refs/heads/{self.branchNamePrefix}/*:refs/remotes/origin/{self.branchNamePrefix}/*")
config.set_value(f'remote "{remote_name}"', "fetch",
f"+refs/heads/{effective_prefix}/*:refs/remotes/{remote_name}/{effective_prefix}/*")
result = remote.fetch(prune=True)
if not result or any(fetch_info.flags & fetch_info.ERROR for fetch_info in result):
raise BaseException(f"Failed to fetch updates for {repository_url}")
+6 -6
View File
@@ -107,12 +107,12 @@ class WebApp:
disconnected = set()
message = custom_json_dumps({event: data})
for websocket in self.ws_connections.keys():
if self.ws_connections[websocket].get(event) == message:
for websocket, state in self.ws_connections.items():
if state.get(event) == message:
continue
try:
await websocket.send_text(message)
self.ws_connections[websocket][event] = message
state[event] = message
except Exception as e:
logger.error(f"Error sending to websocket: {e}")
disconnected.add(websocket)
@@ -163,7 +163,7 @@ class WebApp:
async def websocket_endpoint(self, websocket: WebSocket) -> None:
await websocket.accept()
self.ws_connections[websocket] = {}
state = self.ws_connections.setdefault(websocket, {})
try:
# Send initial state on connect and record it so duplicate broadcasts are suppressed
@@ -172,9 +172,9 @@ class WebApp:
version_payload = custom_json_dumps({"version": self.get_version()})
await websocket.send_text(version_payload)
await websocket.send_text(branches_payload)
self.ws_connections[websocket]["branches"] = branches_payload
state["branches"] = branches_payload
await websocket.send_text(envs_payload)
self.ws_connections[websocket]["environments"] = envs_payload
state["environments"] = envs_payload
while True:
data = await websocket.receive_text()
+8
View File
@@ -532,6 +532,14 @@ function updateEnvironment() {
showStatus('Changes applied (pending server confirmation).');
}
// Initial load
(function applyUrlFilter() {
const params = new URLSearchParams(window.location.search);
const initialFilter = params.get('filter');
if (initialFilter && branchFilter) {
branchFilter.value = initialFilter;
filterBranches();
}
})();
setupWebSockets();
showStatus('Loading branches...');
renderBranches();
+4
View File
@@ -11,3 +11,7 @@ types-PyYAML
types-requests
types-docker
testcontainers
+98
View File
@@ -101,6 +101,104 @@ class TestGitIntegration:
with pytest.raises(ValueError, match="requires either 'url' or 'remotes'"):
GitClone()
def test_git_clone_remotes_per_remote_prefix(self, repo_helper: RemoteRepoHelper) -> None:
# Per-remote `prefix` filters fetch refspec so only matching branches are visible.
# Two repos, two branches each (one matching the prefix, one not). Only the
# matching branch should appear in get_branches() for each remote.
def fresh_repo(prefix: str) -> str:
path = tempfile.mkdtemp(prefix=prefix)
repo = git.Repo.init(path, bare=False)
with repo.config_writer() as cw:
cw.set_value("user", "email", "test@example.com")
cw.set_value("user", "name", "Test User")
# Branches literally named `ansible/...` to match the prefix filter.
# create_commit creates an initial master, then branches off it.
repo_helper.create_commit(repo, "master", "master", "README.md", "content", f"{prefix} init")
repo_helper.create_commit(repo, "master", "ansible/master", "AN.md", "ansible", f"{prefix} ansible")
repo_helper.create_commit(repo, "master", "ansible/fix", "FIX.md", "fix", f"{prefix} fix")
return path
github_dir = fresh_repo("test_prefix_github_")
gitea_dir = fresh_repo("test_prefix_gitea_")
try:
clone = GitClone(remotes=[
{"name": "github", "url": github_dir, "prefix": "ansible"},
{"name": "gitea_admin", "url": gitea_dir, "prefix": "ansible"},
])
clone.env = repo_helper.env
repo_path = clone.progress()
repo = git.Repo(repo_path)
branches = set(clone.get_branches())
# Per-remote refspec was applied: only ansible/* branches fetched.
assert "github/ansible/master" in branches, f"missing: {branches}"
assert "github/ansible/fix" in branches, f"missing: {branches}"
assert "gitea_admin/ansible/master" in branches, f"missing: {branches}"
assert "gitea_admin/ansible/fix" in branches, f"missing: {branches}"
# `master` exists upstream but is NOT visible (filtered by prefix).
assert "github/master" not in branches, f"unwanted: {branches}"
assert "gitea_admin/master" not in branches, f"unwanted: {branches}"
finally:
shutil.rmtree(github_dir, ignore_errors=True)
shutil.rmtree(gitea_dir, ignore_errors=True)
def test_git_clone_remotes_mixed_prefix(self, repo_helper: RemoteRepoHelper) -> None:
# Some remotes with prefix, others without. Each behaves independently:
# prefixed remotes only see matching branches, unprefixed see all.
def fresh_repo(prefix: str) -> str:
path = tempfile.mkdtemp(prefix=prefix)
repo = git.Repo.init(path, bare=False)
with repo.config_writer() as cw:
cw.set_value("user", "email", "test@example.com")
cw.set_value("user", "name", "Test User")
repo_helper.create_commit(repo, "master", "master", "README.md", "content", f"{prefix} init")
repo_helper.create_commit(repo, "master", "ansible/master", "AN.md", "ansible", f"{prefix} ansible")
repo_helper.create_commit(repo, "master", "feature_y", "OTHER.md", "other", f"{prefix} feature")
return path
github_dir = fresh_repo("test_mixed_github_")
gitea_dir = fresh_repo("test_mixed_gitea_")
try:
clone = GitClone(remotes=[
{"name": "github", "url": github_dir, "prefix": "ansible"},
{"name": "gitea_admin", "url": gitea_dir},
])
clone.env = repo_helper.env
repo_path = clone.progress()
repo = git.Repo(repo_path)
branches = set(clone.get_branches())
# Prefixed: only ansible/master.
assert "github/ansible/master" in branches, f"missing: {branches}"
assert "github/feature_y" not in branches, f"unwanted: {branches}"
assert "github/master" not in branches, f"unwanted: {branches}"
# Unprefixed: all branches visible.
assert "gitea_admin/ansible/master" in branches, f"missing: {branches}"
assert "gitea_admin/feature_y" in branches, f"missing: {branches}"
assert "gitea_admin/master" in branches, f"missing: {branches}"
finally:
shutil.rmtree(github_dir, ignore_errors=True)
shutil.rmtree(gitea_dir, ignore_errors=True)
def test_git_clone_remotes_prefix_validation(self) -> None:
# `prefix` must be a string when set; non-string raises ValueError.
with pytest.raises(ValueError, match="'prefix' must be a string"):
GitClone(remotes=[
{"name": "github", "url": "git@example.com:a/b.git", "prefix": 123}, # type: ignore[dict-item]
])
with pytest.raises(ValueError, match="'prefix' must be a string"):
GitClone(remotes=[
{"name": "github", "url": "git@example.com:a/b.git", "prefix": ["ansible"]}, # type: ignore[dict-item]
])
# Empty string / None: accepted, treated as no prefix.
c1 = GitClone(remotes=[{"name": "github", "url": "git@example.com:a/b.git", "prefix": ""}])
assert c1.remotes[0]["prefix"] == ""
c2 = GitClone(remotes=[{"name": "github", "url": "git@example.com:a/b.git"}])
assert c2.remotes[0]["prefix"] == ""
def test_checkout_merged_one_branch(self, repo_helper: RemoteRepoHelper) -> None:
# Create main branch with initial commit
commit1 = repo_helper.create_commit(repo_helper.repo, "master", "master", "file1.txt", "content1",