Author SHA1 Message Date
copilot-swe-agent[bot]andrudolf1 0e43c0fd13 harden playwright test config and assertions
Python package / build (flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics, 3.12.3) (pull_request) Successful in 36s
Python package / build (flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics, 3.13) (pull_request) Successful in 34s
Python package / build (flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics, 3.14) (pull_request) Successful in 36s
Python package / build (flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics, 3.12.3) (pull_request) Successful in 33s
Python package / build (flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics, 3.13) (pull_request) Successful in 35s
Python package / build (flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics, 3.14) (pull_request) Successful in 36s
Python package / build (mypy --no-incremental --config-file mypy.ini backend tests, 3.12.3) (pull_request) Successful in 1m43s
Python package / build (mypy --no-incremental --config-file mypy.ini backend tests, 3.13) (pull_request) Successful in 1m54s
Python package / build (mypy --no-incremental --config-file mypy.ini backend tests, 3.14) (pull_request) Successful in 1m59s
Python package / build (pip list, 3.12.3) (pull_request) Successful in 31s
Python package / build (pip list, 3.13) (pull_request) Successful in 33s
Python package / build (pip list, 3.14) (pull_request) Successful in 31s
Python package / build (python -c "import sys; print(f'Python {sys.version}'); print(f'Build: {sys.version_info}'); print(f'Implementation: {sys.implementation}')", 3.12.3) (pull_request) Successful in 32s
Python package / build (python -c "import sys; print(f'Python {sys.version}'); print(f'Build: {sys.version_info}'); print(f'Implementation: {sys.implementation}')", 3.13) (pull_request) Successful in 36s
Python package / build (python -c "import sys; print(f'Python {sys.version}'); print(f'Build: {sys.version_info}'); print(f'Implementation: {sys.implementation}')", 3.14) (pull_request) Successful in 31s
Python package / tests (pytest, 3.12.3) (pull_request) Canceled after 0s
Python package / tests (pytest, 3.13) (pull_request) Canceled after 0s
Python package / tests (pytest, 3.14) (pull_request) Canceled after 0s
Agent-Logs-Url: https://github.com/rudolf1/brencher/sessions/d019d010-3ca0-4a73-8826-620eef58edab

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-05-19 20:08:12 +00:00
copilot-swe-agent[bot]andrudolf1 ab01fca841 stabilize dry-run playwright integration
Agent-Logs-Url: https://github.com/rudolf1/brencher/sessions/d019d010-3ca0-4a73-8826-620eef58edab

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-05-19 20:06:11 +00:00
copilot-swe-agent[bot]andrudolf1 b2782ee220 clarify async taskgroup behavior
Agent-Logs-Url: https://github.com/rudolf1/brencher/sessions/9e719b58-ad2a-4eca-99bf-43c750f71a72

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-05-19 19:36:56 +00:00
copilot-swe-agent[bot]andrudolf1 1acea85934 add async shutdown cleanup and review fixes
Agent-Logs-Url: https://github.com/rudolf1/brencher/sessions/9e719b58-ad2a-4eca-99bf-43c750f71a72

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-05-19 19:35:59 +00:00
copilot-swe-agent[bot]andrudolf1 6f3dd6ebd8 refactor app concurrency to asyncio taskgroup
Agent-Logs-Url: https://github.com/rudolf1/brencher/sessions/9e719b58-ad2a-4eca-99bf-43c750f71a72

Co-authored-by: rudolf1 <5525769+rudolf1@users.noreply.github.com>
2026-05-19 19:34:46 +00:00
copilot-swe-agent[bot] efe49119a3 Initial plan 2026-05-19 19:02:30 +00:00
admin 2a14d70fab playwright tests. refactor backend (#79)
* refactor backend
* Playwright test
2026-05-10 20:14:50 +02:00
15 changed files with 959 additions and 519 deletions
@@ -0,0 +1,53 @@
---
name: playwright-brencher
description: "Write or refactor Playwright tests for brencher UI. Use when creating page helpers, extracting browser code from tests, or enforcing set_/verify_ method naming."
---
# Playwright Brencher Conventions
Use this skill when writing or refactoring Playwright tests in this repository.
## Required API Shape
Create and use a context manager in this style:
```python
with brencher_page(APP_URL) as page:
page.verify_dry_run_on()
page.set_dry_run_off()
page.verify_dry_run_off()
```
## Naming Rules
1. Action methods that change UI/application state must use `set_...` names.
2. Verification methods (assertions, waits, checks) must use `verify_...` names.
3. Keep low-level selector helpers private (prefix `_`).
4. Prefer one page-object class per screen/area (`BrencherPage` for the current app page).
## Structure Rules
1. Put shared Playwright helpers in `tests/playwright_helper.py`.
2. Keep tests focused on scenario flow; move DOM selectors and waiting logic into page helper methods.
3. Use context manager functions to own Playwright/browser lifecycle.
4. Inline trivial one-use logic where it improves readability and does not duplicate behavior.
## Method Guidance
Recommended style for page object methods:
- `set_*`: click/toggle/type actions.
- `verify_*`: assert visible state, button title/text, or wait until target state is reached.
Example method names:
- `set_dry_run_off`
- `verify_dry_run_on`
- `verify_dry_run_off`
## Review Checklist
1. Test uses `with brencher_page(...) as page` style.
2. No direct `sync_playwright()` block remains in the test body when helper abstraction is intended.
3. Action and verification methods follow `set_` / `verify_` naming.
4. Selector details are encapsulated in private helper methods.
+36 -4
View File
@@ -12,13 +12,12 @@ on:
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [
# "3.11",
# "3.11.3",
"3.12.3",
"3.13",
"3.14",
@@ -31,7 +30,6 @@ jobs:
"flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics",
# "ty check backend",
"mypy --no-incremental --config-file mypy.ini backend tests",
"pytest",
]
steps:
@@ -43,8 +41,42 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest ty
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run checker
run: ${{ matrix.checker }}
tests:
runs-on: ubuntu-latest
needs: build
strategy:
fail-fast: false
matrix:
python-version: [
# "3.11.3",
"3.12.3",
"3.13",
"3.14",
# "3.15",
]
checker: [
"pytest",
]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}, ${{ matrix.checker }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
# sudo apt-get update
# sudo apt-get upgrade -y
# sudo apt-get install -y --no-install-recommends git docker.io
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
playwright install --with-deps chromium
- name: Run checker
run: "pytest"
+25
View File
@@ -0,0 +1,25 @@
FROM nginx:1.27-alpine
# Configure nginx to listen on 5001 to match container port expectations.
RUN sed -i 's/listen 80;/listen 5001;/' /etc/nginx/conf.d/default.conf
# Build a custom landing page directly in the image.
RUN set -eux; \
{ \
echo '<!doctype html>'; \
echo '<html lang="en">'; \
echo '<head>'; \
echo ' <meta charset="utf-8">'; \
echo ' <meta name="viewport" content="width=device-width, initial-scale=1">'; \
echo ' <title>Brencher Test App</title>'; \
echo '</head>'; \
echo '<body>'; \
echo ' <h1>Nginx Is Running</h1>'; \
echo ' <p>Custom landing page for the Brencher test app container.</p>'; \
echo ' <p>Port: 5001</p>'; \
echo ' <p>Git revision: unknown</p>'; \
echo '</body>'; \
echo '</html>'; \
} > /usr/share/nginx/html/index.html
EXPOSE 5001
+142 -390
View File
@@ -1,22 +1,16 @@
import asyncio
import json
import logging
import os
import signal
import sys
import threading
import traceback
from dataclasses import asdict, replace
from typing import Any, Dict, List, Optional, TypeVar, Tuple
from typing import Any, Callable, Dict, List, Optional
from discover_envs import build_environments, parse_arguments
from utils import sigchld_handler
from dotenv import load_dotenv
from fastapi import FastAPI, Response, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse
from enironment import Environment, wrap_in_cached, SharedStateHolder, SharedStateConflictError, SharedState, get_step
from processing import reset_caches
from secondary import SecondaryManager
from enironment import Environment, wrap_in_cached, SharedStateHolder, get_step
from steps.git import GitClone
from steps.step import CachingStep
@@ -29,403 +23,151 @@ load_dotenv("local.env")
load_dotenv('/run/secrets/brencher-secrets')
class DataclassJSONEncoder(json.JSONEncoder):
def default(self, o: Any) -> Any:
if hasattr(o, '__dataclass_fields__'):
return asdict(o)
if isinstance(o, BaseException):
return str(o)
try:
return super().default(o)
except TypeError:
return str(o)
class App:
def __init__(self, environments: Dict[str, Environment]) -> None:
self.environments: Dict[str, Environment] = {id: wrap_in_cached(e) for id, e in environments.items()}
self.environment_update_event = asyncio.Event()
self.environment_update_version = 0
self.emit_callback: Optional[Callable[[], None]] = None
self.reset_requested = False
self.shutdown_event = asyncio.Event()
self.web_app: Any = None
def custom_json_dumps(obj: Any) -> str:
return json.dumps(obj, cls=DataclassJSONEncoder)
def get_local_envs_to_emit(self) -> Dict[str, Dict[str, Any]]:
env_dtos: Dict[str, Dict[str, Any]] = {}
for env in self.environments.values():
pipeline_state: List[Dict[str, Any]] = []
for r in env.pipeline:
try:
if isinstance(r, CachingStep):
result = r._result
else:
result = r.progress()
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
FRONTEND_DIR = os.path.join(os.path.dirname(__file__), '../frontend')
# In-memory state
environments: Dict[str, Environment] = {}
state_lock = threading.Lock()
# Active WebSocket connections
ws_connections: Dict[WebSocket, Dict[str, Any]] = {}
# Event loop reference for thread-safe async calls
_event_loop: Optional[asyncio.AbstractEventLoop] = None
environment_update_event = threading.Event()
# --- Static File Serving ---
@app.get("/")
async def serve_index() -> FileResponse:
return FileResponse(os.path.join(FRONTEND_DIR, 'index.html'))
@app.get("/state")
async def serve_state() -> Response:
return Response(content=custom_json_dumps(get_global_envs_to_emit()), media_type="application/json")
@app.get("/branches")
async def serve_branches_route() -> Response:
return Response(content=custom_json_dumps(get_global_branches_to_emit()), media_type="application/json")
@app.get("/{path:path}")
async def serve_static(path: str) -> FileResponse:
file_path = os.path.join(FRONTEND_DIR, path)
if os.path.exists(file_path) and os.path.isfile(file_path):
return FileResponse(file_path)
return FileResponse(os.path.join(FRONTEND_DIR, 'index.html'))
# --- Utility Functions ---
T = TypeVar('T')
def merge_dicts(a: Dict[str, T], b: Dict[str, T]) -> Dict[str, T]:
result: Dict[str, T] = {}
for k in (a.keys() | b.keys()):
if k not in a or k not in b:
result[k] = a.get(k, b.get(k)) # type: ignore[assignment]
continue
if (
isinstance(a[k], dict)
and isinstance(b[k], dict)
):
result[k] = merge_dicts(a[k], b[k]) # type: ignore[arg-type,assignment]
else:
result[k] = b[k]
return result
def get_local_envs_to_emit() -> Dict[str, Dict[str, Any]]:
env_dtos: Dict[str, Dict[str, Any]] = {}
for env in environments.values():
pipeline_state: List[Dict[str, Any]] = []
for r in env.pipeline:
try:
if isinstance(r, CachingStep):
result = r._result
else:
result = r.progress()
if isinstance(result, BaseException):
stack = traceback.format_exception(type(result), result, result.__traceback__)
if isinstance(result, BaseException):
stack = traceback.format_exception(type(result), result, result.__traceback__)
pipeline_state.append({
"name": r.name,
"status": [str(result), stack],
"error": True,
"is_running": True,
})
else:
pipeline_state.append({
"name": r.name,
"status": result,
"is_running": False,
})
except BaseException as e:
stack = traceback.format_exception(type(e), e, e.__traceback__)
pipeline_state.append({
"name": r.name,
"status": [str(result), stack],
"status": [str(e), stack],
"error": True,
"is_running": True,
})
else:
pipeline_state.append({
"name": r.name,
"status": result,
"is_running": False,
})
env_dtos[env.id] = {'id': env.id }
env_dtos[env.id]['pipeline'] = pipeline_state
try:
shared_state = env.state.progress()
env_dtos[env.id]['branches'] = shared_state.branches
env_dtos[env.id]['branches_token'] = shared_state.token
env_dtos[env.id]['dry'] = shared_state.dry
except BaseException:
pass
return env_dtos
def get_local_branches_to_emit(self) -> Dict[str, Dict[str, List[Any]]]:
branches: Dict[str, Dict[str, List[Any]]] = {}
for k, env in self.environments.items():
branches[k] = {}
try:
step = get_step(env.pipeline, GitClone)
branches[k] = {**step.get_branches()}
except BaseException as e:
stack = traceback.format_exception(type(e), e, e.__traceback__)
pipeline_state.append({
"name": r.name,
"status": [str(e), stack],
"error": True,
"is_running": True,
})
env_dtos[env.id] = {'id': env.id }
env_dtos[env.id]['pipeline'] = pipeline_state
try:
shared_state = env.state.progress()
env_dtos[env.id]['branches'] = shared_state.branches
env_dtos[env.id]['branches_token'] = shared_state.token
env_dtos[env.id]['dry'] = shared_state.dry
except BaseException:
pass
return env_dtos
logger.error(f"Error fetching branches for environment {env.id}: {str(e)}\n{''.join(stack)}")
return branches
def get_global_envs_to_emit() -> Any:
global environments
local_envs = get_local_envs_to_emit()
merged: Dict[str, Any] = local_envs
for connector in secondaryManager or []:
v = connector.environments
common_keys = set(merged.keys()) & set(v.keys())
if len(common_keys) > 0:
_schedule_async(broadcast_error({'message': f"Conflict: both master and secondary have environment with id {common_keys}"}))
merged = merge_dicts(merged, v)
return merged
def get_local_branches_to_emit() -> Dict[str, Dict[str, List[Any]]]:
global environments
branches: Dict[str, Dict[str, List[Any]]] = {}
for k, env in environments.items():
branches[k] = {}
try:
step = get_step(env.pipeline, GitClone)
branches[k] = {**step.get_branches()}
except BaseException as e:
stack = traceback.format_exception(type(e), e, e.__traceback__)
logger.error(f"Error fetching branches for environment {env.id}: {str(e)}\n{''.join(stack)}")
return branches
def get_global_branches_to_emit() -> Dict[str, Dict[str, List[Any]]]:
local_branches: Dict[str, Dict[str, List[Any]]] = get_local_branches_to_emit()
merged: Dict[str, Dict[str, List[Any]]] = local_branches
for connector in secondaryManager or []:
merged = merge_dicts(merged, connector.branches)
return merged
# --- Async Helper for Calling from Sync Threads ---
def _schedule_async(coro: Any) -> None:
"""Schedule a coroutine from a sync thread using the saved event loop."""
if _event_loop is not None and _event_loop.is_running():
asyncio.run_coroutine_threadsafe(coro, _event_loop)
# --- WebSocket Broadcasting Functions ---
async def broadcast(event: str, data: Any) -> None:
"""Broadcast a message to all connected WebSocket clients that have not seen this data yet."""
disconnected = set()
message = custom_json_dumps({event: data})
for websocket in ws_connections.keys():
if ws_connections[websocket].get(event) == message:
continue
try:
await websocket.send_text(message)
ws_connections[websocket][event] = message
except Exception as e:
logger.error(f"Error sending to websocket: {e}")
disconnected.add(websocket)
for ws in disconnected:
ws_connections.pop(ws, None)
async def broadcast_all() -> None:
await broadcast("branches", get_global_branches_to_emit())
await broadcast("environments", get_global_envs_to_emit())
async def broadcast_branches(data: Any) -> None:
await broadcast("branches", data)
async def broadcast_environments(data: Any) -> None:
await broadcast("environments", data)
async def broadcast_error(data: Any) -> None:
await broadcast("error", data)
# --- Single WebSocket Endpoint ---
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket) -> None:
await websocket.accept()
ws_connections[websocket] = {}
try:
# Send initial state on connect and record it so duplicate broadcasts are suppressed
branches_payload = custom_json_dumps({"branches": get_global_branches_to_emit()})
envs_payload = custom_json_dumps({"environments": get_global_envs_to_emit()})
await websocket.send_text(branches_payload)
ws_connections[websocket]["branches"] = branches_payload
await websocket.send_text(envs_payload)
ws_connections[websocket]["environments"] = envs_payload
while True:
data = await websocket.receive_text()
message = json.loads(data)
if "update" in message:
update_data = message.get("update") or {}
logger.info(f"Received environment update: {update_data}")
if secondaryManager:
await secondaryManager.send({"update": update_data})
id = update_data.get('id', '')
if id == '':
reset_caches(list(environments.values()))
elif id not in environments.keys() and id not in { j for it in secondaryManager or [] for j in it.environments.keys()}:
logger.warning(f"Received update for unknown environment id {id}")
continue
elif id in environments.keys():
env = environments.get(id, None)
expected_token = update_data.get('token', '')
if 'branches' in update_data:
if not env:
raise RuntimeError(f"Unknown env {update_data.get('id', '')}")
env.state.set_branches(update_data.get('branches', []), expected_token=expected_token)
if 'dry' in update_data:
if not env:
raise RuntimeError(f"Unknown env {update_data.get('id', '')}")
env.state.set_dry(bool(update_data['dry']), expected_token)
if env:
p = get_step(env.pipeline, type(env.state))
if isinstance(p, CachingStep):
p.reset()
logger.info(f"Updated environment {env.id} branches to {update_data.get('branches')}, dry={update_data.get('dry')}")
environment_update_event.set()
await broadcast_environments(get_global_envs_to_emit())
except WebSocketDisconnect as e:
ws_connections.pop(websocket, None)
except Exception as e:
logger.error(f"WebSocket error: {e}:{traceback.format_exc()}")
await broadcast_error({'message': f'{e}:{traceback.format_exc()}'})
# await websocket.close()
# ws_connections.pop(websocket, None)
# --- SIGCHLD Handler ---
def sigchld_handler(signum: int, frame: Any) -> None:
"""Reap zombie processes."""
while True:
try:
pid, status = os.waitpid(-1, os.WNOHANG)
if pid == 0:
break
except ChildProcessError:
break
# --- Startup Event ---
secondaryManager: Optional[SecondaryManager] = None
@app.on_event("startup")
async def startup_event() -> None:
global _event_loop, secondaryManager
_event_loop = asyncio.get_event_loop()
secondaryManager = SecondaryManager(
urls = os.getenv("SECONDARY_BRENCHER", ""),
on_update=lambda: broadcast_all(),
)
# --- App Class (used by CLI and integration tests) ---
class App:
def __init__(self, cli_env_ids_str: str, dry_run: bool = False) -> None:
global environments
signal.signal(signal.SIGCHLD, sigchld_handler)
import configs.brencher
import configs.brencher2
import configs.brencher_local2
import configs.brencher_local1
import configs.torrserv_proxy
import configs.immich
import configs.registry
import configs.gmail_mcp
environments_l: List[Environment] = [
configs.brencher.brencher,
configs.brencher2.brencher2,
configs.brencher_local2.brencher_local2,
configs.brencher_local1.brencher_local1,
configs.torrserv_proxy.torrserv_proxy,
configs.immich.immich,
configs.registry.registry,
configs.gmail_mcp.gmail_mcp,
]
environments = {e.id: e for e in environments_l}
logger.info(f"cli_env_ids {cli_env_ids_str}")
if len(cli_env_ids_str) > 0 and cli_env_ids_str[0] == '-':
cli_env_ids = cli_env_ids_str[1:].split(',')
cli_env_ids = [x for x in cli_env_ids if len(x) > 0]
logger.info(f"cli_env_ids (minus) {cli_env_ids}")
if cli_env_ids and len(cli_env_ids) > 0:
environments = {k: e for k, e in environments.items() if k not in cli_env_ids}
else:
cli_env_pairs = {x[0]: x[1] if len(x) > 1 else None for x in [x.split(":")
for x in cli_env_ids_str.split(',')
if len(x) > 0
]
}
logger.info(f"cli_env_ids {cli_env_pairs}")
if cli_env_pairs and len(cli_env_pairs) > 0:
environments = {k: e for k, e in environments.items() if k in cli_env_pairs.keys()}
for k, v in cli_env_pairs.items():
if v is not None:
if k in environments:
env = environments[k]
for step in env.pipeline:
resolve_step = step
if isinstance(step, CachingStep):
resolve_step = step._step
if isinstance(resolve_step, SharedStateHolder):
resolve_step.set_branches([(v, 'HEAD')])
logger.info(f"Overriding environment {k} branches to {(v, 'HEAD')}")
else:
logger.warning(f"Environment {k} not found to override branches")
else:
logger.info(f"No branch override for environment {k}")
if dry_run:
for id, e in environments.items():
e.state.set_dry(True)
logger.info(f"Resulting profiles {environments.keys()}")
environments = {id: wrap_in_cached(e) for id, e in environments.items()}
def processing_thread(self) -> None:
while True:
async def processing_loop(self) -> None:
while not self.shutdown_event.is_set():
import processing
def emit_envs() -> None:
_schedule_async(broadcast_branches(get_global_branches_to_emit()))
_schedule_async(broadcast_environments(get_global_envs_to_emit()))
emit = self.emit_callback or (lambda: None)
update_version = self.environment_update_version
with state_lock:
logger.info(f"Processing")
if processing.process_all_jobs(list(environments.values()), lambda: emit_envs()):
environment_update_event.wait(timeout=1 * 5)
else:
environment_update_event.wait(timeout=1 * 60)
environment_update_event.clear()
logger.info("Processing")
if self.reset_requested:
await asyncio.to_thread(processing.reset_caches, list(self.environments.values()))
self.reset_requested = False
has_error = await asyncio.to_thread(processing.process_all_jobs, list(self.environments.values()), emit)
if self.shutdown_event.is_set():
return
# Skip the wait when a newer update arrived while the current processing pass was running.
if self.environment_update_version != update_version:
continue
timeout = 5 if has_error else 60
try:
await asyncio.wait_for(self.environment_update_event.wait(), timeout=timeout)
except asyncio.TimeoutError:
pass
self.environment_update_event.clear()
def run(self) -> None:
processing = threading.Thread(target=self.processing_thread)
processing.daemon = True
processing.start()
def notify_environment_update(self) -> None:
self.environment_update_version += 1
self.environment_update_event.set()
def request_reset(self) -> None:
self.reset_requested = True
def stop(self) -> None:
self.shutdown_event.set()
self.notify_environment_update()
if self.web_app is not None:
self.web_app.stop()
async def runAsync(self) -> None:
await self.processing_loop()
async def runHeadlessAsync(self) -> None:
await self.processing_loop()
def runHeadless(self) -> None:
"""Run the processing loop until cancelled; blocks forever."""
asyncio.run(self.runHeadlessAsync())
async def _run_web_server(self, web_app: Any) -> None:
"""Raise a sentinel exception when the server stops so TaskGroup cancels processing."""
await web_app.start_async()
raise _WebServerStopped()
async def runWebAsync(self, port: int) -> None:
import web
web_app = web.WebApp(core=self, port=port)
self.web_app = web_app
self.emit_callback = web_app.emit_envs
try:
# Keep processing and web serving coupled so shutdown or fatal failure tears down the whole app.
async with asyncio.TaskGroup() as task_group:
task_group.create_task(self.processing_loop())
task_group.create_task(self._run_web_server(web_app))
except* _WebServerStopped:
pass
finally:
self.emit_callback = None
self.web_app = None
def runWeb(self, port: int) -> None:
self.run()
import uvicorn
uvicorn.run(app, host='0.0.0.0', port=port, log_level='info')
asyncio.run(self.runWebAsync(port))
def run(self) -> None:
asyncio.run(self.runAsync())
class _WebServerStopped(Exception):
pass
if __name__ == '__main__':
cli_env_ids_list = sys.argv[1:]
@@ -435,5 +177,15 @@ if __name__ == '__main__':
else:
cli_env_ids_str = cli_env_ids_list[0]
app1 = App(cli_env_ids_str, 'dry' in sys.argv[1:])
app1.runWeb(port=(5007 if 'noweb' in sys.argv[1:] else 5001))
signal.signal(signal.SIGCHLD, sigchld_handler)
environments = build_environments(cli_env_ids_str)
if 'dry' in sys.argv[1:]:
for e in environments.values():
e.state.set_dry(True)
app1 = App(environments)
if 'noweb' in sys.argv[1:]:
app1.runHeadless()
else:
app1.runWeb(5001)
+71
View File
@@ -0,0 +1,71 @@
import importlib
import logging
import pkgutil
from dataclasses import dataclass, field
from typing import Dict, List, Optional
from enironment import Environment, SharedStateHolder
from steps.step import CachingStep
logger = logging.getLogger(__name__)
@dataclass
class ParsedEnvArgs:
excludes: List[str] = field(default_factory=list)
includes: Dict[str, Optional[str]] = field(default_factory=dict)
def parse_arguments(cli_env_ids_str: str) -> ParsedEnvArgs:
logger.info(f"cli_env_ids {cli_env_ids_str}")
if len(cli_env_ids_str) > 0 and cli_env_ids_str[0] == '-':
excludes = [x for x in cli_env_ids_str[1:].split(',') if len(x) > 0]
logger.info(f"cli_env_ids (minus) {excludes}")
return ParsedEnvArgs(excludes=excludes)
includes: Dict[str, Optional[str]] = {
x[0]: x[1] if len(x) > 1 else None
for x in [x.split(":") for x in cli_env_ids_str.split(',') if len(x) > 0]
}
logger.info(f"cli_env_ids {includes}")
return ParsedEnvArgs(includes=includes)
def _discover_environments() -> Dict[str, Environment]:
import configs
found: Dict[str, Environment] = {}
seen: set[int] = set()
for module_info in pkgutil.iter_modules(configs.__path__, prefix=f"{configs.__name__}."):
module = importlib.import_module(module_info.name)
for value in vars(module).values():
if isinstance(value, Environment) and id(value) not in seen:
seen.add(id(value))
found[value.id] = value
return found
def build_environments(cli_env_ids_str: str) -> Dict[str, Environment]:
environments: Dict[str, Environment] = _discover_environments()
args: ParsedEnvArgs = parse_arguments(cli_env_ids_str)
if args.excludes:
environments = {k: e for k, e in environments.items() if k not in args.excludes}
elif args.includes:
environments = {k: e for k, e in environments.items() if k in args.includes}
for k, v in args.includes.items():
if v is None:
logger.info(f"No branch override for environment {k}")
continue
if k not in environments:
logger.warning(f"Environment {k} not found to override branches")
continue
env = environments[k]
for step in env.pipeline:
resolve_step = step
if isinstance(step, CachingStep):
resolve_step = step._step
if isinstance(resolve_step, SharedStateHolder):
resolve_step.set_branches([(v, 'HEAD')])
logger.info(f"Overriding environment {k} branches to {(v, 'HEAD')}")
logger.info(f"Resulting profiles {environments.keys()}")
return environments
+4
View File
@@ -71,6 +71,10 @@ class DockerImageBuild(AbstractStep[DockerImageBuildResult]):
if len(existing_image) > 0 and not self.nocache:
logger.info(f"Image {full_image} already exists locally with ID: {existing_image}")
build_logs: List[str] = []
elif self.env.dry:
logger.info(f"DRY RUN: Would build image {full_image}")
raise BaseException(f"Dry run - skipping actual build for {full_image}")
else:
# Build the image
build_logs = []
+31
View File
@@ -0,0 +1,31 @@
import json
from dataclasses import asdict
import os
from typing import Any
class DataclassJSONEncoder(json.JSONEncoder):
def default(self, o: Any) -> Any:
if hasattr(o, '__dataclass_fields__'):
return asdict(o)
if isinstance(o, BaseException):
return str(o)
try:
return super().default(o)
except TypeError:
return str(o)
def custom_json_dumps(obj: Any) -> str:
return json.dumps(obj, cls=DataclassJSONEncoder)
def sigchld_handler(signum: int, frame: Any) -> None:
"""Reap zombie processes."""
while True:
try:
pid, status = os.waitpid(-1, os.WNOHANG)
if pid == 0:
break
except ChildProcessError:
break
+240
View File
@@ -0,0 +1,240 @@
import asyncio
import json
import logging
import os
import traceback
from contextlib import asynccontextmanager
from typing import Any, AsyncIterator, Dict, List, Optional, TypeVar
from fastapi import FastAPI, Response, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse
from app import App
from utils import custom_json_dumps
from secondary import SecondaryManager
logger = logging.getLogger(__name__)
FRONTEND_DIR = os.path.join(os.path.dirname(__file__), '../frontend')
T = TypeVar('T')
def _merge_dicts(a: Dict[str, T], b: Dict[str, T]) -> Dict[str, T]:
result: Dict[str, T] = {}
for k in (a.keys() | b.keys()):
if k not in a or k not in b:
result[k] = a.get(k, b.get(k)) # type: ignore[assignment]
continue
if (
isinstance(a[k], dict)
and isinstance(b[k], dict)
):
result[k] = _merge_dicts(a[k], b[k]) # type: ignore[arg-type,assignment]
else:
result[k] = b[k]
return result
class WebApp:
def __init__(self, core: App, port: int) -> None:
self.core = core
self.port = port
self.ws_connections: Dict[WebSocket, Dict[str, Any]] = {}
self._event_loop: Optional[asyncio.AbstractEventLoop] = None
self._server: Any = None
self.secondaryManager: Optional[SecondaryManager] = None
@asynccontextmanager
async def lifespan(fastapi_app: FastAPI) -> AsyncIterator[None]:
self._event_loop = asyncio.get_running_loop()
self.secondaryManager = SecondaryManager(
urls=os.getenv("SECONDARY_BRENCHER", ""),
on_update=lambda: self.broadcast_all(),
)
yield
self.app = FastAPI(lifespan=lifespan)
self.app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
self.app.get("/")(self.serve_index)
self.app.get("/state")(self.serve_state)
self.app.get("/branches")(self.serve_branches_route)
self.app.get("/{path:path}")(self.serve_static)
self.app.websocket("/ws")(self.websocket_endpoint)
# --- State assembly (local + secondary) ---
def get_global_envs_to_emit(self) -> Any:
local_envs = self.core.get_local_envs_to_emit()
merged: Dict[str, Any] = local_envs
for connector in self.secondaryManager or []:
v = connector.environments
common_keys = set(merged.keys()) & set(v.keys())
if len(common_keys) > 0:
self._schedule_async(self.broadcast_error({'message': f"Conflict: both master and secondary have environment with id {common_keys}"}))
merged = _merge_dicts(merged, v)
return merged
def get_global_branches_to_emit(self) -> Dict[str, Dict[str, List[Any]]]:
local_branches: Dict[str, Dict[str, List[Any]]] = self.core.get_local_branches_to_emit()
merged: Dict[str, Dict[str, List[Any]]] = local_branches
for connector in self.secondaryManager or []:
merged = _merge_dicts(merged, connector.branches)
return merged
# --- Async scheduling from sync callbacks ---
def _schedule_async(self, coro: Any) -> None:
if self._event_loop is None or not self._event_loop.is_running():
logger.warning("Dropping scheduled coroutine because the web event loop is not running")
coro.close()
return
try:
running_loop = asyncio.get_running_loop()
except RuntimeError:
running_loop = None
if running_loop is self._event_loop:
self._event_loop.create_task(coro)
else:
asyncio.run_coroutine_threadsafe(coro, self._event_loop)
# --- Broadcasting ---
async def broadcast(self, event: str, data: Any) -> None:
"""Broadcast a message to all connected WebSocket clients that have not seen this data yet."""
disconnected = set()
message = custom_json_dumps({event: data})
for websocket in self.ws_connections.keys():
if self.ws_connections[websocket].get(event) == message:
continue
try:
await websocket.send_text(message)
self.ws_connections[websocket][event] = message
except Exception as e:
logger.error(f"Error sending to websocket: {e}")
disconnected.add(websocket)
for ws in disconnected:
self.ws_connections.pop(ws, None)
async def broadcast_all(self) -> None:
await self.broadcast("branches", self.get_global_branches_to_emit())
await self.broadcast("environments", self.get_global_envs_to_emit())
async def broadcast_branches(self, data: Any) -> None:
await self.broadcast("branches", data)
async def broadcast_environments(self, data: Any) -> None:
await self.broadcast("environments", data)
async def broadcast_error(self, data: Any) -> None:
await self.broadcast("error", data)
def emit_envs(self) -> None:
"""Sync callback used by App's processing loop to push updates to clients."""
self._schedule_async(self.broadcast_branches(self.get_global_branches_to_emit()))
self._schedule_async(self.broadcast_environments(self.get_global_envs_to_emit()))
# --- Routes ---
async def serve_index(self) -> FileResponse:
return FileResponse(os.path.join(FRONTEND_DIR, 'index.html'))
async def serve_state(self) -> Response:
return Response(content=custom_json_dumps(self.get_global_envs_to_emit()), media_type="application/json")
async def serve_branches_route(self) -> Response:
return Response(content=custom_json_dumps(self.get_global_branches_to_emit()), media_type="application/json")
async def serve_static(self, path: str) -> FileResponse:
file_path = os.path.join(FRONTEND_DIR, path)
if os.path.exists(file_path) and os.path.isfile(file_path):
return FileResponse(file_path)
return FileResponse(os.path.join(FRONTEND_DIR, 'index.html'))
async def websocket_endpoint(self, websocket: WebSocket) -> None:
await websocket.accept()
self.ws_connections[websocket] = {}
try:
# Send initial state on connect and record it so duplicate broadcasts are suppressed
branches_payload = custom_json_dumps({"branches": self.get_global_branches_to_emit()})
envs_payload = custom_json_dumps({"environments": self.get_global_envs_to_emit()})
await websocket.send_text(branches_payload)
self.ws_connections[websocket]["branches"] = branches_payload
await websocket.send_text(envs_payload)
self.ws_connections[websocket]["environments"] = envs_payload
while True:
data = await websocket.receive_text()
message = json.loads(data)
if "update" in message:
update_data = message.get("update") or {}
logger.info(f"Received environment update: {update_data}")
if self.secondaryManager:
await self.secondaryManager.send({"update": update_data})
id = update_data.get('id', '')
if id == '':
self.core.request_reset()
elif id not in self.core.environments.keys() and id not in {j for it in self.secondaryManager or [] for j in it.environments.keys()}:
logger.warning(f"Received update for unknown environment id {id}")
continue
elif id in self.core.environments.keys():
env = self.core.environments.get(id, None)
expected_token = update_data.get('token', '')
if 'branches' in update_data:
if not env:
raise RuntimeError(f"Unknown env {update_data.get('id', '')}")
env.state.set_branches(update_data.get('branches', []), expected_token=expected_token)
if 'dry' in update_data:
if not env:
raise RuntimeError(f"Unknown env {update_data.get('id', '')}")
env.state.set_dry(bool(update_data['dry']), expected_token)
if env:
self.core.request_reset()
logger.info(f"Updated environment {env.id} branches to {update_data.get('branches')}, dry={update_data.get('dry')}")
self.core.notify_environment_update()
await self.broadcast_environments(self.get_global_envs_to_emit())
except WebSocketDisconnect:
self.ws_connections.pop(websocket, None)
except Exception as e:
logger.error(f"WebSocket error: {e}:{traceback.format_exc()}")
await self.broadcast_error({'message': f'{e}:{traceback.format_exc()}'})
# --- Run ---
async def start_async(self) -> None:
import uvicorn
config = uvicorn.Config(self.app, host='0.0.0.0', port=self.port, log_level='info')
self._server = uvicorn.Server(config)
try:
await self._server.serve()
except asyncio.CancelledError:
self._server.should_exit = True
raise
def start(self) -> None:
asyncio.run(self.start_async())
def stop(self) -> None:
if self._server is None:
return
if self._event_loop is not None and self._event_loop.is_running():
self._event_loop.call_soon_threadsafe(lambda: setattr(self._server, "should_exit", True))
else:
self._server.should_exit = True
+1
View File
@@ -5,6 +5,7 @@ pytest-asyncio
playwright
pytest-playwright
mypy
flake8
ty
types-PyYAML
types-requests
+74
View File
@@ -0,0 +1,74 @@
import subprocess
from pathlib import Path
from enironment import Environment
from steps.docker_plain import DockerImageBuild, DockerContainerCheck, DockerContainerDeploy
from steps.git import GitClone, CheckoutMerged, GitUnmerge
from steps.shared_state import SharedStateHolderInMemory
REPO_ROOT = Path(__file__).resolve().parents[2]
def _git_output(*args: str) -> str:
try:
return subprocess.check_output(
["git", "-C", str(REPO_ROOT), *args],
text=True,
stderr=subprocess.STDOUT,
).strip()
except subprocess.CalledProcessError as exc:
raise RuntimeError(f"Unable to read git metadata for nginx test config: {' '.join(args)}\n{exc.output}") from exc
CURRENT_BRANCH = _git_output("rev-parse", "--abbrev-ref", "HEAD")
CURRENT_COMMIT = _git_output("rev-parse", "HEAD")
clone = GitClone(url="https://github.com/rudolf1/brencher.git")
container_check = DockerContainerCheck(
container_name="test_plain-container"
)
unmerge = GitUnmerge(wd=clone, check=container_check)
state = SharedStateHolderInMemory(unmerge=unmerge)
checkoutMerged = CheckoutMerged(clone,
desired_branches=state,
push=False,
git_user_email="rudolfss13@gmail.com",
git_user_name="test_bot"
)
# Step 1: Build the image
image_build = DockerImageBuild(
wd=checkoutMerged,
dockerfile_path="Dockerfile_test_app",
image_name="test_plain",
image_tag=lambda: "auto-" + checkoutMerged.progress().version,
nocache=False,
)
# Step 3: Deploy the container
container_deploy = DockerContainerDeploy(
image_build=image_build,
container_name="test_plain-container",
ports={"5001/tcp": 5002},
environment={"PROFILES": "no_profiles"},
restart_policy={"Name": "unless-stopped"},
)
__all__ = ["test_local1"]
test_local1 = Environment(
id="test_local1",
state=state,
pipeline=[
clone,
state,
checkoutMerged,
image_build,
container_deploy,
container_deploy,
unmerge
]
)
+25 -11
View File
@@ -4,9 +4,10 @@ Pytest configuration and shared fixtures for integration tests.
This file configures the Python path to allow importing backend modules
and provides shared fixtures for all tests.
"""
from typing import Generator, Callable, Protocol
from typing import Any, Generator, Callable, Protocol, TypeVar
import pytest
import requests
from tests.test_remote_repo import RemoteRepoHelper
@@ -26,25 +27,38 @@ import pytest
class EventuallyFn(Protocol):
def __call__(
self,
assert_fn: Callable[[], bool],
timeout: float = 2.0,
interval: float = 0.05,
assert_fn: Callable[[], None],
timeout: float = 5.0,
interval: float = 0.5,
) -> None: ...
T = TypeVar('T')
def assert_equal(a: T, b:T, message: str) -> None:
if a != b:
raise AssertionError(f"{message}: {a} != {b}")
def check_state(url: str, predicate: Callable[[Any], None]) -> None:
response = requests.get(f"{url}/state", timeout=5)
assert response.status_code == 200, f"Expected status code 200, got {response.status_code}"
state_data = response.json()
try:
predicate(state_data)
except Exception as e:
raise AssertionError(f"Predicate check failed with error: {e}\nState data: {state_data}") from e
@pytest.fixture
def eventually() -> EventuallyFn:
def _eventually(assert_fn: Callable[[], bool], timeout: float = 2.0, interval: float = 0.05) -> None:
def _eventually(assert_fn: Callable[[], None], timeout: float = 5.0, interval: float = 0.5) -> None:
deadline = time.monotonic() + timeout
last_error = None
last_error: BaseException | None = None
while time.monotonic() < deadline:
try:
res = assert_fn()
if res:
return
except AssertionError as err:
assert_fn()
return
except Exception as err:
last_error = err
time.sleep(interval)
time.sleep(interval)
raise AssertionError(f"Condition not met within {timeout:.2f}s") from last_error
+110
View File
@@ -0,0 +1,110 @@
import logging
from contextlib import contextmanager
from typing import Generator
from playwright.sync_api import Locator, Page, sync_playwright
logger = logging.getLogger(__name__)
class BrencherPage:
def __init__(self, page: Page, env_id: str, action_logger: logging.Logger) -> None:
self._page = page
self._env_id = env_id
self._logger = action_logger
def _dry_run_button(self) -> Locator:
button = self._page.locator(f'h3:has-text("{self._env_id}") button.dry-run-btn')
button.wait_for(state="visible", timeout=10000)
return button
def verify_dry_run_on(self) -> None:
btn_title = self._dry_run_button().get_attribute("title") or ""
assert "Dry run on" in btn_title, f"Expected dry run to be active, got title: {btn_title!r}"
self._logger.info(f"Dry-run button title before click: {btn_title!r}")
def set_dry_run_off(self) -> None:
self._dry_run_button().click()
self._logger.info("Clicked dry-run button to disable dry run")
def click_refresh(self) -> None:
self._page.locator("#refresh-branches").click()
self._logger.info("Clicked refresh button")
def verify_dry_run_off(self) -> None:
self._page.wait_for_function(
"""(targetEnvId) => {
const h3s = [...document.querySelectorAll('h3.env-title')];
const h3 = h3s.find(el => el.textContent.includes(targetEnvId));
if (!h3) return false;
const btn = h3.querySelector('button.dry-run-btn');
return btn && !btn.title.includes('Dry run on');
}""",
arg=self._env_id,
timeout=10000,
)
def verify_pipeline_step_status(self, step_name: str, expected_status: str, timeout_ms: int = 10000) -> None:
"""
Verify that a pipeline step reaches the expected status by polling the page DOM.
Args:
step_name: Name of the pipeline step (e.g., "DockerContainerDeploy")
expected_status: Expected status value ("running", "ok", "error", or "dry-run")
timeout_ms: Timeout in milliseconds (default 10000)
"""
# JavaScript to check pipeline step status
js_check = """(stepName, expectedStatus) => {
const jobItems = document.querySelectorAll('.job-item');
for (const item of jobItems) {
const headerText = item.querySelector('.job-header')?.textContent || '';
if (headerText.includes(stepName)) {
const spinner = item.querySelector('.step-spinner');
const errorIndicator = item.querySelector('span[title="Error"]');
if (expectedStatus === 'running') {
return spinner !== null;
} else if (expectedStatus === 'ok') {
return spinner === null && errorIndicator === null;
} else if (expectedStatus === 'error') {
return errorIndicator !== null;
} else if (expectedStatus === 'dry-run') {
// For dry-run, check the spoiler content
const spoiler = item.querySelector('.job-spoiler');
if (spoiler && spoiler.style.display !== 'none') {
const statusText = spoiler.textContent;
return statusText.includes('dry-run') || statusText.includes('dry_run');
}
return false;
}
}
}
return false;
}"""
self._page.wait_for_function(
js_check,
arg=[step_name, expected_status],
timeout=timeout_ms,
)
self._logger.info(f"Pipeline step '{step_name}' reached status '{expected_status}'")
@contextmanager
def brencher_page(
app_link: str,
env_id: str,
action_logger: logging.Logger | None = None,
) -> Generator[BrencherPage, None, None]:
active_logger = action_logger or logger
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True)
page = browser.new_page()
page.goto(app_link)
page.wait_for_load_state("networkidle")
try:
yield BrencherPage(page, env_id, active_logger)
finally:
browser.close()
+52
View File
@@ -0,0 +1,52 @@
import asyncio
import sys
import types
from typing import Callable
import pytest
from enironment import Environment
import processing
from app import App
class TestAppAsyncio:
@pytest.mark.asyncio
async def test_run_web_async_stops_processing_when_server_exits(self, monkeypatch: pytest.MonkeyPatch) -> None:
process_calls = 0
emit_calls = 0
processing_started = asyncio.Event()
loop = asyncio.get_running_loop()
web_app_args: dict[str, object] = {}
def fake_process_all_jobs(environments: list[Environment], onupdate: Callable[[], None]) -> bool:
nonlocal process_calls
process_calls += 1
assert environments == []
onupdate()
loop.call_soon_threadsafe(processing_started.set)
return False
class FakeWebApp:
def __init__(self, core: App, port: int) -> None:
web_app_args["core"] = core
web_app_args["port"] = port
def emit_envs(self) -> None:
nonlocal emit_calls
emit_calls += 1
async def start_async(self) -> None:
await asyncio.wait_for(processing_started.wait(), timeout=1)
monkeypatch.setattr(processing, "process_all_jobs", fake_process_all_jobs)
monkeypatch.setitem(sys.modules, "web", types.SimpleNamespace(WebApp=FakeWebApp))
app = App({})
await asyncio.wait_for(app.runWebAsync(5001), timeout=1)
assert process_calls >= 1
assert emit_calls >= 1
assert web_app_args == {"core": app, "port": 5001}
assert app.emit_callback is None
+24 -23
View File
@@ -1,14 +1,15 @@
import asyncio
import json
import logging
import threading
from contextlib import suppress
from typing import Any, Generator
import docker
import pytest
import requests
from app import App
from .conftest import EventuallyFn
from configs import brencher_local1
from .conftest import EventuallyFn, assert_equal
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
@@ -34,31 +35,31 @@ class TestDockerContainer:
async def test_start(self, eventually: EventuallyFn) -> None:
logger.info(f"Starting")
app = App(cli_env_ids_str="brencher_local1")
# app.runWeb(5001)
processing = threading.Thread(target=lambda: app.runWeb(5001), daemon=True)
processing.start()
app = App({"brencher_local1": brencher_local1.brencher_local1})
processing = asyncio.create_task(app.runWebAsync(5001))
try:
eventually(
lambda: assert_equal(requests.get("http://localhost:5001/state", timeout=5000).status_code, 200, "Server did not respond with 200 OK within timeout"),
20.0,
1.0,
)
# await asyncio.sleep(5000)
state_data = requests.get("http://localhost:5001/state", timeout=5000).json()
logger.info(f"Application state: {json.dumps(state_data, indent=2)}")
eventually(
lambda: requests.get("http://localhost:5001/state", timeout=5000).status_code == 200,
20.0,
1.0,
)
branches_data = requests.get("http://localhost:5001/branches", timeout=5000).json()
logger.info(f"Application branches: {json.dumps(branches_data, indent=2)}")
assert "brencher_local1" in state_data, "Missing 'brencher_local1' field in response"
assert len(state_data["brencher_local1"]["pipeline"]) > 0, "Steps defined for 'brencher_local1' are empty"
state_data = requests.get("http://localhost:5001/state", timeout=5000).json()
logger.info(f"Application state: {json.dumps(state_data, indent=2)}")
branches_data = requests.get("http://localhost:5001/branches", timeout=5000).json()
logger.info(f"Application branches: {json.dumps(branches_data, indent=2)}")
assert "brencher_local1" in state_data, "Missing 'brencher_local1' field in response"
assert len(state_data["brencher_local1"]["pipeline"]) > 0, "Steps defined for 'brencher_local1' are empty"
assert "brencher_local1" in branches_data, "Missing 'brencher_local1' field in response"
assert len(branches_data["brencher_local1"]) > 0, "No branches found for 'brencher_local1'"
assert "brencher_local1" in branches_data, "Missing 'brencher_local1' field in response"
assert len(branches_data["brencher_local1"]) > 0, "No branches found for 'brencher_local1'"
finally:
processing.cancel()
with suppress(asyncio.CancelledError):
await processing
assert processing.cancelled()
# TODO Verify DockerContainerDeploy step status is running and has correct image and ports
# {
+71 -91
View File
@@ -1,124 +1,104 @@
import logging
import threading
from typing import Any, Generator
from typing import Any, Callable, Generator, TypeVar
import docker
import pytest
import requests
from playwright.sync_api import Page, sync_playwright
from app import App
from .conftest import EventuallyFn
from .conftest import EventuallyFn, assert_equal, check_state
from .playwright_helper import brencher_page
from tests.configs import nginx
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
APP_PORT = 5003
APP_URL = f"http://localhost:{APP_PORT}"
ENV_ID = "brencher_local1"
CONTAINER_NAME = "brencher_plain-container"
CONTAINER_NAME = "test_plain-container"
def _get_pipeline_step(state_data: Any, env_id: str, step_name: str) -> Any:
pipeline = state_data[env_id]["pipeline"]
for step in pipeline:
if step["name"] == step_name:
return step
return None
@pytest.mark.skip
class TestDryRunPlaywright:
@pytest.fixture(autouse=True)
def setup_teardown(self) -> Generator[None, Any, None]:
def stop_and_remove_container() -> None:
client = docker.from_env()
try:
container = client.containers.get(CONTAINER_NAME)
container.stop(timeout=10)
container.remove(force=True)
logger.info(f"Container {CONTAINER_NAME} stopped and removed")
except Exception:
logger.info(f"Container {CONTAINER_NAME} not found, no need to stop/remove")
stop_and_remove_container()
yield None
stop_and_remove_container()
client = docker.from_env()
containers = client.containers.list(filters={"name": CONTAINER_NAME}, all=True)
for container in containers:
container.stop(timeout=10)
container.remove(force=True)
logger.info(f"Container {CONTAINER_NAME} stopped and removed")
def test_dry_run_prevents_deploy_until_disabled(self, eventually: EventuallyFn) -> None:
# Start app with brencher_local1 with main branch pre-selected.
# App starts with dry=True by default (SharedStateHolderInMemory default).
app = App(cli_env_ids_str=f"{ENV_ID}:main")
# Start app with nginx_local1 with main branch pre-selected and dry mode on.
env = nginx.test_local1
env.state.set_branches([(nginx.CURRENT_BRANCH, nginx.CURRENT_COMMIT)])
env.state.set_dry(True)
app = App({env.id: env})
server_thread = threading.Thread(target=lambda: app.runWeb(APP_PORT), daemon=True)
server_thread.start()
try:
# Wait for server to be up
eventually(lambda: assert_equal(requests.get(f"{APP_URL}/state", timeout=5).status_code, 200, "Server did not respond with 200 OK within timeout"))
logger.info("Server is up")
# Wait for server to be up
eventually(
lambda: requests.get(f"{APP_URL}/state", timeout=5).status_code == 200,
timeout=20.0,
interval=1.0,
)
logger.info("Server is up")
# Wait for the pipeline to stabilize with dry=True:
# DockerContainerDeploy should return status="dry-run"
with brencher_page(APP_URL, env.id) as page:
eventually(
lambda: check_state(APP_URL, lambda s: assert_equal(next(
p for p in s[env.id]["pipeline"] if p["name"] == "SharedStateHolderInMemory"
)["status"]['dry'], True, "Pipeline step status is not 'running'")),
)
# Wait for the pipeline to stabilize with dry=True:
# DockerContainerDeploy should return status="dry-run"
def _deploy_is_dry_run() -> bool:
state_data = requests.get(f"{APP_URL}/state", timeout=5).json()
step = _get_pipeline_step(state_data, ENV_ID, "DockerContainerDeploy")
if step is None or step.get("error"):
return False
return step["status"].get("status") == "dry-run"
# page.verify_pipeline_step_status("DockerContainerDeploy", "dry-run")
logger.info("Pipeline is in dry-run mode")
eventually(_deploy_is_dry_run, timeout=120.0, interval=2.0)
logger.info("Pipeline is in dry-run mode")
# Verify no real container was created
docker_client = docker.from_env()
real_containers = docker_client.containers.list(filters={"name": CONTAINER_NAME}, all=True)
assert len(real_containers) == 0, "Container should NOT exist during dry run"
# Verify no real container was created
docker_client = docker.from_env()
real_containers = docker_client.containers.list(filters={"name": CONTAINER_NAME}, all=True)
assert len(real_containers) == 0, "Container should NOT exist during dry run"
# Disable dry run from the UI.
page.verify_dry_run_on()
page.set_dry_run_off()
page.verify_dry_run_off()
# Open browser and click the dry-run button to disable dry run
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page: Page = browser.new_page()
page.goto(APP_URL)
page.wait_for_load_state("networkidle")
# Wait for pipeline to re-run and actually deploy the container
eventually(
lambda: check_state(APP_URL, lambda s: assert_equal(next(
p for p in s[env.id]["pipeline"] if p["name"] =="SharedStateHolderInMemory"
)["status"]['dry'], False, "Pipeline step status is not 'running'")),
)
# The dry-run button shows ⏸ when active (dry=True) and ▶ when disabled (dry=False).
# It is inside the h3 heading for brencher_local1.
dry_run_btn = page.locator(f'h3:has-text("{ENV_ID}") button.dry-run-btn')
dry_run_btn.wait_for(state="visible", timeout=10000)
# Wait for the real container to be created and started.
eventually(
lambda: _assert_container_running(docker_client),
# Allow time for a cold image build and first container start on CI runners.
timeout=180,
interval=5.0,
)
# page.verify_pipeline_step_status("DockerContainerDeploy", "running")
logger.info("Container deployed and running after dry run disabled")
# Verify button indicates dry-run is currently active (title contains "Dry run on")
btn_title = dry_run_btn.get_attribute("title") or ""
assert "Dry run on" in btn_title, f"Expected dry run to be active, got title: {btn_title!r}"
logger.info(f"Dry-run button title before click: {btn_title!r}")
# Final sanity: real container must exist and be running
containers = docker_client.containers.list(filters={"name": CONTAINER_NAME})
assert len(containers) == 1, "Container should exist and be running after deploy"
assert containers[0].status == "running"
finally:
app.stop()
server_thread.join(timeout=30)
if server_thread.is_alive():
logger.warning("App server thread did not stop within 30 seconds")
# Click to disable dry run
dry_run_btn.click()
logger.info("Clicked dry-run button to disable dry run")
# Wait for button to update to "Running" state (title no longer says "Dry run on")
page.wait_for_function(
"""(envId) => {
const h3s = [...document.querySelectorAll('h3.env-title')];
const h3 = h3s.find(el => el.textContent.includes(envId));
if (!h3) return false;
const btn = h3.querySelector('button.dry-run-btn');
return btn && !btn.title.includes('Dry run on');
}""",
arg=ENV_ID,
timeout=10000,
)
browser.close()
# Wait for pipeline to re-run and actually deploy the container
def _deploy_is_running() -> bool:
state_data = requests.get(f"{APP_URL}/state", timeout=5).json()
step = _get_pipeline_step(state_data, ENV_ID, "DockerContainerDeploy")
if step is None or step.get("error"):
return False
return step["status"].get("status") == "running"
eventually(_deploy_is_running, timeout=300.0, interval=5.0)
logger.info("Container deployed and running after dry run disabled")
# Final sanity: real container must exist and be running
containers = docker_client.containers.list(filters={"name": CONTAINER_NAME})
assert len(containers) == 1, "Container should exist and be running after deploy"
assert containers[0].status == "running"
def _assert_container_running(docker_client: docker.DockerClient) -> None:
containers = docker_client.containers.list(filters={"name": CONTAINER_NAME}, all=True)
assert len(containers) > 0, "Container should exist after deploy"
assert_equal({container.status for container in containers}, {"running"}, "Container should be running after deploy")