Compare commits

..
36 Commits
Author SHA1 Message Date
hermes e6444ad49c fix(ci): trigger release workflow on auto-* branch pushes
release / build (push) Failing after 32s
Brencher's auto-deploy-via-temp-branch pattern pushes the merge
commit to `auto-<short-sha>` on the origin remote before dispatching
the workflow. Previously the workflow only fired on `v*` tags or
workflow_dispatch, so brencher's auto-* push was invisible to it.

Adding `branches: ['auto-*']` to the trigger list keeps the existing
tag-based manual releases working while letting brencher's temp
branches fire the build on their own push.
2026-09-19 23:49:04 +00:00
hermes eb851d9b1d fix(ci): workflow accepts tag input for brencher dispatch
release / build (push) Successful in 1m14s
2026-09-18 20:43:26 +00:00
hermes c719f6ab8d ci(connectivity_tester): remove debug, voice-gateway style (user-level secrets)
release / build (push) Successful in 1m8s
2026-09-18 12:47:09 +00:00
hermes 6e6f9c67fc fix(ci): use env block for secrets debug (bash doesn't understand templates)
release / build (push) Failing after 30s
2026-09-18 12:44:42 +00:00
hermes 77b3cc2c08 ci(connectivity_tester): voice-gateway-style with debug step
release / build (push) Failing after 19s
2026-09-18 12:43:18 +00:00
hermes d011648e56 fix(ci): remove duplicate runs-on at job-level (yaml unmarshal error)
release / build (ubuntu-24.04, linux/amd64) (push) Failing after 1m25s
release / build (ubuntu-24.04-arm, linux/arm/v7) (push) Canceled after 0s
release / build (ubuntu-24.04-arm, linux/arm64) (push) Canceled after 0s
2026-09-18 12:37:38 +00:00
hermes c22cadc48b ci(connectivity_tester): add tag trigger for v* 2026-09-18 12:35:21 +00:00
hermes 2fa05da943 ci: empty trigger 2026-09-18 12:01:47 +00:00
hermes 3406491dcc ci(connectivity_tester): real workflow with REGISTRY_USERNAME/TOKEN + multi-arch matrix 2026-09-18 11:58:12 +00:00
hermes 7a08d4d022 Обновить .gitea/workflows/release.yml
Build and Push Docker Image / build-and-push (push) Failing after 4m32s
2026-09-18 12:57:49 +02:00
hermes 6e9a998eba ci(connectivity_tester): smoke-test pattern from official docs (gitea.actor + server_url)
release / build (push) Failing after 24s
2026-09-17 22:42:07 +00:00
hermes ef4af24c1e Обновить .gitea/workflows/release.yml
release / build (push) Failing after 23s
2026-09-18 00:34:28 +02:00
hermes f80a524d33 ci(connectivity_tester): smoke-test GITHUB_TOKEN on master branch 2026-09-17 22:10:54 +00:00
hermes faa3e32a53 Reapply "fix(ci): use auto-injected GITHUB_TOKEN instead of REGISTRY_USERNAME/TOKEN secrets"
This reverts commit e6d184bd95.
2026-09-17 22:09:37 +00:00
hermes e6d184bd95 Revert "fix(ci): use auto-injected GITHUB_TOKEN instead of REGISTRY_USERNAME/TOKEN secrets"
This reverts commit d097862c81.
2026-09-17 21:43:55 +00:00
hermes d097862c81 fix(ci): use auto-injected GITHUB_TOKEN instead of REGISTRY_USERNAME/TOKEN secrets 2026-09-17 21:41:13 +00:00
hermes b2e85b686c fix(ci): sanitize platform in image tag (linux/amd64 -> linux-amd64) 2026-09-17 19:06:07 +00:00
admin 932be82ae6 ci(connectivity_tester): drop build block; image pulled from Gitea CR 2026-09-17 19:27:17 +02:00
admin 953e4dcba7 ci(connectivity_tester): switch stack image to Gitea Container Registry 2026-09-17 18:32:39 +02:00
admin 81e0641ab1 fix(ci): use REGISTRY_USERNAME/TOKEN (which are populated); runner OS via matrix.os 2026-09-17 18:19:15 +02:00
admin 8f71c723fb feat(ci): keep PACKAGE_PUSH_* secrets + multi-arch; login continue-on-error so we see what other side fails 2026-09-17 18:13:47 +02:00
admin 7fed0c08cd feat(ci): dump individual secrets, multi-arch matrix, switch to PACKAGE_PUSH_* 2026-09-17 18:05:58 +02:00
admin 4b8ece95c0 feat(ci): dump env+secrets, multi-arch, switch to PACKAGE_PUSH_* 2026-09-17 18:03:55 +02:00
admin 0734adc229 ci(connectivity_tester): copy release.yml to release/0.1.0-connectivity 2026-09-16 23:18:00 +02:00
hermes 8ecc6f0424 ci: drop broken .github/workflows path 2026-09-15 22:21:23 +02:00
admin 4ab39982d1 fix(connectivity_tester): hardcode REGISTRY_USERNAME (missing secret) 2026-09-15 21:20:50 +02:00
admin b5b94b2eee ci(connectivity_tester): rewrite workflow to use REGISTRY_* secrets + multi-arch
Aligns with the proven hermes/reverseproxy@main build.yml: uses
secrets.REGISTRY_USERNAME/REGISTRY_PASSWORD (which already exist in the
cluster-wide Actions runner scope), docker/setup-buildx-action@v3 and
docker/build-push-action@v6 from docker/github-mirror, and targets the
gitea.rudolf.keenetic.link/<image>:auto-<sha> plus :latest tags.

Multi-arch matrix now covers linux/amd64, linux/arm64 and linux/arm/v7
so push to Gitea Container Registry will fan out to rudolf-T (x86_64),
odroid (aarch64) and xu4 (32-bit ARM).
2026-09-15 21:06:24 +02:00
admin bf6b86302f ci(connectivity_tester): add multi-arch Gitea Actions workflow 2026-09-15 19:42:46 +02:00
admin 1e9d33485d feat(connectivity_tester): Endpoint dataclass, proxy-aware checks, public hosts
Replace tuple-of-strings with frozen Endpoint dataclass carrying optional
(proxy_host, proxy_port) tuple. Add network_map.NETWORK_MAP populated from
the live swarm endpoints discovered via the docker socket, and a separate
PUBLIC_SERVICES list sourced from the squid allowlist plus well-known
public hosts. Endpoints with proxy are checked via HTTP CONNECT through
that proxy; endpoints without are probed directly via TCP after attaching
to the relevant overlay network. Cross-validation rules cover leaks and
missing expectations per round.
2026-09-15 10:11:49 +00:00
admin d5b1a23bcb feat(connectivity_tester): rotate networks per-round, cross-validate map
Self-attach/detach to overlay networks via docker socket. At any moment the
container is on exactly one network; per-network list of expected services is
loaded from network_map.NETWORK_MAP. After checks run cross-validation:

- service reachable but not in map  -> ERROR 'LEAK-UNEXPECTED'
- service in map but unreachable    -> ERROR 'MISSING-EXPECTED'
- global service not in any map     -> WARNING 'GLOBAL-UNCOVERED'

Compose uses /var/run/docker.sock bind mount and removes overlay-network
references; EXTRA_ENDPOINTS / HTTP_PROXY / HTTPS_PROXY / NO_PROXY are gone.
endpoints.py removed (its data moved into network_map.py).
2026-09-15 09:23:55 +00:00
admin e94ea37e85 fix(connectivity_tester): drop root name: key from compose
docker stack deploy on this engine rejects 'Additional property name is not allowed'.
Sibling stacks (gitea, hermes, immich) ship without it; the stack name comes from
the brencher 'stack_name' CLI argument instead.
2026-09-15 08:19:19 +00:00
admin c0d53b709c fix(connectivity_tester): COPY . instead of connectivity_tester/
After brencher fix uses context=<repo>/connectivity_tester/ (single-level
now that compose 'dockerfile' is 'Dockerfile'), the previous COPY line
referenced a non-existent connectivity_tester/ subdirectory inside the
build context.
2026-09-15 07:59:06 +00:00
admin 0366a78a96 fix(connectivity_tester): use Dockerfile (single-level) in stack build
Previous 'dockerfile: connectivity_tester/Dockerfile' resolved through
brencher's join to <repo>/connectivity_tester/connectivity_tester/Dockerfile,
which does not exist; build failed with 'Cannot locate specified Dockerfile'.
The Dockerfile sits next to stack-compose.yml at <repo>/connectivity_tester/Dockerfile.
2026-09-15 07:39:43 +00:00
admin c13e9d2ce7 fix(connectivity_tester): define EXTRA_ENDPOINTS empty + add external hosts
brencher DockerComposeBuild raises KeyError: 'EXTRA_ENDPOINTS' when the
env var is not passed via envs=. Hardcode empty default in compose for
deterministic build.

Add google.com and lan gateways to EXPECTED_EXTERNAL so connectivity-tester
probes them on every round.
2026-09-15 05:04:07 +00:00
hermes 020a5092a0 fix(compose): drop shell default syntax from EXTRA_ENDPOINTS
brencher parses compose via re.sub(r'\$\{([^}]+)\}', lambda m: env[m.group(1)]).
With shell default syntax ${VAR:-default} the regex captures 'VAR:-default'
as a single key, env['VAR:-default'] raises KeyError.

Use bare ${EXTRA_ENDPOINTS}; if the env var is missing brencher's
.env lookup (or the dict.get fallback in the deploy step) handles it.
If EXTRA_ENDPOINTS isn't set at deploy time the value is empty, which
the python service treats as 'no extras'.
2026-09-14 19:26:54 +00:00
hermes 5a36921f3b feat(connectivity_tester): new stack for synthetic connectivity checks
Long-running Swarm service that verifies expected endpoints are
reachable from inside the cluster and that the negative-list endpoints
(cloud metadata etc.) stay blocked.

Runs one round every PERIOD_SECONDS (default 600): TCP probes against
EXPECTED_INTERNAL (overlay services), EXPECTED_EXTERNAL (internet
hosts), EXTRA_ENDPOINTS (ad-hoc from env), and NEGATIVE (must NOT
respond; alerts on any leak). Logs structured output, alerts on
failures, container keeps running.

Mode: global — Swarm schedules one task on every node that meets
constraints. Attached to hermes_proxy overlay (for internal DNS) and
default (for egress via squid).

Blocks Org1/hermes-voice-gateway#3 (Path B) until we verify the
overlay + egress paths voice-gateway will depend on.
2026-09-14 09:39:13 +00:00
13 changed files with 478 additions and 177 deletions
+10 -18
View File
@@ -2,14 +2,17 @@ name: release
on:
push:
branches:
- 'auto-*'
tags:
- 'v*'
workflow_dispatch:
inputs:
tag:
description: Tag to build (e.g. v0.1.0-test1)
required: true
default: v0.1.0-test1
description: 'Image tag (e.g. auto-<sha>)'
required: false
default: latest
type: string
jobs:
build:
@@ -28,11 +31,12 @@ jobs:
- name: resolve version
id: version
run: |
if [ -n "${{ github.event.inputs.tag }}" ]; then
echo "VERSION=${{ github.event.inputs.tag }}" >> "$GITHUB_OUTPUT"
if [ -n "${{ inputs.tag }}" ] && [ "${{ inputs.tag }}" != "latest" ]; then
echo "VERSION=${{ inputs.tag }}" >> "$GITHUB_OUTPUT"
else
echo "VERSION=${GITHUB_REF_NAME#refs/tags/}" >> "$GITHUB_OUTPUT"
echo "VERSION=${GITHUB_REF_NAME##*/}" >> "$GITHUB_OUTPUT"
fi
echo "VERSION=$VERSION"
- name: build and push
uses: docker/build-push-action@v6
@@ -43,15 +47,3 @@ jobs:
tags: |
gitea.rudolf.keenetic.link/hermes/connectivity_tester:${{ steps.version.outputs.VERSION }}
gitea.rudolf.keenetic.link/hermes/connectivity_tester:latest
- name: render stack-compose
run: |
mkdir -p out
TAG=${{ steps.version.outputs.VERSION }}
sed "s|\${version}|$TAG|g" connectivity_tester/stack-compose.yml > "out/docker-compose-$TAG.yml"
- name: upload rendered stack-compose
uses: actions/upload-artifact@v4
with:
name: stack-compose
path: out/
+1
View File
@@ -0,0 +1 @@
# test trigger
+12
View File
@@ -0,0 +1,12 @@
FROM python:3.12-slim
WORKDIR /app
COPY . /app/
RUN pip install --no-cache-dir docker==7.1.0
ENV PYTHONUNBUFFERED=1 \
PERIOD_SECONDS=600 \
CHECK_TIMEOUT_S=3
CMD ["python", "-m", "app"]
+276
View File
@@ -0,0 +1,276 @@
"""Connectivity tester.
For every configured overlay network the container attaches itself, runs
TCP checks against its endpoints, then detaches. Endpoints marked with
``proxy=(host, port)`` are probed through that HTTP proxy (CONNECT). After
each round a final loop checks PUBLIC_SERVICES. Cross-validation:
- service reachable but not in map -> ERROR 'LEAK-UNEXPECTED'
- service in map but unreachable -> ERROR 'MISSING-EXPECTED'
- global service not in any map -> WARNING 'GLOBAL-UNCOVERED'
"""
from __future__ import annotations
import asyncio
import logging
import os
import socket
import sys
import time
from typing import Iterable, List, Set, Tuple
import docker
from docker.models.containers import Container
from docker.models.networks import Network
from network_map import Endpoint, GLOBAL_SERVICES, NEGATIVE, NETWORK_MAP, PUBLIC_SERVICES
LOG = logging.getLogger("connectivity-tester")
FALLBACK_TCP_TIMEOUT_S = 3.0
HTTP_CONNECT_TIMEOUT_S = 5.0
def _socket_path() -> str:
return os.environ.get("DOCKER_SOCKET", "/var/run/docker.sock")
def _self_container(client: docker.DockerClient) -> Container:
hostname = socket.gethostname()
for c in client.containers.list():
cn = c.name.lstrip("/")
hn = c.attrs.get("Config", {}).get("Hostname", "")
if hostname == cn or hostname == hn or hostname[:12] == c.id[:12]:
return c
candidates = [c for c in client.containers.list() if hostname[:12] == c.id]
if len(candidates) == 1:
return candidates[0]
raise RuntimeError(f"self container not found for hostname={hostname}")
def _swarm_networks_for(client: docker.DockerClient,
names: Iterable[str]) -> dict:
out = {}
by_name = {n.name: n for n in client.networks.list()}
for name in names:
net = by_name.get(name)
if net is not None and net.attrs.get("Scope") == "swarm":
out[name] = net
return out
def _attach(client: docker.DockerClient, container: Container,
network: Network) -> bool:
current = (container.attrs.get("NetworkSettings", {}) or {}).get("Networks", {})
if network.name in current:
return False
try:
network.connect(container, aliases=["connectivity-tester",
socket.gethostname()])
return True
except Exception as exc:
if "already exists" in str(exc).lower():
return False
raise
def _detach(client: docker.DockerClient, container: Container,
network: Network) -> None:
try:
network.disconnect(container, force=True)
except Exception as exc:
if "not found" in str(exc).lower():
return
raise
async def _tcp_probe(host: str, port: int, timeout: float) -> Tuple[bool, str]:
try:
fut = asyncio.open_connection(host=host, port=port)
reader, writer = await asyncio.wait_for(fut, timeout=timeout)
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
return True, "tcp ok"
except asyncio.TimeoutError:
return False, "timeout"
except OSError as exc:
return False, f"{type(exc).__name__}: {exc}"
async def _proxy_connect(endpoint: Endpoint, timeout: float) -> Tuple[bool, str]:
if endpoint.proxy is None:
return await _tcp_probe(endpoint.name, endpoint.port, timeout=timeout)
proxy_host, proxy_port = endpoint.proxy
target = endpoint.name
target_port = endpoint.port
try:
fut = asyncio.open_connection(host=proxy_host, port=proxy_port)
reader, writer = await asyncio.wait_for(fut, timeout=timeout)
req = (
f"CONNECT {target}:{target_port} HTTP/1.1\r\n"
f"Host: {target}:{target_port}\r\n\r\n"
)
writer.write(req.encode("ascii"))
await writer.drain()
status_line = await asyncio.wait_for(reader.readline(),
timeout=timeout)
if not status_line.startswith(b"HTTP/") or b" 200 " not in status_line:
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
return False, (
"proxy non-200: "
+ status_line.strip().decode(errors="replace")
)
while True:
line = await asyncio.wait_for(reader.readline(), timeout=timeout)
if line in (b"\r\n", b"", b"\n"):
break
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
return True, "proxy-connect ok"
except asyncio.TimeoutError:
return False, "timeout"
except OSError as exc:
return False, f"{type(exc).__name__}: {exc}"
async def _check_endpoint(endpoint: Endpoint, timeout: float) -> bool:
if endpoint.proxy is not None:
ok, detail = await _proxy_connect(endpoint, timeout)
else:
ok, detail = await _tcp_probe(endpoint.name, endpoint.port,
timeout=timeout)
label = f"{endpoint.name}:{endpoint.port}"
src = f"via {endpoint.proxy}" if endpoint.proxy else "direct"
if ok:
LOG.info("PASS %s %s %s", label, src, detail)
else:
LOG.error("FAIL %s %s %s", label, src, detail)
return ok
async def _check_network(network_name: str,
endpoints: List[Endpoint],
timeout: float) -> Tuple[Set[Endpoint], Set[Endpoint]]:
reachable: Set[Endpoint] = set()
unreachable: Set[Endpoint] = set()
for ep in endpoints:
ok = await _check_endpoint(ep, timeout)
(reachable if ok else unreachable).add(ep)
return reachable, unreachable
def _cross_validate(network_name: str,
mapped: List[Endpoint],
reachable: Set[Endpoint],
unreachable: Set[Endpoint],
global_set: Set[Endpoint]) -> None:
mapped_set = set(mapped)
reachable_global = reachable & global_set
for ep in reachable_global - mapped_set:
LOG.error("LEAK-UNEXPECTED %s reachable in %s but not in map",
f"{ep.name}:{ep.port}", network_name)
for ep in mapped_set & unreachable:
LOG.error("MISSING-EXPECTED %s unreachable in %s but listed in map",
f"{ep.name}:{ep.port}", network_name)
for ep in mapped_set & reachable:
LOG.info("OK-MAPPED %s ok in %s", f"{ep.name}:{ep.port}", network_name)
for ep in mapped_set - reachable - unreachable:
LOG.warning("NOT-CHECKED %s in map for %s",
f"{ep.name}:{ep.port}", network_name)
def _consistency_audit(net_names: List[str]) -> None:
flat: Set[Endpoint] = set()
for n in net_names:
flat.update(NETWORK_MAP.get(n, []))
covered = flat | set(PUBLIC_SERVICES)
all_global = set(GLOBAL_SERVICES)
uncovered = all_global - covered
if uncovered:
LOG.warning("GLOBAL-UNCOVERED %s not present in any map",
sorted(f"{e.name}:{e.port}" for e in uncovered))
async def _check_negative(endpoints: List[Endpoint],
timeout: float) -> int:
leaks = 0
for ep in endpoints:
ok = await _tcp_probe(ep.name, ep.port, timeout=timeout)
if ok:
LOG.error("LEAK %s:%d reachable but must NOT be",
ep.name, ep.port)
leaks += 1
else:
LOG.info("OK-NEG %s:%d blocked", ep.name, ep.port)
return leaks
async def main() -> int:
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
stream=sys.stdout,
)
period = int(os.environ.get("PERIOD_SECONDS", "600"))
timeout = float(os.environ.get("CHECK_TIMEOUT_S", "3"))
LOG.info("starting; period=%ds timeout=%.1fs", period, timeout)
client = docker.DockerClient(base_url="unix://" + _socket_path())
self_container = _self_container(client)
nets = _swarm_networks_for(client, NETWORK_MAP.keys())
planned = list(NETWORK_MAP.keys())
LOG.info("configured networks: %d, present in swarm: %d",
len(planned), len(nets))
for n in nets:
LOG.info("plan network: %s", n)
_consistency_audit(planned)
global_set: Set[Endpoint] = set(GLOBAL_SERVICES)
while True:
started = time.monotonic()
try:
for net_name, net in nets.items():
eps_in_map: List[Endpoint] = [
e for e in NETWORK_MAP[net_name] if e.proxy is None
]
LOG.info("--- round begin, attaching to %s (%d endpoints) ---",
net_name, len(eps_in_map))
_attach(client, self_container, net)
try:
reachable, unreachable = await _check_network(
net_name, eps_in_map, timeout)
_cross_validate(net_name, eps_in_map,
reachable, unreachable, global_set)
finally:
_detach(client, self_container, net)
LOG.info("--- round end on %s ---", net_name)
LOG.info("--- public checks (%d endpoints) ---",
len(PUBLIC_SERVICES))
for ep in PUBLIC_SERVICES:
await _check_endpoint(ep, max(timeout, HTTP_CONNECT_TIMEOUT_S))
LOG.info("--- negative checks (%d endpoints) ---", len(NEGATIVE))
await _check_negative(NEGATIVE, timeout)
except Exception:
LOG.exception("round crashed")
elapsed = time.monotonic() - started
sleep_for = max(0.0, period - elapsed)
await asyncio.sleep(sleep_for)
if __name__ == "__main__":
asyncio.run(main())
+122
View File
@@ -0,0 +1,122 @@
from dataclasses import dataclass
from typing import Dict, List, Optional, Tuple
@dataclass(frozen=True)
class Endpoint:
name: str
port: int
proxy: Optional[Tuple[str, int]] = None
tls: bool = False
SWARM_NETWORKS = (
"hermes_infra_hermes_proxy",
"hermes_infra_squid_egress",
"hermes_infra_admin",
"hermes_infra_default",
"hermes_hermes",
"gitea_gitea",
"gmail_mcp_default",
"immich_default",
"torrserv_proxy_default",
"docker_socket_proxy_hermes_proxy",
"connectivity_tester_default",
"brencher_brencher-net",
"brencher2_brencher-net",
)
SQUID_PROXY: Tuple[str, int] = ("squid", 3128)
NETWORK_MAP: Dict[str, List[Endpoint]] = {
"hermes_infra_hermes_proxy": [
Endpoint("hermes-gateway", 8000),
Endpoint("hermes-ui", 8087),
Endpoint("squid", 3128),
Endpoint("docker_socket_proxy", 2375),
Endpoint("brencher-state-proxy", 80),
],
"hermes_infra_squid_egress": [
Endpoint("squid", 3128),
],
"hermes_infra_admin": [
Endpoint("sarg", 80),
],
"hermes_infra_default": [
Endpoint("hermes-ui", 8087),
],
"hermes_hermes": [
Endpoint("hermes-agent", 8000),
Endpoint("hermes-gateway", 8000),
],
"gitea_gitea": [
Endpoint("gitea-server", 3000),
Endpoint("act_runner", 8088),
Endpoint("cache_server", 37051),
],
"gmail_mcp_default": [
Endpoint("gmail-mcp-server", 8000),
Endpoint("telegram-mcp-server", 8000),
],
"immich_default": [
Endpoint("immich-server", 2283),
Endpoint("immich-database", 5432),
Endpoint("immich-redis", 6379),
Endpoint("immich-machine-learning", 8501),
],
"torrserv_proxy_default": [
Endpoint("fileserver", 8000),
Endpoint("reverseproxy", 8080),
],
"docker_socket_proxy_hermes_proxy": [
Endpoint("hermes-gateway", 8000),
],
"connectivity_tester_default": [
Endpoint("connectivity-tester", 8765),
],
"brencher_brencher-net": [
Endpoint("brencher-backend", 5000),
],
"brencher2_brencher-net": [
Endpoint("brencher-backend", 5000),
],
}
PUBLIC_SERVICES: List[Endpoint] = [
Endpoint("gitea.rudolf.keenetic.link", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("gitea.rudolf.keenetic.link", 80, proxy=SQUID_PROXY),
Endpoint("registry.rudolf.keenetic.link", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("registry.rudolf.keenetic.link", 5000, proxy=SQUID_PROXY),
Endpoint("deb.debian.org", 80, proxy=SQUID_PROXY),
Endpoint("deb.debian.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("github.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("githubusercontent.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("telegram.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("t.me", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("minimax.io", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("minimax.chat", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("minimax.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("pypi.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("google.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("cloudflare.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("wikipedia.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("mozilla.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("1.1.1.1", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("8.8.8.8", 53),
Endpoint("8.8.4.4", 53),
]
GLOBAL_SERVICES: List[Endpoint] = sorted(
{ep for eps in NETWORK_MAP.values() for ep in eps} | set(PUBLIC_SERVICES),
key=lambda e: (e.name, e.port),
)
NEGATIVE: List[Endpoint] = [
Endpoint("169.254.169.254", 80),
Endpoint("metadata.google.internal", 80),
]
+25
View File
@@ -0,0 +1,25 @@
services:
connectivity-tester:
image: gitea.rudolf.keenetic.link/hermes/connectivity_tester:${version}
hostname: connectivity-tester
user: "0:0"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
healthcheck:
test: ["CMD-SHELL", "test -S /var/run/docker.sock"]
interval: 60s
timeout: 5s
retries: 3
start_period: 30s
deploy:
mode: global
resources:
limits:
memory: 128M
pids: 64
reservations:
memory: 64M
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
+1 -5
View File
@@ -21,9 +21,6 @@ server {
listen 8080;
server_name _;
resolver 127.0.0.11 192.168.1.1 valid=30s ipv6=off;
resolver_timeout 5s;
# Tiny access log; default nginx format
access_log /var/log/nginx/brencher-state-access.log;
error_log /var/log/nginx/brencher-state-error.log warn;
@@ -40,8 +37,7 @@ server {
# The only allowed path. Exact match: query strings or trailing
# anything else fall through to the default 403.
location = /state {
set $brencher_upstream https://brencher.rudolf.keenetic.link;
proxy_pass $brencher_upstream/state;
proxy_pass https://brencher.rudolf.keenetic.link/state;
proxy_http_version 1.1;
# Upstream is HTTPS; the sidecar trusts the LE-issued cert via
+5 -9
View File
@@ -1,20 +1,16 @@
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.0.0/16
acl hermes_network src 10.0.0.0/8
acl hermes_network src 172.16.0.0/12
acl hermes_network src 192.168.0.0/16
acl allowed_domains dstdomain .minimax.io .minimax.chat .minimax.com .githubusercontent.com deb.debian.org gitea.rudolf.keenetic.link
acl telegram_domains dstdomain .telegram.org .t.me
acl stt_pkg_domains dstdomain .pypi.org .files.pythonhosted.org .huggingface.co .github.com .models.dev .registry.npmjs.org .cdn-lfs.huggingface.co .hf.co .xethub.hf.co .cas-server.xethub.hf.co
acl telegram_ips dst 91.108.4.0/22 91.108.8.0/22 91.108.12.0/22 91.108.16.0/22 91.108.20.0/22 91.108.56.0/22 149.154.160.0/20 185.76.151.0/24
acl CONNECT method CONNECT
acl SSL_ports port 443
acl Safe_ports port 80 443
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet allowed_domains
http_access allow localnet telegram_domains
http_access allow localnet stt_pkg_domains
http_access allow localnet telegram_ips
http_access allow hermes_network allowed_domains
http_access allow hermes_network telegram_domains
http_access deny all
http_port 3128
+7 -7
View File
@@ -169,13 +169,13 @@ services:
# Anything else (POST, CONTAINERS, IMAGES, SERVICES, ...) must be explicitly
# toggled to 1 by a consumer that has a reason to need it. NEVER widen the
# scope from a generic change like this one.
CONTAINERS: 1
IMAGES: 1
NETWORKS: 1
VOLUMES: 1
SERVICES: 1
TASKS: 1
NODES: 1
CONTAINERS: 0
IMAGES: 0
NETWORKS: 0
VOLUMES: 0
SERVICES: 0
TASKS: 0
NODES: 0
POST: 0
ports:
- target: 2375
-39
View File
@@ -1,39 +0,0 @@
#!/usr/bin/env bash
# Custom entrypoint for the act_runner cache-server.
#
# The upstream image's /usr/local/bin/run.sh hardcodes `act_runner daemon`
# and ignores argv. So we replace the entrypoint here.
#
# cache-server also requires cache.external_secret to be set in config.yaml
# as a literal value (not a file path). We keep the value in a Docker
# secret and substitute it into a runtime copy of config.yaml before
# exec'ing act_runner.
set -eu
SECRET_FILE="${ACT_RUNNER_EXTERNAL_SECRET_FILE:-/run/secrets/gitea_cache_secret}"
CONFIG_FILE="${CONFIG_FILE:-/cfg/config.yaml}"
if [[ ! -s "$SECRET_FILE" ]]; then
echo "cache-server-run: secret file $SECRET_FILE is missing or empty" >&2
exit 1
fi
if [[ ! -f "$CONFIG_FILE" ]]; then
echo "cache-server-run: config file $CONFIG_FILE not found" >&2
exit 1
fi
SECRET_VALUE="$(cat "$SECRET_FILE")"
# Render a copy of config.yaml with cache.external_secret replaced by the
# secret's literal value. /cfg/config.yaml is a read-only Swarm config mount,
# so we cannot edit it in place. Use a plain-file replacement — the
# upstream config.yaml has the form `external_secret: ""` (or any literal)
# on a single line, so a simple sed is safe.
RUNTIME_CONFIG="/tmp/config.runtime.yaml"
cp "$CONFIG_FILE" "$RUNTIME_CONFIG"
# Replace the value after `external_secret:` on a single line.
# Use a delimiter unlikely to appear in the secret.
sed -i "s|^ external_secret:.*$| external_secret: \"${SECRET_VALUE}\"|" "$RUNTIME_CONFIG"
exec /usr/local/bin/act_runner cache-server --config "$RUNTIME_CONFIG"
+18 -8
View File
@@ -70,16 +70,26 @@ runner:
- "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
cache:
# Enable cache server to use actions/cache.
enabled: true
# The directory to store the cache data.
# If it's empty, the cache data will be stored in $HOME/.cache/actcache.
dir: ""
# Point at the dedicated gitea_cache_server container on the `gitea` overlay.
# The runner's built-in cache server binds to an interface unreachable from
# per-job container networks, causing ~9 min/job wasted on
# "reserveCache failed: connect ETIMEDOUT" (see gitea/actions/runs/16 job #248).
host: "gitea_cache_server"
port: 37051
external_server: "http://gitea_cache_server:37051/"
# Read from the gitea_cache_secret Docker secret (see stack-compose.yml).
# The host of the cache server.
# It's not for the address to listen, but the address to connect from job containers.
# So 0.0.0.0 is a bad choice, leave it empty to detect automatically.
host: ""
# The port of the cache server.
# 0 means to use a random available port.
port: 0
# The external cache server URL. Valid only when enable is true.
# If it's specified, act_runner will use this URL as the ACTIONS_CACHE_URL rather than start a server by itself.
# The URL should generally end with "/".
# Requires external_secret below to be set to the same value on both this runner and the cache-server.
external_server: ""
# Shared secret between this runner and the external `act_runner cache-server`. Required when external_server
# (or `act_runner cache-server`) is in use: the runner pre-registers each job's ACTIONS_RUNTIME_TOKEN with the
# cache-server, and the cache-server enforces bearer auth + per-repo cache isolation.
external_secret: ""
container:
-36
View File
@@ -1,36 +0,0 @@
#!/usr/bin/env bash
# Custom entrypoint for the gitea_runner service.
#
# Same trick as cache-server-run.sh: act_runner requires cache.external_secret
# to be a literal string in config.yaml, not a file path. We keep the value in
# a Docker secret and substitute it into a runtime copy of config.yaml, then
# exec the image's own /usr/local/bin/run.sh (which handles registration +
# daemon).
set -eu
SECRET_FILE="${ACT_RUNNER_EXTERNAL_SECRET_FILE:-/run/secrets/gitea_cache_secret}"
CONFIG_FILE="${CONFIG_FILE:-/cfg/config.yaml}"
if [[ ! -s "$SECRET_FILE" ]]; then
echo "runner-run: secret file $SECRET_FILE is missing or empty" >&2
exit 1
fi
if [[ ! -f "$CONFIG_FILE" ]]; then
echo "runner-run: config file $CONFIG_FILE not found" >&2
exit 1
fi
SECRET_VALUE="$(cat "$SECRET_FILE")"
# Render a runtime copy with the secret substituted in.
RUNTIME_CONFIG="/tmp/config.runtime.yaml"
cp "$CONFIG_FILE" "$RUNTIME_CONFIG"
sed -i "s|^ external_secret:.*$| external_secret: \"${SECRET_VALUE}\"|" "$RUNTIME_CONFIG"
# The upstream run.sh reads $CONFIG_FILE from the environment to pass
# --config to act_runner. Override it so registration + daemon both pick up
# the runtime copy with the substituted secret.
export CONFIG_FILE="$RUNTIME_CONFIG"
exec /usr/local/bin/run.sh
+1 -55
View File
@@ -1,8 +1,4 @@
# Gitea Actions stack. Uses a dedicated act_runner cache-server so that
# job-container networks can reach the cache backend via Swarm service DNS
# (gitea_cache_server:37051) instead of the unreachable IP the built-in
# cache server binds to per-job.
services:
server:
image: docker.gitea.com/gitea:1.27.0
@@ -23,7 +19,6 @@ services:
runner:
image: docker.io/gitea/act_runner:nightly
entrypoint: ["/usr/local/bin/runner-run.sh"]
environment:
CONFIG_FILE: /cfg/config.yaml
GITEA_INSTANCE_URL: "https://gitea.rudolf.keenetic.link"
@@ -31,51 +26,14 @@ services:
GITEA_RUNNER_NAME: "runner1"
GITEA_RUNNER_LABELS: "runner_label1"
# GITEA_RUNNER_EPHEMERAL: "1"
ACT_RUNNER_EXTERNAL_SECRET_FILE: /run/secrets/gitea_cache_secret
networks:
- gitea
# - nginx_ingress
configs:
- source: runner_config
target: /cfg/config.yaml
mode: 0444
- source: runner_run
target: /usr/local/bin/runner-run.sh
mode: 0755
volumes:
- /var/run/docker.sock:/var/run/docker.sock
secrets:
- gitea_cache_secret
deploy:
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
cache_server:
# The image's /usr/local/bin/run.sh hardcodes `act_runner daemon` and
# ignores argv, so passing `cache-server` via `command` doesn't help —
# the daemon path is always taken. We replace the entrypoint by
# mounting our own script via a Swarm config (file mount), then have it
# substitute the Docker secret literal into a runtime copy of
# config.yaml's cache.external_secret and exec act_runner.
image: docker.io/gitea/act_runner:nightly
entrypoint: ["/usr/local/bin/cache-server-run.sh"]
environment:
ACT_RUNNER_EXTERNAL_SECRET_FILE: /run/secrets/gitea_cache_secret
networks:
- gitea
volumes:
- cache_data:/root/.cache/actcache
secrets:
- gitea_cache_secret
configs:
- source: runner_config
target: /cfg/config.yaml
mode: 0444
- source: cache_server_run
target: /usr/local/bin/cache-server-run.sh
mode: 0755
networks:
nginx_ingress:
@@ -86,17 +44,6 @@ networks:
configs:
runner_config:
file: ./config.yaml
cache_server_run:
file: ./cache-server-run.sh
runner_run:
file: ./runner-run.sh
secrets:
gitea_cache_secret:
# Create once before first deploy (or to rotate):
# echo -n "$SECRET" | docker secret create gitea_cache_secret -
# Then `docker stack deploy -c stack-compose.yml gitea`.
external: true
volumes:
data:
@@ -105,5 +52,4 @@ volumes:
type: nfs
o: addr=192.168.1.96
device: :/root/volumes/gitea
# device: :/media/DATA/runtime/gitea
cache_data:
# device: :/media/DATA/runtime/gitea