Compare commits

..
Author SHA1 Message Date
hermes 6848d114a7 chore(hermes_squid): remove brencher-state-proxy
Drop the brencher-state-proxy sidecar from hermes_infra -- it is
no longer needed. The sidecar existed to relay /state from
brencher.rudolf.keenetic.link into the admin overlay; that relay is
unnecessary since brencher ingress handles the path itself.

Changes vs the brencher-state-proxy commit that was merged in #4:
- delete hermes_squid/brencher-state-proxy.conf
- remove the brencher-state-proxy: service block from
  hermes_squid/stack-compose.yml
- remove the brencher_state_proxy_conf: top-level config object
- drop brencher-state-proxy from the NO_PROXY / no_proxy lists
- update the surrounding comments
2026-08-29 22:47:36 +00:00
c3b7128006 hermes_squid: open docker_socket_proxy to full read-only scope (#4)
Flip CONTAINERS/IMAGES/NETWORKS/VOLUMES/SERVICES/TASKS/NODES from 0 to 1 so hermes-agent can inspect running stacks (read container logs, enumerate swarm services) without going through each node's shell.

POST stays 0 — no container creation, no image builds, no exec into running containers. Anything that mutates state remains blocked at the proxy level.

---------

Co-authored-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/4
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-08-30 00:09:56 +02:00
hermesandhermes 0c95d8e1ee feat(hermes_squid): enable local STT via faster-whisper (runtime install) (#21)
## What

Wires up local STT (faster-whisper) into the hermes-gateway container so voice messages on Telegram transcribe without depending on external services.

Closes #19. Supersedes the closed #20 (which used a custom Dockerfile-hermes; this version uses a bind-mounted entrypoint script instead since pypi.org is now whitelisted).

## Changes

1. **`hermes_squid/squid.conf`** — expands the egress whitelist:
   - new `stt_pkg_domains` for `pypi.org`, `files.pythonhosted.org`, `huggingface.co`, `github.com`, `models.dev`, `registry.npmjs.org`, `cdn-lfs.huggingface.co` — needed for runtime `pip install faster-whisper` and HF model fetch.
   - new `telegram_ips` covering AS62041 (91.108.4.0/22, 91.108.8.0/22, 91.108.12.0/22, 91.108.16.0/22, 91.108.20.0/22, 91.108.56.0/22, 149.154.160.0/20, 185.76.151.0/24) as a backstop for the `dstdomain` ACL. squid's `dstdomain` does not match HTTPS CONNECT requests without `ssl_bump`, so the recent log shows 24 of 9083 `api.telegram.org` requests getting 403 in a 5-minute window.
   - renamed `hermes_network` -> `localnet` (squid default name, no behavior change).

2. **`hermes_squid/hermes-gateway-entrypoint.sh`** (new) — bind-mounted into the container at `/opt/data/hermes-gateway-entrypoint.sh` and set as the service entrypoint. The script:
   - is idempotent: on first start, runs `apt-get install -y python3-pip ffmpeg` and `pip3 install --break-system-packages faster-whisper`. Subsequent restarts skip install if `faster_whisper` is already importable.
   - exports `STT_PROVIDER=local`, `STT_LOCAL_MODEL=base`, `STT_LOCAL_DEVICE=cpu`, `STT_LOCAL_COMPUTE_TYPE=int8`.
   - `exec`s the original gateway command (`gateway run`).

3. **`hermes_squid/stack-compose.yml`** — `hermes-gateway` gets:
   - `entrypoint: ["/opt/data/hermes-gateway-entrypoint.sh"]`
   - `command: ["gateway", "run"]`
   - bind-mount `./hermes-gateway-entrypoint.sh:/opt/data/hermes-gateway-entrypoint.sh:ro`
   - new env: `STT_PROVIDER`, `STT_LOCAL_MODEL`, `STT_LOCAL_DEVICE`, `STT_LOCAL_COMPUTE_TYPE`

No other service in the compose file is touched.

## Why runtime install, not a custom image

- `pypi.org` is now whitelisted, so `pip3 install` works at runtime through the same squid.
- Avoids maintaining a custom hermes-agent image and a local registry tag.
- The upstream `nousresearch/hermes-agent:v2026.8.3` image is kept as-is.

## Deploy notes

- brencher apply will trigger `docker service update` for hermes-gateway; the new entrypoint runs the apt+pip install on first start (~1-2 min).
- First start also downloads the `base` faster-whisper model (~150 MB) from `huggingface.co` — covered by `stt_pkg_domains`.
- squid config is reread on `docker service update` for the squid service; new ACLs apply without a full stack restart.
- Subsequent restarts of hermes-gateway skip the install path entirely (idempotent guard).

## Verified locally

- `git diff --stat upstream/master` shows only the three files above (+25 entrypoint, +14 squid, +8 compose).
- The new worktree is `/opt/data/work/thread-114-Alex-InfraAlexBot/ansible` (per-session working dir).

## Risks

- `apt-get update` inside the container needs `deb.debian.org` reachable — already in `allowed_domains`.
- `STT_LOCAL_MODEL=base` is ~150 MB and runs on CPU. If latency on Raspberry-class nodes is bad, switch to `tiny`.
- `--break-system-packages` is required for PEP 668 (Debian trixie + Python 3.11+). If the upstream image switches to PEP 668 strictly, this flag is the right escape hatch.

## Related

- #19 — investigation that found missing faster-whisper + squid deny pattern
- #20 — superseded (used Dockerfile-hermes instead of runtime install)

---------

Co-authored-by: hermes <hermes@rudolf.keenetic.link>
Reviewed-on: http://192.168.1.96/admin/ansible/pulls/21
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-08-29 22:53:46 +02:00
9 changed files with 27 additions and 639 deletions
-49
View File
@@ -1,49 +0,0 @@
name: release
on:
push:
branches:
- 'auto-*'
tags:
- 'v*'
workflow_dispatch:
inputs:
tag:
description: 'Image tag (e.g. auto-<sha>)'
required: false
default: latest
type: string
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v4
- name: log into Gitea Packages
uses: docker/login-action@v3
with:
registry: gitea.rudolf.keenetic.link
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: resolve version
id: version
run: |
if [ -n "${{ inputs.tag }}" ] && [ "${{ inputs.tag }}" != "latest" ]; then
echo "VERSION=${{ inputs.tag }}" >> "$GITHUB_OUTPUT"
else
echo "VERSION=${GITHUB_REF_NAME##*/}" >> "$GITHUB_OUTPUT"
fi
echo "VERSION=$VERSION"
- name: build and push
uses: docker/build-push-action@v6
with:
context: connectivity_tester
file: connectivity_tester/Dockerfile
push: true
tags: |
gitea.rudolf.keenetic.link/hermes/connectivity_tester:${{ steps.version.outputs.VERSION }}
gitea.rudolf.keenetic.link/hermes/connectivity_tester:latest
-1
View File
@@ -1 +0,0 @@
# test trigger
-12
View File
@@ -1,12 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
COPY . /app/
RUN pip install --no-cache-dir docker==7.1.0
ENV PYTHONUNBUFFERED=1 \
PERIOD_SECONDS=600 \
CHECK_TIMEOUT_S=3
CMD ["python", "-m", "app"]
-276
View File
@@ -1,276 +0,0 @@
"""Connectivity tester.
For every configured overlay network the container attaches itself, runs
TCP checks against its endpoints, then detaches. Endpoints marked with
``proxy=(host, port)`` are probed through that HTTP proxy (CONNECT). After
each round a final loop checks PUBLIC_SERVICES. Cross-validation:
- service reachable but not in map -> ERROR 'LEAK-UNEXPECTED'
- service in map but unreachable -> ERROR 'MISSING-EXPECTED'
- global service not in any map -> WARNING 'GLOBAL-UNCOVERED'
"""
from __future__ import annotations
import asyncio
import logging
import os
import socket
import sys
import time
from typing import Iterable, List, Set, Tuple
import docker
from docker.models.containers import Container
from docker.models.networks import Network
from network_map import Endpoint, GLOBAL_SERVICES, NEGATIVE, NETWORK_MAP, PUBLIC_SERVICES
LOG = logging.getLogger("connectivity-tester")
FALLBACK_TCP_TIMEOUT_S = 3.0
HTTP_CONNECT_TIMEOUT_S = 5.0
def _socket_path() -> str:
return os.environ.get("DOCKER_SOCKET", "/var/run/docker.sock")
def _self_container(client: docker.DockerClient) -> Container:
hostname = socket.gethostname()
for c in client.containers.list():
cn = c.name.lstrip("/")
hn = c.attrs.get("Config", {}).get("Hostname", "")
if hostname == cn or hostname == hn or hostname[:12] == c.id[:12]:
return c
candidates = [c for c in client.containers.list() if hostname[:12] == c.id]
if len(candidates) == 1:
return candidates[0]
raise RuntimeError(f"self container not found for hostname={hostname}")
def _swarm_networks_for(client: docker.DockerClient,
names: Iterable[str]) -> dict:
out = {}
by_name = {n.name: n for n in client.networks.list()}
for name in names:
net = by_name.get(name)
if net is not None and net.attrs.get("Scope") == "swarm":
out[name] = net
return out
def _attach(client: docker.DockerClient, container: Container,
network: Network) -> bool:
current = (container.attrs.get("NetworkSettings", {}) or {}).get("Networks", {})
if network.name in current:
return False
try:
network.connect(container, aliases=["connectivity-tester",
socket.gethostname()])
return True
except Exception as exc:
if "already exists" in str(exc).lower():
return False
raise
def _detach(client: docker.DockerClient, container: Container,
network: Network) -> None:
try:
network.disconnect(container, force=True)
except Exception as exc:
if "not found" in str(exc).lower():
return
raise
async def _tcp_probe(host: str, port: int, timeout: float) -> Tuple[bool, str]:
try:
fut = asyncio.open_connection(host=host, port=port)
reader, writer = await asyncio.wait_for(fut, timeout=timeout)
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
return True, "tcp ok"
except asyncio.TimeoutError:
return False, "timeout"
except OSError as exc:
return False, f"{type(exc).__name__}: {exc}"
async def _proxy_connect(endpoint: Endpoint, timeout: float) -> Tuple[bool, str]:
if endpoint.proxy is None:
return await _tcp_probe(endpoint.name, endpoint.port, timeout=timeout)
proxy_host, proxy_port = endpoint.proxy
target = endpoint.name
target_port = endpoint.port
try:
fut = asyncio.open_connection(host=proxy_host, port=proxy_port)
reader, writer = await asyncio.wait_for(fut, timeout=timeout)
req = (
f"CONNECT {target}:{target_port} HTTP/1.1\r\n"
f"Host: {target}:{target_port}\r\n\r\n"
)
writer.write(req.encode("ascii"))
await writer.drain()
status_line = await asyncio.wait_for(reader.readline(),
timeout=timeout)
if not status_line.startswith(b"HTTP/") or b" 200 " not in status_line:
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
return False, (
"proxy non-200: "
+ status_line.strip().decode(errors="replace")
)
while True:
line = await asyncio.wait_for(reader.readline(), timeout=timeout)
if line in (b"\r\n", b"", b"\n"):
break
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
return True, "proxy-connect ok"
except asyncio.TimeoutError:
return False, "timeout"
except OSError as exc:
return False, f"{type(exc).__name__}: {exc}"
async def _check_endpoint(endpoint: Endpoint, timeout: float) -> bool:
if endpoint.proxy is not None:
ok, detail = await _proxy_connect(endpoint, timeout)
else:
ok, detail = await _tcp_probe(endpoint.name, endpoint.port,
timeout=timeout)
label = f"{endpoint.name}:{endpoint.port}"
src = f"via {endpoint.proxy}" if endpoint.proxy else "direct"
if ok:
LOG.info("PASS %s %s %s", label, src, detail)
else:
LOG.error("FAIL %s %s %s", label, src, detail)
return ok
async def _check_network(network_name: str,
endpoints: List[Endpoint],
timeout: float) -> Tuple[Set[Endpoint], Set[Endpoint]]:
reachable: Set[Endpoint] = set()
unreachable: Set[Endpoint] = set()
for ep in endpoints:
ok = await _check_endpoint(ep, timeout)
(reachable if ok else unreachable).add(ep)
return reachable, unreachable
def _cross_validate(network_name: str,
mapped: List[Endpoint],
reachable: Set[Endpoint],
unreachable: Set[Endpoint],
global_set: Set[Endpoint]) -> None:
mapped_set = set(mapped)
reachable_global = reachable & global_set
for ep in reachable_global - mapped_set:
LOG.error("LEAK-UNEXPECTED %s reachable in %s but not in map",
f"{ep.name}:{ep.port}", network_name)
for ep in mapped_set & unreachable:
LOG.error("MISSING-EXPECTED %s unreachable in %s but listed in map",
f"{ep.name}:{ep.port}", network_name)
for ep in mapped_set & reachable:
LOG.info("OK-MAPPED %s ok in %s", f"{ep.name}:{ep.port}", network_name)
for ep in mapped_set - reachable - unreachable:
LOG.warning("NOT-CHECKED %s in map for %s",
f"{ep.name}:{ep.port}", network_name)
def _consistency_audit(net_names: List[str]) -> None:
flat: Set[Endpoint] = set()
for n in net_names:
flat.update(NETWORK_MAP.get(n, []))
covered = flat | set(PUBLIC_SERVICES)
all_global = set(GLOBAL_SERVICES)
uncovered = all_global - covered
if uncovered:
LOG.warning("GLOBAL-UNCOVERED %s not present in any map",
sorted(f"{e.name}:{e.port}" for e in uncovered))
async def _check_negative(endpoints: List[Endpoint],
timeout: float) -> int:
leaks = 0
for ep in endpoints:
ok = await _tcp_probe(ep.name, ep.port, timeout=timeout)
if ok:
LOG.error("LEAK %s:%d reachable but must NOT be",
ep.name, ep.port)
leaks += 1
else:
LOG.info("OK-NEG %s:%d blocked", ep.name, ep.port)
return leaks
async def main() -> int:
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
stream=sys.stdout,
)
period = int(os.environ.get("PERIOD_SECONDS", "600"))
timeout = float(os.environ.get("CHECK_TIMEOUT_S", "3"))
LOG.info("starting; period=%ds timeout=%.1fs", period, timeout)
client = docker.DockerClient(base_url="unix://" + _socket_path())
self_container = _self_container(client)
nets = _swarm_networks_for(client, NETWORK_MAP.keys())
planned = list(NETWORK_MAP.keys())
LOG.info("configured networks: %d, present in swarm: %d",
len(planned), len(nets))
for n in nets:
LOG.info("plan network: %s", n)
_consistency_audit(planned)
global_set: Set[Endpoint] = set(GLOBAL_SERVICES)
while True:
started = time.monotonic()
try:
for net_name, net in nets.items():
eps_in_map: List[Endpoint] = [
e for e in NETWORK_MAP[net_name] if e.proxy is None
]
LOG.info("--- round begin, attaching to %s (%d endpoints) ---",
net_name, len(eps_in_map))
_attach(client, self_container, net)
try:
reachable, unreachable = await _check_network(
net_name, eps_in_map, timeout)
_cross_validate(net_name, eps_in_map,
reachable, unreachable, global_set)
finally:
_detach(client, self_container, net)
LOG.info("--- round end on %s ---", net_name)
LOG.info("--- public checks (%d endpoints) ---",
len(PUBLIC_SERVICES))
for ep in PUBLIC_SERVICES:
await _check_endpoint(ep, max(timeout, HTTP_CONNECT_TIMEOUT_S))
LOG.info("--- negative checks (%d endpoints) ---", len(NEGATIVE))
await _check_negative(NEGATIVE, timeout)
except Exception:
LOG.exception("round crashed")
elapsed = time.monotonic() - started
sleep_for = max(0.0, period - elapsed)
await asyncio.sleep(sleep_for)
if __name__ == "__main__":
asyncio.run(main())
-122
View File
@@ -1,122 +0,0 @@
from dataclasses import dataclass
from typing import Dict, List, Optional, Tuple
@dataclass(frozen=True)
class Endpoint:
name: str
port: int
proxy: Optional[Tuple[str, int]] = None
tls: bool = False
SWARM_NETWORKS = (
"hermes_infra_hermes_proxy",
"hermes_infra_squid_egress",
"hermes_infra_admin",
"hermes_infra_default",
"hermes_hermes",
"gitea_gitea",
"gmail_mcp_default",
"immich_default",
"torrserv_proxy_default",
"docker_socket_proxy_hermes_proxy",
"connectivity_tester_default",
"brencher_brencher-net",
"brencher2_brencher-net",
)
SQUID_PROXY: Tuple[str, int] = ("squid", 3128)
NETWORK_MAP: Dict[str, List[Endpoint]] = {
"hermes_infra_hermes_proxy": [
Endpoint("hermes-gateway", 8000),
Endpoint("hermes-ui", 8087),
Endpoint("squid", 3128),
Endpoint("docker_socket_proxy", 2375),
Endpoint("brencher-state-proxy", 80),
],
"hermes_infra_squid_egress": [
Endpoint("squid", 3128),
],
"hermes_infra_admin": [
Endpoint("sarg", 80),
],
"hermes_infra_default": [
Endpoint("hermes-ui", 8087),
],
"hermes_hermes": [
Endpoint("hermes-agent", 8000),
Endpoint("hermes-gateway", 8000),
],
"gitea_gitea": [
Endpoint("gitea-server", 3000),
Endpoint("act_runner", 8088),
Endpoint("cache_server", 37051),
],
"gmail_mcp_default": [
Endpoint("gmail-mcp-server", 8000),
Endpoint("telegram-mcp-server", 8000),
],
"immich_default": [
Endpoint("immich-server", 2283),
Endpoint("immich-database", 5432),
Endpoint("immich-redis", 6379),
Endpoint("immich-machine-learning", 8501),
],
"torrserv_proxy_default": [
Endpoint("fileserver", 8000),
Endpoint("reverseproxy", 8080),
],
"docker_socket_proxy_hermes_proxy": [
Endpoint("hermes-gateway", 8000),
],
"connectivity_tester_default": [
Endpoint("connectivity-tester", 8765),
],
"brencher_brencher-net": [
Endpoint("brencher-backend", 5000),
],
"brencher2_brencher-net": [
Endpoint("brencher-backend", 5000),
],
}
PUBLIC_SERVICES: List[Endpoint] = [
Endpoint("gitea.rudolf.keenetic.link", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("gitea.rudolf.keenetic.link", 80, proxy=SQUID_PROXY),
Endpoint("registry.rudolf.keenetic.link", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("registry.rudolf.keenetic.link", 5000, proxy=SQUID_PROXY),
Endpoint("deb.debian.org", 80, proxy=SQUID_PROXY),
Endpoint("deb.debian.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("github.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("githubusercontent.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("telegram.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("t.me", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("minimax.io", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("minimax.chat", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("minimax.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("pypi.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("google.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("cloudflare.com", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("wikipedia.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("mozilla.org", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("1.1.1.1", 443, proxy=SQUID_PROXY, tls=True),
Endpoint("8.8.8.8", 53),
Endpoint("8.8.4.4", 53),
]
GLOBAL_SERVICES: List[Endpoint] = sorted(
{ep for eps in NETWORK_MAP.values() for ep in eps} | set(PUBLIC_SERVICES),
key=lambda e: (e.name, e.port),
)
NEGATIVE: List[Endpoint] = [
Endpoint("169.254.169.254", 80),
Endpoint("metadata.google.internal", 80),
]
-25
View File
@@ -1,25 +0,0 @@
services:
connectivity-tester:
image: gitea.rudolf.keenetic.link/hermes/connectivity_tester:${version}
hostname: connectivity-tester
user: "0:0"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
healthcheck:
test: ["CMD-SHELL", "test -S /var/run/docker.sock"]
interval: 60s
timeout: 5s
retries: 3
start_period: 30s
deploy:
mode: global
resources:
limits:
memory: 128M
pids: 64
reservations:
memory: 64M
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
-93
View File
@@ -1,93 +0,0 @@
# nginx config for the brencher-state-proxy sidecar.
#
# Path-locked reverse proxy for the brencher /state endpoint:
# - /state -> proxied to https://brencher.rudolf.keenetic.link/state
# - everything else -> 403 Forbidden
#
# This lets hermes-agent reach /state without going through squid's
# host-level allowlist (which can't see inside TLS) and without exposing
# the rest of brencher's surface area.
#
# Listen on 8080 (plain HTTP). No TLS -- this is a private internal
# API on the admin overlay, reachable only from inside the swarm. No
# host port is published.
#
# Upstream is the public URL on purpose -- the sidecar reaches the public
# brencher.rudolf.keenetic.link through the nginx-ingress overlay the same
# way any external client would, which means it doesn't need to be on the
# brencher container's private networks.
server {
listen 8080;
server_name _;
# Tiny access log; default nginx format
access_log /var/log/nginx/brencher-state-access.log;
error_log /var/log/nginx/brencher-state-error.log warn;
# Healthcheck for swarm -- not /state, deliberately, so a 200 here
# doesn't accidentally imply /state is reachable from outside the
# allowlist.
location = /healthz {
access_log off;
return 200 "ok\n";
add_header Content-Type text/plain;
}
# The only allowed path. Exact match: query strings or trailing
# anything else fall through to the default 403.
location = /state {
proxy_pass https://brencher.rudolf.keenetic.link/state;
proxy_http_version 1.1;
# Upstream is HTTPS; the sidecar trusts the LE-issued cert via
# the system CA bundle. SNI explicitly named so any future
# transparent-proxy middlebox doesn't break the handshake.
proxy_ssl_server_name on;
proxy_ssl_name brencher.rudolf.keenetic.link;
proxy_set_header Host brencher.rudolf.keenetic.link;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_connect_timeout 5s;
proxy_read_timeout 30s;
proxy_send_timeout 10s;
# Don't cache -- state should be live.
proxy_buffering off;
add_header Cache-Control "no-store" always;
}
# Secondary brencher (runs on the docker host via host.docker.internal,
# reached as http://host.docker.internal:5002 -- same trick the brencher
# container itself uses in docker-compose.yml's extra_hosts). Only the
# /state2 path is exposed; everything else returns 403 like /state.
location = /state2 {
proxy_pass http://host.docker.internal:5002/state;
proxy_http_version 1.1;
# Cleartext HTTP upstream (port 5002), no TLS to wrangle.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_connect_timeout 5s;
proxy_read_timeout 30s;
proxy_send_timeout 10s;
proxy_buffering off;
add_header Cache-Control "no-store" always;
}
# Catch-all: deny everything else, including /state/ (with trailing
# slash), /state?foo=bar (with query), and any other path.
location / {
access_log off;
return 403 "forbidden: only /state is allowed through this proxy\n";
add_header Content-Type text/plain;
}
}
+9 -5
View File
@@ -1,16 +1,20 @@
acl hermes_network src 10.0.0.0/8
acl hermes_network src 172.16.0.0/12
acl hermes_network src 192.168.0.0/16
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.0.0/16
acl allowed_domains dstdomain .minimax.io .minimax.chat .minimax.com .githubusercontent.com deb.debian.org gitea.rudolf.keenetic.link
acl telegram_domains dstdomain .telegram.org .t.me
acl stt_pkg_domains dstdomain .pypi.org .files.pythonhosted.org .huggingface.co .github.com .models.dev .registry.npmjs.org .cdn-lfs.huggingface.co .hf.co .xethub.hf.co .cas-server.xethub.hf.co
acl telegram_ips dst 91.108.4.0/22 91.108.8.0/22 91.108.12.0/22 91.108.16.0/22 91.108.20.0/22 91.108.56.0/22 149.154.160.0/20 185.76.151.0/24
acl CONNECT method CONNECT
acl SSL_ports port 443
acl Safe_ports port 80 443
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow hermes_network allowed_domains
http_access allow hermes_network telegram_domains
http_access allow localnet allowed_domains
http_access allow localnet telegram_domains
http_access allow localnet stt_pkg_domains
http_access allow localnet telegram_ips
http_access deny all
http_port 3128
+18 -56
View File
@@ -64,10 +64,10 @@ services:
command: gateway run
user: "0:0"
networks:
# No admin overlay membership here: the sidecars (brencher-state-proxy,
# docker_socket_proxy) are on hermes_proxy too, so DNS resolves
# without expanding hermes-gateway's L3 footprint. The NO_PROXY
# list below ensures those requests skip squid.
# No admin overlay membership here: the docker_socket_proxy sidecar
# is on hermes_proxy too, so DNS resolves without expanding
# hermes-gateway's L3 footprint. The NO_PROXY list below ensures
# those requests skip squid.
- hermes_proxy
volumes:
- hermes_gateway_data:/opt/data
@@ -84,21 +84,21 @@ services:
HTTPS_PROXY: http://squid:3128
http_proxy: http://squid:3128
https_proxy: http://squid:3128
# The sidecars added in this PR (brencher-state-proxy, docker_socket_proxy)
# are reachable on the hermes_proxy overlay. They are NOT public services, so
# traffic to them MUST NOT be routed through squid (which can't reach
# them anyway and would 403 the request).
# The docker_socket_proxy sidecar is reachable on the hermes_proxy
# overlay. It is NOT a public service, so traffic to it MUST NOT be
# routed through squid (which can't reach it anyway and would 403
# the request).
# Comma-separated list. Format accepted by curl/requests/urllib:
# bare hostnames, IPs, CIDR ranges. Order doesn't matter; case-insensitive.
# 127.0.0.1 - loopback
# .local - mDNS / link-local (covers host.docker.internal)
# squid, hermes-gateway, hermes-agent - same-overlay services that
# hermes-gateway itself needs to talk to without going through squid
# brencher-state-proxy, docker_socket_proxy - the new sidecars
# docker_socket_proxy - the sidecar
# 10.0.0.0/8 - the swarm's typical overlay subnet (defensive: covers
# any IP-direct call that bypasses DNS)
NO_PROXY: localhost,127.0.0.1,.local,squid,hermes-gateway,hermes-agent,brencher-state-proxy,docker_socket_proxy,10.0.0.0/8
no_proxy: localhost,127.0.0.1,.local,squid,hermes-gateway,hermes-agent,brencher-state-proxy,docker_socket_proxy,10.0.0.0/8
NO_PROXY: localhost,127.0.0.1,.local,squid,hermes-gateway,hermes-agent,docker_socket_proxy,10.0.0.0/8
no_proxy: localhost,127.0.0.1,.local,squid,hermes-gateway,hermes-agent,docker_socket_proxy,10.0.0.0/8
depends_on:
- squid
deploy:
@@ -169,13 +169,13 @@ services:
# Anything else (POST, CONTAINERS, IMAGES, SERVICES, ...) must be explicitly
# toggled to 1 by a consumer that has a reason to need it. NEVER widen the
# scope from a generic change like this one.
CONTAINERS: 0
IMAGES: 0
NETWORKS: 0
VOLUMES: 0
SERVICES: 0
TASKS: 0
NODES: 0
CONTAINERS: 1
IMAGES: 1
NETWORKS: 1
VOLUMES: 1
SERVICES: 1
TASKS: 1
NODES: 1
POST: 0
ports:
- target: 2375
@@ -193,42 +193,6 @@ services:
delay: 5s
max_attempts: 3
brencher-state-proxy:
# Path-locked reverse proxy for brencher.rudolf.keenetic.link/state.
# Lets hermes-agent reach /state without going through squid (which
# can't see inside TLS) and without exposing the rest of brencher.
# See hermes_squid/brencher-state-proxy.conf for the path rules.
image: nginx:alpine
hostname: brencher-state-proxy
extra_hosts:
# Lets the sidecar reach the secondary brencher at
# http://host.docker.internal:5002/state -- same trick the brencher
# container itself uses in its own docker-compose.yml.
- "host.docker.internal:host-gateway"
networks:
# hermes_proxy: same as docker_socket_proxy. hermes-gateway is
# already on this overlay, so DNS works. Egress is allowed despite
# the `internal: true` flag (which only blocks external ingress,
# not outbound traffic). The sidecar's upstream
# proxy_pass https://brencher.rudolf.keenetic.link/state is a
# public-internet HTTPS call, not a service-name lookup, so
# nginx_ingress membership is not required.
- hermes_proxy
configs:
- source: brencher_state_proxy_conf
target: /etc/nginx/conf.d/brencher-state-proxy.conf
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.hostname == odroid # must run where the secret is mounted (and where the public ingress routes back to)
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
networks:
hermes_proxy:
driver: overlay
@@ -247,8 +211,6 @@ configs:
file: ./squid.conf
nginx_hermes_conf:
file: ./nginx-hermes.conf
brencher_state_proxy_conf:
file: ./brencher-state-proxy.conf
volumes:
squid_etc: