Files
hermesandHermes Agent 0f9c5fa105 fix(ingress): isolate HTTP redirect from ACME (#22)
See commit 1af4832. Removes the broken global redirect that exposed internal :8443, pins TLS routes to websecure, and adds a dedicated web redirect middleware so ACME HTTP-01 remains isolated.

---------

Co-authored-by: Hermes Agent <hermes@localhost>
Reviewed-on: org1/infra#22
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-11-22 17:49:41 +01:00

5.0 KiB

Shared public ingress

The common Traefik entrypoint routes public domains to Kubernetes and Swarm backends.

Domain Backend
hermes.rudolf.keenetic.link Kubernetes Service hermes:9119
brencher.rudolf.keenetic.link Swarm 192.168.1.96:5001
immich.rudolf.keenetic.link Swarm 192.168.1.96:2283
gitea.rudolf.keenetic.link Swarm 192.168.1.96:3002

The Swarm backends are represented by selectorless Kubernetes Services and static Endpoints. The Swarm ports must remain reachable from the Traefik node at 192.168.1.96.

HTTPS target configuration

The target design is TCP forwarding to Traefik, with TLS termination in Traefik:

WAN TCP 80  -> 192.168.1.114:80
WAN TCP 443 -> 192.168.1.114:443

Configure the Keenetic entries as raw TCP forwarding. Do not use the KeenDNS HTTPS proxy, terminate TLS on the router, rewrite SNI, or replace the HTTP Host header.

Traefik must provide:

  • web on TCP 80 for HTTP and ACME HTTP-01;
  • websecure on TCP 443 for TLS termination;
  • a persistent ACME storage file, for example /data/acme.json;
  • an ACME resolver with httpChallenge.entryPoint: web;
  • a non-placeholder registration email supplied outside Git.

The exact HelmChartConfig values must match the installed K3s Traefik chart version. Inspect the live HelmChart/traefik values before applying a chart override; do not assume that a values schema from another K3s release is compatible.

For each public Ingress, pin the TLS route to websecure. The separate http-redirect.yaml resource attaches a redirectScheme middleware to web-only routes. Keeping the HTTP redirect separate prevents the internal ACME challenge router from being sent to a backend.

The resulting TLS Ingress shape is:

metadata:
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
    traefik.ingress.kubernetes.io/router.tls.certresolver: letsencrypt
spec:
  tls:
    - hosts:
        - immich.rudolf.keenetic.link

With a Traefik ACME resolver, the certificate is stored in the configured persistent ACME storage rather than committed to Git. If the installed chart requires a Kubernetes Secret for a different TLS mode, provision that Secret through the approved secret-management path. Do not commit acme.json, a private key, an API token, or an unencrypted Kubernetes Secret.

Declarative traffic configuration

The repository configuration is in traefik-https-config.yaml. It extends the installed K3s HelmChart/traefik with:

  • web exposed on TCP 80 for the HTTP redirect and ACME HTTP-01;
  • websecure exposed on TCP 443 for TLS termination;
  • a permanent HTTP-to-HTTPS redirectScheme middleware on web-only routes;
  • the letsencrypt ACME resolver using HTTP-01 through web (Traefik's ACME challenge handling remains available on the web entrypoint);
  • persistent /data/acme.json storage through a 1Gi PVC.

The resolver uses the operator-supplied registration email from the configuration. The email is not a credential; the ACME account state and certificate private keys remain in persistent /data/acme.json storage and must not be committed to Git. The TLS annotations and host lists are present in both the shared public Ingress and Hermes-specific Ingress.

Rollout procedure

  1. Verify that every hostname resolves to the router's current public address.

  2. Configure Keenetic WAN TCP 80 and 443 forwarding to 192.168.1.114:80 and 192.168.1.114:443.

  3. Confirm from an external network that TCP 80 reaches Traefik. The KeenDNS default 404 is not proof of correct forwarding.

  4. Configure the Traefik websecure entrypoint, persistent ACME storage, and HTTP-01 resolver through the repository's GitOps path.

  5. Add TLS annotations and host entries to the public Ingress resources.

  6. Apply through Flux only after rendering and reviewing the complete diff.

  7. Check the Traefik logs and ACME resource/state for the challenge result.

  8. Test HTTP-01 reachability:

    curl -4v http://immich.rudolf.keenetic.link/.well-known/acme-challenge/test
    

    This request must reach Traefik, not the KeenDNS web proxy.

  9. Test TLS and SNI directly against the WAN address:

    curl -4vk \
      --resolve immich.rudolf.keenetic.link:443:<wan-ip> \
      https://immich.rudolf.keenetic.link/
    
  10. Verify the certificate SAN, the Traefik route, the HTTP status, and the Immich backend separately. Repeat the TLS/SNI test for every public hostname.

Failure boundaries

  • DNS resolves incorrectly: fix DNS before changing Kubernetes.
  • TCP 80 or 443 is refused: fix Keenetic forwarding or WAN reachability.
  • KeenDNS certificate/default 404 appears: traffic still uses the router proxy instead of TCP forwarding.
  • ACME challenge fails: inspect port 80, Host preservation, and Traefik resolver/storage before changing backend routes.
  • TLS succeeds but the route is 404/502: inspect the Ingress, Service, Endpoints, and Traefik access log separately.