Files
hermesandHermes Agent 0d29c7d7af fix(ingress): let Traefik own HTTP-01 challenge path (#23)
## Problem

Let's Encrypt HTTP-01 requests were reaching ordinary Kubernetes HTTP routes instead of Traefik's internal ACME challenge handler. The CA observed backend responses (404/401/403), so Traefik continued serving its default certificate.

## Change

- remove the Kubernetes `web` catch-all redirect Ingress and its backend routes;
- restore Traefik entrypoint HTTP-to-HTTPS redirect;
- set the redirect port explicitly to public `443`, preventing internal `:8443` in `Location`;
- keep `web` available for the built-in HTTP-01 challenge router;
- update the ingress decision/runbook.

## Verification

- `git diff --check` passes;
- `kubectl kustomize k3s_rencher/infrastructure/ingress` renders successfully;
- rendered resources no longer include the Kubernetes HTTP redirect catch-all;
- the Traefik redirect explicitly targets public port `443`.

## Post-merge acceptance

Verify externally that HTTP redirects to `https://<host>/` without `:8443`, a real HTTP-01 attempt succeeds, and the served certificate has Let's Encrypt issuer/SANs for all four domains.

---------

Co-authored-by: Hermes Agent <hermes@localhost>
Reviewed-on: #23
Reviewed-by: admin <rudolfss13@gmail.com>
Co-authored-by: hermes <rudolfss13+hermes@gmail.com>
2026-11-22 22:29:19 +01:00
..
2026-10-23 23:13:12 +02:00

Install the server on odroid

curl -sfL https://get.k3s.io | sh -s - server \
  --node-ip="100.70.193.97" \
  --node-external-ip="100.70.193.97" \
  --tls-san="100.70.193.97" \
  --flannel-iface="tailscale0" \
  --flannel-external-ip \
  --flannel-backend="vxlan"

Get the join token from odroid:

sudo systemctl is-active k3s sudo cat /var/lib/rancher/k3s/server/node-token

Copy that value into K3S_TOKEN on each node only when running its join command. Do not commit the token to this file.

Join rudolf-T

Run on rudolf-T:

export K3S_TOKEN='<TODO>'
sudo env K3S_TOKEN="$K3S_TOKEN" sh -c '
set -eu

install -d -m 0755 /etc/rancher/k3s
install -d -m 0755 /etc/systemd/system/k3s-agent.service.d

printf "%s\\n" \
  "nameserver 192.168.1.1" \
  "options timeout:2 attempts:2" \
  > /etc/rancher/k3s-resolv.conf

printf "%s\\n" \
  "[Service]" \
  "BindReadOnlyPaths=/etc/rancher/k3s-resolv.conf:/etc/resolv.conf" \
  > /etc/systemd/system/k3s-agent.service.d/10-registry-dns.conf

systemctl daemon-reload

curl -sfL https://get.k3s.io | \
  K3S_URL="https://100.70.193.97:6443" \
  K3S_TOKEN="$K3S_TOKEN" \
  sh -s - agent \
  --node-ip="100.105.201.15" \
  --node-external-ip="100.105.201.15" \
  --flannel-iface="tailscale0"
'

The resolver override is mounted only into the k3s-agent service namespace. It does not replace the Ubuntu host resolver for other services. The embedded containerd inherits it for image pulls.

Join xu4

Run on xu4:

export K3S_TOKEN='<TODO>'
curl -sfL https://get.k3s.io | \
  K3S_URL="https://100.70.193.97:6443" \
  K3S_TOKEN="$K3S_TOKEN" \
  sh -s - agent \
  --node-ip="100.96.92.102" \
  --node-external-ip="100.96.92.102" \
  --flannel-iface="tailscale0"

Verify from odroid

sudo k3s kubectl get nodes -o wide
sudo k3s kubectl get pods -A -o wide
sudo k3s kubectl get --raw='/readyz?verbose'
sudo k3s kubectl get events -A --sort-by=.lastTimestamp | tail -n 50

Health

API server health

kubectl get --raw='/readyz?verbose' kubectl get --raw='/livez?verbose'

Node health and scheduling status

kubectl get nodes -o wide

All system and workload pods

kubectl get pods -A -o wide

Recent warnings/errors across namespaces

kubectl get events -A --sort-by=.lastTimestamp | tail -n 50

Request a fresh Git fetch. The source update triggers its dependent Kustomization.

kubectl annotate -n flux-system --overwrite gitrepository/k3s reconcile.fluxcd.io/requestedAt="$(date +%s)"

Compare the latest fetched commit with the last successful application.

kubectl get kustomization k3s -n flux-system -o jsonpath='applied: {.status.lastAppliedRevision}{"\nattempted: "}{.status.lastAttemptedRevision}{"\n"}'

Show reconciliation conditions and recent errors/events.

kubectl describe kustomization k3s -n flux-system