G1: Veron 1 host live behind Cloudflare Tunnel

app.windygit.com / api.windygit.com / models.windygit.com are serving over
HTTPS with ZERO inbound ports open on Grant's network.

  - tunnel 4e856c5d, 4 registered edge connections, systemd-managed and bounded
  - three proxied single-level CNAMEs (Free Universal SSL covers them; a
    two-level name would need ACM and would die in the TLS handshake)
  - services bound to 127.0.0.1 with configurable host ports — Veron 1 is
    Grant's workstation and 3000/3300 belong to other projects
  - docs/RUNBOOK-VERON.md

I-12 PROVEN IN PRODUCTION: /version reports source=baked with a sha equal to
the deployed HEAD.

Also fixed: the tunnel health probe targeted localhost from inside a container,
so it was permanently red. A check that is always red is as useless as one that
is always green — it is how a fleet canary goes 37 days dead unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Grant Whitmer
2026-08-11 14:34:18 -04:00
parent 67753497f8
commit a68261a563
5 changed files with 102 additions and 3 deletions

View File

@@ -20,7 +20,7 @@ boot guard in `api/app/main.py` that refuses to start there in production.
| Port | Service | | Port | Service |
|---|---| |---|---|
| 8600 | `windy-git-api` — our plane | | 8600 | `windy-git-api` — our plane |
| 3000 | Gitea | | **3080** | Gitea — host 3000 and 3300 are taken by resident projects on Veron 1 |
| 5432 | Postgres | | 5432 | Postgres |
| 2000 | cloudflared metrics (probe target) | | 2000 | cloudflared metrics (probe target) |
@@ -30,7 +30,7 @@ Zone `windygit.com` = `9d8637dcac3415607b2116e6099fe567` · account `193b347aede
| Hostname | → | | Hostname | → |
|---|---| |---|---|
| `app.windygit.com` | Gitea :3000 — UI **and** git over HTTPS | | `app.windygit.com` | Gitea :3080 — UI **and** git over HTTPS |
| `api.windygit.com` | our plane :8600 | | `api.windygit.com` | our plane :8600 |
| `models.windygit.com` | HF-compatible endpoint :8600 (v2) | | `models.windygit.com` | HF-compatible endpoint :8600 (v2) |
| `windygit.com` | Cloudflare Pages — marketing, Grant-gated | | `windygit.com` | Cloudflare Pages — marketing, Grant-gated |

View File

@@ -50,6 +50,12 @@ class Settings(BaseSettings):
# ---- account-server OIDC (human identity) ----------------------------- # ---- account-server OIDC (human identity) -----------------------------
account_server_base_url: str = "https://account.windyword.ai" account_server_base_url: str = "https://account.windyword.ai"
# cloudflared binds its metrics on the HOST, so from inside a container
# `localhost` is the wrong box. A health check that is permanently red is as
# useless as one that is permanently green -- it trains people to ignore the
# dashboard, which is how a 37-day-dead fleet canary goes unnoticed.
tunnel_metrics_url: str = "http://host.docker.internal:2000/metrics"
# ---- storage law (I-3, G4.4) ------------------------------------------ # ---- storage law (I-3, G4.4) ------------------------------------------
# Git object databases MUST live on a POSIX filesystem. A test asserts this # Git object databases MUST live on a POSIX filesystem. A test asserts this
# path does not resolve to a network mount. # path does not resolve to a network mount.

View File

@@ -122,7 +122,7 @@ class TunnelProvider(Provider):
async def probe(self) -> ProbeResult: async def probe(self) -> ProbeResult:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client: async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
try: try:
r = await client.get("http://localhost:2000/metrics") r = await client.get(self._s.tunnel_metrics_url)
except httpx.RequestError as exc: except httpx.RequestError as exc:
return ProbeResult(False, f"cloudflared metrics unreachable: {exc}") return ProbeResult(False, f"cloudflared metrics unreachable: {exc}")
return ProbeResult(r.status_code == 200, f"cloudflared metrics -> {r.status_code}", True) return ProbeResult(r.status_code == 200, f"cloudflared metrics -> {r.status_code}", True)

View File

@@ -24,6 +24,9 @@ services:
# nothing needs to be reachable from the LAN, let alone the internet (G1.6). # nothing needs to be reachable from the LAN, let alone the internet (G1.6).
ports: ["127.0.0.1:${API_PORT:-8600}:8600"] ports: ["127.0.0.1:${API_PORT:-8600}:8600"]
depends_on: {db: {condition: service_healthy}} depends_on: {db: {condition: service_healthy}}
# cloudflared runs on the host, not in this network. Without this the tunnel
# probe is permanently red and stops meaning anything.
extra_hosts: ["host.docker.internal:host-gateway"]
restart: unless-stopped restart: unless-stopped
gitea: gitea:

90
docs/RUNBOOK-VERON.md Normal file
View File

@@ -0,0 +1,90 @@
# RUNBOOK — Windy Git on Veron 1 (rung R0)
Host `Veron-1-5090`, WireGuard `10.10.0.6`, alias `wg-veron`. Passwordless sudo.
⛔ **Kit 0 is never a host for this service** (D-4). `api/app/main.py` refuses to
boot in production if it finds itself on `72.60.118.54`.
## Layout
| Path | Holds |
|---|---|
| `/srv/windygit/src` | the deploy checkout (clone of `sneakyfree/windy-git`) |
| `/srv/windygit/git` | **git object databases + Gitea data** — local NVMe, truth (I-3) |
| `/srv/windygit/src/data/pg` | Postgres data |
| `/etc/cloudflared/config.yml` | tunnel ingress |
| `/etc/cloudflared/windy-git.json` | tunnel credentials, mode 600 |
| `/srv/windygit/src/.env` | secrets, mode 600, **never committed** |
## Ports — all loopback, on purpose
| Port | Service |
|---|---|
| `127.0.0.1:3080` | Gitea (host 3000 is a resident node dev server; 3300 is nginx — **do not fight them for a port**) |
| `127.0.0.1:8600` | windy-git API |
| `127.0.0.1:2000` | cloudflared metrics |
**No inbound port is opened.** cloudflared dials out, so the dynamic residential
IP is irrelevant and there is no firewall hole to maintain.
## Start / stop
```bash
ssh wg-veron
cd /srv/windygit/src
sudo docker compose ps
sudo docker compose logs -f api
sudo systemctl status windygit-tunnel
```
## Deploy
```bash
ssh wg-veron
cd /srv/windygit/src && git pull
export COMMIT_SHA_BUILD=$(git rev-parse HEAD) BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
sudo -E docker compose up -d --build
curl -s https://api.windygit.com/version # MUST equal git rev-parse HEAD
```
⚠️ **Never put `COMMIT_SHA` in `.env`.** It does nothing here — the sha is baked
into the image and a runtime override is ignored with a warning (I-12). That env
pin is the documented root cause of nine sibling services misreporting their
commit, and one reporting another repo's commit entirely.
## Verify (the four things that must be true)
```bash
curl -s https://api.windygit.com/version | jq # source must be "baked"
curl -s https://api.windygit.com/health/full | jq # degraded is HONEST, not broken
curl -sI https://app.windygit.com/ | head -1 # Gitea, 200
sudo ss -tlnp | grep -E "3080|8600" # both must be 127.0.0.1
```
## Troubleshooting
**A hostname returns 530 or won't resolve** — the tunnel is down. `sudo systemctl
restart windygit-tunnel`, then `journalctl -u windygit-tunnel -n 50`.
**TLS handshake fails with `curl` exit 35 and no HTTP status at all** — someone
added a **two-level** hostname. Free Universal SSL covers `windygit.com` and
`*.windygit.com` only. The request dies before the tunnel is consulted, so it
presents as "the app is broken" when the app is perfect. Either go back to a
single level or buy Advanced Certificate Manager ($10/mo).
**Port bind fails on `docker compose up`** — a resident project took the port.
Set `GITEA_PORT` / `API_PORT` in `.env` and update `/etc/cloudflared/config.yml`
to match. **Never stop another project's container to free a port.**
**`/health/full` says degraded** — that is the design (I-8). Read `checks`: an
unconfigured provider is honest, not broken. R2, Gitea admin token and Eternitas
are wired in strands G2–G4.
## Promotion to R1 (first external push)
R0's honest limits: no SLA, it is Grant's workstation, and there are no
VPS-style snapshots. All acceptable while Grant is the only user; all
disqualifying the moment a stranger depends on it. **The trigger is not a date —
it is the first external push.** Move the control plane to a dedicated VPS (not
Kit 0), keep Veron 1 as the runner. It is an rsync, a Postgres dump and three
DNS record edits.