125 Commits

Author SHA1 Message Date
Kit OC5
6f19e23eaf lockbox-names: names-only lister (section + label + resolvable yes/no/dup), never a value
All checks were successful
check / gate (push) Successful in 14s
canary / probe (push) Successful in 6s
So lanes can discover what the lockbox holds without opening it (Super Admin 50 request).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 19:23:47 -04:00
ef96051d6f Merge pull request #2 from sneakyfree/runner-guard
All checks were successful
check / gate (push) Successful in 14s
canary / probe (push) Successful in 6s
runner-guard: no stranger code on self-hosted runners (PR check + nightly sweep)
2026-10-01 05:29:07 -04:00
Kit OC5
a0dc6f8568 runner-guard: block workflows that hand a self-hosted runner to strangers (Boss 10-01)
R1 pull_request_target; R2 fork pull_request on self-hosted without a same-repo/environment
gate; R3 outsider events (issue_comment, workflow_run, ...) on self-hosted; R0 unparseable.
PR mode in the 5-min sync posts windy-git/runner-guard on open PRs of public sneakyfree repos
that change a workflow (each status once). Nightly sweep (Windy 0 timer) over every public
repo: page + BOARD line on new hits + red windy-job heartbeat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 05:28:37 -04:00
Kit OC5
51e0b9d30b bridge + sync: onboard windy-calendar-site (static, no workflows; guards only)
All checks were successful
check / gate (push) Successful in 28s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 05:04:49 -04:00
Kit OC5
51777b0ad0 bridge + sync: onboard windy-hand-site (static site, no workflows yet; guards only)
All checks were successful
check / gate (push) Successful in 13s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 04:50:15 -04:00
Kit OC5
cbcb4c206b docs: CUTOVER-PRIMARY.md checklist (draft, nothing flipped)
All checks were successful
check / gate (push) Successful in 45s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 04:03:09 -04:00
Kit OC5
2c62e2834a secret-guard allow: windy-agent #412 synthetic Z.ai fixture (hash not in lockbox)
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:56:48 -04:00
3a9b7ecab7 Merge pull request #1 from sneakyfree/veron-ollama-warn
All checks were successful
check / gate (push) Successful in 22s
canary / probe (push) Successful in 8s
compute-guard: WARN on new Veron Ollama (:11434) references
2026-10-01 03:36:32 -04:00
Kit OC5
cd0400c371 compute-guard: WARN (never red) on NEW references to Veron Ollama :11434
Grant via Boss 10-01: compute = Windy Mind (endpoint + key); do not call Veron Ollama directly.
Judged on lines a PR adds only (not the baseline tree), never blocks even in MODE=block,
windy-mind (the compute door) allowed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:36:11 -04:00
Kit OC5
4e7ab667ed backup_state: pin restic cache dir (systemd has no HOME); init only on a MISSING repo, log other errors
All checks were successful
check / gate (push) Successful in 38s
canary / probe (push) Successful in 10s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:23:41 -04:00
Kit OC5
f10db19316 drill_cross_host.sh: read the R2 pair + endpoint from the lockbox (drill PASSED 10-01)
All checks were successful
check / gate (push) Successful in 18s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:19:56 -04:00
Kit OC5
3503894a1e state backup: systemd units (NOT enabled) + cross-host drill script
All checks were successful
check / gate (push) Successful in 23s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:59:19 -04:00
Kit OC5
fbab5c4669 secret-guard/secret-scan: PyPI API token shape (pypi-AgE macaroon), WARN-first
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 13s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:47:53 -04:00
Kit OC5
1b552c0882 docs: RESTORE-DRILL.md (state backup + restore procedure)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:46:53 -04:00
Kit OC5
8ebc18c3bc gitea 1.24.6 -> 1.24.7 (security: LFS auth bypass, symlink bypass, OAuth2 missed return)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:45:28 -04:00
Kit OC5
9566826ce9 lockbox-put: append-only lockbox PR tool (no one reads/greps the lockbox); installer updated
All checks were successful
check / gate (push) Successful in 28s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:40:31 -04:00
Kit OC5
160a3d69ba backup_state.sh: encrypted restic backup of Postgres + Gitea state to R2 (SOTU 10-01 gap: DB was not backed up)
All checks were successful
check / gate (push) Successful in 23s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:35:09 -04:00
Kit OC5
7e28a23c22 secret-scan + env-names: shared hash-only tools (Boss 10-01: lanes printed secrets while hunting them)
All checks were successful
check / gate (push) Successful in 9s
canary / probe (push) Successful in 5s
secret-scan reports file:line/commit + lockbox KEY NAME or shape, never a value or fragment;
env-names lists variable names/length/hash only. Same shapes as secret-guard. Seeded-fake tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 01:17:30 -04:00
Kit OC5
1da4d39c0b bridge + sync: onboard windy-text, windy-call, windy-cell (telephony in scope, GitHub Actions dead since 08-14; deploy.yml disabled)
All checks were successful
check / gate (push) Successful in 14s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 01:00:48 -04:00
Kit OC5
53fbcfe78f secret-guard allow: windy-code VS Code public aiKey, windytalk redaction fixture (hash not in lockbox)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 00:59:59 -04:00
Kit OC5
1c552e94de secret-guard: Twilio shapes (SID/API key, 32-hex secret assignment) + per-kind warn mode
Windy Text 10-01: a live Twilio auth token sat in bridged-repo tests. Auth tokens are bare
32-hex, so they are matched only when assigned to a name containing token/secret/key/password;
hash is of the value alone. SECRET_GUARD_WARN_KINDS lets a new shape warn before it blocks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 00:59:05 -04:00
Kit OC5
8690c4f8d3 secret-guard allow: 5 windy-agent #395 FAKE fixtures (weekly scan 09-28; verified never in the lockbox)
All checks were successful
check / gate (push) Successful in 17s
canary / probe (push) Successful in 5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 06:13:22 -04:00
Kit OC5
313f41b49c deploy: secret-guard BLOCK drop-in (orchestrator 09-24)
All checks were successful
check / gate (push) Successful in 8s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:08:11 -04:00
Kit OC5
ea02ade0cb weekly public secret scan (all 5 GitHub accounts, full history, hash-only)
All checks were successful
check / gate (push) Successful in 13s
canary / probe (push) Successful in 4s
scripts/public_secret_scan.py: every PUBLIC repo, every object (incl.
unreachable + PR refs), secret_shapes patterns, excused by the secret-guard
allow list. Output JSON with hash8 + location only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:05:16 -04:00
Kit OC5
497222094f secret-guard: triaged allow list (fakes by hash, test PEMs by path)
Private-key matches are only the BEGIN line (same hash everywhere), so they
are allowed by path+kind; everything else by hash. Real revoked tokens
(1354fc9b, d49dc2ba) are pinned by a test to never be allowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:04:13 -04:00
Kit OC5
d28da26000 secret-guard: no live credentials in bridged repos (hash-only findings)
All checks were successful
check / gate (push) Successful in 22s
New guard windy-git/secret-guard (warn-only) over EVERY text file: Telegram,
GitHub, AWS, Slack, Anthropic, OpenAI, Stripe live, Google API keys and
private-key blocks (scripts/secret_shapes.py, shared with the weekly public
scan). A finding carries "<kind> #<sha256[:8]>", never the value (house rule
10). Known fakes allowed BY HASH (ci/secret-guard-allow.yml). GUARDS_STATUS
gets a secrets column. Leak hunt 09-24: @Windy_0_bot token in a public fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:01:59 -04:00
Kit OC5
04c857654a rerun_ci.sh: find wg-q in the invoking user home under sudo
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 5s
Under sudo ~ is /root, so the final run listing failed (windy-inbox #14).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 02:38:25 -04:00
Kit OC5
32512a32fc bridge windy-inbox (new private repo; Windy Drops is build lead)
All checks were successful
check / gate (push) Successful in 17s
canary / probe (push) Successful in 4s
sync REPOS + BRIDGE_REPOS + guards page (owner Windy Drops). Imported
writable into Gitea first (repo 164).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:51:09 -04:00
Kit OC5
b52e6b4784 bridge: post no-Docker smoke jobs (name regex skipped "no Docker")
All checks were successful
check / gate (push) Successful in 9s
canary / probe (push) Successful in 5s
windy-search #96 "Boot smoke (no Docker)" passed but was hidden by the
image-build name filter. Negative lookbehind for no/no-/without.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:11:55 -04:00
Kit OC5
6440c7d5f4 deploy: compute-guard BLOCK drop-in (live on Veron since 09-24 00:0xZ)
All checks were successful
check / gate (push) Successful in 12s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:05:34 -04:00
Kit OC5
17acae6af6 guards: Grant-owned findings never block (compute-guard + ci-hygiene)
status_for(lane, whole_tree, grant=...): only lane-owned findings fail in
MODE=block; Grant-owned (ci/grant-owned.yml) post WARN. The bridge splits via
guards_report.split_grant; if the split cannot run it WARNs (never blocks).
Orchestrator 09-23: block compute-guard for lane-owned paths only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:04:01 -04:00
Kit OC5
790d794900 bridge: drop eternitas ci/build from BRIDGE_NO_DAEMON (converted to ci/smoke, #179)
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:51:26 -04:00
Kit OC5
1b8cebbfe1 branding: friendly invite-only page instead of Gitea's raw Registration is disabled
A Windy account with no forge account (registration CLOSED at launch, Grant
09-23) lands on /user/link_account. Override shows invite-only + signed-in name
+ link to the dashboard; other cases = Gitea 1.24.6 template verbatim. No change
to who can register. Orchestrator ask 09-23.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:49:34 -04:00
Kit OC5
83fbf1f992 guards_report: chat lane is now the Windy Chat session
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:12:40 -04:00
Kit OC5
9b5334fee7 test: allow-list entries must be line-scoped (replaces is-empty check)
f71a5aa pushed with this test red (tail hid pytest rc); fixed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:11:53 -04:00
Kit OC5
f71a5aab39 ci-hygiene allow: windy-pro disabled deploy job (needs-docker false positive)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:11:20 -04:00
Kit OC5
f219437282 ci-hygiene report: same disabled-workflow filter as check()
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:10:35 -04:00
Kit OC5
b15584d33a ci-hygiene: needs-docker skips workflows disabled on Windy Git
deploy/release workflows run on the target host (real daemon) and are
disabled here (repo_unit DisabledWorkflows); one bounded query per process,
flag everything if it fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:10:11 -04:00
Kit OC5
0a57d96f2e bridge + ci-hygiene: Docker-needing CI jobs (option A)
- bridge: BRIDGE_NO_DAEMON names image-build jobs whose name lacks docker
  (default eternitas:ci/build); never posted, like the docker-named ones.
- ci-hygiene: flag docker build/buildx/run/compose, docker-compose and
  docker/build-push-action in workflow steps ("needs docker") with the fix:
  job services: + a no-Docker smoke test; the image builds at deploy.
- test_guards_report: owner column (14ed23a broke it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:09:09 -04:00
Kit OC5
8b1ae4ac60 branding: home page says invite only (forge registration closed at launch)
Some checks failed
check / gate (push) Failing after 7s
canary / probe (push) Successful in 5s
Site-honesty re-audit 3.17 (Traveler): the home page promised sign-in with any
Windy account while ENABLE_AUTO_REGISTRATION=false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:02:55 -04:00
Kit OC5
14ed23a9fe guards_report: owner-lane column (session names: 8c -> Windy Hub, calendar -> Windy Calender)
Some checks failed
check / gate (push) Failing after 10s
canary / probe (push) Failing after 1m5s
Orchestrator routing 09-23: hub/account-server/dashboard/site -> Windy Hub;
windy-calendar only -> Windy Calender; telemetry -> Windy Admin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 18:44:11 -04:00
Kit OC5
c4799dcc13 ci: mount windy-pro's read-only build inputs into dind; allow exactly that path for jobs
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 5s
Non-secret inputs git-ignored in windy-pro (models, linux-x64 portable
bundle, enter-monitor build) that build-desktop needs. Mounted :ro into
dind; valid_volumes allows only /ci-inputs/windy-pro; refresh-ci-inputs.sh
copies them from the frozen release clone (read-only on the source).
Invariant I-5 narrowed, not dropped: exactly that one path, read-only in
dind, no other service mounts it, still no docker socket (proven to fail
on :rw). Orchestrator-approved (option a). Applied in an idle window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 17:28:48 -04:00
Kit OC5
683c01ec7d rerun_ci: Gitea stores repos lowercased on disk (WindyCloud failed)
All checks were successful
check / gate (push) Successful in 13s
canary / probe (push) Successful in 5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 17:22:17 -04:00
Kit OC5
4bb4893131 bridge: replace the mirror PR when a GitHub PR is retargeted to another base
All checks were successful
check / gate (push) Successful in 8s
canary / probe (push) Successful in 5s
The mirror kept its creation-time base forever. eternitas #167 was stacked
on fix/one-hallway, retargeted to main after #166 merged; ci.yml
(pull_request: branches [main]) then silently never ran for it, while
unfiltered workflows did. An edited event triggers nothing, so close the
stale mirror and open a fresh one on the new base (runs CI at once).
An unknown base is left alone, never guessed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 16:16:15 -04:00
Kit OC5
9e9eb08d96 compute guard: line-scoped allow entries; allow MindKeychain.jsx's two OAuth endpoints only
All checks were successful
check / gate (push) Successful in 12s
canary / probe (push) Successful in 4s
Allow entries may carry matches: (regexes); then only matching lines are
allowed, so an allowed file can't smuggle in a new call. windy-pro #609
MindKeychain.jsx: openrouter.ai/auth? and /api/v1/auth/keys (BYOK key
acquisition via OAuth PKCE, no inference; successor of the MindPanel
allow, ADR-064). An inference call in the same file still flags (tested).
Orchestrator-approved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:46:53 -04:00
Kit OC5
dece301bf0 grant-owned: windy-pro ROOT .env.example (desktop BYOK dev fallback; hub reads neither)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:45:33 -04:00
Kit OC5
87dcddb87d guards_report: one live status page for compute-guard + ci-hygiene
All checks were successful
check / gate (push) Successful in 15s
Scans every bridged default branch with both guards; lane-owned vs
Grant-owned (ci/grant-owned.yml: windy-pro desktop paths + its desktop CI
jobs, attributed per job) so Grant's code never holds up a block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:38:21 -04:00
Kit OC5
a5f7b036a8 guards: skip a commit the sync hasn't fetched yet, quietly (race, not error)
All checks were successful
check / gate (push) Successful in 15s
canary / probe (push) Successful in 4s
The bridge reads PR/default heads from GitHub after the sync's fetch; a
push in between isn't in the clone until the next cycle. Both guards logged
a CalledProcessError for it (windy-pro main 40 s after the fetch). Now
None = nothing posted this cycle; the next one scans it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:36:19 -04:00
Kit OC5
255aa58b35 ci-hygiene guard: lockfile-only installs, pinned images, no host-port services (warn-only)
All checks were successful
check / gate (push) Successful in 19s
House rule 6 (09-23). The bridge now also posts windy-git/ci-hygiene on
every PR head (added lines) and default branch (whole files), scanning CI
workflows and Dockerfiles for: floating pip / uv pip installs (not -r,
not --no-deps, not exact pins), uv sync without --locked/--frozen,
npm install instead of npm ci (unless every package is exact-pinned),
yarn/pnpm without a frozen lockfile, :latest images and COPY lock* globs
(Windy Mail #147), and CI services publishing a HOST port (every job
shares one dind: Windy Mind runs 147/176 died on 5432). Warn-only;
CI_HYGIENE_MODE=block later. Allow-list ci/ci-hygiene-allow.yml (empty).

compute_guard's walker is now parameterised (line_fn / path_ok /
prefilter) so both guards share one scanner, cache and allow loader; the
bridge posts both through one _post_guard. Today: 95 issues in 21 repos;
windy-git, calendar, traveler, traveler-site clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:30:51 -04:00
Kit OC5
e64a1b5fcb deps: install from uv.lock (hash-pinned) in the image and CI, never floating
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 5s
The API image did 'pip install -e .' and CI 'pip install -e ".[dev]"':
every rebuild could ship newer fastapi/starlette/pydantic than CI tested
(Windy Cloud hit exactly this 09-23). uv.lock is cut to EXACTLY what prod
runs now (42 runtime pkgs, 0 differences; botocore/pyjwt held back to
prod's versions). Image: uv 0.12.5 exports, pip --require-hashes installs,
same layout. CI: uv sync --locked (also fails on a stale lock).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:25:18 -04:00
Kit OC5
1bc55eaec9 compute guard: flag direct AI-provider use (Windy Mind is the only door), warn-only
All checks were successful
check / gate (push) Successful in 28s
canary / probe (push) Successful in 14s
Grant's rule (09-23): every model call goes through Windy Mind. The bridge
now posts windy-git/compute-guard on every PR head (lines the PR ADDS vs its
merge-base) and default-branch head (whole tree): provider hosts, provider
SDK imports/deps and raw provider key names. Warn-only: success + "⚠ WARN"
and a link to the first hit; COMPUTE_GUARD_MODE=block turns it red later.

Exceptions live in ci/compute-guard-allow.yml, each with a reason (Mind
itself, user-BYOK windy-agent / windy-code extension / windy-pro desktop +
MindPanel, windy-connect config writers). Tests, docs, comments, lockfiles,
vendored code and CI config are never scanned. Reads the sync's bare clones
(no docker exec); cached per (repo, sha, rules). Non-fatal; never a fake OK.

First cases = COMPUTE_BYPASS_AUDIT.md. Today on default branches: 38
findings in 3 repos (windy-chat audit #2, windy-pro account-server #3/#4,
windytalk reference/), 0 elsewhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:42:10 -04:00
Kit OC5
fdb0f5989e ci: run dind under Sysbox, not privileged (rollback override kept)
Some checks failed
canary / probe (push) Has been cancelled
check / gate (push) Has been cancelled
dind was privileged: true, so a job that escaped into dind was root on
Veron 1, which is Grant's workstation. Under sysbox-runc (sysbox-ce 0.7.1,
installed 09-23 with no docker restart) dind root is an unprivileged host
uid. Smoke-tested standalone: nested containers, internet, a services-style
postgres on a private network and a python image all pass unprivileged.
Fresh volume dind-storage-sysbox; the old dind-storage stays for
docker-compose.privileged.yml, the one-command rollback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:23:55 -04:00
Kit OC5
9be2952ff8 rerun_ci: detect a running oneshot sync correctly (is-active lies for oneshot)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:22:59 -04:00
Kit OC5
fe3bce39ff bridge: post pending for queued jobs a runner can take
All checks were successful
check / gate (push) Successful in 28s
Gitea 1.24 lists only picked-up jobs, so a queued PR showed NOTHING on
GitHub and lanes asked whether their push was lost (Windy Mind #131,
Windy Cloud today). The bridge now reads waiting jobs from the gitea DB
and posts pending where nothing newer was picked up; a queued re-run
supersedes the stale failure it replaces.

Only status 5 jobs whose runs-on labels a live runner has: blocked jobs
often end skipped and label-unrunnable jobs are cancelled unpicked, and
neither ever reaches /actions/tasks, so their pending would never resolve.
Lookup is bounded (30 s) and non-fatal: the IO-stall lesson.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:20:32 -04:00
Kit OC5
6b0b60eb4a scripts: rerun_ci.sh — re-fire a PR's CI without the web button
Gitea 1.24 has no rerun API and the web button needs Grant's SSO identity.
Guarded branch rewind that the next sync undoes; restores the branch itself
on timeout. Used today for eternitas #166 and windy-mind #131 after the
Veron IO stall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:17:10 -04:00
Kit OC5
60708dd8db push velocity: correlate the SSO-id subquery on the grouped column
All checks were successful
check / gate (push) Successful in 21s
canary / probe (push) Successful in 9s
Dry-run against the real gitea DB: 'subquery uses ungrouped column u.id'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:06:27 -04:00
Kit OC5
aedc772d29 push velocity: isolate its query so a failure never costs ci.run rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:06:27 -04:00
Kit OC5
acb8ec3a16 push velocity: key humans on windy_identity_id (SSO link); no id -> system + caller
Telemetry UPDATE 2 actor rule: agent/human rows without actor_id are
quarantined. Forge humans sign in only via Windy SSO, so Gitea's
external_login_user.external_id is their windy_identity_id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:06:27 -04:00
Kit OC5
0051c72037 telemetry: detect push velocity from Gitea's action table (alert only)
git push never touches our API, so throttle.py can't see it. Gitea's
action table records every push; the 5-min sync-side emitter now reads
it and emits forge.push_velocity when an account crosses 60 pushes/1h,
500 pushes/24h (standard-band base) or 10 ref deletes/24h. One row per
account per rule per window while over; windyadmin (the sync) exempt.
Nothing sits in the push path and nothing is refused. HOLD until
Telemetry Boss declares the shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:06:27 -04:00
Kit OC5
4c76b1b12a canary: end the hub session the login probe opens (journey cleanup rule)
All checks were successful
check / gate (push) Successful in 32s
canary / probe (push) Successful in 12s
identity.login created a live hub session every 10 min and never ended it.
It now logs out with the token it got: retried on 5xx / no response
(8 x 15 s), 401/404/410 = already over, any other 4xx fails fast, and a
cleanup it can't finish is reported as identity.logout DOWN "CLEANUP
FAILED" (alerts + red run). The hub's /auth/logout revokes every refresh
token of the account (verified live), so the next run's logout heals a
leftover; no ledger needed. Proven end to end: login 200, logout 200,
10/10 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 13:39:55 -04:00
Kit OC5
2d4fadb090 sync: bound the janitor and telemetry steps (docker exec hangs in an IO stall)
Some checks failed
canary / probe (push) Has been cancelled
check / gate (push) Has been cancelled
09-23 16:43Z the Veron data2 SMR stall left runc exec in D state; the
janitor's docker exec never returned, so the sync sat 'activating' and no
repo mirrored or got a status for any lane. Both steps are non-fatal;
now they time out (120 s / 180 s) and the run continues.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:54:34 -04:00
Kit OC5
e3b69fa759 telemetry: UPDATE 7 — read the ingest body; count quarantined + dropped on heartbeats
Some checks failed
canary / probe (push) Has been cancelled
check / gate (push) Has been cancelled
The ledger answers 202 even when it quarantines rows. Both emitters now log
a warning with the reasons and report service.health.telemetry_quarantined
and telemetry_dropped (API: buffer overflow; sync: 0 by construction, since
a failed send keeps cursor + spool). HOLD until Telemetry Boss declares both
keys on windy-git's two service.health shapes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:32:28 -04:00
Kit OC5
4acf50d9ef bridge tests: fake serves workflow contents; cover invalid-workflow status
All checks were successful
check / gate (push) Successful in 23s
b7a7e94 made the bridge read workflow files, which the strict fake Gitea
refused (7 red). The fake now serves contents (404 when absent), and new
tests cover: error posted with no runs, valid files add nothing, no repost,
.gitea/workflows wins over .github/workflows, and each workflow_problem
shape. pyyaml declared in dev extras (the bridge imports it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:31:19 -04:00
Kit OC5
b7a7e94df0 bridge: post an error status when Windy Git ignores an invalid workflow
Gitea drops an invalid workflow file with one log line and fires no run, so
the GitHub PR showed nothing and lanes waited for CI that never came
(windytalk #100). The bridge now reads each workflow file at the commit it
reports on and posts windy-git/<wf>/workflow = error with the reason.
Verified: 0 false positives on all 23 bridged repos' main; catches
windytalk #100's broken commits (invalid YAML at line 12), fix commit clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:29:12 -04:00
c83f808a60 bridge: retry transport blips (TLS timeout/reset), never HTTP errors
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 9s
A single GitHub TLS handshake timeout failed the whole sync, flipped its
windy-job heartbeat to ok:false and would page for nothing. Up to 3
attempts with backoff for URLError/timeout/reset; HTTP errors return
immediately as before. Test covers both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:15:22 -04:00
eb27e63db3 telemetry: adopt the end-to-end synthetic convention (UPDATE 4)
All checks were successful
check / gate (push) Successful in 24s
canary / probe (push) Successful in 9s
Replaces the keyed marker from 1c3b5b0 with the ecosystem convention:
any X-Windy-Synthetic value marks the request synthetic; the flag lives in
a per-request contextvar, labels this request's rows, and is FORWARDED on
downstream calls (Eternitas trust lookup, Gitea API). The canary sends
"1". Rows are still recorded; the label separates, never suppresses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:06:51 -04:00
1c3b5b0638 telemetry: synthetic:true on canary refusals (keyed, not a bare flag)
All checks were successful
check / gate (push) Successful in 25s
canary / probe (push) Successful in 7s
The canary deliberately sends forged tokens every 10 min; those refusal
rows read as attacks. It now sends X-Windy-Synthetic carrying a shared
secret (Gitea repo secret CANARY_SYNTHETIC_KEY = WINDYGIT_SYNTHETIC_KEY in
Veron .env); the API marks the row synthetic only on a constant-time
match, so an attacker cannot label their own refusals synthetic to hide.
synthetic is declared on forge.auth.failed (Telemetry Boss, UPDATE 3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:54:56 -04:00
90643fe48e telemetry step 2: API boot/health + forge.auth.failed (declared)
All checks were successful
check / gate (push) Successful in 25s
canary / probe (push) Successful in 6s
Membrane first: I-2 and MEMBRANE.v1 now list the windy-admin ledger
(POST /v1/events). api/app/telemetry.py: service.boot once per start
(commit_sha omitted when unknown, I-12), an hourly in-process
service.health with the shared keys (requests, errors_5xx/4xx,
refusals_4xx, p95_ms only when there was traffic), and one
forge.auth.failed row per refused request: declared 13-code enum,
http_status, caller class, route TEMPLATE (never the concrete path),
actor_type system with no actor_id (all-lanes rule). No token = nothing
sent or buffered; flush failures keep rows (bounded) and never raise.
Token from root-only /etc/windygit/telemetry.env (optional env_file).
8 behavioural tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:47:29 -04:00
00ec963f82 telemetry: fix ci.run completeness — cursor on (finish time, job id)
Some checks failed
check / gate (push) Has been cancelled
Telemetry Boss found jobs_finished=43 vs 8 ci.run rows. Root cause: the
high-water mark was the job id, but jobs FINISH out of id order, so every
long job that started before the mark and finished after it was silently
never emitted. Now a (finish time, id) cursor; finish = stopped, or
updated for skipped jobs with no stop time. Heartbeat finished/failed/
cancelled counts are derived from exactly the rows emitted, so
sum(jobs_finished) == count(ci.run) by construction (dry run on real
data: 97 == 97, failed 2 == 2, cancelled 13 == 13). posted_to_github now
set from the bridge's own rules. duration_ms = Gitea whole seconds x 1000.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:42:36 -04:00
b5e4eaf57a telemetry: ci.job_cancelled from the janitor; interval_s on heartbeat
All checks were successful
check / gate (push) Successful in 22s
canary / probe (push) Successful in 6s
The janitor now returns one JSON line per job it cancels (repo, workflow,
job, reason, runs_on, waited_s) into a spool; the emitter ships them as
ci.job_cancelled (declared with Telemetry Boss) and truncates the spool
only after a 2xx. Run status recompute folded into the same statement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:27:53 -04:00
baaa542bae docs: audit disposition 09-23 — R2 god token replaced by bucket-scoped token
All checks were successful
check / gate (push) Successful in 43s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:15:35 -04:00
0634a6cb1b style: ruff fix in telemetry_emit
All checks were successful
check / gate (push) Successful in 29s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:08:21 -04:00
1a171eabd5 telemetry: CI emitter for admin.windyword.ai (inert until token)
Some checks failed
check / gate (push) Failing after 20s
canary / probe (push) Successful in 6s
ci.run (one row per finished job, exactly once via a high-water mark;
branch_kind default|pr|other so the dashboard can show "main is red") and
service.health (interval counts: finished/failed/cancelled, waiting,
running, runners online, oldest wait). Shapes declared with Windy
Telemetry 40; sends nothing until WINDYGIT_TELEMETRY_TOKEN exists.
State is only advanced after a 2xx, so a failed post retries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:06:54 -04:00
28c31236b8 ci: janitor also clears jobs blocked forever on failed needs
When a needed job fails, Gitea leaves dependants BLOCKED (7) even after
the run finishes; eternitas build jobs sat there 8h. Mark them skipped
(what GitHub shows) once the run is done and 30 min have passed.
Found by the new telemetry dry run (oldest_waiting_s = 29160).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:06:33 -04:00
cd5967031b ci: janitor cancels jobs no runner can ever take
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 6s
windy-pro alone left ~4 jobs per run waiting forever (build-electron on
macos/windows/ubuntu-latest, deploy if:false): Gitea evaluates job if:
only at pick time, the labels do not exist here, and waiting jobs are
invisible in /actions/tasks. 37 such jobs across 10 runs today. After
30 min they are cancelled and the run status recomputed. Runs each sync,
non-fatal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 10:58:53 -04:00
f246417095 ci: windy-pro desktop/installer jobs are NON-BLOCKING (Grant, 09-23)
build-desktop, test-installer and reality-check still run on Windy Git
and stay visible there, but the bridge no longer posts them to GitHub, so
they cannot turn windy-pro's combined status red. Windy Git side only;
the desktop code is Grant's to fix. BRIDGE_NON_BLOCKING, per repo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 10:58:07 -04:00
50c1464043 security: never bundle credential repos to R2 in plaintext
All checks were successful
check / gate (push) Successful in 23s
canary / probe (push) Successful in 7s
kit-army-config (the lockbox) and every *-soul / anima repo carry
credentials; the nightly R2 bundles are unencrypted, so the R2 key was a
key to every secret. Excluded by name (BACKUP_EXCLUDE); they are backed up
encrypted by the Windy Drops lane (restic) and stay mirrored on Veron.
Behavioural test runs the script's own exclusion function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 10:48:42 -04:00
7a63f90da3 ci: bridge windy-mind (private) verdicts to GitHub
All checks were successful
check / gate (push) Successful in 23s
canary / probe (push) Successful in 6s
windy-mind has been writable + CI on Windy Git since 08-13 (deploy.yml
disabled, uv pinned); it only lacked GitHub commit statuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 09:59:28 -04:00
b8f97f0731 ops: sync never pushes archive/* branches to Windy Git
All checks were successful
check / gate (push) Successful in 20s
archive/<machine>-<date>/<branch> are off-machine safety copies of
unpushed work (one-repo doctrine). GitHub holds them; running CI on them
is waste. Negative refspec ^refs/heads/archive/* on the push (git 2.43
on Veron). Requested by 8c for windy-pro's Mac mini archive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 09:57:57 -04:00
95c33c8004 ops: host systemd units in git; windy-pro tags never reach Windy Git
All checks were successful
check / gate (push) Successful in 22s
- deploy/systemd/: sync/backup timers+services, tunnel, and the windy-job
  heartbeat drop-ins (silent-failure audit). They existed only on Veron,
  the same drift that left the runbook wrong. GITHUB_TOKEN is stripped
  (repo is public); it stays in the root-only unit on the host.
- sync: SYNC_NO_TAGS (default windy-pro). build-electron fires on v* tags
  and targets ubuntu/macos/windows-latest, labels no runner has, so it
  would queue forever, invisibly. Desktop releases are built elsewhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 09:51:51 -04:00
d5181f1c6d ci: G11.5 resolved — onboard windy-pro; cloud cells + windytalk; promote script
All checks were successful
check / gate (push) Successful in 21s
canary / probe (push) Successful in 6s
- import_from_github.py no longer refuses windy-pro: lane 8c audited all
  14 checkouts (WINDYPRO_CHECKOUTS.md), GitHub main is canonical.
- sync + bridge: windy-cloud-domains, windy-cloud-vps, windytalk (default
  branch master), windy-pro; windy-cloud-sites added to the bridge (it was
  synced but never bridged).
- scripts/promote_to_ci.sh: the mirror->writable procedure as one script,
  with the delete-before-import hazard documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 09:49:15 -04:00
e6530d3171 test: behavioral G3.5 webhook tests (audit: tests were source-string asserts)
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 9s
Seven HTTP-level tests through the real route: sha256= prefix and bare
digests accepted, digest of re-serialised JSON refused, forged/wrong-key/
missing signatures refused, unset secret -> 503, a revocation with a bad
signature never reaches the handler, the reachability ping never acts.
Mutation-checked: dropping the prefix strip fails the behavioral test
while the old string-grep invariant still passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 03:26:14 -04:00
419443573a ci: onboard windy-hand; stop running windy-agent CI twice
All checks were successful
check / gate (push) Successful in 23s
windy-hand promoted to writable + bridged. windy-agent is PUBLIC and its
GitHub Actions already run on Veron's GitHub runner; running its 3-version
pytest matrix here too was pure duplicate load (3 x ~4.5 cores for 25+
min, host load 64 on 24 cores). Actions are now off for windy-agent on
Windy Git; it stays in REPOS as a synced copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 03:23:25 -04:00
4a34b35441 security: CI egress filter — jobs reach the internet, never Veron/LAN
Measured: an unprivileged job container inside the CI dind could open SSH,
Ollama and dev servers on Veron (192.168.1.73) and the rest of the LAN,
WireGuard and Tailscale — lateral movement for any malicious dependency,
no escape needed. egress.sh (idempotent; windygit-ci-egress.service at
boot) hooks the jobs bridge: runner<->dind, replies, DNS and public
egress allowed; RFC1918, CGNAT, link-local and the host itself dropped.
Verified from a job container: 6/6 private targets blocked, DNS,
internet and the public forge OK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 03:20:45 -04:00
dfe5543eda docs: runbook + AGENTS match reality (item 5 of the launch bar)
Some checks failed
check / gate (push) Has been cancelled
canary / probe (push) Successful in 34s
RUNBOOK-VERON: deploy uses fetch + merge --ff-only and api-only rebuilds
(the old text used git pull, contradicting its own warning); new sections
for host timers, CI (6 runners x1, windyadmin-scoped, 90m ceiling, queue
truth in the gitea DB, logs in R2), sign-in posture and break-glass;
standing checkout = OC5. AGENTS.md no longer says GENESIS / no code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 03:19:12 -04:00
40cb455d0d ci: onboard windy-translate, windytranslate-site, windytraveler-site
Some checks failed
check / gate (push) Has been cancelled
Promoted to writable; the two sites' CF deploy.yml disabled (they would
need the CF god token in a job container). All three only have
ubuntu-latest workflows today, so nothing runs until their lanes switch
runs-on to [self-hosted, linux, x64].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 03:13:40 -04:00
8c404eb410 security: turn off Gitea OAuth auto-registration
Any Windy Word account (public signup, unverified email) auto-registered a
forge account on first sign-in — reproduced with a throwaway account —
and the act runners are instance-wide, so a stranger's workflow would run
on Veron's privileged dind beside the R2 god token. §7 makes opening the
forge to non-Grant users Grant's call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 03:09:43 -04:00
5b16114b98 auth: token contract v1 (aud windy_git, both issuers); CI for eternitas
Some checks failed
check / gate (push) Successful in 37s
canary / probe (push) Has been cancelled
- hub_jwt: aud list is ["windy_git"] (contract v1 array). Dropped
  "windy-git": that is Gitea's OIDC client_id, so a forge id_token would
  have passed the aud check. `type: human` is now REQUIRED (id_tokens have
  none), which makes accepting the discovery-URL issuer safe.
- runner job ceiling 30m -> 90m: eternitas's serial pytest is ~50 min and
  would have been killed mid-suite.
- eternitas (private) added to the GitHub status bridge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:58:56 -04:00
18ea9a4686 auth: G3.2 hub JWKS verifier — humans can sign in to the plane (SSO #14)
The human path refused every token in production (503
human_signin_not_ready) because no verifier existed. api/app/hub_jwt.py
verifies hub access tokens against account.windyword.ai's JWKS:

- RS256 only (closes alg:none and HS256-with-public-key confusion)
- iss must be "windy-identity" — what hub ACCESS tokens carry (observed
  live); id_tokens (discovery-URL issuer) are not accepted as bearers
- aud optional today, must name Windy Git when present; hub_require_aud
  flips it mandatory once the hub emits it. PyJWT's own aud check is off
  on purpose: it rejects ANY aud-bearing token when no audience is given.
- type must be human; identity = windy_identity_id, never sub (per-row id)
- production verifies even if require_verified_jwt is off

11 behavioral tests sign real RS256 tokens with a local key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:55:17 -04:00
32e8ac8474 ci: bridge windy-registry (private) PR/main verdicts to GitHub
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:53:20 -04:00
8e9fa3116c ci: six runners; SSO #8 Gitea sign-in hardening (staged)
- runner-5/6: 50+ jobs were queued with ~11 private repos onboarded. dind
  keeps the 12-core ceiling, so this adds concurrency, not CPU.
- Gitea: password + passkey sign-in forms off (break-glass = CLI), and
  ACCOUNT_LINKING auto -> login. auto linked any hub login whose email
  matched an existing account, and SITE ADMIN windyadmin carries Grant's
  email. Grant is linked by the hub's stable sub, which matches first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:51:56 -04:00
74ad4950b2 ci: onboard windy-drops, windy-code-web, windy-code, windy-traveler
All checks were successful
check / gate (push) Successful in 1m1s
canary / probe (push) Successful in 38s
All four promoted from pull mirrors to writable. windy-code keeps only
canonical-domains-lint active: its other 15 workflows target hosted
macOS/Windows/ubuntu-latest runners and would queue forever here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:49:00 -04:00
e7bbf9af51 ci: prune.sh must address dind over TCP (it has no unix socket)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:47:35 -04:00
45686283be ci: bound CI storage; don't bridge image-build jobs
- deploy/runner/prune.sh + windygit-ci-prune.timer (6h): age-based prune
  of the CI-only dind (containers, finished-job volumes, images/builder
  cache >7d) plus a hard 60 GB cap. Only that daemon, over its own TCP
  socket; never the host's Docker. It was 38 GB and unbounded — the same
  class of growth that filled Kit 0 on 09-01.
- pr_status_bridge: jobs named *docker* are not posted. Job containers
  have no daemon by design (I-5), so they are red on every commit; a
  permanent red X teaches everyone to ignore red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:47:28 -04:00
e4a15869c0 ci: onboard windy-connect + windy-search to private-repo CI
Some checks failed
check / gate (push) Has been cancelled
windy-connect promoted from pull mirror to writable (release.yml, which
publishes to PyPI on tag push, disabled — the sync pushes tags).
windy-search was already writable; its scheduled drift-check is disabled
because it now runs as cron on Kit 0. Both added to BRIDGE_REPOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:44:19 -04:00
dcf9286f16 ci: make Windy Git CI permanent for the private repos
All checks were successful
check / gate (push) Successful in 21s
canary / probe (push) Successful in 6s
- Four runners x capacity 1 instead of one x capacity 4. Concurrent jobs in
  one act_runner share /root/.cache/act; a refresh racing a copy killed 3 of
  windy-chat's ~20 jobs at setup-node (lstat ... no such file). Separate
  processes have separate caches. Same parallelism, same capped dind.
- Behavioral tests for pr_status_bridge (latest verdict wins, no reposting,
  skipped never painted green, fork PRs never run, pagination, PR lifecycle).
- import_from_github.py reads IMPORT_GITEA_URL, not GITEA_BASE_URL: sourcing
  the deploy .env pointed it at http://gitea:3000 and it died on DNS after the
  mirror it replaces had already been deleted.
- CUTOVER.md: the private-repo CI path, onboarding steps, and the
  /actions/tasks-hides-queued-runs trap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:18:04 -04:00
1b09b9b0d3 ci: sync windy-chat + windy-mail, bridge PR CI verdicts back to GitHub
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 6s
GitHub Actions can't run on the private platform repos. Windy Git already
has their code and a working runner, so:

- windy-chat and windy-mail were read-only pull mirrors (which can never
  run Actions); they are now writable, deploy.yml/build-image.yml disabled,
  and synced from GitHub like the others.
- scripts/pr_status_bridge.py mirrors open same-repo GitHub PRs into Windy
  Git (so pull_request workflows fire) and posts each job's result back as
  a GitHub commit status (windy-git/<workflow>/<job>) on PR heads and the
  default-branch head. Fork PRs are never run. Runs after every sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 02:13:21 -04:00
390c1e7479 ops: move tunnel metrics to 2001, sync windy-git into itself
Some checks failed
check / gate (push) Successful in 19s
canary / probe (push) Failing after 7s
windygit-tunnel had crash-looped ~91k times: another project's
cornercall-tunnel holds 127.0.0.1:2000, and cloudflared exits when it
cannot bind its metrics port. Ingress only survived because a stray
cloudflared.service ran the same config. That unit is now disabled and
/etc/cloudflared/config.yml uses metrics 127.0.0.1:2001.

Also add windy-git to the GitHub->Windy Git sync list; its self-hosted
copy was stuck 3 commits behind (only check + canary workflows, no
deploys, so syncing is safe).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 01:29:57 -04:00
2b30b0ac99 docs: record the runner bump, the act cache flake, and the remaining reds
Second root cause found and fixed: act_runner 0.2.11 predates
`runs.using: node24`, so any repo on actions/checkout@v5 died before its first
step. Bumped to 0.6.1; Windy-Clone went 4/4 red to 4/4 green.

Also upgrades the act-cache note from "watch item" to a confirmed job failure
(lstat on a vanished file mid-tar), with the wipe command and the annotated-tag
dead end that looks like a wrong checkout but isn't.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 19:09:26 -04:00
cd7dd9b7ae ci: bump act_runner 0.2.11 -> 0.6.1 for node24 action support
0.2.11's bundled act only knows runs.using node12/node16/node20, so any repo
pinning a current action major (actions/checkout@v5, actions/setup-python@v6)
fails before its first step with "The runs.using key in action.yml must be one
of: [...], got node24". Windy-Clone is how this surfaced.

Verified node24 is absent from the 0.2.11 binary and present in 0.6.1, and that
every key in deploy/runner/config.yaml still exists in 0.6.1's schema.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 19:00:07 -04:00
9fc27eabd6 docs: record the real CI root cause — setup-uv resolves uv via the forge API
The localhost->postgres fix was correct but was never what failed these jobs;
they died at step 2. setup-uv v4+ resolves "latest" through GITHUB_API_URL,
which act_runner points at our own forge, so it 404s and every later step is
skipped by success(). Pinned an explicit uv version across 11 repos.

Also records the diagnosis traps that cost the most time: Gitea's job status
enum (1=success, 2=failure), act misattributing the error to the previous step,
the jobs-log API needing a repo-matched id, and Secure cookies defeating a
loopback curl login.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 17:57:04 -04:00
Grant Whitmer
db055a1922 docs: refresh the turnover prompt for the actual next task
Some checks failed
check / gate (push) Successful in 18s
canary / probe (push) Failing after 6s
Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-14 17:31:23 -04:00
Grant Whitmer
86326ca6a7 docs: record three-repo CI fix results — 1 of 3 verified
All checks were successful
check / gate (push) Successful in 19s
windy-registry's postgres integration went failure -> success, proving the fix.
windy-mind and WindyCloud still fail for a cause I could not determine: the
jobs API returns 'job not found' for the ids the runs report, so logs were not
retrievable that way. Next session should read them from the Gitea web UI.

Records the trap that the three repos did NOT share one pattern — a naive
localhost->postgres swap would have left WindyCloud on port 15432.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-14 17:30:44 -04:00
Grant Whitmer
e0d5d118be docs: turnover for a fresh session
All checks were successful
check / gate (push) Successful in 35s
canary / probe (push) Successful in 8s
State, the immediate task (three-repo localhost->service-name CI fix), the traps
already paid for, and a copy-paste prompt.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-14 15:00:28 -04:00
Grant Whitmer
01e36155a3 ci: remove the service-networking probe; scope the python-pin test
Some checks failed
check / gate (push) Successful in 20s
canary / probe (push) Failing after 1m33s
The probe's own log was never retrievable through the jobs API, but the
question it asked was answered better by a direct comparison of two real
workflows on the same runner and image:

  windy-git gate       @postgres:5432   -> passes its migration round-trip
  eternitas migrations @localhost:5432  -> failed

Also scopes test_g73 to workflows that actually run Python. It failed the probe
for not pinning a version when the probe only shelled out to psql — the test
being wrong rather than the workflow.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-14 14:41:14 -04:00
Grant Whitmer
fe8f84bbdf ci: probe how service containers are addressed on this runner
Some checks failed
check / gate (push) Failing after 17s
canary / probe (push) Successful in 16s
Several migrated workflows hardcode postgres at localhost:5432, which is
correct on GitHub-hosted runners (services are port-mapped to the VM) and
suspect on Gitea Actions (the job runs IN a container, so localhost is the job).
Prove which form works before rewriting anyone's workflow.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-14 13:14:24 -04:00
Grant Whitmer
eae1bff50b G2.3: Windy Git branding — and get it out of one host's disk into the repo
All checks were successful
check / gate (push) Successful in 19s
canary / probe (push) Successful in 8s
The front end was 100% stock Gitea: green teacup, "Gitea: Git with a cup of
tea", "A painless, self-hosted Git service". G2.3 was specified in the plan with
an acceptance test and never executed, and nothing enforced it.

Now: Windy Git name, wind-mark logo, brand-blue accent, and a landing page that
says what this actually is. Uses Gitea's SUPPORTED surface (custom templates +
public assets) so upstream upgrades keep arriving — no source modified (D-2/I-1).

Two traps this cost, both now documented and tested:

1. GITEA__DEFAULT__APP_NAME does not work. Gitea reads APP_NAME from the TOP
   LEVEL of app.ini; the env var created a literal [default] section that Gitea
   ignores, so the installer's stock APP_NAME kept winning while the config
   looked correct. The env-to-ini pass also APPENDED a second APP_NAME rather
   than replacing the first — a new variant of the documented G4A.3 trap.

2. Cloudflare caches /assets/* for 6h and no token in this stack can purge, so
   the new logo and CSS were invisible while being correct at origin. Brand
   assets now carry a VERSION IN THE FILENAME; bump it on every change.

Committed with an idempotent apply.sh, because applying it straight to Veron's
disk first was itself the config-drift trap this project documents: a rebuild
would have silently reverted to stock Gitea.

85 tests green.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-14 09:08:48 -04:00
Grant Whitmer
e7dee39151 throttle: stop claiming to limit pushes we cannot see
All checks were successful
check / gate (push) Successful in 19s
canary / probe (push) Successful in 9s
I reintroduced the exact defect I had just criticised. ACTION_BASE listed
"push" and "push.force", but git push goes straight to Gitea over HTTPS and
never touches this API — so nothing records a push, a count would be zero
forever, and enforce() would look up a limit, count nothing, and allow
everything. A silent no-op wearing the costume of a control, made worse by a
config name that implies the protection exists.

Split into ACTION_BASE (actually enforced: repo.create, grant.create) and
NOT_ENFORCED_HERE (push, push.force) with the reason and the remedy written
down: enforcing push velocity needs a Gitea-side pre-receive or push webhook
reporting into agent_actions.

enforce("push") now raises rather than silently allowing, and a test asserts the
two sets stay disjoint.

Found by auditing whether the auth fix could be walked around — every
/api/v1/repos/* route does require a caller, and the only unauthenticated
endpoints are /health, /version and the HMAC-verified webhook.

83 tests green.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:33:30 -04:00
Grant Whitmer
25368547cb docs: runbook — never pull -q, never force-push a deployed branch
All checks were successful
check / gate (push) Successful in 21s
canary / probe (push) Successful in 11s
Two deploy traps paid for on 2026-08-14: 'git pull -q' hid a divergent-branch
error so a deploy ran against stale code while reporting success, and the
divergence came from amending a commit a deploy checkout already tracked.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:28:57 -04:00
Grant Whitmer
c60bfb2b89 I-12: fail the build when COMMIT_SHA is empty
All checks were successful
check / gate (push) Successful in 19s
/version went null after a deploy — the exact "service cannot name its own
commit" defect this project was built to prevent, caught by its own honesty
check.

Cause: the sed replaced "" with "" (a no-op when COMMIT_SHA is empty) and the
grep then matched that same empty string, so the guard verified nothing. A build
with no COMMIT_SHA passed and shipped a container reporting commit_sha: null.

Now the build fails loudly instead.

Second cause of the stale deploy, and it was mine: an earlier `git commit
--amend` + force-push rewrote history the Veron deploy checkout was already
sitting on, leaving it divergent so `git pull -q` failed SILENTLY (-q hid
"Need to specify how to reconcile divergent branches"). Two lessons: do not
force-push a branch a deploy checkout tracks, and do not pull with -q in a
deploy script.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:27:38 -04:00
Grant Whitmer
a0ed4a5ec0 SECURITY: make EPT routing independent of signature well-formedness
All checks were successful
check / gate (push) Successful in 20s
looks_like_ept used jwt.get_unverified_header, which validates the WHOLE token
and therefore rejects anything with a malformed signature segment. Routing
consequently depended on signature well-formedness: an EPT-shaped token with a
bad signature fell through to the HUMAN path, where it was refused for the wrong
reason and — with require_verified_jwt off (dev) — could have been read as a
human identity via its `sub` claim.

Now the header segment is decoded directly, so routing depends only on what the
token CLAIMS to be; whether it is authentic remains verify_ept's job.

Also routes alg:none to the EPT verifier regardless of typ, since a `none`
token is never valid for any caller. Both forged shapes now return 401
ept_invalid — the honest code — instead of 503 "feature not ready".

Found by noticing a forged EPT returned 503 where the verifier should have
answered 401, rather than accepting "it was refused, close enough".

80 tests green.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:23:50 -04:00
Grant Whitmer
830ca48705 docs: mark revocation finding resolved — it was critical, not low
All checks were successful
check / gate (push) Successful in 19s
Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:20:46 -04:00
Grant Whitmer
deb3a8fafc test: prove the revocation wiring end-to-end (resolve_passport -> decide_trust)
Some checks failed
check / gate (push) Has been cancelled
Monkeypatched httpx so resolve_passport sees a real revoked trust body
(status=revoked, band=unproven, allowed=[]) and must raise
PassportNotInGoodStanding — proving the WIRING, not just the decision. This is
the path that stops a validly-signed EPT that outlived its passport's
revocation (~365-day tokens).

Live-confirmed alongside: Eternitas refuses to mint EPTs for revoked bots
("credentials are not issued for non-active bots"), so the only exposure was a
pre-existing token — exactly what this now catches. The active agent's EPT still
returns 200. A fully-live revoked test would require revoking a real fleet
passport (destructive), so the wiring is proven deterministically instead.

79 tests green.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:20:44 -04:00
Grant Whitmer
587fb05265 SECURITY: enforce passport status — revocation now takes effect live
All checks were successful
check / gate (push) Successful in 21s
A revoked passport returns HTTP 200, status=revoked, band=unproven,
allowed_actions=[] (verified live 2026-08-13). resolve_passport keyed refusal
only on HTTP 4xx and band=="untrusted", so it returned band 'unproven' and the
agent was seated. Revocation was NOT enforced on the live auth path at all — and
now that agent auth actually works, a revoked agent could authenticate and act.

Extracts decide_trust(body) -> (band, actions) | raise. Only status=="active"
is allowed; revoked/suspended/frozen/unknown all refuse, fail-closed on the
field that carries the most consequential fact about an identity. The agent call
site turns that into a clean 403 passport_revoked.

This is the REAL revocation gate — the token cannot be un-issued, but its
standing is re-checked on every request, so revocation takes effect on the next
call with no webhook required. The Eternitas webhook remains useful for
invalidating locally-issued credentials/grants (G6.3, not built yet), but it was
never the primary gate and its being unwired is no longer a live exposure.

Behavioral tests: revoked body refused, active accepted, unknown/missing status
fails closed.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:18:16 -04:00
Grant Whitmer
c96d1d3102 canary: guard both forgery shapes now that real verification is live
All checks were successful
check / gate (push) Successful in 20s
The EPT-shaped forgery is the one that matters after G3.2 — it is what
signature verification actually guards. The JWT-shaped one still exercises the
human gate. Both must_refuse; a 2xx on either pages.

Verified live: forged EPT -> 401 ept_invalid, forged JWT -> 503,
genuine EPT -> 200.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:13:44 -04:00
Grant Whitmer
79dcea4d3e G3.2/G3.4: real EPT verification + wire the throttle, reopening agent auth
All checks were successful
check / gate (push) Successful in 19s
canary / probe (push) Successful in 10s
REOPENS the agent path — but only because possession is now actually proven.

EPT verification (api/app/ept.py): ES256 against Eternitas's published key set
at /.well-known/eternitas-keys. algorithms=["ES256"] makes alg:none and
algorithm confusion unrepresentable rather than merely unlikely; issuer and exp
are enforced by the library; an unknown kid is refused.

Order is deliberate: signature FIRST, trust lookup second. These EPTs live ~365
days and carry rev/tru baked in at issuance, so a year-old "rev: false" proves
nothing — revocation and band still come from a live lookup on every request.

Found while building it: real EPTs put the passport in the "sub" claim. The old
code read "passport"/"sub_passport", which no genuine EPT carries — so real
agents were never recognised and ONLY forged tokens ever authenticated. The
bypass was not just a hole, it was the only thing that worked.

Throttle (api/app/throttle.py): BAND_MULTIPLIER and rate_*_per_day were defined
and read by nothing. Now enforced on repo.create and grant.create, counted
against agent_actions (one source of truth, not a private counter that drifts
from the audit log). Fails CLOSED — a limiter that fails open protects you until
the moment something is wrong. Untrusted band is 403 read-only, not 429, because
"slow down" would be a lie.

Tests: 14 behavioral, signing real ES256 tokens with a locally-generated key so
they exercise the crypto path with no network dependency — genuine tokens
accepted, and alg:none / foreign key / tampered payload / expired / wrong issuer
/ unknown kid / missing claims all refused. 74 green.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 23:09:47 -04:00
Grant Whitmer
c257cc55c6 docs: second-auditor review (Fable) — 1 critical fixed live, 4 open
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 11s
Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 22:54:15 -04:00
Grant Whitmer
85fa65a52b canary: continuously verify the forged-token bypass stays closed
All checks were successful
check / gate (push) Successful in 18s
Adds must_refuse checks: a 2xx is the alarm, a 401/403/503 is health. The
forged alg:none agent token is probed every 10 min; if it ever returns 2xx the
canary goes red and pages. Proven both ways — 503 reads ok, a 200 endpoint
alarms 'ACCEPTED — this MUST be refused'. The security property is now enforced
by a running check, not assumed.

Co-Authored-By: Claude (Fable 5) <noreply@anthropic.com>
2026-08-13 22:52:56 -04:00
Grant Whitmer
d8deffe4db SECURITY: close the agent-auth bypass — trust is not authentication
All checks were successful
check / gate (push) Successful in 18s
canary / probe (push) Successful in 11s
Verified live 2026-08-13: a forged 'alg:none' token naming a passport lifted
from the logs returned HTTP 200 as that agent. The agent path read the passport
without verifying the EPT signature, asked Eternitas 'is this passport
reputable?', and seated the caller on a yes. That answers reputation, not
possession — anyone who knows a passport number could impersonate that agent on
the public API.

The human path already failed closed for exactly this reason
(require_verified_jwt). The gate was on the wrong path: it sat AFTER the agent
branch returned. The agent path now fails closed too, BEFORE the trust lookup,
so a forged token never even reaches Eternitas. Reopens automatically when the
ES256/JWKS verifier (G3.2/G9.1) exists and this gate consults it.

Adds BEHAVIORAL tests (not string-grep): a forged alg:none token exercised
through the real get_caller must raise, not authenticate. This is the test that
would have caught the bypass; the suite had 86 source-string assertions and
zero that ran the auth decision.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 22:49:45 -04:00
Grant Whitmer
83047def94 docs: whole account on Windy Git — 143 repos, 1.58 GB, two tiers
131 read-only mirrors (cannot run Actions, zero deploy risk) + 12 writable with
CI and deploys disabled. Total size matches the measured GitHub archive exactly,
which is the confirmation the copy is complete.

Splits the two concerns cleanly: having a copy is safe and should cover
everything now; running code needs judgement and happens per repo.

windy-pro IS included as a mirror — the G11.5 caution is about making it
writable while six checkouts disagree on HEAD, not about holding a read-only
copy. The DR copy is now complete.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:49:48 -04:00
Grant Whitmer
711c47d8c9 fix: the --all-as-mirrors flag itself was never added
The function landed but the argparse anchor did not match the real formatting,
so the command existed and could not be invoked. Caught by running it rather
than assuming the patch applied.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:43:22 -04:00
Grant Whitmer
ae6ca8b9d2 G11.3: bulk DR copy — every repo as a read-only mirror
Grant's plan: clone the whole account, let it circulate, reverse direction later
when things are clean. Right plan, with one change that matters.

Sampling 40 repos found 18 carrying deploy/release/publish workflows that
trigger on push: — roughly 63 across the account. Importing those writable with
Actions enabled would arm sixty-odd production deploy triggers on Veron 1, each
needing disarming by hand.

So the bulk goes in as READ-ONLY pull mirrors. A mirror cannot run Actions at
all, so this carries zero deploy risk, and Gitea syncs them itself with no
script and no timer. What you get is a complete, current second copy of the
whole account — the disaster-recovery half — with none of the execution risk.

Converting one to writable + CI stays a deliberate per-repo act: re-import,
review its workflows, disable the deploying ones. That judgement belongs at the
moment you want CI on that repo, not in bulk sixty times by accident.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:40:52 -04:00
Grant Whitmer
89723c6ebd safety: disable deploy workflows on Windy Git before they can fire
Six workflows deploy to production on push:. Windy Git now has a working
runner, so the next synced commit to main would have attempted a production
deploy FROM VERON 1. Their secrets are unset here so they would have failed —
but loudly, on every push, with any pre-SSH step still running.

All six now disabled_manually. Tests, lints and migration checks stay active:
they need no secrets, which is exactly why Phase 1 delivers CI value with
nothing to configure.

Same class of mistake as the push-mirror direction, caught before firing this
time rather than after.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:30:56 -04:00
Grant Whitmer
b2f00821d7 docs: replace the cutover plan with the phased one that matches reality
Phase 1 requires nothing from anyone: agents keep pushing to GitHub, a timer
syncs GitHub -> Windy Git every 15 minutes, CI runs on Veron against current
code. Phase 2 flips one repo at a time, only when that repo is idle.

Records the direction mistake honestly: the source of truth is wherever people
are actually typing, not wherever the plan says it should be.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:14:21 -04:00
Grant Whitmer
51acf9f86e URGENT FIX: reverse the sync — GitHub is the source of truth, not Windy Git
I migrated nine repos writable with push-mirrors pointed AT GitHub. That was
wrong for the actual situation: a dozen agent sessions on the Mac mini are
pushing to GitHub continuously, so GitHub is where the live work is.

A push-mirror force-updates refs. On its 8-hour timer it would have pushed
Windy Git's stale copy over live work — silently, no conflict, nothing to
notice. Removed all nine before the first timer fired; verified no GitHub repo
had been touched (latest push predated the mirrors).

Replaced with the correct Phase 1 direction:

  agents --push--> GitHub --sync--> Windy Git --> CI on Veron

It requires NOTHING from anyone. No remote changes, no coordination, no
'everybody stop pushing'. Agents keep working exactly as they are and CI starts
running on 24 cores.

Windy Git is force-updated on purpose: in Phase 1 it holds nothing anyone
depends on, so GitHub always wins and there is no merge to reconcile.

Phase 2 is per-repo and only when that repo is idle. Never a big-bang cutover
across a dozen live sessions.

Fetches +refs/heads/* and tags explicitly rather than --mirror, which would drag
GitHub's refs/pull/* that Gitea rejects and bury the real errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:11:34 -04:00
102 changed files with 9680 additions and 134 deletions

View File

@@ -51,9 +51,10 @@ jobs:
- name: install
run: |
python3 --version
python3 -m venv .venv
.venv/bin/pip install -q --upgrade pip
.venv/bin/pip install -e ".[dev]"
# From uv.lock, never floating: CI tests exactly what the image ships.
# --locked also FAILS if pyproject.toml changed without re-locking.
python3 -m pip install -q uv==0.12.5
uv sync --locked --extra dev
- name: lint
run: .venv/bin/ruff check api scripts

View File

@@ -2,11 +2,17 @@
Read this before touching anything. Then read `DNA_STRAND_MASTER_PLAN.md`, which is the source of truth.
## Current state
## Current state (2026-09-23)
**GENESIS.** No code. No `make dev` yet — building it is codon **G0.8**.
**LIVE on Veron 1** — `app.windygit.com` (Gitea 1.24.6, Windy SSO only),
`api.windygit.com` (our plane: humans via hub JWKS, agents via Eternitas EPT),
`models.windygit.com`. Strands G0–G5, G7, G11 done; see the plan for the rest.
The next work is Strand **G0** (cell substrate), then **G1** (Veron 1 host + Cloudflare Tunnel), then **G2** (Gitea, stock and branded), then **G3** (identity), then **G4** (storage). G0–G4 are sequential. G5–G12 are concurrent once G4 lands.
It is also **the permanent CI for the private platform repos** (GitHub Actions
cannot run on them): `scripts/sync_from_github.sh` + `scripts/pr_status_bridge.py`,
onboarding in `docs/CUTOVER.md`, operations in `docs/RUNBOOK-VERON.md`.
Standing dev checkout: **OC5 `~/windy-git`**. Deploy copy: Veron `/srv/windygit/src`.
## The rules that will get you reverted if you break them
@@ -28,7 +34,7 @@ The next work is Strand **G0** (cell substrate), then **G1** (Veron 1 host + Clo
- Errors are 4-field repair pointers: `{code, speak, machine_cause, remediation_tool}`. No exceptions, including validation errors.
- Every tool response carries `state_proof` + `next_actions`.
- Telemetry `actor_type` comes from the enum `{human, agent, system}`. **`'service'` is not legal** — it 422s and silently drops the whole batch. A sibling service is losing telemetry to exactly this today.
- Runner labels are explicit and pinned. **`ubuntu-latest` is banned** — all four `windy-registry` workflows use it and every run fails.
- Runner labels are explicit and pinned: `[self-hosted, linux, x64]` or `veron-1`. **`ubuntu-latest` is banned** — no runner here has it, so the job queues forever.
## Membrane

View File

@@ -81,7 +81,7 @@ Numbered because code cites them. Changing one requires an ADR that names it.
1. **I-1 · Gitea is a component, never a merged tree.** Our code lives in our services and calls Gitea's REST API. Any patch to Gitea source lives in `patches/` as a numbered, rebasable diff with a one-line justification, and `make check` fails if `patches/` grows past **3** files without an ADR.
2. **I-2 · The membrane is ENUMERATED.**
**Calls out:** `windy-cloud` kernel `GET /api/v1/storage/objects` + `HEAD` (read user objects to version them) · `windy-cloud` `POST /api/v1/storage/quota/check` · `eternitas` `GET /api/v1/trust/{passport}` (band + allowed_actions) · `eternitas` `GET /api/v1/registry/{passport}/integrity` · `account-server` OIDC discovery + JWKS · `windy-cloud-sites` `POST /api/v1/sites/{id}/versions` (publish docs from a repo).
**Calls out:** `windy-cloud` kernel `GET /api/v1/storage/objects` + `HEAD` (read user objects to version them) · `windy-cloud` `POST /api/v1/storage/quota/check` · `eternitas` `GET /api/v1/trust/{passport}` (band + allowed_actions) · `eternitas` `GET /api/v1/registry/{passport}/integrity` · `account-server` OIDC discovery + JWKS · `windy-cloud-sites` `POST /api/v1/sites/{id}/versions` (publish docs from a repo). · windy-admin ledger `POST https://admin.windyword.ai/v1/events` (field telemetry, 2026-09-23: `ci.run`, `ci.job_cancelled`, `service.boot`, `service.health`, `forge.auth.failed` — shapes declared with the ledger owner first; no content, no passports, no emails)
**Calls in:** `POST /internal/repo-from-folder` (Cloud portal: git-enable a folder) · `POST /internal/mirror-status` (ops).
**Events out:** `repo.created`, `repo.pushed`, `release.published`, `model.published`, `ci.completed`.
**Events in:** `passport.revoked` (fail-closed), `storage.quota.exceeded`, `identity.created`.

View File

@@ -10,15 +10,31 @@ WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends git curl \
&& rm -rf /var/lib/apt/lists/*
COPY pyproject.toml ./
RUN pip install --no-cache-dir -e .
# Dependencies come from uv.lock, hash-pinned, never "latest at build time".
# Floating installs meant a rebuild could ship different fastapi/starlette/
# pydantic than CI tested (Windy Cloud's OpenAPI drift, 09-23). The lock was
# cut to exactly what prod ran then. uv only exports; pip installs, so the
# image layout (system python, uvicorn on PATH) is unchanged.
COPY --from=ghcr.io/astral-sh/uv:0.12.5 /uv /usr/local/bin/uv
COPY pyproject.toml uv.lock ./
RUN uv export --frozen --no-dev --no-emit-project -o /tmp/requirements.txt \
&& pip install --no-cache-dir --require-hashes -r /tmp/requirements.txt \
&& rm /tmp/requirements.txt
COPY api ./api
RUN pip install --no-cache-dir --no-deps -e .
COPY alembic ./alembic
COPY alembic.ini ./
COPY scripts ./scripts
RUN sed -i "s|^BAKED_COMMIT_SHA: str = \"\"|BAKED_COMMIT_SHA: str = \"${COMMIT_SHA}\"|" api/app/buildinfo.py \
# I-12: an EMPTY COMMIT_SHA must fail the build, not sail through it.
# Previously the sed replaced "" with "" (a no-op) and the grep then matched
# that same empty string, so a build with no COMMIT_SHA passed and shipped a
# container reporting commit_sha: null — exactly the "service cannot name its
# own commit" defect this project exists to prevent. Caught 2026-08-14 when
# /version went null after a deploy.
RUN test -n "${COMMIT_SHA}" || (echo "FATAL: COMMIT_SHA build arg is empty (I-12)" && false) \
&& sed -i "s|^BAKED_COMMIT_SHA: str = \"\"|BAKED_COMMIT_SHA: str = \"${COMMIT_SHA}\"|" api/app/buildinfo.py \
&& sed -i "s|^BAKED_BUILT_AT: str = \"\"|BAKED_BUILT_AT: str = \"${BUILT_AT}\"|" api/app/buildinfo.py \
&& grep -q "BAKED_COMMIT_SHA: str = \"${COMMIT_SHA}\"" api/app/buildinfo.py

View File

@@ -22,7 +22,7 @@ boot guard in `api/app/main.py` that refuses to start there in production.
| 8600 | `windy-git-api` — our plane |
| **3080** | Gitea — host 3000 and 3300 are taken by resident projects on Veron 1 |
| 5432 | Postgres |
| 2000 | cloudflared metrics (probe target) |
| 2001 | cloudflared metrics — NOT 2000: `cornercall-tunnel` (another project) takes 2000, and a metrics bind failure kills the whole tunnel |
## Ingress — Cloudflare Tunnel `windy-git`

View File

@@ -25,7 +25,10 @@ import httpx
from fastapi import Header, Request
from api.app.config import Settings
from api.app.ept import EptInvalid, looks_like_ept, verify_ept
from api.app.errors import RepairPointer, passport_unresolvable
from api.app.hub_jwt import HubTokenInvalid, verify_hub_token
from api.app.telemetry import synthetic_headers
log = logging.getLogger(__name__)
@@ -72,6 +75,36 @@ BAND_MULTIPLIER: dict[str, float] = {
}
class PassportNotInGoodStanding(Exception):
"""The passport resolved, but Eternitas does not list it as active
(revoked / suspended / frozen / unknown status)."""
def __init__(self, passport: str, status: str) -> None:
self.passport = passport
self.status = status
super().__init__(f"{passport} status={status!r}")
def decide_trust(body: dict, passport: str) -> tuple[str, tuple[str, ...]]:
"""The trust body -> (band, allowed_actions), or refuse.
THE GATE THAT WAS MISSING. A revoked passport returns HTTP 200 with
`status: revoked`, `band: unproven`, `allowed_actions: []` — verified live
2026-08-13. The previous code keyed refusal only on HTTP 4xx and on
band=="untrusted", so a revoked agent (200, band unproven) authenticated and
acted normally. Revocation was not enforced on the live path at all; the
webhook that was supposed to be the backup was never the primary gate.
Only `status == "active"` is allowed. Anything else — including a status
Eternitas invents tomorrow — refuses. Fail-closed on the field that carries
the most consequential fact about an identity.
"""
status = str(body.get("status", "")).lower()
if status != "active":
raise PassportNotInGoodStanding(passport, status or "missing")
return body.get("band", "unproven"), tuple(body.get("allowed_actions", ()))
async def resolve_passport(settings: Settings, passport: str) -> tuple[str, tuple[str, ...]]:
"""G3.6 — THE STATUS-CODE LAW.
@@ -93,7 +126,7 @@ async def resolve_passport(settings: Settings, passport: str) -> tuple[str, tupl
)
url = f"{settings.eternitas_base_url}/api/v1/trust/{passport}"
headers = {"X-API-Key": settings.eternitas_platform_api_key}
headers = {"X-API-Key": settings.eternitas_platform_api_key, **synthetic_headers()}
last_status = 0
for attempt in range(3):
async with httpx.AsyncClient(timeout=httpx.Timeout(8.0, connect=3.0)) as client:
@@ -105,8 +138,9 @@ async def resolve_passport(settings: Settings, passport: str) -> tuple[str, tupl
continue
last_status = r.status_code
if r.status_code == 200:
body = r.json()
return body.get("band", "unproven"), tuple(body.get("allowed_actions", []))
# decide_trust raises PassportNotInGoodStanding on a non-active
# status; that propagates past the retry loop as a hard refusal.
return decide_trust(r.json(), passport)
if r.status_code in (400, 404):
# Malformed or not-issued. Refuse immediately — retrying cannot help
# and pretending it might is how a soft-allow gets written.
@@ -159,42 +193,77 @@ async def get_caller(
token = authorization.split(" ", 1)[1].strip()
# --- agent (Eternitas EPT) --------------------------------------------
# An EPT names its passport; the trust API is the authority on whether that
# passport may act. We never read a band out of the token itself.
passport = _unverified_claim(token, "passport") or _unverified_claim(token, "sub_passport")
if passport:
band, actions = await resolve_passport(settings, passport)
# Possession FIRST, reputation second. The signature proves the caller holds
# this passport; the trust lookup then says what it may do. Doing only the
# second was the 2026-08-13 impersonation bypass.
if looks_like_ept(token):
try:
verified = verify_ept(token, settings.eternitas_base_url)
except EptInvalid as exc:
raise RepairPointer(
status_code=401,
code="ept_invalid",
speak="We couldn't confirm that helper's ID, so we didn't let it in.",
machine_cause=f"EPT verification failed: {exc}",
remediation_tool="windy_git.reissue_agent_token",
) from exc
# The token is authentic. It is NOT evidence of current standing: these
# EPTs live ~365 days and carry `rev`/`tru` baked in at issuance, so a
# year-old `rev: false` proves nothing. Revocation and band come from a
# live lookup, every time.
try:
band, actions = await resolve_passport(settings, verified.passport)
except PassportNotInGoodStanding as exc:
# The signature is authentic, but the identity is no longer good.
# Revocation takes effect here, live, on the next request — no
# webhook required. That is the honest place for it: the token can't
# be un-issued, but its standing is checked every time.
raise RepairPointer(
status_code=403,
code="passport_revoked",
speak="That helper's access has been turned off.",
machine_cause=f"eternitas status for {verified.passport} is {exc.status!r}, not active",
remediation_tool=None,
) from exc
if band.lower() == "untrusted":
raise RepairPointer(
status_code=403,
code="agent_read_only",
speak="That helper can look, but it isn't allowed to make changes yet.",
machine_cause=f"passport {passport} band=untrusted is read-only",
machine_cause=f"passport {verified.passport} band=untrusted is read-only",
remediation_tool=None,
)
return Caller(
actor_type=ActorType.agent,
passport=passport,
passport=verified.passport,
band=band,
allowed_actions=actions,
)
# --- human (account-server RS256) -------------------------------------
if settings.is_production and settings.require_verified_jwt:
# I-8, applied to ourselves. G3.2's JWKS verifier is not written yet, and
# an unverified JWT is an authentication bypass rather than a shortcut.
# Refusing is the only honest answer until the verifier exists.
raise RepairPointer(
status_code=503,
code="human_signin_not_ready",
speak="Signing in isn't switched on yet. Nothing you have is affected.",
machine_cause=(
"JWKS verification (G3.2) is not implemented; refusing to accept "
"an unverified human token in production"
),
remediation_tool=None,
)
# --- human (hub RS256 access token, G3.2) ------------------------------
# Production ALWAYS verifies, whatever require_verified_jwt says: the flag
# only exists to let local dev run against unsigned fixture tokens.
if settings.require_verified_jwt or settings.is_production:
try:
human = verify_hub_token(
token,
settings.account_server_base_url,
issuers=tuple(settings.hub_issuers),
audiences=tuple(settings.hub_audiences),
require_aud=settings.hub_require_aud,
)
except HubTokenInvalid as exc:
raise RepairPointer(
status_code=401,
code="token_invalid",
speak="We couldn't confirm that sign-in. Try signing in again.",
machine_cause=f"hub token verification failed: {exc}",
remediation_tool=None,
) from exc
return Caller(actor_type=ActorType.human, identity_id=human.identity_id)
# Local dev only (require_verified_jwt=False outside production).
identity_id = _unverified_claim(token, "windy_identity_id") or _unverified_claim(token, "sub")
if not identity_id:
raise RepairPointer(
@@ -214,10 +283,8 @@ def _unverified_claim(token: str, claim: str) -> str | None:
on the result re-establishes trust independently: an agent's authority comes
from a live Eternitas trust lookup, never from the token's own assertions.
⚠️ Full RS256/ES256 JWKS verification for the human path lands in G3.2's
verifier and MUST be in place before `api.windygit.com` accepts a human
token from outside. Until then the human path is reachable only from inside
the tunnel, and `settings.require_verified_jwt` refuses it in production.
Humans are verified by hub_jwt.verify_hub_token (G3.2); this reader backs
only the local-dev path, which production never takes.
"""
import base64
import json

View File

@@ -53,16 +53,33 @@ class Settings(BaseSettings):
# ---- account-server OIDC (human identity) -----------------------------
account_server_base_url: str = "https://account.windyword.ai"
# G3.2 — what a hub ACCESS token must say about itself (see hub_jwt.py).
# Token contract v1 (lane 8c, 2026-09-23): access tokens may carry either
# issuer. id_tokens are kept out by `type` + `windy_identity_id` + aud, not
# by issuer.
hub_issuers: list[str] = ["windy-identity", "https://account.windyword.ai"]
# Contract v1: aud is an ARRAY; first-party tokens list every product, and
# Windy Git's entry is `windy_git` (underscore). ⚠️ NEVER add "windy-git"
# (hyphen): that is Gitea's OIDC client_id, so an id_token minted for the
# forge would carry it and pass as a bearer here.
hub_audiences: list[str] = ["windy_git"]
# Flip to True once the hub emits aud on every access token.
hub_require_aud: bool = False
# ---- field telemetry (admin.windyword.ai ledger) ----------------------
# Unset token = nothing sent, nothing buffered. The token lives in the
# root-only /etc/windygit/telemetry.env on Veron, never in the repo.
windygit_telemetry_token: str = ""
telemetry_ingest_url: str = "https://admin.windyword.ai/v1/events"
# Internal callers (the Cloud portal calling /internal/*). A first-class
# caller class, not a bypass: unset means service calls are REFUSED.
service_token: str = ""
# ⚠️ FAIL-CLOSED GATE. Full RS256/ES256 JWKS verification lands in G3.2.
# Until it does, the human token path must not be reachable in production —
# accepting an unverified JWT is not a shortcut, it is an authentication
# bypass. Agents are unaffected: their authority comes from a live Eternitas
# trust lookup, not from anything the token asserts about itself.
# ⚠️ FAIL-CLOSED GATE. Human tokens are verified against the hub's JWKS
# (G3.2, hub_jwt.py). False only enables the unverified local-dev path, and
# production verifies regardless — an unverified JWT is a bypass, not a
# shortcut.
require_verified_jwt: bool = True
# ---- storage law (I-3, G4.4) ------------------------------------------

157
api/app/ept.py Normal file
View File

@@ -0,0 +1,157 @@
"""EPT signature verification (G3.2 / G9.1) — the gate that makes an agent an agent.
Trust is not authentication. A trust lookup answers *"is this passport
reputable?"*; only a signature answers *"does this caller actually hold it?"*.
Skipping the second question was a live impersonation bypass on 2026-08-13 — a
forged `alg:none` token naming a passport read out of the logs returned HTTP 200.
What this module refuses, deliberately and by construction:
* **`alg: none`** — the original exploit. `algorithms=["ES256"]` makes it
unrepresentable rather than merely unlikely.
* **Algorithm confusion.** Only ES256 is accepted. If an attacker presents an
HS256 token, PyJWT will not try to use an EC public key as an HMAC secret,
which is the classic way "verified" JWTs get forged.
* **An unknown `kid`.** The key must be one Eternitas currently publishes.
* **A wrong issuer or an expired token** — checked by the library, not by us.
What it deliberately does NOT decide: whether the agent is *allowed* to act.
The EPT carries `rev` and `tru` claims baked in at issuance, and these tokens
live for a year (observed `exp` ≈ 365 days). A year-old `rev: false` is not
evidence of anything. **Revocation and trust must come from a live lookup**, so
this module returns only identity and the caller re-checks standing.
"""
from __future__ import annotations
import base64
import json
import logging
import time
from dataclasses import dataclass
import httpx
import jwt
from jwt import PyJWKClient
log = logging.getLogger(__name__)
ISSUER = "eternitas.ai"
ALGORITHMS = ["ES256"] # exactly one. Never widen this list.
_jwks_client: PyJWKClient | None = None
_jwks_url: str | None = None
class EptInvalid(Exception):
"""The token is not a valid, currently-signed Eternitas EPT."""
@dataclass(frozen=True)
class VerifiedEpt:
passport: str
operator: str | None
bot_name: str | None
issued_at: int | None
expires_at: int | None
def _client(base_url: str) -> PyJWKClient:
"""One cached JWKS client. PyJWKClient caches keys and refetches on an
unknown kid, so a key rotation heals itself without a redeploy."""
global _jwks_client, _jwks_url
url = f"{base_url.rstrip('/')}/.well-known/eternitas-keys"
if _jwks_client is None or _jwks_url != url:
_jwks_client = PyJWKClient(url, cache_keys=True, lifespan=300)
_jwks_url = url
return _jwks_client
def verify_ept(token: str, eternitas_base_url: str) -> VerifiedEpt:
"""Verify an EPT's signature and claims. Raises EptInvalid on ANY doubt.
There is no partial success and no "probably fine" path: every failure mode
below produces the same refusal, because a caller that cannot prove
possession is indistinguishable from an attacker.
"""
try:
signing_key = _client(eternitas_base_url).get_signing_key_from_jwt(token)
except Exception as exc: # noqa: BLE001 - unknown kid, unreachable JWKS, malformed
raise EptInvalid(f"no usable signing key: {type(exc).__name__}: {exc}") from exc
try:
claims = jwt.decode(
token,
signing_key.key,
algorithms=ALGORITHMS, # ES256 only — closes alg:none and alg confusion
issuer=ISSUER,
options={
"require": ["sub", "iss", "exp"],
"verify_signature": True,
"verify_exp": True,
"verify_iss": True,
},
)
except jwt.PyJWTError as exc:
raise EptInvalid(f"{type(exc).__name__}: {exc}") from exc
# The REAL claim name. Eternitas puts the passport in `sub`; the previous
# code looked for `passport` / `sub_passport`, which no genuine EPT carries —
# so real agents were never recognised and only forged tokens ever "worked".
passport = claims.get("sub")
if not isinstance(passport, str) or not passport.strip():
raise EptInvalid("EPT carried no passport in `sub`")
return VerifiedEpt(
passport=passport,
operator=claims.get("ope"),
bot_name=claims.get("bot"),
issued_at=claims.get("iat"),
expires_at=claims.get("exp"),
)
def looks_like_ept(token: str) -> bool:
"""Cheap, unauthenticated triage: is this token even *claiming* to be an EPT?
Used ONLY to route a token to the right verifier. It decides nothing about
trust — an attacker controls every byte it reads.
Decodes the header segment directly rather than via
`jwt.get_unverified_header`, which validates the whole token structure and
therefore rejects anything with a malformed SIGNATURE. That made routing
depend on signature well-formedness: an EPT-shaped token with a bad
signature fell through to the human path, where it was refused for the wrong
reason ("signing in isn't switched on") and — with `require_verified_jwt`
off — could have been read as a human identity via its `sub` claim.
Routing must depend only on what the token claims to be. Whether it is
authentic is `verify_ept`'s job, and it says no.
"""
try:
head_b64 = token.split(".", 1)[0]
head_b64 += "=" * (-len(head_b64) % 4)
header = json.loads(base64.urlsafe_b64decode(head_b64))
except Exception: # noqa: BLE001
return False
if not isinstance(header, dict):
return False
return header.get("typ") == "EPT" or header.get("alg") in ("ES256", "none")
async def eternitas_reachable(base_url: str) -> bool:
"""Whether the key set can be fetched at all. Used by /health/full so an
unreachable JWKS is reported rather than discovered during an outage."""
try:
async with httpx.AsyncClient(timeout=httpx.Timeout(5.0, connect=3.0)) as c:
r = await c.get(
f"{base_url.rstrip('/')}/.well-known/eternitas-keys",
headers={"User-Agent": "windy-git/1.0"},
)
return r.status_code == 200 and "keys" in r.json()
except Exception: # noqa: BLE001
return False
def seconds_until_expiry(ept: VerifiedEpt) -> int | None:
return None if ept.expires_at is None else int(ept.expires_at - time.time())

133
api/app/hub_jwt.py Normal file
View File

@@ -0,0 +1,133 @@
"""Human token verification (G3.2) — hub access tokens from account.windyword.ai.
Until this existed the human path refused every token in production (503
`human_signin_not_ready`), because reading an unverified JWT's claims is an
authentication bypass, not a shortcut. This module is what lets it say yes.
The token it accepts is the hub's ACCESS token, as observed live 2026-09-23:
header {alg: RS256, typ: JWT, kid: <published at /.well-known/jwks.json>}
claims iss = "windy-identity" (contract v1 also allows the discovery URL)
type = "human", exp - iat = 900 s
sub = per-row user id ← NOT the cross-product identity
windy_identity_id = the Windy Account UUID (what Gitea's OIDC links on)
no `aud` yet
What it refuses, by construction:
* **Anything but RS256.** One algorithm, never a list. Closes `alg: none` and
HS256-with-the-public-key confusion.
* **An unknown `kid`**, a wrong issuer, an expired token — library-checked.
* **An id_token used as a bearer.** id_tokens prove a login happened to a
relying party (for the forge: aud `windy-git`), not that this caller may act
here. They carry no `type` and no `windy_identity_id`, and their aud is a
client id, not the product name `windy_git` — any one of the three refuses.
* **A non-human `type`.** An agent's authority comes from its EPT and a live
Eternitas lookup, never from a hub token dressed as a person.
* **A token with no `windy_identity_id`.** `sub` is a different namespace (the
per-row user id); falling back to it would silently mint identities that
match nothing Gitea knows.
`aud` (token contract v1, lane 8c): an array; first-party tokens list every
product and Windy Git's is `windy_git`. Optional until the hub emits it; when
present it MUST include `windy_git`.
`hub_require_aud=True` makes it mandatory — flip it once the hub emits it.
"""
from __future__ import annotations
from dataclasses import dataclass
import jwt
from jwt import PyJWKClient
ALGORITHMS = ["RS256"] # exactly one. Never widen this list.
_jwks_client: PyJWKClient | None = None
_jwks_url: str | None = None
class HubTokenInvalid(Exception):
"""Not a valid, currently-signed hub access token for a human."""
@dataclass(frozen=True)
class VerifiedHuman:
identity_id: str
email: str | None
expires_at: int | None
def _client(base_url: str) -> PyJWKClient:
"""Cached JWKS client; refetches on an unknown kid so rotation self-heals."""
global _jwks_client, _jwks_url
url = f"{base_url.rstrip('/')}/.well-known/jwks.json"
if _jwks_client is None or _jwks_url != url:
_jwks_client = PyJWKClient(url, cache_keys=True, lifespan=300)
_jwks_url = url
return _jwks_client
def verify_hub_token(
token: str,
base_url: str,
*,
issuers: tuple[str, ...],
audiences: tuple[str, ...],
require_aud: bool,
signing_key=None,
) -> VerifiedHuman:
"""Verify a hub access token. Raises HubTokenInvalid on ANY doubt.
`signing_key` exists for tests only (a locally generated key, no network).
"""
try:
key = (
signing_key
if signing_key is not None
else _client(base_url).get_signing_key_from_jwt(token).key
)
except Exception as exc: # noqa: BLE001 - unknown kid, unreachable JWKS, malformed
raise HubTokenInvalid(f"no usable signing key: {type(exc).__name__}: {exc}") from exc
try:
claims = jwt.decode(
token,
key,
algorithms=ALGORITHMS,
issuer=list(issuers),
options={
"require": ["iss", "exp", "iat"],
"verify_signature": True,
"verify_exp": True,
"verify_iss": True,
# Checked by hand below: PyJWT rejects any token CARRYING aud
# when no audience is passed, which would break the moment the
# hub starts emitting it — the exact trap the SSO matrix names.
"verify_aud": False,
},
)
except jwt.PyJWTError as exc:
raise HubTokenInvalid(f"{type(exc).__name__}: {exc}") from exc
aud = claims.get("aud")
if aud is None:
if require_aud:
raise HubTokenInvalid("token carries no aud and hub_require_aud is on")
else:
presented = {aud} if isinstance(aud, str) else set(aud) if isinstance(aud, list) else set()
if not presented & set(audiences):
raise HubTokenInvalid(f"aud {sorted(presented)} does not name Windy Git")
# REQUIRED, not defaulted: id_tokens carry no `type`, and this is one of the
# two claims (with windy_identity_id) that keep them from acting as bearers.
if claims.get("type") != "human":
raise HubTokenInvalid(f"token type {claims.get('type')!r} is not a human access token")
identity = claims.get("windy_identity_id") or claims.get("windyIdentityId")
if not isinstance(identity, str) or not identity.strip():
raise HubTokenInvalid("token carries no windy_identity_id")
return VerifiedHuman(
identity_id=identity, email=claims.get("email"), expires_at=claims.get("exp")
)

View File

@@ -6,11 +6,13 @@ component and is reached only over its REST API (D-2 / I-1).
from __future__ import annotations
import asyncio
import logging
import socket
import time
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
@@ -25,6 +27,7 @@ from api.app.providers.registry import (
R2Provider,
)
from api.app.routes import health, repos, webhooks
from api.app.telemetry import SYNTHETIC, Telemetry, caller_class, is_synthetic
logging.basicConfig(
level=logging.INFO,
@@ -44,9 +47,7 @@ def _refuse_kit_zero(settings) -> None:
if not settings.is_production:
return
try:
local_ips = {
info[4][0] for info in socket.getaddrinfo(socket.gethostname(), None)
}
local_ips = {info[4][0] for info in socket.getaddrinfo(socket.gethostname(), None)}
except socket.gaierror:
return
if settings.kit0_host in local_ips:
@@ -98,8 +99,23 @@ async def lifespan(app: FastAPI):
# systemd Restart=always, plus the runbook's `systemctl status`.
]
telemetry = Telemetry(
settings.telemetry_ingest_url,
settings.windygit_telemetry_token,
environment=settings.environment,
commit_sha=info.commit_sha,
version=info.version,
)
app.state.telemetry = telemetry
telemetry.boot()
await telemetry.flush()
task = asyncio.create_task(telemetry.run()) if telemetry.enabled else None
yield
if task is not None:
task.cancel()
await telemetry.flush()
if engine is not None:
await engine.dispose()
@@ -119,8 +135,44 @@ app.include_router(repos.router)
app.include_router(webhooks.router)
@app.middleware("http")
async def _count_requests(request: Request, call_next):
"""Heartbeat counts (requests, 4xx/5xx, refusals, p95). Never raises."""
start = time.perf_counter()
marker = SYNTHETIC.set(is_synthetic(request.headers))
try:
response = await call_next(request)
finally:
SYNTHETIC.reset(marker)
tel = getattr(request.app.state, "telemetry", None)
if tel is not None:
tel.record_request(
response.status_code,
(time.perf_counter() - start) * 1000,
refused=getattr(request.state, "refused", False),
)
return response
@app.exception_handler(RepairPointer)
async def _repair_pointer_handler(_, exc: RepairPointer) -> JSONResponse:
async def _repair_pointer_handler(request: Request, exc: RepairPointer) -> JSONResponse:
tel = getattr(request.app.state, "telemetry", None)
detail = exc.detail if isinstance(exc.detail, dict) else {}
code = detail.get("code")
if tel is not None and code in tel.auth_codes:
# A refusal is a failure row (field-visibility rule 1). The caller is
# unauthenticated by definition, so: system actor, no actor_id, and
# the route TEMPLATE, never the concrete path.
request.state.refused = True
route = request.scope.get("route")
tel.auth_failed(
code=code,
http_status=exc.status_code,
caller=caller_class(request.headers),
route=getattr(route, "path", None),
upstream_status=getattr(exc, "upstream_status", None),
synthetic=is_synthetic(request.headers),
)
return JSONResponse(status_code=exc.status_code, content=exc.detail)

View File

@@ -105,7 +105,7 @@ class DatabaseProvider(Provider):
# TunnelProvider was removed deliberately. See the note in main.py: cloudflared
# binds 127.0.0.1:2000 on the HOST, and this process runs in a container whose
# binds 127.0.0.1:2001 on the HOST, and this process runs in a container whose
# only route to the host is the bridge gateway (172.17.0.1), where nothing is
# listening. Binding the metrics endpoint wider would fix the probe and make a
# metrics bind failure able to take down ingress -- a worse trade than losing

View File

@@ -26,6 +26,7 @@ from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from api.app import throttle
from api.app.auth import ActorType, Caller, get_caller
from api.app.errors import RepairPointer
from api.app.models.core import (
@@ -208,6 +209,11 @@ async def create_repo(
remediation_tool=None,
)
# Throttle before any side effect. Checking after would let a rate-limited
# agent still create the Gitea repo and only then be told no.
async with _sessionmaker(request)() as session:
await throttle.enforce(session, settings, caller, "repo.create")
gitea = GiteaClient(settings)
owner = _owner_login(caller)
await gitea.ensure_user(owner, f"{owner}@windygit.com")
@@ -234,6 +240,8 @@ async def create_repo(
),
)
session.add(repo)
await session.flush()
await throttle.record(session, caller, "repo.create", repo_id=repo.id)
await session.commit()
await session.refresh(repo)
@@ -337,6 +345,7 @@ async def create_grant(
"""
settings = request.app.state.settings
async with _sessionmaker(request)() as session:
await throttle.enforce(session, settings, caller, "grant.create")
repo = await _load_repo(session, repo_id, caller)
is_owner = (caller.identity_id and repo.identity_id == caller.identity_id) or (
caller.passport and repo.passport == caller.passport
@@ -364,6 +373,8 @@ async def create_grant(
expires_at=expires,
)
session.add(grant)
await session.flush()
await throttle.record(session, caller, "grant.create", repo_id=repo.id)
await session.commit()
await session.refresh(grant)
grant_id, role = grant.id, grant.role

View File

@@ -20,6 +20,7 @@ import httpx
from api.app.config import Settings
from api.app.errors import RepairPointer, provider_unconfigured
from api.app.telemetry import synthetic_headers
_TIMEOUT = httpx.Timeout(20.0, connect=5.0)
@@ -36,6 +37,7 @@ class GiteaClient:
return {
"Authorization": f"token {self._s.gitea_admin_token}",
"Content-Type": "application/json",
**synthetic_headers(), # end-to-end synthetic convention (Telemetry UPDATE 4)
}
async def _request(self, method: str, path: str, **kw: Any) -> httpx.Response:

255
api/app/telemetry.py Normal file
View File

@@ -0,0 +1,255 @@
"""Field telemetry to the admin ledger (admin.windyword.ai) — step 2, 2026-09-23.
Shapes are DECLARED with Telemetry Boss (the ledger owner); the server
quarantines any row that doesn't match, so never add a key or a code here
without re-declaring it first:
service.boot once per process start {commit_sha, version, environment}
service.health hourly, in-process interval_s, uptime_s, requests,
errors_5xx, errors_4xx,
refusals_4xx, p95_ms
forge.auth.failed every refused request {code, http_status, caller,
route?, upstream_status?}
Refusals come first: a refused caller is the most expensive silent failure
("the button did nothing"). An UNAUTHENTICATED caller has no trustworthy id, so
per the all-lanes actor rule the row is actor_type "system", no actor_id, and
the caller class goes in metadata.caller.
Privacy: codes, statuses, route TEMPLATES, counts, durations. Never a passport
number, an email, a token fragment or a concrete path with names in it.
No token → nothing is sent and nothing is buffered. A failed flush keeps the
rows (bounded) and retries on the next tick; it never raises into a request.
"""
from __future__ import annotations
import asyncio
import contextvars
import json
import logging
import time
import urllib.request
from datetime import UTC, datetime
log = logging.getLogger("windy-git.telemetry")
PLATFORM, SERVICE = "windy-git", "api"
FLUSH_EVERY_S = 60
HEALTH_EVERY_S = 3600
MAX_BUFFER = 5000
MAX_LATENCY_SAMPLES = 20000
# The declared forge.auth.failed code enum (Telemetry Boss, 2026-09-23). A code
# outside this set is NOT a refusal row — it counts in errors_* instead.
AUTH_CODES = frozenset(
{
"not_signed_in",
"token_invalid",
"token_unrecognised",
"ept_invalid",
"passport_revoked",
"passport_unresolvable",
"agent_read_only",
"agent_rate_limited",
"quota_exceeded",
"trust_unavailable",
"throttle_unavailable",
"service_token_invalid",
"service_auth_unconfigured",
}
)
def _iso(epoch: float) -> str:
return datetime.fromtimestamp(epoch, UTC).isoformat().replace("+00:00", "Z")
# Ecosystem convention (Telemetry UPDATE 4): synthetic traffic travels END TO
# END. Originators (canaries, probes, journeys) send `X-Windy-Synthetic: 1`;
# every service marks all of that request's rows synthetic:true AND forwards the
# header on every downstream call. Absent = real. Never strip it, never set it
# on real traffic. The label separates rows — it never suppresses them.
SYNTHETIC: contextvars.ContextVar[bool] = contextvars.ContextVar("windy_synthetic", default=False)
def is_synthetic(headers) -> bool:
return bool((headers.get("x-windy-synthetic") or "").strip())
def synthetic_headers() -> dict:
"""Merge into every downstream request made while serving this one."""
return {"X-Windy-Synthetic": "1"} if SYNTHETIC.get() else {}
def caller_class(headers) -> str:
"""Declared values: anonymous_human | anonymous_agent | unknown."""
from api.app.ept import looks_like_ept
if headers.get("x-service-token"):
return "unknown"
auth = headers.get("authorization") or ""
if not auth.lower().startswith("bearer "):
return "unknown"
return "anonymous_agent" if looks_like_ept(auth.split(" ", 1)[1].strip()) else "anonymous_human"
class Telemetry:
def __init__(
self,
url: str,
token: str,
*,
environment: str = "",
commit_sha: str | None = None,
version: str = "",
) -> None:
self.url, self.token = url, token
self.environment, self.commit_sha, self.version = environment, commit_sha, version
self.started = time.time()
self.buffer: list[dict] = []
self.auth_codes = AUTH_CODES
self._reset_window()
@property
def enabled(self) -> bool:
return bool(self.token)
def _reset_window(self) -> None:
self.window_start = time.time()
self.requests = self.errors_5xx = self.errors_4xx = self.refusals_4xx = 0
self.latencies_ms: list[float] = []
# UPDATE 7: rows the ledger quarantined (it still answers 202) and rows
# this process lost (buffer overflow). Non-zero = a bug in this emitter.
self.quarantined = self.dropped = 0
# ---- recording (never raises into a request) --------------------------
def record_request(self, status: int, duration_ms: float, *, refused: bool = False) -> None:
self.requests += 1
if status >= 500:
self.errors_5xx += 1
elif refused:
self.refusals_4xx += 1
elif status >= 400:
self.errors_4xx += 1
if len(self.latencies_ms) < MAX_LATENCY_SAMPLES:
self.latencies_ms.append(duration_ms)
def _event(self, event_type: str, metadata: dict, *, ts: float | None = None) -> None:
if not self.enabled:
return
self.buffer.append(
{
"ts": _iso(ts or time.time()),
"platform": PLATFORM,
"service": SERVICE,
"event_type": event_type,
"actor_type": "system",
"metadata": metadata,
}
)
if len(self.buffer) > MAX_BUFFER:
self.dropped += len(self.buffer) - MAX_BUFFER
del self.buffer[: len(self.buffer) - MAX_BUFFER]
def boot(self) -> None:
meta = {"version": self.version, "environment": self.environment}
if self.commit_sha: # unknown is absent, never invented (I-12)
meta["commit_sha"] = self.commit_sha
self._event("service.boot", meta, ts=self.started)
def auth_failed(
self,
*,
code: str,
http_status: int,
caller: str,
route: str | None = None,
upstream_status: int | None = None,
synthetic: bool = False,
) -> None:
if code not in AUTH_CODES:
return
meta: dict = {
"code": code,
"http_status": int(http_status),
"caller": caller,
"synthetic": bool(synthetic),
}
if route:
meta["route"] = route
if upstream_status is not None:
meta["upstream_status"] = int(upstream_status)
self._event("forge.auth.failed", meta)
def health_row(self) -> dict:
now = time.time()
meta = {
"interval_s": int(now - self.window_start),
"uptime_s": int(now - self.started),
"requests": self.requests,
"errors_5xx": self.errors_5xx,
"errors_4xx": self.errors_4xx,
"refusals_4xx": self.refusals_4xx,
"telemetry_quarantined": self.quarantined,
"telemetry_dropped": self.dropped,
}
if self.latencies_ms: # no traffic = no p95, not a fake 0
s = sorted(self.latencies_ms)
meta["p95_ms"] = int(s[min(len(s) - 1, int(0.95 * (len(s) - 1) + 0.5))])
return meta
def health(self) -> None:
self._event("service.health", self.health_row())
self._reset_window()
# ---- sending ------------------------------------------------------------
def _post(self, batch: list[dict]) -> tuple[int, dict]:
req = urllib.request.Request(
self.url,
data=json.dumps({"events": batch}).encode(),
method="POST",
headers={
"Authorization": f"Bearer {self.token}",
"Content-Type": "application/json",
"User-Agent": "windy-git-api-telemetry/1",
},
)
with urllib.request.urlopen(req, timeout=20) as r:
try:
body = json.loads(r.read() or b"{}")
except ValueError:
body = {}
return r.status, body if isinstance(body, dict) else {}
async def flush(self) -> None:
if not self.enabled or not self.buffer:
return
batch = self.buffer[:500]
try:
status, body = await asyncio.to_thread(self._post, batch)
except Exception as exc: # noqa: BLE001 - telemetry must never take the API down
log.warning("telemetry flush failed (%d rows kept): %s", len(self.buffer), exc)
return
if 200 <= status < 300:
del self.buffer[: len(batch)]
self.note_quarantine(body)
def note_quarantine(self, body: dict) -> None:
# 202 does NOT mean every row landed: refused rows are dead-lettered.
q = body.get("quarantined")
if isinstance(q, int) and q > 0:
self.quarantined += q
log.warning("telemetry: %d row(s) QUARANTINED by the ledger: %s", q,
"; ".join(map(str, body.get("rejections") or [])) or "no reason given")
async def run(self) -> None:
"""The one in-process timer: flush every minute, heartbeat every hour."""
last_health = time.monotonic()
while True:
await asyncio.sleep(FLUSH_EVERY_S)
if time.monotonic() - last_health >= HEALTH_EVERY_S:
self.health()
last_health = time.monotonic()
await self.flush()

187
api/app/throttle.py Normal file
View File

@@ -0,0 +1,187 @@
"""EI-band velocity limits (G3.4) — throttle by trust, never by omission.
`BAND_MULTIPLIER` and the `rate_*_per_day` settings existed since G0 and were
**read by nothing**: a documented invariant with no implementation, which is the
exact failure pattern the ecosystem audits kept finding. This module is the
missing consumer.
The doctrine (§0.6) is *capability-completeness*: an agent is never denied a
capability it should have, it is **rate-limited by how much it has proven**.
Platinum gets 10x, gold 4x, standard 1x, watch 0.5x, and untrusted is read-only.
Counting is done against `agent_actions`, which is already the append-only record
of every agent write. That is deliberate: a limiter with its own private counter
disagrees with the audit log the moment either is restarted, and then nobody can
say what actually happened. One source of truth, queried.
**Fail-closed.** If the count cannot be taken, the action is refused. A limiter
that fails open is decoration — it protects you right up until the moment
something is wrong, which is the only moment it matters.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from api.app.auth import BAND_MULTIPLIER, ActorType, Caller
from api.app.config import Settings
from api.app.errors import RepairPointer
from api.app.models.core import AgentAction
log = logging.getLogger(__name__)
WINDOW = timedelta(days=1)
# Actions this module ACTUALLY enforces: they pass through our API, so we can
# both count and refuse them.
ACTION_BASE: dict[str, str] = {
"repo.create": "rate_repo_creates_per_day",
"grant.create": "rate_grants_per_day",
}
# ⚠️ DECLARED BUT NOT ENFORCEABLE HERE — and named, rather than quietly listed
# alongside the real ones.
#
# `git push` goes straight to Gitea over HTTPS and never touches this API, so
# nothing records a `push` action and a count of them would be zero forever.
# Listing these in ACTION_BASE (as this module first did) would make `enforce`
# look up a limit, count nothing, and allow everything — a silent no-op wearing
# the costume of a control. That is the same dead-code pattern this module was
# written to remove, and it is worse here because the config name implies the
# protection exists.
#
# Enforcing push velocity requires a Gitea-side hook (pre-receive or the push
# webhook) that reports into `agent_actions`. Until that exists these settings
# are inert, and saying so is the honest option.
NOT_ENFORCED_HERE: dict[str, str] = {
"push": "rate_pushes_per_day",
"push.force": "rate_force_pushes_per_day",
}
def limit_for(settings: Settings, action: str, band: str | None) -> int:
"""Effective per-day allowance. Unknown bands get the standard rate.
Unknown-band handling is a real decision, not a default. Eternitas began
emitting `unproven` on 2026-07-30 without it appearing in the documented
enum. Treating an unrecognised band as untrusted would lock out every freshly
hatched agent the day Eternitas adds a name; treating it as platinum would be
a hole. Standard-with-no-bonus is the honest middle.
"""
base = getattr(settings, ACTION_BASE[action])
mult = BAND_MULTIPLIER.get((band or "").lower(), 1.0)
return int(base * mult)
async def enforce(
session: AsyncSession,
settings: Settings,
caller: Caller,
action: str,
) -> None:
"""Raise 429 if this agent has spent its allowance. No-op for humans.
Humans are governed by account tier and their own session; this is the
agent-velocity control specifically (I-6: parity in capability, asymmetry in
throttle).
"""
if caller.actor_type != ActorType.agent or not caller.passport:
return
if action not in ACTION_BASE: # unknown action = unlimited would be a hole
raise RepairPointer(
status_code=500,
code="throttle_unknown_action",
speak="Something went wrong on our side. Nothing was changed.",
machine_cause=f"no rate base configured for action {action!r}",
remediation_tool=None,
)
band = (caller.band or "").lower()
# Untrusted is read-only. This is a capability decision, not a rate: it gets
# a 403 with a different explanation, because "slow down" would be a lie.
if BAND_MULTIPLIER.get(band, 1.0) <= 0:
raise RepairPointer(
status_code=403,
code="agent_read_only",
speak="That helper can look, but it isn't allowed to make changes yet.",
machine_cause=f"passport {caller.passport} band={band!r} is read-only",
remediation_tool=None,
)
allowed = limit_for(settings, action, band)
since = datetime.now(UTC) - WINDOW
try:
used = (
await session.execute(
select(func.count())
.select_from(AgentAction)
.where(
AgentAction.passport == caller.passport,
AgentAction.action == action,
AgentAction.result == "ok",
AgentAction.ts >= since,
)
)
).scalar_one()
except Exception as exc: # noqa: BLE001
# FAIL CLOSED. A limiter that fails open protects you until the moment
# something is wrong, which is the only moment it matters.
log.warning("throttle count failed for %s/%s: %s", caller.passport, action, exc)
raise RepairPointer(
status_code=503,
code="throttle_unavailable",
speak="We couldn't check that helper's limits, so we didn't make the change.",
machine_cause=f"agent_actions count failed: {type(exc).__name__}",
remediation_tool=None,
) from exc
if used >= allowed:
raise RepairPointer(
status_code=429,
code="agent_rate_limited",
speak=(
"That helper has done a lot in the last day, so we've paused it. "
"It'll be able to continue shortly."
),
machine_cause=(
f"passport {caller.passport} used {used}/{allowed} of {action} "
f"in 24h (band={band or 'unknown'})"
),
remediation_tool=None,
used=used,
allowed=allowed,
band=band or "unknown",
)
async def record(
session: AsyncSession,
caller: Caller,
action: str,
result: str = "ok",
repo_id=None,
) -> None:
"""Append the action that was just allowed.
Written AFTER the work succeeds, on purpose: counting attempts would let a
failing agent exhaust its own allowance by retrying, turning a transient
error into a lockout.
"""
if caller.actor_type != ActorType.agent or not caller.passport:
return
session.add(
AgentAction(
passport=caller.passport,
repo_id=repo_id,
action=action,
ei_at_action=caller.band,
result=result,
)
)
await session.flush()

View File

@@ -0,0 +1,92 @@
"""The canary's login probe must end the session it opens (journey cleanup rule)."""
from __future__ import annotations
import importlib.util
import io
import sys
import urllib.error
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
_spec = importlib.util.spec_from_file_location("canary", ROOT / "scripts" / "canary.py")
canary = importlib.util.module_from_spec(_spec)
sys.modules["canary"] = canary # dataclasses resolve their module by name
_spec.loader.exec_module(canary)
class _Resp:
def __init__(self, status=200, body=b"{}"):
self.status, self._body = status, body
def read(self):
return self._body
def __enter__(self):
return self
def __exit__(self, *a):
return False
def _err(code):
return urllib.error.HTTPError(canary.LOGOUT_URL, code, "x", {}, io.BytesIO(b""))
def _script(monkeypatch, outcomes):
calls = []
def fake(req, timeout=None):
calls.append((req.get_method(), req.full_url, req.get_header("Authorization")))
o = outcomes.pop(0)
if isinstance(o, Exception):
raise o
return o
monkeypatch.setattr(canary.urllib.request, "urlopen", fake)
return calls
def test_logout_ends_the_session(monkeypatch):
calls = _script(monkeypatch, [_Resp(200)])
r = canary.logout("tok", sleep=lambda s: None)
assert r.status == "ok"
assert calls == [("POST", canary.LOGOUT_URL, "Bearer tok")]
def test_5xx_and_no_response_are_retried_then_succeed(monkeypatch):
calls = _script(monkeypatch, [_err(502), OSError("reset"), _Resp(200)])
assert canary.logout("tok", sleep=lambda s: None).status == "ok"
assert len(calls) == 3
def test_already_over_counts_as_done(monkeypatch):
_script(monkeypatch, [_err(401)])
assert canary.logout("tok", sleep=lambda s: None).status == "ok"
def test_other_4xx_fails_fast_and_honestly(monkeypatch):
calls = _script(monkeypatch, [_err(400)])
r = canary.logout("tok", sleep=lambda s: None)
assert r.status == "down" and r.detail.startswith("CLEANUP FAILED") and len(calls) == 1
def test_retries_are_bounded_and_reported(monkeypatch):
calls = _script(monkeypatch, [_err(503)] * 8)
r = canary.logout("tok", attempts=8, sleep=lambda s: None)
assert r.status == "down" and "CLEANUP FAILED after 8 tries" in r.detail and len(calls) == 8
def test_login_probe_logs_out_with_the_token_it_got(monkeypatch):
calls = _script(monkeypatch, [_Resp(200, b'{"token": "abc"}'), _Resp(200)])
c = canary.Check("identity.login", "https://account.windyword.ai/api/v1/auth/login", "x",
method="POST", body={"email": "e", "password": "p"},
after=canary._logout_after_login)
r = canary._probe(c)
assert r.status == "ok" and [f.status for f in r.followups] == ["ok"]
assert calls[1] == ("POST", canary.LOGOUT_URL, "Bearer abc")
def test_login_without_token_is_a_cleanup_failure_not_a_pass():
[f] = canary._logout_after_login(b"{}")
assert f.status == "down" and "CLEANUP FAILED" in f.detail

View File

@@ -0,0 +1,164 @@
"""CI hygiene guard (house rule 6): lockfile-only installs, no host-port services."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "scripts"))
_spec = importlib.util.spec_from_file_location("ci_hygiene", ROOT / "scripts" / "ci_hygiene.py")
hy = importlib.util.module_from_spec(_spec)
sys.modules["ci_hygiene"] = hy
_spec.loader.exec_module(hy)
WF = ".github/workflows/ci.yml"
@pytest.mark.parametrize("path, text", [
(WF, " .venv/bin/pip install -e \".[dev]\""), # windy-git's own, before e64a1b5
("Dockerfile", "RUN pip install --no-cache-dir -e ."), # windy-git image, before e64a1b5
(WF, " - run: uv pip install -e \".[dev]\""), # WindyCloud #109's CI
(WF, " run: pip install fastapi uvicorn"),
(WF, " - run: uv sync --all-extras"), # windy-mind style, not locked
(WF, " - run: npm install"), # windy-drops / windytalk
(WF, " - run: npm install --no-save --no-audit --no-fund jsdom"), # windy-pro reality-check
(WF, " - run: yarn install"),
(WF, " - run: cd web && pnpm install"),
("docker/api.Dockerfile", "RUN apt-get update && pip install requests"),
])
def test_floating_installs_are_flagged(path, text):
assert [k for k, _ in hy.scan_line(path, text)] == ["floating install"]
@pytest.mark.parametrize("path, text", [
(WF, " python3 -m pip install -q uv==0.12.5"), # exact tool pin
(WF, " uv sync --locked --extra dev"),
(WF, " - run: uv sync --frozen"),
(WF, " - run: npm ci"),
(WF, " - run: npm install --no-save jsdom@24.1.0"),
(WF, " - run: pip install -r requirements.lock --require-hashes"),
(WF, " - run: pip install -r requirements.txt"),
("Dockerfile", " && pip install --no-cache-dir --require-hashes -r /tmp/requirements.txt \\\\"),
("Dockerfile", "RUN pip install --no-cache-dir --no-deps -e ."), # project only, deps from the lock
(WF, " .venv/bin/pip install -q --upgrade pip"),
(WF, " - run: yarn install --frozen-lockfile"),
(WF, " # - run: npm install (commented out)"),
(WF, " - run: echo 'pip is great'"),
])
def test_locked_or_pinned_installs_pass(path, text):
assert hy.scan_line(path, text) == []
@pytest.mark.parametrize("text, port", [
(" - 5432:5432", "5432"), # windy-mind / eternitas (collided 09-23)
(" - '15432:5432'", "15432"), # WindyCloud
(' - "6379:6379"', "6379"),
])
def test_services_publishing_a_host_port_are_flagged(text, port):
[(kind, match)] = hy.scan_line(WF, text)
assert kind == "host port" and port in match
def test_host_port_rule_is_for_workflows_only():
assert hy.scan_line("docker-compose.yml", " - 5432:5432") == []
@pytest.mark.parametrize("path, ok", [
(".github/workflows/ci.yml", True), (".gitea/workflows/check.yaml", True),
("Dockerfile", True), ("api/Dockerfile.prod", True), ("docker/web.Dockerfile", True),
("scripts/setup.sh", False), ("README.md", False), ("node_modules/x/Dockerfile", False),
(".github/lint/x.yml", False),
])
def test_scope_is_ci_workflows_and_dockerfiles(path, ok):
assert hy.path_ok(path) is ok
def test_warn_mode_never_turns_red(monkeypatch):
monkeypatch.setattr(hy, "MODE", "warn")
state, desc, f = hy.status_for([hy.cg.Finding(WF, 12, "floating install", "npm install (use npm ci)")], True)
assert state == "success" and desc.startswith("⚠ WARN (not blocking): 1 CI hygiene issue in CI/Dockerfiles")
def test_allow_file_is_line_scoped_exceptions_only():
"""Every exception is line-scoped (`matches`), so an allowed file can't hide a
NEW floating install or docker step. Today: windy-pro's if:false deploy job."""
allow = hy.cg.load_allow(hy.ALLOW_FILE)
assert [(e["repo"], e["paths"]) for e in allow] == [("windy-pro", [".github/workflows/ci.yml"])]
assert all(e.get("matches") for e in allow)
ok = " run: docker build -f account-server/Dockerfile -t windy-pro:${{ github.sha }} ."
new = " run: docker build -t windy-pro-api ."
assert hy.cg.allowed("windy-pro", WF, allow, ok)
assert not hy.cg.allowed("windy-pro", WF, allow, new)
assert not hy.cg.allowed("windy-chat", WF, allow, ok)
@pytest.mark.parametrize("path, text, want", [
("Dockerfile", "COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv", "ghcr.io/astral-sh/uv:latest"), # Mail #147
("Dockerfile", "FROM python:latest", "python:latest"),
("Dockerfile", "FROM --platform=linux/amd64 node:latest AS web", "node:latest"),
(WF, " image: postgres:latest", "postgres:latest"),
(WF, " - uses: docker://ghcr.io/foo/bar:latest", "ghcr.io/foo/bar:latest"),
])
def test_latest_images_are_flagged(path, text, want):
hits = hy.scan_line(path, text)
assert ("floating image", want) in hits
@pytest.mark.parametrize("text", [
"COPY pyproject.toml uv.lock* ./", # Windy Mail #147
"COPY package.json package-lock.json* ./",
])
def test_optional_lock_globs_are_flagged(text):
assert [k for k, _ in hy.scan_line("Dockerfile", text)] == ["optional lock"]
@pytest.mark.parametrize("path, text", [
("Dockerfile", "COPY --from=ghcr.io/astral-sh/uv:0.12.5 /uv /usr/local/bin/uv"),
("Dockerfile", "FROM python:3.12-slim"),
("Dockerfile", "COPY pyproject.toml uv.lock ./"),
("Dockerfile", "COPY src/*.py ./src/"),
("Dockerfile", "RUN echo latest release notes"),
])
def test_pinned_images_and_real_locks_pass(path, text):
assert hy.scan_line(path, text) == []
@pytest.mark.parametrize("text", [
" - run: docker compose -f docker-compose.yml -f docker-compose.ci.yml build", # eternitas ci/build
" run: docker build -t windy-mail .",
" - run: docker-compose up -d",
" run: docker buildx build --load .",
" - uses: docker/build-push-action@v6",
])
def test_docker_in_ci_is_flagged_with_the_fix(text):
hits = hy.scan_line(WF, text)
assert [k for k, _ in hits] == ["needs docker"]
assert "no-Docker smoke test" in hits[0][1]
@pytest.mark.parametrize("path, text", [
("Dockerfile", "RUN docker build ."), # not a workflow
(WF, " run: ssh host 'docker compose up -d'"), # remote host has a daemon
(WF, " # docker compose build"), # comment
(WF, " run: echo docker build"),
])
def test_docker_not_flagged_outside_ci_steps(path, text):
assert [k for k, _ in hy.scan_line(path, text) if k == "needs docker"] == []
def test_needs_docker_skips_workflows_disabled_on_windy_git(monkeypatch):
"""deploy.yml runs on the target host (a real daemon); Gitea has it disabled here."""
F = hy.cg.Finding
monkeypatch.setattr(hy, "_DISABLED", {"eternitas": {"deploy.yml"}})
got = hy._runs_here("Eternitas", [
F(".github/workflows/deploy.yml", 70, "needs docker", "docker compose in CI"),
F(".github/workflows/ci.yml", 176, "needs docker", "docker compose in CI"),
F(".github/workflows/deploy.yml", 12, "floating install", "npm install"),
])
assert [(f.path.rsplit("/", 1)[1], f.kind) for f in got] == [
("ci.yml", "needs docker"), ("deploy.yml", "floating install")]
assert hy._runs_here("eternitas", None) is None

View File

@@ -0,0 +1,252 @@
"""Compute guard: Windy Mind is the only door to AI compute (warn-only today)."""
from __future__ import annotations
import importlib.util
import subprocess
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[2]
_spec = importlib.util.spec_from_file_location("compute_guard", ROOT / "scripts" / "compute_guard.py")
cg = importlib.util.module_from_spec(_spec)
sys.modules["compute_guard"] = cg
_spec.loader.exec_module(cg)
ALLOW = cg.load_allow(ROOT / "ci" / "compute-guard-allow.yml")
@pytest.mark.parametrize(
"path, text, kind",
[
# audit #1 (windy-search, closed) and #2 (windy-chat, live): the shapes they had
("service/app/anthropic_client.py", 'URL = "https://api.anthropic.com/v1/messages"', "provider host"),
("service/app/config.py", 'token = os.environ["ANTHROPIC_OAUTH_TOKEN"]', "provider key"),
("services/agent-roster/lib/llm.js", "const url = 'https://api.groq.com/openai/v1/chat/completions'", "provider host"),
("docker-compose.yml", " GROQ_API_KEY: ${GROQ_API_KEY}", "provider key"),
# audit #3/#4 (windy-pro account-server)
("account-server/src/routes/transcription.ts", "const r = await fetch('https://api.openai.com/v1/audio/transcriptions'", "provider host"),
("account-server/src/config.ts", "openaiKey: process.env.OPENAI_API_KEY,", "provider key"),
# SDKs and deps
("app/llm.py", "from anthropic import Anthropic", "provider SDK"),
("app/llm.py", "import openai", "provider SDK"),
("app/llm.py", "import google.generativeai as genai", "provider SDK"),
("src/ai.ts", 'import Anthropic from "@anthropic-ai/sdk";', "provider SDK"),
("src/ai.js", "const Groq = require('groq-sdk')", "provider SDK"),
("package.json", ' "openai": "^4.52.0",', "provider SDK dep"),
("requirements.txt", "anthropic>=0.40", "provider SDK dep"),
("pyproject.toml", ' "google-generativeai>=0.8",', "provider SDK dep"),
],
)
def test_audit_shapes_are_flagged(path, text, kind):
assert kind in [k for k, _ in cg.scan_line(path, text)]
@pytest.mark.parametrize(
"path, text",
[
("app/mind.py", 'MIND = "https://mind.windyword.ai/v1/chat/completions"'), # the door itself
("app/models.py", "openai_compatible = True # Mind speaks the OpenAI wire format"),
("app/x.py", "from app.openai_shim import x"), # a local module, not the SDK
("package.json", ' "openai-types-lite": "1.0.0",'), # a different package
("README.txt", "set OPENAI_API_KEY"), # scanned-by-rule, excluded by SKIP separately
],
)
def test_near_misses_are_not_flagged(path, text):
if cg.SKIP.search(path):
return
assert cg.scan_line(path, text) == []
@pytest.mark.parametrize(
"path",
["tests/test_llm.py", "api/tests/x.py", "src/ai.test.ts", "web/foo.spec.js", "docs/setup.md",
"README.md", "package-lock.json", "uv.lock", "node_modules/openai/index.js", ".github/workflows/ci.yml",
"conftest.py", "app/llm_test.py"],
)
def test_tests_docs_lockfiles_vendored_ci_are_never_scanned(path):
assert cg.SKIP.search(path)
def test_allow_list_needs_a_reason_per_entry(tmp_path):
bad = tmp_path / "a.yml"
bad.write_text("allow:\n - repo: x\n paths: ['*']\n")
with pytest.raises(ValueError):
cg.load_allow(bad)
@pytest.mark.parametrize(
"repo, path, ok",
[
("windy-mind", "app/providers/anthropic.py", True),
("windy-agent", "agent/providers.py", True),
("windy-code", "extensions/windy-ai/src/aiProvider.ts", True),
("windy-code", "web/server/llm.ts", False), # BYOK is the extension only
("windy-connect", "backend/src/writers/claude_code.py", True),
("windy-chat", "services/agent-roster/lib/llm.js", False), # audit #2: must be flagged
("windy-pro", "account-server/src/routes/translations.ts", False),
],
)
def test_allow_list_entries(repo, path, ok):
assert cg.allowed(repo, path, ALLOW) is ok
DIFF = """diff --git a/app/llm.py b/app/llm.py
--- a/app/llm.py
+++ b/app/llm.py
@@ -10,0 +11,2 @@
+import anthropic
+client = anthropic.Anthropic()
diff --git a/tests/test_llm.py b/tests/test_llm.py
--- /dev/null
+++ b/tests/test_llm.py
@@ -0,0 +1 @@
+import anthropic
@@ -40 +42 @@
-x = 1
+x = 2
"""
def test_only_added_non_test_lines_are_findings():
fs = cg.parse_added("windy-chat", DIFF, ALLOW)
assert [(f.path, f.line, f.kind) for f in fs] == [("app/llm.py", 11, "provider SDK")]
def _repo(tmp_path, files: dict[str, str]) -> tuple[Path, str]:
work = tmp_path / "w"
work.mkdir()
run = lambda *a: subprocess.run(["git", *a], cwd=work, check=True, capture_output=True) # noqa: E731
run("init", "-q", "-b", "main")
for p, text in files.items():
(work / p).parent.mkdir(parents=True, exist_ok=True)
(work / p).write_text(text)
run("add", "-A")
run("-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm", "x")
bare = tmp_path / "r.git"
subprocess.run(["git", "clone", "-q", "--bare", str(work), str(bare)], check=True)
sha = subprocess.run(["git", "--git-dir", str(bare), "rev-parse", "main"],
capture_output=True, text=True, check=True).stdout.strip()
return bare, sha
def test_tree_scan_on_a_real_git_repo(tmp_path):
bare, sha = _repo(tmp_path, {
"app/llm.py": "import os\nKEY = os.environ['OPENAI_API_KEY']\n",
"app/ok.py": "MIND = 'https://mind.windyword.ai'\n",
"tests/test_llm.py": "import anthropic\n",
"docs/x.md": "api.anthropic.com\n",
})
fs = cg.scan_tree("windy-chat", bare, sha, ALLOW)
assert [(f.path, f.line, f.kind) for f in fs] == [("app/llm.py", 2, "provider key")]
def test_warn_mode_never_turns_red(monkeypatch):
monkeypatch.setattr(cg, "MODE", "warn")
state, desc, f = cg.status_for([cg.Finding("a.py", 3, "provider host", "api.openai.com")], whole_tree=False)
assert state == "success" and desc.startswith("⚠ WARN (not blocking): 1 direct AI-provider use added")
assert "a.py:3" in desc and f.path == "a.py"
def test_block_mode_fails(monkeypatch):
monkeypatch.setattr(cg, "MODE", "block")
state, desc, _ = cg.status_for([cg.Finding("a.py", 3, "provider host", "x")], whole_tree=True)
assert state == "failure" and desc.startswith("BLOCKED")
def test_clean_is_ok():
assert cg.status_for([], whole_tree=True)[:2] == (
"success", "OK: no direct AI-provider use in tree (Windy Mind is the only door)")
@pytest.mark.parametrize(
"text",
[
" # The ANTHROPIC_OAUTH_TOKEN setting was removed on 2026-09-23 ON PURPOSE", # windy-search
"# ANTHROPIC_API_KEY=",
" // fallback used to call https://api.groq.com directly",
" * @see https://api.openai.com/v1/audio",
"<!-- api.anthropic.com -->",
],
)
def test_comments_are_not_calls(text):
assert cg.scan_line("service/app/config.py", text) == []
def test_code_with_a_trailing_comment_still_counts():
assert cg.scan_line("a.js", "fetch('https://api.openai.com/v1') // TODO move to Mind")
def test_windy_pro_desktop_is_byok_but_the_account_server_is_not():
assert cg.allowed("windy-pro", "src/client/desktop/main.js", ALLOW)
assert not cg.allowed("windy-pro", "account-server/src/routes/translations.ts", ALLOW)
def test_a_commit_not_fetched_yet_is_skipped_not_an_error(tmp_path, monkeypatch):
bare, sha = _repo(tmp_path, {"app/llm.py": "import anthropic\n"})
monkeypatch.setattr(cg, "WORK", tmp_path)
monkeypatch.setattr(cg, "CACHE", tmp_path / "cache.json")
(tmp_path / "windy-chat.git").symlink_to(bare)
assert cg.check("windy-chat", "f" * 40, "main", True) is None # pushed after the fetch
assert [f.kind for f in cg.check("windy-chat", sha, "main", True)] == ["provider SDK"]
KEYCHAIN = "src/client/web/src/pages/panels/MindKeychain.jsx"
def test_scoped_allow_admits_only_the_oauth_endpoints():
ok = [
" window.location.href = `https://openrouter.ai/auth?callback_url=${encodeURIComponent(callback)}`",
" const res = await fetch('https://openrouter.ai/api/v1/auth/keys', {",
]
for line in ok:
assert cg.allowed("windy-pro", KEYCHAIN, ALLOW, line)
# an inference call smuggled into the same file still flags
assert not cg.allowed("windy-pro", KEYCHAIN, ALLOW,
" await fetch('https://openrouter.ai/api/v1/chat/completions', {")
# a scoped entry never allows a line it can't see
assert not cg.allowed("windy-pro", KEYCHAIN, ALLOW)
def test_scoped_allow_in_a_real_diff():
diff = f"""--- /dev/null
+++ b/{KEYCHAIN}
@@ -0,0 +1,3 @@
+ window.location.href = `https://openrouter.ai/auth?callback_url=x`
+ const res = await fetch('https://openrouter.ai/api/v1/auth/keys', {{
+ await fetch('https://openrouter.ai/api/v1/chat/completions', {{
"""
fs = cg.parse_added("windy-pro", diff, ALLOW)
assert [(f.line, f.match) for f in fs] == [(3, "openrouter.ai")]
def test_block_mode_never_blocks_grant_owned(monkeypatch):
monkeypatch.setattr(cg, "MODE", "block")
g = cg.Finding("src/client/desktop/x.js", 9, "provider host", "api.openai.com")
state, desc, f = cg.status_for([], whole_tree=True, grant=[g])
assert state == "success" and desc.startswith("⚠ WARN (Grant-owned, not blocking): 1") and f is g
lane = cg.Finding("a.py", 3, "provider host", "x")
assert cg.status_for([lane], whole_tree=True, grant=[g])[0] == "failure"
def test_veron_ollama_warns_on_added_lines_and_never_blocks(monkeypatch):
hits = cg.scan_line("app/llm.py", 'OLLAMA = "http://192.168.1.73:11434/api/generate"')
assert [k for k, _ in hits] == ["veron ollama"]
assert cg.scan_line("app/llm.py", 'port = 114345') == [] # not the port
assert cg.scan_line("app/llm.py", "# was http://x:11434 (removed)") == [] # a comment is not a call
f = cg.Finding("app/llm.py", 7, "veron ollama", ":11434")
monkeypatch.setattr(cg, "MODE", "block")
state, desc, _ = cg.status_for([f], whole_tree=False)
assert state == "success" and desc.startswith("⚠ WARN: new Veron Ollama ref app/llm.py:7")
assert "Windy Mind" in desc and len(desc) <= 140
hard = cg.Finding("app/llm.py", 1, "provider host", "api.openai.com")
assert cg.status_for([f, hard], whole_tree=False)[0] == "failure" # a real violation still blocks
def test_ollama_in_added_pr_lines_only():
diff = ("+++ b/svc/client.py\n@@ -0,0 +1,2 @@\n+import httpx\n"
"+URL = 'http://veron:11434/api/chat'\n")
got = cg.parse_added("some-repo", diff, [])
assert [(f.kind, f.line) for f in got] == [("veron ollama", 2)]

View File

@@ -0,0 +1,312 @@
"""Behavioral tests for EPT verification and EI throttling.
These sign real ES256 tokens with a locally-generated key and verify against a
locally-served key set, so they exercise the ACTUAL crypto path with no network
dependency and no reliance on Eternitas being reachable.
This file exists because the suite it joins was ~86 "does the source contain
this string" assertions and zero that ran the auth decision — which is how a
live impersonation bypass passed every test on 2026-08-13.
"""
from __future__ import annotations
import time
import jwt
import pytest
from cryptography.hazmat.primitives.asymmetric import ec
from api.app import ept as ept_mod
from api.app.auth import BAND_MULTIPLIER
from api.app.config import Settings
from api.app.ept import EptInvalid, verify_ept
from api.app.throttle import ACTION_BASE, limit_for
ISSUER = "eternitas.ai"
KID = "test-key-1"
@pytest.fixture
def signing(monkeypatch):
"""A real EC keypair; point the verifier's JWKS lookup at its public half."""
key = ec.generate_private_key(ec.SECP256R1())
class _FakeJWK:
def __init__(self, k):
self.key = k
class _FakeClient:
def __init__(self, *a, **kw):
pass
def get_signing_key_from_jwt(self, token):
header = jwt.get_unverified_header(token)
if header.get("kid") != KID:
raise Exception(f"unknown kid {header.get('kid')!r}")
return _FakeJWK(key.public_key())
monkeypatch.setattr(ept_mod, "_jwks_client", None)
monkeypatch.setattr(ept_mod, "PyJWKClient", _FakeClient)
return key
def _sign(key, claims, alg="ES256", kid=KID):
return jwt.encode(claims, key, algorithm=alg, headers={"kid": kid, "typ": "EPT"})
def _claims(**over):
c = {
"sub": "ET26-TEST-0001",
"iss": ISSUER,
"iat": int(time.time()) - 10,
"exp": int(time.time()) + 3600,
}
c.update(over)
return c
# ---- the property that was broken ----------------------------------------
def test_genuine_ept_is_accepted(signing):
v = verify_ept(_sign(signing, _claims()), "https://api.eternitas.ai")
assert v.passport == "ET26-TEST-0001"
def test_passport_comes_from_sub_not_a_passport_claim(signing):
"""Real EPTs put the passport in `sub`. The pre-fix code read `passport` /
`sub_passport`, which no genuine EPT carries — so real agents were never
recognised and only forged tokens ever worked."""
tok = _sign(signing, _claims(sub="ET26-REAL-9999", passport="ET26-LIES-0000"))
assert verify_ept(tok, "https://api.eternitas.ai").passport == "ET26-REAL-9999"
def test_alg_none_is_refused(signing):
"""The exact 2026-08-13 exploit."""
import base64
import json as _j
def seg(d):
return base64.urlsafe_b64encode(_j.dumps(d).encode()).rstrip(b"=").decode()
forged = f"{seg({'alg':'none','typ':'EPT','kid':KID})}.{seg(_claims())}."
with pytest.raises(EptInvalid):
verify_ept(forged, "https://api.eternitas.ai")
def test_signature_from_a_different_key_is_refused(signing):
attacker = ec.generate_private_key(ec.SECP256R1())
with pytest.raises(EptInvalid):
verify_ept(_sign(attacker, _claims()), "https://api.eternitas.ai")
def test_tampered_payload_is_refused(signing):
tok = _sign(signing, _claims())
h, _p, s = tok.split(".")
import base64
import json as _j
evil = base64.urlsafe_b64encode(
_j.dumps(_claims(sub="ET26-EVIL-0000")).encode()
).rstrip(b"=").decode()
with pytest.raises(EptInvalid):
verify_ept(f"{h}.{evil}.{s}", "https://api.eternitas.ai")
def test_expired_token_is_refused(signing):
"""Genuinely signed, genuinely expired — proves exp is enforced rather than
the token merely failing to parse."""
tok = _sign(signing, _claims(exp=int(time.time()) - 5, iat=int(time.time()) - 100))
with pytest.raises(EptInvalid):
verify_ept(tok, "https://api.eternitas.ai")
def test_wrong_issuer_is_refused(signing):
"""Correctly signed by a trusted key but claiming another issuer."""
with pytest.raises(EptInvalid):
verify_ept(_sign(signing, _claims(iss="evil.example.com")), "https://api.eternitas.ai")
def test_unknown_kid_is_refused(signing):
with pytest.raises(EptInvalid):
verify_ept(_sign(signing, _claims(), kid="attacker-key"), "https://api.eternitas.ai")
def test_missing_required_claims_are_refused(signing):
for missing in ("sub", "exp"):
c = _claims()
c.pop(missing)
with pytest.raises(EptInvalid):
verify_ept(_sign(signing, c), "https://api.eternitas.ai")
def test_only_es256_is_ever_accepted():
"""Widening this list reopens algorithm confusion."""
assert ept_mod.ALGORITHMS == ["ES256"]
# ---- the throttle that used to be dead code ------------------------------
def test_band_multiplier_is_actually_consumed():
"""BAND_MULTIPLIER was defined and read by nothing before this."""
s = Settings()
assert limit_for(s, "repo.create", "platinum") == s.rate_repo_creates_per_day * 10
assert limit_for(s, "repo.create", "gold") == s.rate_repo_creates_per_day * 4
assert limit_for(s, "repo.create", "standard") == s.rate_repo_creates_per_day
assert limit_for(s, "repo.create", "watch") == s.rate_repo_creates_per_day // 2
def test_unknown_band_gets_standard_not_unlimited_and_not_zero():
s = Settings()
assert limit_for(s, "repo.create", "a-band-invented-tomorrow") == s.rate_repo_creates_per_day
assert limit_for(s, "repo.create", None) == s.rate_repo_creates_per_day
def test_untrusted_band_is_read_only():
assert BAND_MULTIPLIER["untrusted"] == 0
def test_every_throttled_action_has_a_configured_base():
s = Settings()
for action, field in ACTION_BASE.items():
assert getattr(s, field) > 0, f"{action} has no positive base rate"
# ---- revocation enforced on the live trust path (not just the webhook) ----
def test_revoked_passport_is_refused_by_trust_decision():
"""A revoked passport returns HTTP 200, status=revoked, band=unproven,
allowed=[] (verified live 2026-08-13). The decision must refuse it — the
old code returned band 'unproven' and seated the agent."""
from api.app.auth import PassportNotInGoodStanding, decide_trust
revoked = {"status": "revoked", "band": "unproven", "allowed_actions": []}
with pytest.raises(PassportNotInGoodStanding):
decide_trust(revoked, "ET26-NJQT-QMR0")
def test_active_passport_is_accepted_by_trust_decision():
from api.app.auth import decide_trust
active = {"status": "active", "band": "gold", "allowed_actions": ["read", "send"]}
band, actions = decide_trust(active, "ET26-1EF9-VJAN")
assert band == "gold" and actions == ("read", "send")
def test_unknown_or_missing_status_fails_closed():
from api.app.auth import PassportNotInGoodStanding, decide_trust
for body in ({"band": "gold"}, {"status": "suspended"}, {"status": "frozen"},
{"status": ""}, {}):
with pytest.raises(PassportNotInGoodStanding):
decide_trust(body, "ET26-X")
@pytest.mark.asyncio
async def test_resolve_passport_raises_on_revoked_status_wiring(monkeypatch):
"""Proves the WIRING, not just the decision: resolve_passport must feed the
real trust body through decide_trust and propagate the refusal. A validly
minted EPT can outlive the passport by ~a year, so this is the path that
stops a revoked-but-still-signed token."""
from api.app import auth
from api.app.auth import PassportNotInGoodStanding, resolve_passport
from api.app.config import Settings
class _Resp:
status_code = 200
def json(self):
return {"status": "revoked", "band": "unproven", "allowed_actions": []}
class _Client:
def __init__(self, *a, **k):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *a):
return False
async def get(self, *a, **k):
return _Resp()
monkeypatch.setattr(auth.httpx, "AsyncClient", _Client)
settings = Settings(eternitas_platform_api_key="x", eternitas_base_url="https://api.eternitas.ai")
with pytest.raises(PassportNotInGoodStanding):
await resolve_passport(settings, "ET26-NJQT-QMR0")
@pytest.mark.asyncio
async def test_resolve_passport_returns_band_on_active_wiring(monkeypatch):
import httpx # noqa: F401
from api.app import auth
from api.app.auth import resolve_passport
from api.app.config import Settings
class _Resp:
status_code = 200
def json(self):
return {"status": "active", "band": "gold", "allowed_actions": ["read"]}
class _Client:
def __init__(self, *a, **k): pass
async def __aenter__(self): return self
async def __aexit__(self, *a): return False
async def get(self, *a, **k): return _Resp()
monkeypatch.setattr(auth.httpx, "AsyncClient", _Client)
settings = Settings(eternitas_platform_api_key="x")
band, actions = await resolve_passport(settings, "ET26-1EF9-VJAN")
assert band == "gold" and actions == ("read",)
def test_routing_does_not_depend_on_signature_wellformedness():
"""An EPT-shaped token must route to the EPT verifier even when its
signature is malformed. jwt.get_unverified_header() validates the whole
token and rejects bad signature padding, which used to push such tokens to
the human path — refused for the wrong reason, and readable as a human
identity via `sub` whenever require_verified_jwt was off."""
import base64
import json as _j
from api.app.ept import looks_like_ept
def seg(d):
return base64.urlsafe_b64encode(_j.dumps(d).encode()).rstrip(b"=").decode()
for hdr in ({"alg": "none", "typ": "EPT"}, {"alg": "ES256", "typ": "EPT"},
{"alg": "ES256"}):
tok = f"{seg(hdr)}.{seg({'sub': 'ET26-X'})}.x" # deliberately bad signature
assert looks_like_ept(tok), f"{hdr} did not route to the EPT verifier"
assert not looks_like_ept("not-a-token")
assert not looks_like_ept("")
def test_only_actions_that_route_through_this_api_are_claimed_enforced():
"""git push never touches this API, so a push limit here would count zero
forever and allow everything — a silent no-op wearing the costume of a
control. Push limits must stay in NOT_ENFORCED_HERE until a Gitea-side hook
reports pushes into agent_actions."""
from api.app.throttle import ACTION_BASE, NOT_ENFORCED_HERE
assert "push" not in ACTION_BASE
assert "push.force" not in ACTION_BASE
assert "push" in NOT_ENFORCED_HERE
assert not set(ACTION_BASE) & set(NOT_ENFORCED_HERE)
@pytest.mark.asyncio
async def test_enforce_refuses_an_action_it_cannot_actually_limit():
"""Asking to throttle 'push' must raise, not silently allow."""
from api.app.auth import ActorType, Caller
from api.app.config import Settings
from api.app.errors import RepairPointer
from api.app.throttle import enforce
caller = Caller(actor_type=ActorType.agent, passport="ET26-X", band="gold")
with pytest.raises(RepairPointer) as exc:
await enforce(None, Settings(), caller, "push")
assert exc.value.code == "throttle_unknown_action"

View File

@@ -0,0 +1,77 @@
"""guards_report: job attribution and the Grant-owned split."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
import yaml
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "scripts"))
_spec = importlib.util.spec_from_file_location("guards_report", ROOT / "scripts" / "guards_report.py")
gr = importlib.util.module_from_spec(_spec)
sys.modules["guards_report"] = gr
_spec.loader.exec_module(gr)
OWNED = yaml.safe_load((ROOT / "ci" / "grant-owned.yml").read_text())["grant_owned"]
WF = """name: CI
on:
push:
jobs:
reality-check:
runs-on: x
steps:
- run: npm install jsdom
test-backend:
runs-on: x
steps:
- run: pip install pytest
"""
def test_job_of_attributes_lines_to_their_job():
assert gr.job_of(WF, 3) is None # `on:` block, not a job
assert gr.job_of(WF, 8) == "reality-check"
assert gr.job_of(WF, 12) == "test-backend"
def test_windy_pro_desktop_jobs_and_paths_are_grant_owned():
ci = ".github/workflows/ci.yml"
assert gr.grant_owned("windy-pro", ci, "reality-check", OWNED)
assert gr.grant_owned("windy-pro", ci, "build-electron", OWNED)
assert not gr.grant_owned("windy-pro", ci, "test-backend", OWNED) # server side: 8c
assert gr.grant_owned("windy-pro", ".github/workflows/release-mac.yml", None, OWNED)
assert gr.grant_owned("windy-pro", "src/client/desktop/main.js", None, OWNED)
assert not gr.grant_owned("windy-pro", "services/account-server/Dockerfile", None, OWNED)
assert not gr.grant_owned("windy-chat", "src/client/desktop/main.js", None, OWNED)
def test_render_splits_lane_and_grant_counts():
F = gr.cg.Finding
res = {"windy-pro": {"sha": "a" * 40, "compute": [],
"hygiene": [(F("ci.yml", 8, "floating install", "npm install"), "reality-check", True),
(F("ci.yml", 12, "floating install", "pip x"), "test-backend", False)]},
"windy-git": {"sha": "b" * 40, "compute": [], "hygiene": []}}
md = gr.render(res)
assert "| ci-hygiene (house rule 6) | 1 | 1 | ❌ not yet |" in md
assert "| compute-guard (Mind is the only door) | 0 | 0 | ✅ YES |" in md
assert "| windy-git | Windy Git | bbbbbbb | 0 | 0 | 0 | clean ✅ |" in md
assert "| windy-pro | Windy Hub | aaaaaaa |" in md # owner = session to message
assert "(job reality-check)" in md
def test_windy_pro_root_env_example_is_grant_owned_but_not_the_account_servers():
assert gr.grant_owned("windy-pro", ".env.example", None, OWNED)
assert not gr.grant_owned("windy-pro", "account-server/.env.example", None, OWNED)
def test_split_grant_sends_desktop_code_to_grant(monkeypatch):
F = gr.cg.Finding
fs = [F("src/client/desktop/main.js", 3, "provider host", "x"),
F("account-server/src/llm.ts", 5, "provider host", "y")]
lane, grant = gr.split_grant("windy-pro", "a" * 40, fs)
assert [f.path for f in grant] == ["src/client/desktop/main.js"]
assert [f.path for f in lane] == ["account-server/src/llm.ts"]
assert gr.split_grant("windy-chat", "a" * 40, fs) == (fs, [])

169
api/tests/test_hub_jwt.py Normal file
View File

@@ -0,0 +1,169 @@
"""G3.2 / I-8 — human tokens are verified, never read (SSO #14, 2026-09-23).
Behavioral: every case signs a real RS256 token with a locally generated key
and drives `get_caller`, so a green run means the gate refuses what it must —
not that some string appears in auth.py.
"""
from __future__ import annotations
import base64
import hashlib
import hmac
import json
import time
import jwt
import pytest
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from api.app import hub_jwt
from api.app.config import Settings
from api.app.errors import RepairPointer
KEY = rsa.generate_private_key(public_exponent=65537, key_size=2048)
OTHER = rsa.generate_private_key(public_exponent=65537, key_size=2048)
IDENTITY = "5e1b9569-7f01-489d-bf14-6fe5a367fa3f"
def _claims(**over):
now = int(time.time())
c = {
"iss": "windy-identity",
"type": "human",
"sub": "row-id-not-identity",
"windy_identity_id": IDENTITY,
"email": "grant@example.com",
"iat": now,
"exp": now + 900,
}
c.update(over)
return {k: v for k, v in c.items() if v is not None}
def _sign(claims, key=KEY, alg="RS256"):
return jwt.encode(claims, key, algorithm=alg, headers={"kid": "test"})
class _Req:
def __init__(self, settings):
self.app = type("A", (), {"state": type("S", (), {"settings": settings})()})()
@pytest.fixture(autouse=True)
def _local_jwks(monkeypatch):
"""The hub's JWKS, served from KEY's public half — no network."""
class _Key:
key = KEY.public_key()
class _Client:
def get_signing_key_from_jwt(self, token):
return _Key()
monkeypatch.setattr(hub_jwt, "_client", lambda base_url: _Client())
async def _caller(token, **settings):
from api.app.auth import get_caller
s = Settings(environment="production", **settings)
return await get_caller(_Req(s), authorization=f"Bearer {token}", x_service_token=None)
async def _refused(token, **settings):
with pytest.raises(RepairPointer) as exc:
await _caller(token, **settings)
assert exc.value.status_code == 401 and exc.value.code == "token_invalid"
return exc.value
@pytest.mark.asyncio
async def test_genuine_hub_token_is_a_human_named_by_windy_identity_id():
c = await _caller(_sign(_claims()))
assert c.actor_type == "human"
assert c.identity_id == IDENTITY # NOT `sub`, which is the per-row user id
@pytest.mark.asyncio
async def test_forged_signature_is_refused():
await _refused(_sign(_claims(), key=OTHER))
@pytest.mark.asyncio
async def test_expired_token_is_refused():
await _refused(_sign(_claims(iat=int(time.time()) - 2000, exp=int(time.time()) - 60)))
@pytest.mark.asyncio
async def test_wrong_issuer_and_id_tokens_are_refused():
await _refused(_sign(_claims(iss="https://evil.example")))
# Contract v1: the discovery-URL issuer is legal for ACCESS tokens...
c = await _caller(_sign(_claims(iss="https://account.windyword.ai")))
assert c.identity_id == IDENTITY
# ...but an id_token minted for the forge (aud = Gitea's client id
# "windy-git", no type, sub = identity) must never act as a bearer here.
id_token = _claims(
iss="https://account.windyword.ai",
aud="windy-git",
type=None,
windy_identity_id=None,
sub=IDENTITY,
)
await _refused(_sign(id_token))
await _refused(_sign(dict(id_token, windy_identity_id=IDENTITY, type="human")))
@pytest.mark.asyncio
async def test_hs256_confusion_is_refused():
# The classic forgery: HMAC the token with the PUBLIC key as the secret.
pub = KEY.public_key().public_bytes(
serialization.Encoding.PEM, serialization.PublicFormat.SubjectPublicKeyInfo
)
header = base64.urlsafe_b64encode(json.dumps({"alg": "HS256", "typ": "JWT"}).encode()).rstrip(
b"="
)
body = base64.urlsafe_b64encode(json.dumps(_claims()).encode()).rstrip(b"=")
sig = base64.urlsafe_b64encode(
hmac.new(pub, header + b"." + body, hashlib.sha256).digest()
).rstrip(b"=")
await _refused((header + b"." + body + b"." + sig).decode())
@pytest.mark.asyncio
async def test_non_human_or_missing_type_is_refused():
await _refused(_sign(_claims(type="agent")))
await _refused(_sign(_claims(type=None)))
@pytest.mark.asyncio
async def test_missing_windy_identity_is_refused_not_read_from_sub():
await _refused(_sign(_claims(windy_identity_id=None)))
@pytest.mark.asyncio
async def test_aud_is_tolerated_when_it_names_windy_git_and_refused_otherwise():
"""PyJWT rejects ANY aud-bearing token when no audience is configured — the
trap that would break the day the hub starts emitting aud."""
c = await _caller(_sign(_claims(aud=["windy_chat", "windy_git", "windy_mail"])))
assert c.identity_id == IDENTITY
await _refused(_sign(_claims(aud=["windy_chat"])))
@pytest.mark.asyncio
async def test_require_aud_refuses_tokens_without_it():
await _refused(_sign(_claims()), hub_require_aud=True)
c = await _caller(_sign(_claims(aud=["windy_git"])), hub_require_aud=True)
assert c.identity_id == IDENTITY
@pytest.mark.asyncio
async def test_production_verifies_even_if_the_flag_is_off():
"""require_verified_jwt=False is a local-dev convenience; production must
never take the unverified path."""
await _refused(_sign(_claims(), key=OTHER), require_verified_jwt=False)
def test_algorithm_list_is_exactly_rs256():
assert hub_jwt.ALGORITHMS == ["RS256"]

View File

@@ -10,12 +10,16 @@ happened somewhere in this ecosystem and cost real time.
from __future__ import annotations
import base64 as _b64
import json as _json
import re
import subprocess
import sys
import types as _types
from pathlib import Path
import pytest
import pytest as _pytest
ROOT = Path(__file__).resolve().parents[2]
@@ -304,12 +308,13 @@ def test_g36_trust_client_never_soft_allows():
def test_g36_unverified_human_jwt_is_refused_in_production():
"""I-8 applied to ourselves: an unverified JWT is an authentication bypass,
not a shortcut. Until G3.2's JWKS verifier exists, production refuses."""
not a shortcut. G3.2's verifier now exists; behavioral proof that forged,
expired, mis-issued and mis-audienced tokens are refused lives in
test_hub_jwt.py. Here: the gate defaults closed."""
from api.app.config import Settings
assert Settings().require_verified_jwt is True
src = (ROOT / "api" / "app" / "auth.py").read_text()
assert "human_signin_not_ready" in src
assert "windy-git" not in Settings().hub_audiences # Gitea's client_id: id_token confusion
def test_no_auth_bypass_env_var_anywhere():
@@ -420,9 +425,28 @@ def test_i05_jobs_get_a_network_per_job_not_a_shared_bridge():
def test_i05_jobs_cannot_bind_mount_from_the_daemon_host():
cfg = (ROOT / "deploy" / "runner" / "config.yaml").read_text()
assert "valid_volumes: []" in cfg
assert 'docker_host: "-"' in cfg
"""Narrowed 2026-09-23 (orchestrator-approved): a job may bind-mount EXACTLY
one daemon path, windy-pro's non-secret build inputs, and only because dind
itself has that path READ-ONLY. Anything more (a second path, a writable
one, a glob) reopens the host to CI code. Still no docker socket for jobs."""
import re
import yaml
rd = ROOT / "deploy" / "runner"
cfg = yaml.safe_load((rd / "config.yaml").read_text())
allowed = cfg["container"]["valid_volumes"]
assert allowed in ([], ["/ci-inputs/windy-pro"]), f"I-5: jobs may mount nothing else: {allowed}"
assert cfg["container"]["docker_host"] == "-"
if allowed:
compose = yaml.safe_load((rd / "docker-compose.yml").read_text())
binds = [v for v in compose["services"]["dind"]["volumes"] if v.startswith("/")]
assert binds == ["/home/user1-gpu/ci-inputs/windy-pro:/ci-inputs/windy-pro:ro"], (
f"I-5: dind's only host bind must be the ci-inputs path, READ-ONLY: {binds}")
for name, svc in compose["services"].items():
if name != "dind":
for v in svc.get("volumes") or []:
assert not re.match(r"^/home/user1-gpu/ci-inputs", v), f"I-5: {name} mounts ci-inputs"
def test_i05_no_ci_container_can_reach_the_forge_network():
@@ -488,6 +512,11 @@ def test_g73_workflow_pins_a_python_that_satisfies_requires_python():
for wf in ROOT.rglob(".gitea/workflows/*.y*ml"):
text = wf.read_text()
# Only workflows that actually RUN Python need to pin it. A workflow
# that shells out to psql or curl does not, and demanding a pin from
# it is the test being wrong rather than the workflow.
if not _re.search(r"\bpython3?\b|pytest|pip ", text):
continue
# Either a pinned container image or an explicit setup-python version.
pin = _re.search(r"image:\s*python:(\d+)\.(\d+)", text) or _re.search(
r'python-version:\s*"?(\d+)\.(\d+)"?', text
@@ -633,3 +662,112 @@ def test_g09_backup_fails_loudly():
src = (ROOT / "scripts" / "backup.sh").read_text()
assert "COMPLETED WITH FAILURES" in src
assert "refusing to report a backup that did not happen" in src
# --------------------------------------------------------------------------
# SECURITY (behavioral, not string-grep): the agent path must not authenticate
# an unverified token. Regression guard for the 2026-08-13 forged-token bypass.
# --------------------------------------------------------------------------
def _forged_bearer(passport: str) -> str:
def seg(d):
return _b64.urlsafe_b64encode(_json.dumps(d).encode()).rstrip(b"=").decode()
return f"{seg({'alg':'none','typ':'JWT'})}.{seg({'passport':passport})}.not-a-signature"
def _fake_request(settings):
app = _types.SimpleNamespace(state=_types.SimpleNamespace(settings=settings))
return _types.SimpleNamespace(app=app)
@_pytest.mark.asyncio
async def test_security_forged_agent_token_is_refused_in_production():
"""The 2026-08-13 exploit: an alg:none token naming a real passport returned
HTTP 200 as that agent. It must now be refused whichever gate catches it —
an EPT-shaped forgery by signature verification, a JWT-shaped one by the
human gate. What is asserted is REFUSAL, not a particular error code."""
from api.app.auth import get_caller
from api.app.config import Settings
from api.app.errors import RepairPointer
settings = Settings(environment="production", require_verified_jwt=True,
eternitas_platform_api_key="x")
req = _fake_request(settings)
# Both shapes now route to the EPT verifier, because alg:none is never
# valid for ANY caller — so 401 "your token is bad" is the honest answer,
# not 503 "that feature isn't ready".
for typ, expected in (("JWT", "ept_invalid"), ("EPT", "ept_invalid")):
def seg(d):
return _b64.urlsafe_b64encode(_json.dumps(d).encode()).rstrip(b"=").decode()
forged = (f"{seg({'alg':'none','typ':typ})}"
f".{seg({'passport':'ET26-1EF9-VJAN','sub':'ET26-1EF9-VJAN'})}.sig")
with _pytest.raises(RepairPointer) as exc:
await get_caller(req, authorization=f"Bearer {forged}", x_service_token=None)
assert exc.value.status_code in (401, 403, 503), f"{typ} was not refused"
assert exc.value.code == expected, f"{typ} -> {exc.value.code}"
@_pytest.mark.asyncio
async def test_security_no_bearer_is_still_401():
from api.app.auth import get_caller
from api.app.config import Settings
from api.app.errors import RepairPointer
req = _fake_request(Settings(environment="production"))
with _pytest.raises(RepairPointer) as exc:
await get_caller(req, authorization=None, x_service_token=None)
assert exc.value.status_code == 401
def test_i12_build_fails_when_commit_sha_is_empty():
"""The sed+grep pair silently accepted an empty COMMIT_SHA: it replaced ""
with "" and then matched that same empty string, shipping a container that
reported commit_sha: null. That is the exact defect I-12 exists to prevent,
and it happened on 2026-08-14."""
df = (ROOT / "Dockerfile").read_text()
assert 'test -n "${COMMIT_SHA}"' in df
# --------------------------------------------------------------------------
# G2.3 — branding lives in the repo, not only on one host's disk
# --------------------------------------------------------------------------
def test_g23_branding_is_version_controlled():
"""It was applied directly to Veron's disk first, which is the config-drift
trap this project documents: the running system and the repo disagree, and
a rebuild silently reverts to stock Gitea."""
b = ROOT / "deploy" / "branding"
for f in ("apply.sh", "README.md", "templates/home.tmpl",
"templates/custom/header.tmpl"):
assert (b / f).exists(), f"missing {f}"
def test_g23_brand_css_filename_is_versioned():
"""Cloudflare caches /assets/* for 6h and no token here can purge, so a
fixed filename leaves stale bytes live for hours."""
import re as _re
hdr = (ROOT / "deploy" / "branding" / "templates" / "custom" / "header.tmpl").read_text()
m = _re.search(r"theme-windy\.v(\d+)\.css", hdr)
assert m, "brand CSS must carry a version in its FILENAME"
assert (ROOT / "deploy" / "branding" / "public" / "assets" / "css"
/ f"theme-windy.v{m.group(1)}.css").exists()
def test_backup_never_bundles_credential_repos_to_r2():
"""kit-army-config (the lockbox) and the *-soul / anima repos carry
credentials; the R2 bundles are plaintext. Behavioural: run the script's
own exclusion function against the names."""
import subprocess
script = (ROOT / "scripts" / "backup.sh").read_text()
fn = script[script.index('EXCLUDE="'):script.index("cleanup()")]
# A file named like a pattern in cwd must not break the match (glob expansion).
probe = "cd \"$(mktemp -d)\" && touch x-soul && " + fn + (
'for n in kit-army-config anima windy-0-soul kit-0c5-soul herm-0-soul '
'soulsafe windy-chat eternitas; do excluded "$n" && echo "X $n" || echo "- $n"; done'
)
out = subprocess.run(["bash", "-c", probe], capture_output=True, text=True, check=True).stdout
skipped = {ln[2:] for ln in out.splitlines() if ln.startswith("X ")}
assert skipped == {"kit-army-config", "anima", "windy-0-soul", "kit-0c5-soul", "herm-0-soul"}

View File

@@ -0,0 +1,64 @@
"""lockbox-names: headings + labels + resolvable, NEVER a value or prose after a label."""
from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
V1 = "Qm7xT2vLp9RkZw4HnB8dYc3S" # synthetic values
V2 = "other-fake-value-1234567890"
V3 = "dup-one-aaaaaaaaaaaaaaaa"
V4 = "dup-two-bbbbbbbbbbbbbbbb"
PROSE = "ZZPROSEZZ-not-for-printing"
def make(tmp_path):
r = tmp_path / "kit"
(r / "secrets" / "x").mkdir(parents=True)
(r / "ACCESS_LOCKBOX.md").write_text(
"# LOCKBOX\n\n## 🔷 AZURE signing (added 10-01)\n"
f"- **Tenant:** {PROSE} lives in the portal\n"
f"- **`AZURE_CLIENT_ID`**: `{V1}`\n"
f"- **Secret (AZURE_CLIENT_SECRET):** `{V2}`\n"
"## GOOGLE oauth\n"
f"GOOGLE_OAUTH_CLIENT_ID={V2}\n"
f"- **`DUP_KEY`**: `{V3}`\n- **`DUP_KEY`**: `{V4}`\n"
f"## stray\n**{V1}** is a heading-like bold that is secret shaped? no, just label\n")
(r / "secrets" / "x" / "a.env").write_text(f"FILE_KEY={V1}\n")
subprocess.run(["git", "init", "-q", "-b", "main"], cwd=r, check=True)
return r
def run(r, *args):
e = {**os.environ, "LOCKBOX_REPO": str(r), "LOCKBOX_REF": "WORKTREE"}
p = subprocess.run([sys.executable, str(ROOT / "scripts" / "lockbox_names.py"), *args],
capture_output=True, text=True, env=e)
return p.returncode, p.stdout + p.stderr
def test_lists_labels_and_resolvable_without_values_or_prose(tmp_path):
r = make(tmp_path)
rc, out = run(r, "AZURE|GOOGLE|FILE|DUP")
assert rc == 0
assert "AZURE signing (added 10-01) | AZURE_CLIENT_ID | md | yes" in out
assert "| GOOGLE_OAUTH_CLIENT_ID | env | yes" in out
assert "| FILE_KEY | file | yes" in out
assert "| DUP_KEY | md | dup" in out
# a prose label is listed but never resolvable, and nothing after the label leaks
assert "| Tenant: " not in out or "| Tenant" in out
assert "| label | no" in out
for secret in (V1, V2, V3, V4, PROSE, "lives in the portal"):
assert secret not in out
for i in range(0, len(secret) - 7):
assert secret[i:i + 8] not in out
def test_filter_and_bad_regex(tmp_path):
r = make(tmp_path)
rc, out = run(r, "NOSUCHTHING")
assert rc == 0 and "0 entries" in out
rc, out = run(r, "(")
assert rc == 2 and "bad regex" in out

View File

@@ -0,0 +1,77 @@
"""lockbox-put against a LOCAL fake lockbox repo only (never the real one)."""
from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
FAKE = "Zk3vQ8mT1pLw7Xn2Rb5Hd9Yc" # synthetic
def sh(*a, cwd=None):
return subprocess.run(a, cwd=cwd, check=True, capture_output=True, text=True)
def make_remote(tmp_path):
work = tmp_path / "seed"
work.mkdir()
sh("git", "init", "-q", "-b", "main", cwd=work)
(work / "ACCESS_LOCKBOX.md").write_text("# LOCKBOX\n\n- **`EXISTING_KEY`**: `abcdefgh12345678`\nOLD_ENV_KEY=whatever123456\n")
sh("git", "add", "-A", cwd=work)
sh("git", "-c", "user.name=t", "-c", "user.email=t@t", "commit", "-qm", "seed", cwd=work)
bare = tmp_path / "remote.git"
sh("git", "clone", "-q", "--bare", str(work), str(bare))
return bare
def put(tmp_path, bare, key, content, mode=0o600):
f = tmp_path / "val"
f.write_text(content)
os.chmod(f, mode)
e = {**os.environ, "LOCKBOX_PUT_REPO": str(bare), "LOCKBOX_PUT_NO_PR": "1", "HOME": str(tmp_path / "h")}
(tmp_path / "h" / ".cache").mkdir(parents=True, exist_ok=True)
r = subprocess.run([sys.executable, str(ROOT / "scripts" / "lockbox_put.py"), key, str(f), "--lane", "test", "--note", "n"],
capture_output=True, text=True, env=e)
return r.returncode, r.stdout + r.stderr
def test_appends_one_key_by_branch_and_never_echoes_value(tmp_path):
bare = make_remote(tmp_path)
rc, out = put(tmp_path, bare, "NEW_TEST_KEY", FAKE)
assert rc == 0 and "pushed branch lockbox-put/new_test_key-" in out
assert FAKE not in out and FAKE[:6] not in out
br = [b.strip() for b in sh("git", "branch", "--list", "lockbox-put/*", cwd=bare).stdout.splitlines()]
assert len(br) == 1
diff = sh("git", "diff", "--numstat", f"main..{br[0]}", cwd=bare).stdout.split()
assert diff[1] == "0" and diff[2] == "ACCESS_LOCKBOX.md" # additions only
content = sh("git", "show", f"{br[0]}:ACCESS_LOCKBOX.md", cwd=bare).stdout
assert f"- **`NEW_TEST_KEY`**: `{FAKE}`" in content and "EXISTING_KEY" in content
# main is untouched
assert FAKE not in sh("git", "show", "main:ACCESS_LOCKBOX.md", cwd=bare).stdout
def test_refuses_existing_key_both_formats_and_bad_input(tmp_path):
bare = make_remote(tmp_path)
for k in ("EXISTING_KEY", "OLD_ENV_KEY"):
rc, out = put(tmp_path, bare, k, FAKE)
assert rc == 3 and "already exists" in out and FAKE not in out
assert put(tmp_path, bare, "lower_case", FAKE)[0] == 2
assert put(tmp_path, bare, "OK_KEY_1", FAKE, mode=0o644)[0] == 2 # not 0600
assert put(tmp_path, bare, "OK_KEY_2", "has space `tick`")[0] == 2 # unsafe value
assert not sh("git", "branch", "--list", "lockbox-put/*", cwd=bare).stdout.strip() # nothing pushed
def test_symlink_refused(tmp_path):
bare = make_remote(tmp_path)
real = tmp_path / "real"
real.write_text(FAKE)
os.chmod(real, 0o600)
link = tmp_path / "link"
link.symlink_to(real)
e = {**os.environ, "LOCKBOX_PUT_REPO": str(bare), "LOCKBOX_PUT_NO_PR": "1"}
r = subprocess.run([sys.executable, str(ROOT / "scripts" / "lockbox_put.py"), "SYM_KEY", str(link)],
capture_output=True, text=True, env=e)
assert r.returncode == 2 and FAKE not in r.stdout + r.stderr

View File

@@ -0,0 +1,463 @@
"""Behavioral tests for scripts/pr_status_bridge.py.
The bridge is the ONLY CI signal the private platform repos get on GitHub, so
these drive its real functions against fake Gitea/GitHub APIs rather than
grepping its source: a status painted green that nobody tested is worse than
no status at all.
"""
from __future__ import annotations
import base64
import importlib.util
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[2]
_spec = importlib.util.spec_from_file_location(
"pr_status_bridge", ROOT / "scripts" / "pr_status_bridge.py"
)
bridge = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(bridge)
SHA = "a" * 40
def _run(i, wf, job, status, sha=SHA, n=1):
return {
"id": i,
"workflow_id": wf,
"name": job,
"status": status,
"head_sha": sha,
"run_number": n,
}
class Fake:
def __init__(self, runs=(), statuses=(), gh_prs=(), wg_prs=(), workflows=None):
self.runs, self.statuses = list(runs), list(statuses)
self.workflows = workflows or {} # {path: yaml text} at every commit
self.gh_prs, self.wg_prs = list(gh_prs), list(wg_prs)
self.posted, self.opened, self.closed = [], [], []
def gitea(self, method, path, body=None):
if "/contents/" in path:
want = path.split("/contents/", 1)[1].split("?", 1)[0]
if want in self.workflows:
return 200, {"content": base64.b64encode(self.workflows[want].encode()).decode()}
files = [
{"type": "file", "name": k.rsplit("/", 1)[1], "path": k}
for k in self.workflows
if k.rsplit("/", 1)[0] == want
]
return (200, files) if files else (404, None)
if "/actions/tasks" in path:
page = int(path.rsplit("page=", 1)[1])
return 200, {"workflow_runs": self.runs[(page - 1) * 50 : page * 50]}
if method == "GET" and path.endswith("/pulls?state=open&limit=50"):
return 200, self.wg_prs
if method == "POST" and path.endswith("/pulls"):
self.opened.append(body)
return 201, {}
if method == "PATCH":
self.closed.append(path)
return 201, {}
raise AssertionError(path)
def github(self, method, path, body=None):
if "/statuses" in path and method == "GET":
return 200, self.statuses
if "/statuses/" in path and method == "POST":
self.posted.append(body)
return 201, {}
if "/pulls?" in path:
return 200, self.gh_prs
raise AssertionError(path)
@pytest.fixture
def fake(monkeypatch):
def make(queued=(), **kw):
f = Fake(**kw)
monkeypatch.setattr(bridge, "gitea", f.gitea)
monkeypatch.setattr(bridge, "github", f.github)
monkeypatch.setattr(bridge, "queued_jobs", lambda repo, sha: list(queued))
return f
return make
def test_posts_latest_verdict_per_job(fake):
f = fake(runs=[_run(1, "ci.yml", "test", "failure"), _run(2, "ci.yml", "test", "success", n=2)])
bridge.post_statuses("r", SHA)
assert [(p["context"], p["state"]) for p in f.posted] == [("windy-git/ci/test", "success")]
assert f.posted[0]["target_url"].endswith("/actions/runs/2")
def test_unchanged_state_is_not_reposted(fake):
f = fake(
runs=[_run(1, "ci.yml", "test", "success")],
statuses=[{"context": "windy-git/ci/test", "state": "success"}],
)
bridge.post_statuses("r", SHA)
assert f.posted == []
def test_skipped_job_is_never_painted_green(fake):
f = fake(runs=[_run(1, "substrate-drift.yml", "check", "skipped")])
bridge.post_statuses("r", SHA)
assert f.posted == []
def test_other_commits_runs_are_ignored(fake):
f = fake(runs=[_run(1, "ci.yml", "test", "failure", sha="b" * 40)])
bridge.post_statuses("r", SHA)
assert f.posted == []
def test_runs_past_the_first_page_are_seen(fake):
noise = [_run(100 + i, "drift.yml", "x", "skipped", sha="c" * 40) for i in range(50)]
f = fake(runs=noise + [_run(1, "ci.yml", "test", "success")])
bridge.post_statuses("r", SHA)
assert [p["state"] for p in f.posted] == ["success"]
def _gh_pr(n, repo="sneakyfree/r"):
return {
"number": n,
"title": "t",
"html_url": "u",
"head": {"ref": f"b{n}", "sha": SHA, "repo": {"full_name": repo} if repo else None},
"base": {"ref": "main"},
}
def test_fork_prs_are_never_mirrored(fake, monkeypatch):
monkeypatch.setattr(bridge, "GH_OWNER", "sneakyfree")
f = fake(gh_prs=[_gh_pr(1, repo="stranger/r"), _gh_pr(2, repo=None)])
assert bridge.sync_prs("r") == []
assert f.opened == []
def test_pr_mirror_opened_once_and_closed_when_github_closes(fake, monkeypatch):
monkeypatch.setattr(bridge, "GH_OWNER", "sneakyfree")
f = fake(gh_prs=[_gh_pr(7)], wg_prs=[{"number": 3, "title": "[GH#5] gone"}])
assert bridge.sync_prs("r") == [SHA]
assert [o["head"] for o in f.opened] == ["b7"]
assert f.closed == ["/repos/windyadmin/r/pulls/3"]
f2 = fake(gh_prs=[_gh_pr(7)], wg_prs=[{"number": 4, "title": "[GH#7] t"}])
bridge.sync_prs("r")
assert f2.opened == [] and f2.closed == []
def test_image_build_jobs_are_not_posted(fake):
"""No Docker daemon in job containers (I-5): a build job's red is structural."""
f = fake(
runs=[_run(1, "ci.yml", "Docker Build", "failure"), _run(2, "ci.yml", "docker", "failure")]
)
bridge.post_statuses("r", SHA)
assert f.posted == []
def test_non_blocking_jobs_are_not_posted_for_that_repo_only(fake, monkeypatch):
"""Grant ruled windy-pro's desktop/installer jobs non-blocking: they must not
reach GitHub for windy-pro, and the rule must not leak to other repos."""
monkeypatch.setattr(bridge, "NON_BLOCKING", {"windy-pro": {"ci/build-desktop"}})
runs = [_run(1, "ci.yml", "build-desktop", "failure"), _run(2, "ci.yml", "test", "success")]
f = fake(runs=runs)
bridge.post_statuses("windy-pro", SHA)
assert [p["context"] for p in f.posted] == ["windy-git/ci/test"]
f2 = fake(runs=runs)
bridge.post_statuses("windy-chat", SHA)
assert sorted(p["context"] for p in f2.posted) == [
"windy-git/ci/build-desktop",
"windy-git/ci/test",
]
def test_default_non_blocking_is_grants_ruling():
assert bridge.NON_BLOCKING.get("windy-pro") == {
"ci/build-desktop",
"ci/test-installer",
"ci/reality-check",
}
def test_transport_blips_are_retried_but_http_errors_are_not(monkeypatch):
import urllib.error
calls = {"n": 0}
class _R:
status = 200
def read(self):
return b"{}"
def __enter__(self):
return self
def __exit__(self, *a):
return False
def flaky(req, timeout):
calls["n"] += 1
if calls["n"] < 3:
raise urllib.error.URLError("_ssl.c:983: The handshake operation timed out")
return _R()
monkeypatch.setattr(bridge.urllib.request, "urlopen", flaky)
monkeypatch.setattr(bridge.time, "sleep", lambda s: None)
assert bridge._call("http://x", "t", "GET", "/p") == (200, {})
assert calls["n"] == 3
def forbidden(req, timeout):
calls["n"] += 1
raise urllib.error.HTTPError("http://x/p", 403, "no", {}, None)
calls["n"] = 0
monkeypatch.setattr(bridge.urllib.request, "urlopen", forbidden)
assert bridge._call("http://x", "t", "GET", "/p") == (403, None)
assert calls["n"] == 1
GOOD = "on: push\njobs:\n test:\n runs-on: ubuntu-latest\n steps: []\n"
BROKEN = "on: push\njobs:\n test:\n runs-on: x\n steps: [\n"
def test_invalid_workflow_gets_an_error_status_even_with_no_runs(fake):
# Gitea fires NO run for an invalid file: without this the PR shows nothing.
f = fake(workflows={".github/workflows/ci.yml": BROKEN})
bridge.post_statuses("windy-chat", SHA)
assert [(p["context"], p["state"]) for p in f.posted] == [("windy-git/ci/workflow", "error")]
assert "invalid YAML at line 5" in f.posted[0]["description"]
assert f.posted[0]["target_url"].endswith(f"/src/commit/{SHA}/.github/workflows/ci.yml")
def test_valid_workflows_post_nothing_extra(fake):
f = fake(runs=[_run(1, "ci.yml", "test", "success")], workflows={".github/workflows/ci.yml": GOOD})
bridge.post_statuses("windy-chat", SHA)
assert [p["context"] for p in f.posted] == ["windy-git/ci/test"]
def test_workflow_error_is_not_reposted(fake):
f = fake(
workflows={".github/workflows/ci.yml": BROKEN},
statuses=[{"context": "windy-git/ci/workflow", "state": "error"}],
)
bridge.post_statuses("windy-chat", SHA)
assert f.posted == []
def test_gitea_dir_wins_over_github_dir(fake):
# Gitea runs .gitea/workflows when it has files and ignores .github/workflows.
f = fake(workflows={".gitea/workflows/ci.yml": GOOD, ".github/workflows/old.yml": BROKEN})
bridge.post_statuses("windy-chat", SHA)
assert f.posted == []
@pytest.mark.parametrize(
"text, problem",
[
(GOOD, None),
("on: push\njobs:\n a:\n uses: ./x.yml\n", None),
(BROKEN, "invalid YAML at line 5"),
("jobs:\n a:\n runs-on: x\n", "no `on:` trigger"),
("on: push\n", "no `jobs:`"),
("on: push\njobs:\n a:\n steps: []\n", "job `a` has no `runs-on:`"),
("- a\n", "not a YAML mapping"),
],
)
def test_workflow_problem(text, problem):
assert bridge.workflow_problem(text) == problem
def _q(n, wf, job):
return {"run_number": n, "workflow_id": wf, "name": job}
def test_queued_job_shows_pending_instead_of_nothing(fake):
f = fake(queued=[_q(5, "ci.yml", "test")])
bridge.post_statuses("windy-chat", SHA)
assert [(p["context"], p["state"]) for p in f.posted] == [("windy-git/ci/test", "pending")]
assert f.posted[0]["target_url"].endswith("/actions/runs/5")
def test_queued_rerun_supersedes_the_stale_failure(fake):
f = fake(runs=[_run(1, "ci.yml", "test", "failure", n=4)], queued=[_q(7, "ci.yml", "test")])
bridge.post_statuses("windy-chat", SHA)
assert [(p["context"], p["state"]) for p in f.posted] == [("windy-git/ci/test", "pending")]
def test_older_queued_job_never_overrides_a_newer_verdict(fake):
f = fake(runs=[_run(1, "ci.yml", "test", "success", n=9)], queued=[_q(3, "ci.yml", "test")])
bridge.post_statuses("windy-chat", SHA)
assert [(p["context"], p["state"]) for p in f.posted] == [("windy-git/ci/test", "success")]
def test_queued_docker_and_non_blocking_jobs_stay_unposted(fake):
f = fake(queued=[_q(2, "ci.yml", "docker-build"), _q(2, "ci.yml", "build-desktop")])
bridge.post_statuses("windy-pro", SHA)
assert f.posted == []
def test_queued_lookup_refuses_unsafe_input():
assert bridge.queued_jobs("x'; drop table t;--", SHA) == []
assert bridge.queued_jobs("windy-chat", "not-a-sha") == []
def _db(monkeypatch, jobs, labels):
import json as _json
import subprocess as _sp
payload = _json.dumps({"jobs": jobs, "labels": [_json.dumps(x) for x in labels]})
monkeypatch.setattr(
bridge.subprocess, "run",
lambda *a, **k: _sp.CompletedProcess(a, 0, stdout=payload, stderr=""),
)
RUNNER = ["veron-1", "linux-x64", "self-hosted", "linux", "x64"]
def test_only_jobs_a_runner_can_take_are_pending(monkeypatch):
# macos-latest is cancelled unpicked by the janitor: pending would never resolve.
_db(monkeypatch, [
{"run_number": 3, "workflow_id": "ci.yml", "name": "test", "runs_on": '["self-hosted","linux","x64"]'},
{"run_number": 3, "workflow_id": "ci.yml", "name": "mac", "runs_on": '["macos-latest"]'},
], [RUNNER])
assert [j["name"] for j in bridge.queued_jobs("windy-chat", SHA)] == ["test"]
def test_lookup_failure_is_non_fatal(monkeypatch):
import subprocess as _sp
def boom(*a, **k):
raise _sp.TimeoutExpired("docker", 30)
monkeypatch.setattr(bridge.subprocess, "run", boom)
assert bridge.queued_jobs("windy-chat", SHA) == []
class _Guard:
def __init__(self, findings):
self.findings = findings
def check(self, repo, sha, default_branch, is_default_head):
return self.findings
@staticmethod
def status_for(findings, whole_tree, grant=()):
if not findings and grant:
return "success", f"GRANT-WARN {len(grant)}", grant[0]
if not findings:
return "success", "OK: clean", None
return "failure", f"BLOCK {len(findings)}", findings[0]
class _F:
path, line = "app/llm.py", 7
def test_guard_posts_warn_with_a_link_to_the_first_finding(fake, monkeypatch):
f = fake()
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
bridge.post_compute_guard("windy-chat", SHA, "main", False)
assert [(p["context"], p["state"], p["description"]) for p in f.posted] == [
("windy-git/compute-guard", "failure", "BLOCK 1")]
assert f.posted[0]["target_url"].endswith(f"/src/commit/{SHA}/app/llm.py#L7")
def test_guard_same_status_is_not_reposted(fake, monkeypatch):
f = fake(statuses=[{"context": "windy-git/compute-guard", "state": "failure", "description": "BLOCK 1"}])
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
bridge.post_compute_guard("windy-chat", SHA, "main", False)
assert f.posted == []
def test_guard_that_cannot_run_posts_nothing(fake, monkeypatch):
f = fake()
monkeypatch.setitem(sys.modules, "compute_guard", _Guard(None))
bridge.post_compute_guard("windy-chat", SHA, "main", True)
assert f.posted == []
def test_ci_hygiene_posts_under_its_own_context(fake, monkeypatch):
f = fake(statuses=[{"context": "windy-git/compute-guard", "state": "failure", "description": "BLOCK 1"}])
monkeypatch.setitem(sys.modules, "ci_hygiene", _Guard([_F()]))
bridge.post_ci_hygiene("windy-chat", SHA, "main", True)
# the compute-guard status with the same description must not suppress it
assert [(p["context"], p["description"]) for p in f.posted] == [("windy-git/ci-hygiene", "BLOCK 1")]
def test_retargeted_pr_gets_a_fresh_mirror_on_the_new_base(fake):
# eternitas #167: stacked on fix/one-hallway, retargeted to main on GitHub.
gh = [{"number": 167, "title": "feat", "html_url": "u",
"head": {"ref": "feat/x", "sha": SHA, "repo": {"full_name": f"{bridge.GH_OWNER}/eternitas"}},
"base": {"ref": "main"}}]
wg = [{"number": 9, "title": "[GH#167] feat", "base": {"ref": "fix/one-hallway"}}]
f = fake(gh_prs=gh, wg_prs=wg)
assert bridge.sync_prs("eternitas") == [SHA]
assert f.closed == [f"/repos/{bridge.WG_OWNER}/eternitas/pulls/9"]
assert [(o["base"], o["head"]) for o in f.opened] == [("main", "feat/x")]
def test_unchanged_base_leaves_the_mirror_alone(fake):
gh = [{"number": 5, "title": "t", "html_url": "u",
"head": {"ref": "b", "sha": SHA, "repo": {"full_name": f"{bridge.GH_OWNER}/windy-chat"}},
"base": {"ref": "main"}}]
f = fake(gh_prs=gh, wg_prs=[{"number": 3, "title": "[GH#5] t", "base": {"ref": "main"}}])
bridge.sync_prs("windy-chat")
assert f.closed == [] and f.opened == []
def test_named_no_daemon_job_is_not_posted_for_that_repo_only(fake, monkeypatch):
"""eternitas ci/build needs Docker but its name doesn't say so (option A, 09-23)."""
monkeypatch.setattr(bridge, "NO_DAEMON_NAMED", {"eternitas": {"ci/build"}})
runs = [_run(1, "ci.yml", "build", "failure"), _run(2, "ci.yml", "test", "success")]
f = fake(runs=runs)
bridge.post_statuses("eternitas", SHA)
assert [p["context"] for p in f.posted] == ["windy-git/ci/test"]
f2 = fake(runs=runs)
bridge.post_statuses("windy-chat", SHA)
assert sorted(p["context"] for p in f2.posted) == ["windy-git/ci/build", "windy-git/ci/test"]
def test_default_no_daemon_named_is_empty():
"""eternitas converted its ci/build to a no-Docker ci/smoke (#179); nothing left."""
assert bridge.NO_DAEMON_NAMED == {}
def test_grant_owned_findings_never_block(fake, monkeypatch):
"""Orchestrator 09-23: compute-guard blocks lane-owned code only."""
f = fake()
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
monkeypatch.setitem(sys.modules, "guards_report",
type("GR", (), {"split_grant": staticmethod(lambda r, s, fs: ([], list(fs)))}))
bridge.post_compute_guard("windy-pro", SHA, "main", True)
assert [(p["state"], p["description"]) for p in f.posted] == [("success", "GRANT-WARN 1")]
def test_failed_grant_split_warns_instead_of_blocking(fake, monkeypatch):
def boom(*a):
raise RuntimeError("no bare clone")
f = fake()
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
monkeypatch.setitem(sys.modules, "guards_report", type("GR", (), {"split_grant": staticmethod(boom)}))
bridge.post_compute_guard("windy-pro", SHA, "main", True)
assert [p["state"] for p in f.posted] == ["success"]
@pytest.mark.parametrize("name, hidden", [
("Docker Build", True), ("docker-build", True), ("Docker build", True), ("docker", True),
("Boot smoke (no Docker)", False), ("smoke (no-docker)", False), ("boot without Docker", False),
("smoke", False),
])
def test_no_docker_smoke_jobs_are_posted(name, hidden):
"""windy-search #96: its replacement job says "no Docker" and was hidden."""
assert bridge.needs_daemon("windy-search", "ci", name) is hidden

View File

@@ -0,0 +1,82 @@
"""Push-velocity detection (scripts/telemetry_emit.py): detect + alert only.
Driven through the real function with rows shaped like the Gitea query's.
"""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "scripts"))
_spec = importlib.util.spec_from_file_location("telemetry_emit", ROOT / "scripts" / "telemetry_emit.py")
te = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(te)
NOW = 1_800_000_000.0
def row(login="agent-et26abcd1234", uid=7, p1h=0, p24h=0, d24h=0, repos=1, wid=None):
return {"uid": uid, "login": login, "wid": wid, "p1h": p1h, "p24h": p24h, "d24h": d24h, "repos": repos}
def test_under_every_threshold_emits_nothing():
ev, keep = te.push_velocity_events([row(p1h=60, p24h=500, d24h=10)], NOW, {})
assert ev == [] and keep == {}
def test_burst_emits_one_declared_row_with_the_passport():
ev, keep = te.push_velocity_events([row(p1h=61, p24h=61, repos=3)], NOW, {})
assert len(ev) == 1
e = ev[0]
assert e["event_type"] == "forge.push_velocity" and e["service"] == "forge"
assert e["actor_type"] == "agent" and e["actor_id"] == "ET26-ABCD-1234"
assert e["metadata"] == {
"rule": "pushes_1h", "window_s": 3600, "count": 61, "threshold": 60,
"repos": 3, "gitea_user_id": 7,
}
assert keep == {"7:pushes_1h": NOW}
def test_still_over_is_reported_once_per_window_not_every_run():
_, keep = te.push_velocity_events([row(p1h=90)], NOW, {})
ev, keep = te.push_velocity_events([row(p1h=95)], NOW + 300, keep)
assert ev == [] and keep == {"7:pushes_1h": NOW}
ev, _ = te.push_velocity_events([row(p1h=95)], NOW + 3601, keep)
assert len(ev) == 1
def test_dropping_back_under_rearms():
_, keep = te.push_velocity_events([row(p1h=90)], NOW, {})
_, keep = te.push_velocity_events([row(p1h=5)], NOW + 300, keep)
assert keep == {}
ev, _ = te.push_velocity_events([row(p1h=90)], NOW + 600, keep)
assert len(ev) == 1
def test_the_sync_account_is_exempt():
ev, _ = te.push_velocity_events([row(login="windyadmin", uid=1, p1h=9999, p24h=9999)], NOW, {})
assert ev == []
def test_sso_human_is_keyed_on_windy_identity_id():
ev, _ = te.push_velocity_events([row(login="u-5e1b9569abc", wid="5e1b9569-full-id", d24h=11)], NOW, {})
assert [(e["actor_type"], e["actor_id"], e["metadata"]["rule"]) for e in ev] == [
("human", "5e1b9569-full-id", "ref_deletes_24h")
]
assert "caller" not in ev[0]["metadata"]
def test_no_provable_id_is_system_plus_caller_never_an_invented_id():
# UPDATE 2 actor rule: agent/human rows without an actor_id are quarantined.
for login in ("u-nolink", "agent-weird"):
ev, _ = te.push_velocity_events([row(login=login, p24h=501)], NOW, {})
assert ev[0]["actor_type"] == "system" and "actor_id" not in ev[0]
assert ev[0]["metadata"]["caller"] == "unknown"
def test_passport_round_trip():
assert te.passport_from_login("agent-et26p1zgttp8") == "ET26-P1ZG-TTP8"
assert te.passport_from_login("u-abc") is None

View File

@@ -0,0 +1,70 @@
"""runner-guard: workflow shapes that hand a self-hosted runner to strangers."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
_spec = importlib.util.spec_from_file_location("runner_guard", ROOT / "scripts" / "runner_guard.py")
rg = importlib.util.module_from_spec(_spec)
sys.modules["runner_guard"] = rg
_spec.loader.exec_module(rg)
def rules(text):
return [(r, ln) for _p, ln, r, _m in rg.lint_text("w.yml", text)]
def test_pull_request_target_always_fails():
assert rules("on: pull_request_target\njobs:\n a:\n runs-on: ubuntu-latest\n steps: []\n")[0][0] == "R1"
def test_fork_pr_on_self_hosted_fails_and_points_at_runs_on():
wf = "on:\n pull_request:\njobs:\n t:\n runs-on: [self-hosted, linux, x64]\n steps: []\n"
assert rules(wf) == [("R2", 5)]
def test_same_repo_gate_or_environment_passes():
gated = ("on: [pull_request]\njobs:\n t:\n if: github.event.pull_request.head.repo.full_name == github.repository\n"
" runs-on: [self-hosted]\n steps: []\n")
env = "on: [pull_request]\njobs:\n t:\n environment: ci\n runs-on: self-hosted\n steps: []\n"
assert rules(gated) == [] and rules(env) == []
def test_outsider_events_on_self_hosted_fail_but_writer_events_pass():
wf = "on:\n issue_comment:\n workflow_run:\n workflows: [x]\njobs:\n t:\n runs-on: self-hosted\n steps: []\n"
assert sorted(r for r, _ in rules(wf)) == ["R3", "R3"]
ok = "on:\n push:\n tags: ['v*']\n workflow_dispatch:\n schedule:\n - cron: '0 3 * * *'\njobs:\n t:\n runs-on: self-hosted\n steps: []\n"
assert rules(ok) == []
def test_expression_runs_on_is_treated_as_self_hosted_and_hosted_runner_is_fine():
expr = "on: pull_request\njobs:\n t:\n runs-on: ${{ matrix.os }}\n steps: []\n"
hosted = "on: pull_request\njobs:\n t:\n runs-on: ubuntu-latest\n steps: []\n"
assert rules(expr) == [("R2", 4)] and rules(hosted) == []
def test_broken_yaml_is_a_finding_and_non_workflows_are_ignored():
assert rules("on: [push\njobs: {")[0][0] == "R0"
assert rules("name: just a file\n") == []
def test_pr_mode_posts_each_status_once(monkeypatch, tmp_path):
calls = []
monkeypatch.setattr(rg, "STATE", str(tmp_path / "s.json"))
monkeypatch.setattr(rg, "public_repos", lambda owners: [("o/r", "main")])
monkeypatch.setattr(rg, "file_at", lambda *a: "on: push\\njobs:\\n t:\\n runs-on: self-hosted\\n steps: []\\n")
def fake_gh(*args, check=True):
if args[0].startswith("repos/o/r/pulls?"):
return "7 abc123 o/r\\n"
if args[0].endswith("/files?per_page=100"):
return ".github/workflows/ci.yml\\n"
calls.append(args[0])
return ""
monkeypatch.setattr(rg, "gh", fake_gh)
rg.cmd_pr(["o"], post=True)
rg.cmd_pr(["o"], post=True)
assert calls == ["repos/o/r/statuses/abc123"]

View File

@@ -0,0 +1,156 @@
"""Secret guard: shapes, hash-only findings, allow by hash."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "scripts"))
_spec = importlib.util.spec_from_file_location("secret_guard", ROOT / "scripts" / "secret_guard.py")
sg = importlib.util.module_from_spec(_spec)
sys.modules["secret_guard"] = sg
_spec.loader.exec_module(sg)
ss = sg.ss
# Synthetic shapes only: none of these is a real credential.
TG = "1234567890:" + "A" * 35
CASES = [
("telegram bot token", f"TELEGRAM_BOT_TOKEN={TG}"),
("github token", "token = 'ghp_" + "a1" * 18 + "'"),
("aws access key", "aws_access_key_id = AKIA" + "ABCDEFGHIJKLMNOP"),
("slack token", "xoxb-" + "1234567890-abcdefghij"),
("anthropic key", "ANTHROPIC_API_KEY=sk-ant-" + "x" * 30),
("openai key", "OPENAI_API_KEY=sk-proj-" + "y" * 40),
("stripe live key", "STRIPE=sk_live_" + "z" * 24),
("google api key", "key=AIza" + "B" * 35),
("private key block", "-----BEGIN OPENSSH PRIVATE KEY-----"),
]
@pytest.mark.parametrize("kind, text", CASES)
def test_each_shape_is_found_and_only_its_hash_is_kept(kind, text):
hits = sg.scan_line("app.py", text)
assert [k for k, _ in hits] == [kind]
match = hits[0][1]
assert match.startswith(f"{kind} #") and len(match.rsplit("#", 1)[1]) == 8
# house rule 10: the value itself must never appear in a finding
secret = text.split("=", 1)[-1].strip(" '")
assert secret not in match
@pytest.mark.parametrize("text", [
"sha512-" + "Q" * 86 + "==", # lockfile integrity
"version: 12345678:abc", # short, not a token
"sk-ant-short", # too short
"re_test_register_sends_verification", # windy-pro's fake Resend key
"ANTHROPIC_API_KEY=", # a name, not a value
])
def test_non_secrets_are_not_flagged(text):
assert sg.scan_line("x", text) == []
def test_anthropic_key_is_not_double_counted_as_openai():
assert [k for k, _ in sg.scan_line("x", "sk-ant-" + "q" * 40)] == ["anthropic key"]
def test_allow_is_by_hash_only():
F = sg.cg.Finding
fake = F("tests/t.py", 3, "telegram bot token", f"telegram bot token #{ss.h8(TG)}")
real = F("tests/t.py", 9, "telegram bot token", "telegram bot token #deadbeef")
allow = {"windy-chat": {"hashes": {ss.h8(TG)}, "paths": []}}
assert sg._drop_allowed("windy-chat", [fake, real], allow) == [real]
assert sg._drop_allowed("windy-mail", [fake], allow) == [fake]
def test_private_key_blocks_are_allowed_by_path_never_by_hash():
F = sg.cg.Finding
hdr = "private key block #" + ss.h8("-----BEGIN PRIVATE KEY-----")
test_key = F("tests/keys/test.pem", 1, "private key block", hdr)
prod_key = F("deploy/prod.pem", 1, "private key block", hdr)
allow = {"r": {"hashes": {hdr.rsplit("#", 1)[1]}, "paths": [("tests/keys/*", {"private key block"})]}}
assert sg._drop_allowed("r", [test_key, prod_key], allow) == [prod_key]
def test_path_allow_cannot_cover_real_token_kinds(tmp_path):
bad = tmp_path / "a.yml"
bad.write_text("allow:\n - repo: r\n paths: [tests/*]\n kinds: [telegram bot token]\n reason: no\n")
with pytest.raises(ValueError):
sg.load_allow(bad)
def test_shipped_allow_file_never_excuses_the_real_leaked_tokens():
a = sg.load_allow()
every = set().union(*(v["hashes"] for v in a.values())) if a else set()
assert not {"1354fc9b", "d49dc2ba"} & every # real (now revoked) credentials: remove, never allow
def test_allow_file_loads_and_needs_reasons(tmp_path):
assert isinstance(sg.load_allow(), dict)
bad = tmp_path / "a.yml"
bad.write_text("allow:\n - repo: r\n hashes: [abcd1234]\n")
with pytest.raises(ValueError):
sg.load_allow(bad)
def test_block_and_warn(monkeypatch):
f = sg.cg.Finding("a.py", 1, "github token", "github token #abcd1234")
monkeypatch.setattr(sg, "MODE", "block")
assert sg.status_for([f], False)[0] == "failure"
assert sg.status_for([], False, grant=[f])[0] == "success"
monkeypatch.setattr(sg, "MODE", "warn")
state, desc, _ = sg.status_for([f], True)
assert state == "success" and desc.startswith("⚠ WARN (not blocking): 1 secret-shaped string in tree")
def test_public_scan_excuses_by_hash_and_by_path():
spec = importlib.util.spec_from_file_location("public_secret_scan", ROOT / "scripts" / "public_secret_scan.py")
ps = importlib.util.module_from_spec(spec)
spec.loader.exec_module(ps)
allow = {"windy-agent": {"hashes": {"aaaa1111"}, "paths": [("tests/keys/*", {"private key block"})]}}
assert ps.excused("windy-agent", "openai key", "aaaa1111", ["tests/x.py"], allow)
assert not ps.excused("windy-agent", "telegram bot token", "1354fc9b", ["tests/test_log_redaction.py"], allow)
assert ps.excused("windy-agent", "private key block", "ffff0000", ["tests/keys/a.pem"], allow)
# a PEM header anywhere outside the allowed paths still counts
assert not ps.excused("windy-agent", "private key block", "ffff0000", ["tests/keys/a.pem", "deploy/k.pem"], allow)
assert not ps.excused("other", "openai key", "aaaa1111", ["x"], allow)
HEX32 = "0123456789abcdef" * 2 # synthetic
def test_twilio_shapes_hash_only_and_no_md5_noise():
kinds = lambda t: [k for k, _ in ss.find(t)] # noqa: E731
assert kinds(f'TWILIO_AUTH_TOKEN = "{HEX32}"') == ["32-hex secret assignment"]
assert kinds(f"auth_token: {HEX32}") == ["32-hex secret assignment"]
assert kinds("AC" + HEX32) == ["twilio sid/api key"]
assert kinds("SK" + HEX32) == ["twilio sid/api key"]
# plain md5 / uuid-without-dashes / a 64-hex sha256 are NOT secrets by shape
assert kinds(f"md5 = {HEX32}") == []
assert kinds(f"checksum_key = {HEX32}{HEX32}") == []
assert kinds(f"name = 'x{HEX32}'") == []
# the hash is of the value alone, so renaming the variable keeps the same allow hash
a = ss.find(f"A_TOKEN={HEX32}")[0][1]
b = ss.find(f"OTHER_SECRET: '{HEX32}'")[0][1]
assert a == b == ss.h8(HEX32)
assert HEX32 not in repr(ss.find(f"A_TOKEN={HEX32}"))
def test_warn_kinds_do_not_block(monkeypatch):
f = sg.cg.Finding("a.py", 1, "32-hex secret assignment", "32-hex secret assignment #abcd1234")
monkeypatch.setattr(sg, "MODE", "block")
monkeypatch.setattr(sg, "WARN_KINDS", {"32-hex secret assignment"})
assert sg.status_for([f], True)[0] == "success"
monkeypatch.setattr(sg, "WARN_KINDS", set())
assert sg.status_for([f], True)[0] == "failure"
def test_pypi_token_shape_hash_only():
tok = "pypi-AgE" + "Ab1_-" * 20 # synthetic
got = ss.find(f"password = {tok}")
assert [k for k, _ in got] == ["pypi token"] and got[0][1] == ss.h8(tok)
assert tok not in repr(got)
assert ss.find("pypi-AgE-too-short") == []

View File

@@ -0,0 +1,106 @@
"""secret-scan + env-names: findings carry label/location/hash, NEVER a value or fragment."""
from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
SCRIPTS = ROOT / "scripts"
# Synthetic, random-looking, never real. FAKE_LB is in the fake lockbox; TWI matches a shape.
FAKE_LB = "k7Xv2QpLm9RtZw4HnB8dYc3S"
TWI = "9f3a7c1e5b2d48806a1f4e7d2c9b0835"
def run(script, *args, env=None):
e = {**os.environ, **(env or {})}
r = subprocess.run([sys.executable, str(SCRIPTS / script), *args], capture_output=True, text=True, env=e)
return r.returncode, r.stdout + r.stderr
def no_fragment(out: str, value: str, n: int = 6):
assert value not in out
for i in range(len(value) - n + 1):
assert value[i:i + n] not in out, f"fragment of the value leaked at {i}"
def seeded(tmp_path):
lb = tmp_path / "lockbox.md"
lb.write_text(f"FAKE_VENDOR_API_KEY={FAKE_LB}\nNOTE: nothing here\n")
repo = tmp_path / "repo"
repo.mkdir()
g = lambda *a: subprocess.run(["git", "-C", str(repo), *a], check=True, capture_output=True) # noqa: E731
g("init", "-q", "-b", "main")
g("config", "user.email", "t@t")
g("config", "user.name", "t")
(repo / "app.py").write_text(f'KEY = "{FAKE_LB}"\nTWILIO_AUTH_TOKEN = "{TWI}"\n')
g("add", "-A")
g("commit", "-qm", "add secrets")
(repo / "app.py").write_text("KEY = None\n") # removed from HEAD, still in history
g("commit", "-qam", "remove")
return lb, repo
def test_tree_scan_labels_locations_no_value(tmp_path):
lb, repo = seeded(tmp_path)
(repo / "live.py").write_text(f'x = "{FAKE_LB}"\n')
rc, out = run("secret_scan.py", str(repo / "live.py"), env={"SECRET_SCAN_LOCKBOX_PATHS": str(lb)})
assert rc == 1
assert "live.py:1" in out and "lockbox:FAKE_VENDOR_API_KEY" in out
no_fragment(out, FAKE_LB)
def test_history_finds_removed_secret_with_commit(tmp_path):
lb, repo = seeded(tmp_path)
rc, out = run("secret_scan.py", str(repo), "--history", env={"SECRET_SCAN_LOCKBOX_PATHS": str(lb)})
assert rc == 1
assert "app.py:1" in out and "commit=" in out and "lockbox:FAKE_VENDOR_API_KEY" in out
assert "app.py:2" in out and "32-hex secret assignment" in out
no_fragment(out, FAKE_LB)
no_fragment(out, TWI)
def test_repo_flag_clones_scans_and_cleans_up(tmp_path):
lb, repo = seeded(tmp_path)
cache = tmp_path / "home"
(cache / ".cache").mkdir(parents=True)
rc, out = run("secret_scan.py", "--repo", str(repo),
env={"SECRET_SCAN_LOCKBOX_PATHS": str(lb), "HOME": str(cache)})
assert rc == 1 and "app.py:1" in out
no_fragment(out, FAKE_LB)
assert not [p for p in (cache / ".cache").iterdir() if p.name.startswith("secret-scan-")]
def test_clean_tree_and_bad_input(tmp_path):
(tmp_path / "ok.txt").write_text("hello world\n")
rc, out = run("secret_scan.py", str(tmp_path / "ok.txt"), "--no-lockbox")
assert rc == 0 and "0 finding(s)" in out
rc, out = run("secret_scan.py", str(tmp_path), "--history", "--no-lockbox")
assert rc == 2 and "needs a git repo" in out
def test_env_names_never_prints_values(tmp_path):
a = tmp_path / "a.env"
b = tmp_path / "b.env"
a.write_text(f"# c\nexport DB_PASSWORD={FAKE_LB}\nTOKEN=\"{TWI}\"\nEMPTY=\nONLY_A=1\n")
b.write_text(f"DB_PASSWORD={FAKE_LB}\nTOKEN=different-value-here\nONLY_B=2\n")
rc, out = run("env_names.py", str(a), "--hash")
assert rc == 0 and "DB_PASSWORD" in out and "set" in out and "empty" in out and "len=24" in out
assert "sha256:" in out
no_fragment(out, FAKE_LB)
no_fragment(out, TWI, 7)
rc, out = run("env_names.py", str(a), "--compare", str(b))
assert "DB_PASSWORD" in out and "SAME" in out and "DIFFERENT" in out
assert "only-in-A" in out and "only-in-B" in out
no_fragment(out, FAKE_LB)
rc, out = run("env_names.py", str(tmp_path / "missing.env"))
assert rc == 2
def test_shapes_are_the_guards_shapes():
sys.path.insert(0, str(SCRIPTS))
import secret_scan as sc
import secret_shapes as ss
assert sc.ss is ss # one source of shapes: a new guard shape is automatically a scan shape

202
api/tests/test_telemetry.py Normal file
View File

@@ -0,0 +1,202 @@
"""Field telemetry from the API (step 2): behavioural, no network.
A refusal must become exactly one forge.auth.failed row in the DECLARED shape
(the ledger quarantines anything else); non-refusal errors must not; the
heartbeat must count what happened and never invent a p95 for no traffic.
"""
from __future__ import annotations
import httpx
import pytest
from fastapi import Depends, FastAPI
from api.app import telemetry as tmod
from api.app.errors import RepairPointer
DECLARED_AUTH_KEYS = {"code", "http_status", "caller", "route", "upstream_status", "synthetic"}
def _app(tel: tmod.Telemetry) -> FastAPI:
"""The real middleware + handler, re-registered on a bare app (no DB)."""
from api.app import main
app = FastAPI()
app.state.telemetry = tel
app.middleware("http")(main._count_requests)
app.exception_handler(RepairPointer)(main._repair_pointer_handler)
def refuse(code: str, status: int):
def dep():
raise RepairPointer(
status_code=status,
code=code,
speak="no",
machine_cause="test",
remediation_tool=None,
)
return dep
@app.get("/api/v1/repos/{repo}/grants", dependencies=[Depends(refuse("passport_revoked", 403))])
async def grants(repo: str):
return {}
@app.get("/api/v1/nope", dependencies=[Depends(refuse("repo_not_found", 404))])
async def nope():
return {}
@app.get("/ok")
async def ok():
return {"ok": True}
return app
async def _get(app, path, headers=None):
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app), base_url="http://t") as c:
return await c.get(path, headers=headers or {})
def _tel():
return tmod.Telemetry(
"http://ledger.invalid/v1/events",
"tok",
environment="test",
commit_sha="abc1234",
version="0.1.0",
)
@pytest.mark.asyncio
async def test_refusal_emits_one_declared_row_with_route_template_not_path():
tel = _tel()
r = await _get(
_app(tel),
"/api/v1/repos/grandmas-secret-project/grants",
{"Authorization": "Bearer eyJhbGciOiJFUzI1NiIsInR5cCI6IkVQVCJ9.e30.x"},
)
assert r.status_code == 403
rows = [e for e in tel.buffer if e["event_type"] == "forge.auth.failed"]
assert len(rows) == 1
row = rows[0]
assert row["actor_type"] == "system" and "actor_id" not in row
assert set(row["metadata"]) <= DECLARED_AUTH_KEYS
assert row["metadata"]["code"] == "passport_revoked"
assert row["metadata"]["http_status"] == 403
assert row["metadata"]["caller"] == "anonymous_agent"
assert row["metadata"]["route"] == "/api/v1/repos/{repo}/grants"
assert "grandmas-secret-project" not in str(row)
@pytest.mark.asyncio
async def test_non_auth_errors_are_counted_but_not_refusal_rows():
tel = _tel()
await _get(_app(tel), "/api/v1/nope")
assert not [e for e in tel.buffer if e["event_type"] == "forge.auth.failed"]
assert tel.errors_4xx == 1 and tel.refusals_4xx == 0
@pytest.mark.asyncio
async def test_heartbeat_counts_requests_refusals_and_p95():
tel = _tel()
app = _app(tel)
for _ in range(3):
await _get(app, "/ok")
await _get(app, "/api/v1/repos/x/grants")
meta = tel.health_row()
assert meta["requests"] == 4 and meta["refusals_4xx"] == 1 and meta["errors_5xx"] == 0
assert isinstance(meta["p95_ms"], int)
assert {"interval_s", "uptime_s"} <= set(meta)
def test_no_traffic_means_no_p95_not_a_fake_zero():
assert "p95_ms" not in _tel().health_row()
def test_no_token_sends_and_buffers_nothing():
tel = tmod.Telemetry("http://ledger.invalid", "")
tel.boot()
tel.auth_failed(code="token_invalid", http_status=401, caller="unknown")
assert tel.buffer == []
def test_unknown_code_is_never_sent_as_a_refusal():
tel = _tel()
tel.auth_failed(code="made_up_code", http_status=401, caller="unknown")
assert tel.buffer == []
def test_boot_omits_an_unknown_commit_rather_than_inventing_one():
tel = tmod.Telemetry("http://x", "tok", commit_sha=None, version="0.1.0")
tel.boot()
assert "commit_sha" not in tel.buffer[0]["metadata"]
def test_caller_classes_are_the_declared_three():
assert tmod.caller_class({}) == "unknown"
assert tmod.caller_class({"x-service-token": "s"}) == "unknown"
assert (
tmod.caller_class({"authorization": "Bearer eyJhbGciOiJSUzI1NiJ9.e30.x"})
== "anonymous_human"
)
@pytest.mark.asyncio
async def test_synthetic_header_marks_the_row_and_absent_means_real():
async def refusal(headers):
tel = _tel()
await _get(_app(tel), "/api/v1/repos/x/grants", headers)
return [e for e in tel.buffer if e["event_type"] == "forge.auth.failed"][0]["metadata"]["synthetic"]
assert await refusal({"X-Windy-Synthetic": "1"}) is True
assert await refusal({}) is False
def test_synthetic_is_forwarded_downstream_only_for_synthetic_requests():
token = tmod.SYNTHETIC.set(True)
try:
assert tmod.synthetic_headers() == {"X-Windy-Synthetic": "1"}
finally:
tmod.SYNTHETIC.reset(token)
assert tmod.synthetic_headers() == {}
# ---- UPDATE 7: the ledger answers 202 even when it quarantines rows ----------
@pytest.mark.asyncio
async def test_quarantined_rows_are_warned_and_counted_on_the_next_heartbeat(monkeypatch, caplog):
tel = _tel()
tel.boot()
monkeypatch.setattr(
tel, "_post", lambda b: (202, {"accepted": 0, "quarantined": 1, "rejections": ["undeclared key"]})
)
with caplog.at_level("WARNING", logger="windy-git.telemetry"):
await tel.flush()
assert tel.buffer == [] # sent; the ledger dead-lettered it, retrying won't help
assert "QUARANTINED" in caplog.text and "undeclared key" in caplog.text
tel.health()
assert tel.buffer[-1]["metadata"]["telemetry_quarantined"] == 1
assert tel.health_row()["telemetry_quarantined"] == 0 # reset per heartbeat window
@pytest.mark.asyncio
async def test_clean_send_reports_zero_and_logs_nothing(monkeypatch, caplog):
tel = _tel()
tel.boot()
monkeypatch.setattr(tel, "_post", lambda b: (202, {"accepted": 1, "quarantined": 0, "rejections": []}))
with caplog.at_level("WARNING", logger="windy-git.telemetry"):
await tel.flush()
assert caplog.text == ""
row = tel.health_row()
assert row["telemetry_quarantined"] == 0 and row["telemetry_dropped"] == 0
def test_buffer_overflow_is_counted_as_dropped(monkeypatch):
monkeypatch.setattr(tmod, "MAX_BUFFER", 3)
tel = _tel()
for _ in range(5):
tel.boot()
assert len(tel.buffer) == 3
assert tel.health_row()["telemetry_dropped"] == 2

View File

@@ -0,0 +1,106 @@
"""G3.5 — the Eternitas webhook receiver, driven over HTTP (audit 2026-08-13).
The G3.5 invariants in test_invariants.py grep webhooks.py for strings; a
refactor that kept the strings and broke the behaviour would pass them all.
These send real requests through the real route (no DB: every case here stops
before the revocation handler) and assert what the receiver DOES.
"""
from __future__ import annotations
import hashlib
import hmac
import json
import httpx
import pytest
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from api.app.config import Settings
from api.app.errors import RepairPointer
from api.app.routes import webhooks
SECRET = "s" * 64
URL = "/api/v1/webhooks/eternitas"
def _app(secret: str = SECRET) -> FastAPI:
app = FastAPI()
app.include_router(webhooks.router)
app.state.settings = Settings(eternitas_webhook_secret=secret)
@app.exception_handler(RepairPointer)
async def _h(_, exc: RepairPointer) -> JSONResponse:
return JSONResponse(status_code=exc.status_code, content=exc.detail)
return app
async def _post(body: bytes, headers: dict, secret: str = SECRET) -> httpx.Response:
transport = httpx.ASGITransport(app=_app(secret))
async with httpx.AsyncClient(transport=transport, base_url="http://t") as c:
return await c.post(
URL, content=body, headers={"content-type": "application/json", **headers}
)
def _sig(raw: bytes, secret: str = SECRET) -> str:
return hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest()
# Deliberately odd spacing/key order: a receiver that re-serialises before
# hashing produces a different digest and must fail.
RAW = b'{"event":"windygit.selftest", "data": {"b": 2, "a": 1}}'
EVENT = {"x-eternitas-event": "windygit.selftest"}
@pytest.mark.asyncio
async def test_prefixed_and_bare_digests_are_both_accepted():
for header in (f"sha256={_sig(RAW)}", _sig(RAW)):
r = await _post(RAW, {**EVENT, "x-eternitas-signature": header})
assert r.status_code == 200, r.text
assert r.json()["acted"] is False # unknown event: received, nothing done
@pytest.mark.asyncio
async def test_digest_of_reserialised_json_is_refused():
reserialised = json.dumps(json.loads(RAW)).encode()
assert reserialised != RAW
r = await _post(RAW, {**EVENT, "x-eternitas-signature": f"sha256={_sig(reserialised)}"})
assert r.status_code == 401 and r.json()["code"] == "webhook_signature_invalid"
@pytest.mark.asyncio
async def test_forged_or_wrong_key_signature_is_refused():
for header in ("sha256=" + "0" * 64, f"sha256={_sig(RAW, 'other-secret')}", "garbage"):
r = await _post(RAW, {**EVENT, "x-eternitas-signature": header})
assert r.status_code == 401, header
@pytest.mark.asyncio
async def test_signed_event_without_signature_is_refused():
r = await _post(RAW, EVENT)
assert r.status_code == 401
@pytest.mark.asyncio
async def test_unset_secret_refuses_rather_than_accepts():
r = await _post(RAW, {**EVENT, "x-eternitas-signature": f"sha256={_sig(RAW)}"}, secret="")
assert r.status_code == 503 and r.json()["code"] == "webhook_secret_unset"
@pytest.mark.asyncio
async def test_revocation_with_bad_signature_never_reaches_the_handler():
body = b'{"event":"passport.revoked","passport":"ET26-TEST-GOOD"}'
r = await _post(
body,
{"x-eternitas-event": "passport.revoked", "x-eternitas-signature": "sha256=" + "f" * 64},
)
assert r.status_code == 401 # refused before any DB work
@pytest.mark.asyncio
async def test_reachability_ping_acknowledges_but_never_acts():
r = await _post(b'{"anything": "at all"}', {"x-eternitas-event": "platform.test_ping"})
assert r.status_code == 200 and r.json()["acted"] is False

13
ci/ci-hygiene-allow.yml Normal file
View File

@@ -0,0 +1,13 @@
# CI hygiene allow-list: installs that may float, or services that may publish
# a host port. House rule 6 (09-23): installs come from a lockfile. Every entry
# is an exception and MUST say why. Paths are fnmatch globs from the repo root.
# Owner: Windy Git lane (13); changes go through the orchestrator.
allow:
- repo: windy-pro
paths: [".github/workflows/ci.yml"]
# ONLY the old deploy job's two docker lines. That job is `if: false`
# (CD boundary, 2026-07), and the compose line runs ON windyword.ai inside
# the ssh string. Any other docker step in ci.yml still flags.
matches: ['docker build -f account-server/Dockerfile -t windy-pro:', 'docker compose down && docker compose up -d --build']
reason: "needs-docker false positive: the deploy job is if: false and its compose runs on the remote host over ssh. Added with the needs-docker rule (orchestrator option A, 09-23)."

View File

@@ -0,0 +1,52 @@
# Compute guard allow-list: code that MAY talk to an AI provider directly.
# Windy Mind is the ONLY door to AI compute (Grant, 2026-09-23). Every entry
# here is an exception to that rule and MUST say why. Paths are fnmatch globs
# relative to the repo root. Owner of this file: Windy Git lane (13); changes
# go through the orchestrator. Source of the first entries: COMPUTE_BYPASS_AUDIT.md.
allow:
- repo: windy-mind
paths: ["*"]
reason: "Windy Mind IS the door: provider clients belong here by definition."
- repo: windy-agent
paths: ["*"]
reason: >-
User BYOK: self-hosted agents call providers on the USER's own keys.
Mind stays opt-in there, or every self-hosted user's inference lands on
Grant's bill (no-cloud-cost-liability rule; audit #7).
- repo: windy-code
paths: ["extensions/windy-ai/*"]
reason: "User BYOK AI extension: the user's own provider keys; Mind is one opt-in provider (audit #8)."
- repo: windy-connect
paths: ["*writers/*"]
reason: "Writes client configs that NAME the user's own provider env vars; makes no provider calls (audit #11)."
- repo: windy-pro
paths: ["src/client/desktop/*"]
reason: >-
User BYOK desktop client: cloud STT/translate keys come from what the USER
enters (renderer localStorage -> electron-store; env var only for dev), and
the CSP line allows exactly those user-keyed hosts (audit #10). The
account-server is NOT covered: server-side calls go through Mind.
- repo: windy-pro
paths: ["src/client/web/src/pages/panels/MindPanel.jsx"]
reason: "Validates the USER's own OpenRouter key for BYOK (audit #10); spends no house money."
- repo: windy-pro
paths: ["src/client/web/src/pages/panels/MindKeychain.jsx"]
# ONLY these two endpoints: any other openrouter.ai call in this file (e.g.
# /api/v1/chat, i.e. inference) still flags. Orchestrator-approved 09-23.
matches: ['openrouter\.ai/auth\?', 'openrouter\.ai/api/v1/auth/keys']
reason: "BYOK key acquisition via OpenRouter OAuth PKCE; no inference; successor of MindPanel allow (ADR-064)."
- repo: windy-git
paths: ["scripts/compute_guard.py", "ci/compute-guard-allow.yml"]
reason: "The guard's own pattern list and this file."
- repo: windy-mind
paths: ["*"]
matches: [':11434']
reason: "Windy Mind IS the compute door (endpoint + key); it may call Ollama. Only the Ollama port is allowed here, any provider host/SDK in Mind still flags."

19
ci/grant-owned.yml Normal file
View File

@@ -0,0 +1,19 @@
# Code Grant owns directly (orchestrator, 09-23): guard findings here are listed
# SEPARATELY in the guards status page and never hold up "block". Changes to
# these files are proposals for Grant / Windy Word 44, not a lane's fix.
grant_owned:
- repo: windy-pro
reason: "Windy Word desktop (Electron) + its release/installer builds: Grant's, built from the Mac mini."
paths:
- "src/client/desktop/*"
# ROOT .env.example only (exact path): documents the desktop app's BYOK dev
# fallback keys; the hub reads neither (8c, 09-23). account-server/.env.example
# is NOT matched and stays the hub lane's.
- ".env.example"
- "installer-v2/*"
- ".github/workflows/build-windows.yml"
- ".github/workflows/release-mac.yml"
- ".github/workflows/build-installer.yml"
- ".github/workflows/build-offline-installers.yml"
jobs:
".github/workflows/ci.yml": [reality-check, build-desktop, test-installer, build-electron]

62
ci/secret-guard-allow.yml Normal file
View File

@@ -0,0 +1,62 @@
# Secret guard allow-list: KNOWN FAKE values that look like secrets (test
# fixtures, docs). Allowed BY HASH (sha256[:8] of the value), so a real secret
# in the same file still flags. Private-key blocks (the match is only the BEGIN
# line, same hash everywhere) are allowed by PATH + kind instead.
# Every entry MUST say why. Owner: Windy Git lane (13); changes via the orchestrator.
# Triage 09-24 (values never printed): each hash checked against every version of
# the lockbox; fakes judged by impossible length for the kind (real Anthropic keys
# ~108 chars, OpenAI 51 or 160+), fake-words, or identity with upstream public
# fixtures. NOT allowed, remove instead: 1354fc9b (old @Windy_0_bot token) and
# d49dc2ba (old Anthropic key), both real and revoked, in public windy-agent.
allow:
- repo: windy-code
hashes: [ac9265e5, 46eb1235]
reason: "Upstream microsoft/vscode terminalEnvironment.test.ts fixtures (identical hash upstream; public)."
- repo: windy-code
paths: ["build/azure-pipelines/common/publish.ts"]
kinds: [private key block]
reason: "Upstream VS Code build script (PEM header string in code, not a key)."
- repo: windy-agent
hashes: [a9235a6d, dd6a2baa, 02c362d8, a6f6ff79, f7503b21, d0c94833, 4ab092e3, e3aa1eb8, 833382ee]
reason: "Redaction/sanitizer test fixtures; lengths impossible for real Anthropic/OpenAI keys; never in the lockbox."
- repo: windy-agent
hashes: [89bd408f, b8c94b72, bf23cdf5, d1d85dfe, e3e07f06]
reason: "09-24 #395 replacement fixtures (each contains FAKE; token-SHAPED on purpose so redaction tests prove real tokens are scrubbed); verified by Windy Agent sha-for-sha against every lockbox version: never real. Weekly scan 09-28."
- repo: windy-agent
paths: ["tests/test_agent_keys.py"]
kinds: [private key block]
reason: "Test-generated key material for agent-key tests."
- repo: windy-mind
hashes: [f8a630b2]
reason: "Provider test fixture (27 chars; a real Anthropic key is ~108)."
- repo: windy-pro
hashes: [756de8d8, 7828319d, 1a5d44a2]
reason: ".env.production.example placeholder + crash-summary test fixtures (AWS doc EXAMPLE key shape, short fake Slack token)."
- repo: windy-pro
paths: ["account-server/docs/oauth-providers.md"]
kinds: [private key block]
reason: "Docs show the PEM header format; no key material."
- repo: windytalk
hashes: [baf8656a]
reason: "Diagnostics redaction test fixture (fake-word in value)."
- repo: eternitas
paths: ["tests/golden_vectors/**", "tests/test_soul_vault_key_separation.py"]
kinds: [private key block]
reason: "Test vectors and throwaway keys for signature/vault tests."
- repo: windy-drops
paths: ["tools/conformance/test-keys/*"]
kinds: [private key block]
reason: "Conformance-suite test keys (named test-private.pem)."
- repo: windy-git
paths: ["api/tests/test_secret_guard.py"]
kinds: [private key block]
reason: "The guard's own test uses a PEM header string as a sample."
- repo: windy-code
hashes: ["23f32607"]
reason: "VS Code OSS extensions' package.json aiKey: Microsoft's public telemetry (App Insights) key, shipped in every VS Code build; not a Windy credential."
- repo: windytalk
hashes: ["c4189d79"]
reason: "apps/desktop/test/diagnostics.test.ts redaction fixture (hexSecret beside a fake sk-ant token); hash checked against the lockbox 10-01: not present."
- repo: windy-agent
hashes: ["84e0c0ea"]
reason: "tests/test_log_redaction.py:56 Z.ai redaction fixture REPLACED by windy-agent #412 with a synthetic value; hash checked against the lockbox 10-01: not present."

28
deploy/branding/README.md Normal file
View File

@@ -0,0 +1,28 @@
# Windy Git branding (G2.3)
Gitea's **supported** customisation surface: custom templates and public assets.
No Gitea source is modified, so upstream upgrades keep arriving (D-2, I-1).
deploy/branding/ → $GITEA_CUSTOM (/data/gitea) in the container
→ /srv/windygit/git/gitea/ on Veron 1
Apply with `./deploy/branding/apply.sh`.
## Two traps this cost, both worth knowing
**1. `GITEA__DEFAULT__APP_NAME` does not work.** Gitea reads `APP_NAME` from the
*top level* of `app.ini` (before any `[section]`). The env var instead created a
literal `[default]` section, which Gitea ignores — and the installer's stock
`APP_NAME` kept winning, so the site said "Gitea: Git with a cup of tea" while
the config looked correct. Worse, the env-to-ini pass **appended** a second
`APP_NAME` rather than replacing the first. `apply.sh` sets it at the top level
directly.
**2. Cloudflare caches `/assets/*` for 6 hours and no token in this stack can
purge.** Editing a fixed filename leaves the old bytes live for hours — the new
logo and CSS were both invisible while being correct at origin. **Version the
filename** (`theme-windy.v2.css`) on every brand change; a `?query` is not
enough because some caches ignore it.
The nav logo is swapped in CSS rather than by overriding Gitea's navbar
template — a one-line rule instead of a forked template that would drift.

23
deploy/branding/apply.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Apply Windy Git branding to the Gitea custom tree. Idempotent.
set -euo pipefail
CUSTOM="${GITEA_CUSTOM_HOST:-/srv/windygit/git/gitea}"
HERE="$(cd "$(dirname "$0")" && pwd)"
sudo mkdir -p "$CUSTOM"/templates/custom "$CUSTOM"/public/assets/img "$CUSTOM"/public/assets/css
sudo cp -r "$HERE"/templates/. "$CUSTOM"/templates/
sudo cp -r "$HERE"/public/. "$CUSTOM"/public/
sudo chown -R 1000:1000 "$CUSTOM"/templates "$CUSTOM"/public
# APP_NAME must sit at the TOP LEVEL. See README trap #1.
INI="$CUSTOM/conf/app.ini"
sudo cp "$INI" "$INI.bak-brand-$(date +%s)"
sudo python3 - "$INI" <<'PY'
import sys
p = sys.argv[1]
lines = [l for l in open(p).read().splitlines()
if not l.strip().startswith(("APP_NAME", "APP_SLOGAN"))]
lines.insert(0, "APP_NAME = Windy Git")
open(p, "w").write("\n".join(lines) + "\n")
PY
echo "applied. restart gitea to pick it up."

View File

@@ -0,0 +1,27 @@
/* Windy Git brand accent. Layered on top of Gitea's theme rather than
replacing it, so upstream theme fixes keep arriving. */
:root {
--color-primary: #0ea5e9;
--color-primary-dark-1: #0284c7;
--color-primary-dark-2: #0369a1;
--color-primary-light-1: #38bdf8;
--color-primary-light-2: #7dd3fc;
}
.wg-hero { max-width: 780px; margin: 4rem auto 2rem; padding: 0 1.5rem; text-align: center; }
.wg-hero h1 { font-size: 2.6rem; margin: 1.2rem 0 .4rem; letter-spacing: -.02em; }
.wg-hero .wg-sub { font-size: 1.15rem; opacity: .78; margin-bottom: 2.2rem; }
.wg-grid { display: grid; gap: 1.1rem; grid-template-columns: repeat(auto-fit,minmax(230px,1fr));
max-width: 900px; margin: 0 auto 2.5rem; padding: 0 1.5rem; text-align: left; }
.wg-card { border: 1px solid var(--color-secondary); border-radius: 8px; padding: 1.1rem 1.2rem; }
.wg-card h3 { margin: 0 0 .35rem; font-size: 1.02rem; }
.wg-card p { margin: 0; opacity: .74; font-size: .9rem; line-height: 1.5; }
.wg-cta { margin-bottom: 3rem; }
/* Logo swap via CSS.
Cloudflare cached the stock /assets/img/logo.svg for 6h and no available API
token can purge. Pointing at a NEW filename sidesteps the stale object
without touching Gitea's own templates — the supported customisation surface,
per D-2 (membrane, not merge). */
img[src$="/assets/img/logo.svg"] {
content: url("/assets/img/wg-mark.svg");
}

View File

@@ -0,0 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32" width="32" height="32">
<defs><linearGradient id="w" x1="0" y1="0" x2="1" y2="1">
<stop offset="0%" stop-color="#38bdf8"/><stop offset="100%" stop-color="#0ea5e9"/>
</linearGradient></defs>
<circle cx="16" cy="16" r="15" fill="#0b1220"/>
<path d="M5 11h13a3.2 3.2 0 1 0-3.1-4" fill="none" stroke="url(#w)" stroke-width="2.4" stroke-linecap="round"/>
<path d="M5 16h17a3.6 3.6 0 1 1-3.5 4.5" fill="none" stroke="url(#w)" stroke-width="2.4" stroke-linecap="round"/>
<path d="M5 21h9a2.8 2.8 0 1 1-2.7 3.5" fill="none" stroke="url(#w)" stroke-width="2.4" stroke-linecap="round"/>
</svg>

After

Width:  |  Height:  |  Size: 660 B

View File

@@ -0,0 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32" width="32" height="32">
<defs><linearGradient id="w" x1="0" y1="0" x2="1" y2="1">
<stop offset="0%" stop-color="#38bdf8"/><stop offset="100%" stop-color="#0ea5e9"/>
</linearGradient></defs>
<circle cx="16" cy="16" r="15" fill="#0b1220"/>
<path d="M5 11h13a3.2 3.2 0 1 0-3.1-4" fill="none" stroke="url(#w)" stroke-width="2.4" stroke-linecap="round"/>
<path d="M5 16h17a3.6 3.6 0 1 1-3.5 4.5" fill="none" stroke="url(#w)" stroke-width="2.4" stroke-linecap="round"/>
<path d="M5 21h9a2.8 2.8 0 1 1-2.7 3.5" fill="none" stroke="url(#w)" stroke-width="2.4" stroke-linecap="round"/>
</svg>

After

Width:  |  Height:  |  Size: 660 B

View File

@@ -0,0 +1,5 @@
{{/* Windy Git brand layer.
The filename carries a version: Cloudflare caches /assets/* for 6h with no
purge token available to this stack, so editing a fixed filename leaves the
old bytes live for hours. Bump the suffix on every brand change. */}}
<link rel="stylesheet" href="{{AssetUrlPrefix}}/css/theme-windy.v2.css">

View File

@@ -0,0 +1,28 @@
{{template "base/head" .}}
<div role="main" aria-label="{{ctx.Locale.Tr "home"}}" class="page-content home">
<div class="wg-hero">
<img src="{{AssetUrlPrefix}}/img/logo.svg" alt="Windy Git" width="72" height="72">
<h1>Windy Git</h1>
<p class="wg-sub">Your work, every version — and agents as citizens.</p>
<div class="wg-cta">
{{if not .IsSigned}}
<a class="ui primary button" href="{{AppSubUrl}}/user/login">Sign in with Windy</a>
<p class="wg-note" style="margin-top:.9rem;font-size:.95rem;opacity:.75">Invite only at launch: new accounts are not open yet.</p>
{{else}}
<a class="ui primary button" href="{{AppSubUrl}}/{{.SignedUser.Name}}?tab=repositories">Your repositories</a>
{{end}}
<a class="ui button" href="{{AppSubUrl}}/explore/repos">Explore</a>
</div>
</div>
<div class="wg-grid">
<div class="wg-card"><h3>Code and models, one host</h3>
<p>Git and LFS over Windy Cloud storage. Source, weights and adapters live side by side.</p></div>
<div class="wg-card"><h3>Agents are first-class</h3>
<p>An agent signs in with its own Eternitas passport — not a human's borrowed token — and its work is attributable to it.</p></div>
<div class="wg-card"><h3>Invite only, for now</h3>
<p>Windy Git accounts are by invitation while we launch. If you've been invited, you sign in with your Windy account, with no second password.</p></div>
<div class="wg-card"><h3>Kept, and kept elsewhere</h3>
<p>Every repository is bundled nightly to off-site storage, and the restore is rehearsed rather than assumed.</p></div>
</div>
</div>
{{template "base/footer" .}}

View File

@@ -0,0 +1,61 @@
{{/* Windy Git override of Gitea 1.24.6 templates/user/auth/link_account.tmpl.
Forge registration is CLOSED at launch (Grant, 09-23), so a Windy account
with no forge account used to land on Gitea's raw "Registration is disabled"
error. That case now gets a plain invite-only page; every other case is
Gitea's template unchanged (re-diff it on Gitea upgrades). No change to who
can register. */}}
{{if and .DisableRegistration (not .user_exists)}}
{{template "base/head" .}}
<div role="main" aria-label="Windy Git is invite-only" class="page-content user link-account">
<div class="ui middle very relaxed page grid">
<div class="column tw-my-5">
<div class="tw-flex tw-flex-col tw-gap-4 tw-max-w-2xl tw-m-auto">
<h4 class="ui top attached header center">Windy Git is invite-only while we launch</h4>
<div class="ui attached segment">
<p>You're signed in with your Windy account{{if or .user_name .email}} as <strong>{{or .user_name .email}}</strong>{{end}}. Nothing is wrong with it: Windy Git just isn't open to every account yet.</p>
<p>We'll let you know when it opens.</p>
<a class="ui primary button" href="https://app.windyword.ai/dashboard">Back to your dashboard</a>
</div>
</div>
</div>
</div>
</div>
{{template "base/footer" .}}
{{else}}
{{template "base/head" .}}
<div role="main" aria-label="{{.Title}}" class="page-content user link-account">
<overflow-menu class="ui secondary pointing tabular top attached borderless menu secondary-nav">
<div class="overflow-menu-items tw-justify-center">
<!-- TODO handle .ShowRegistrationButton once other login bugs are fixed -->
{{if not .AllowOnlyInternalRegistration}}
<a class="item {{if not .user_exists}}active{{end}}"
data-tab="auth-link-signup-tab">
{{ctx.Locale.Tr "auth.oauth_signup_tab"}}
</a>
{{end}}
<a class="item {{if .user_exists}}active{{end}}"
data-tab="auth-link-signin-tab">
{{ctx.Locale.Tr "auth.oauth_signin_tab"}}
</a>
</div>
</overflow-menu>
<div class="ui middle very relaxed page grid">
<div class="column tw-my-5">
{{/* these styles are quite tricky but it needs to be the same as the signin page */}}
<div class="ui tab {{if not .user_exists}}active{{end}}" data-tab="auth-link-signup-tab">
<div class="tw-flex tw-flex-col tw-gap-4 tw-max-w-2xl tw-m-auto">
{{if .AutoRegistrationFailedPrompt}}<div class="ui message">{{.AutoRegistrationFailedPrompt}}</div>{{end}}
{{template "user/auth/signup_inner" .}}
</div>
</div>
<div class="ui tab {{if .user_exists}}active{{end}}" data-tab="auth-link-signin-tab">
<div class="tw-flex tw-flex-col tw-gap-4 tw-max-w-2xl tw-m-auto">
{{template "user/auth/signin_inner" .}}
</div>
</div>
</div>
</div>
</div>
{{template "base/footer" .}}
{{end}}

View File

@@ -11,8 +11,9 @@ log:
runner:
file: /data/.runner
capacity: 4 # concurrent jobs; Veron has 24 cores, dind is capped at 12
timeout: 30m
capacity: 1 # per runner; parallelism = number of runner services (6). See docker-compose.yml
timeout: 90m # hard ceiling per job. eternitas's serial pytest is ~50 min; keep
# timeout-minutes in each workflow — a hang still reads as a hang
shutdown_timeout: 3m
insecure: false
fetch_timeout: 5s
@@ -53,6 +54,9 @@ container:
privileged: false
options:
workdir_parent: /workspace
valid_volumes: [] # a job cannot bind-mount anything from the daemon host
# A job may bind-mount exactly ONE daemon path: the read-only windy-pro build
# inputs (mounted :ro into dind itself). Per-runner, not per-repo (act_runner
# limit): any windyadmin repo could mount it; it is non-secret and read-only.
valid_volumes: ["/ci-inputs/windy-pro"]
docker_host: "-" # do NOT expose the runner's own docker socket to jobs
force_pull: false

View File

@@ -0,0 +1,15 @@
# ROLLBACK ONLY: the pre-Sysbox dind (privileged: true), kept one command away.
# Use it if CI breaks under Sysbox:
#
# cd /srv/windygit/src/deploy/runner
# sudo docker compose -f docker-compose.yml -f docker-compose.privileged.yml up -d dind
#
# (then restart the runners while idle). Compose merges `volumes` by container
# path, so this puts back the old `dind-storage` volume with its image cache.
# Going forward again: the same command without the second -f.
services:
dind:
runtime: runc
privileged: true
volumes:
- dind-storage:/var/lib/docker

View File

@@ -26,8 +26,12 @@
# * `dind` and every job container it spawns are UNTRUSTED. They are on a
# private network with no access to the forge, its database, or its .env.
#
# dind itself is privileged — that is the cost, and it is the reason a job
# escape lands in a disposable daemon rather than on Grant's workstation.
# dind is NOT privileged (2026-09-23): it runs under the Sysbox runtime
# (sysbox-ce on Veron, `runtime: sysbox-runc`), a user-namespaced system
# container whose root is an unprivileged host uid. A job that escapes its own
# container lands in dind as a nobody on the host, not as root on Grant's
# workstation. Before Sysbox, dind was `privileged: true`; that config is kept
# as docker-compose.privileged.yml (ROLLBACK ONLY, one command, see that file).
#
# ⚠️ Do NOT "simplify" this by mounting the host docker socket.
@@ -36,21 +40,49 @@ name: windy-git-runner
services:
dind:
image: docker.io/library/docker:27-dind
privileged: true
runtime: sysbox-runc # NOT privileged: see the I-5 note above
environment:
DOCKER_TLS_CERTDIR: "" # plain TCP on an isolated network, no host route
command: ["dockerd", "--host=tcp://0.0.0.0:2375", "--tls=false"]
networks: [jobs]
volumes:
- dind-storage:/var/lib/docker
# A fresh volume: Sysbox shifts ownership to its own uid range. The old
# `dind-storage` is kept untouched for the privileged rollback.
- dind-storage-sysbox:/var/lib/docker
# READ-ONLY, non-secret build inputs for windy-pro's desktop jobs (models,
# linux-x64 portable bundle, enter-monitor build), copied from the frozen
# release clone by deploy/runner/refresh-ci-inputs.sh. Jobs may mount ONLY
# this path (config.yaml valid_volumes). Orchestrator-approved 09-23.
- /home/user1-gpu/ci-inputs/windy-pro:/ci-inputs/windy-pro:ro
# G1.5 — bounded so a fork-bomb workflow cannot starve Grant's interactive
# session. Veron 1 is his workstation, not a dedicated build box.
cpus: 12.0 # 12 of 24 cores
mem_limit: 64g
restart: unless-stopped
runner:
image: docker.io/gitea/act_runner:0.2.11
# ── FOUR runners × capacity 1, not one runner × capacity 4 (2026-09-23) ──
#
# act caches every action repo at /root/.cache/act/<hash> INSIDE the runner
# process and re-fetches it at the start of each job. With capacity 4, four
# concurrent jobs share that one directory: one job's refresh rewrites it while
# another is tarring it into its job container, and the job dies with
# `lstat /root/.cache/act/<hash>/…: no such file or directory` on
# `actions/setup-node` / `setup-uv` — a failure that reads like a broken
# workflow. windy-chat (~20 jobs per push) hit it on 3 jobs in its first run.
# `rm -rf /root/.cache/act` only reset the clock. Separate processes get
# separate caches, so the race cannot occur. Same total parallelism, same
# single capped dind — the blast radius is unchanged.
runner: &runner
# 0.2.11 -> 0.6.1 on 2026-08-14. The bundled act in 0.2.11 only knows
# `runs.using: node12|node16|node20`, so ANY repo pinning a current action
# major dies before its first step with "The runs.using key in action.yml
# must be one of: [...], got node24" — Windy-Clone on actions/checkout@v5
# is how this surfaced. Verified: `node24` is absent from the 0.2.11 binary
# and present in 0.6.1. Every key in this directory's config.yaml still
# exists in 0.6.1's schema (0.6.1 only ADDS keys), so the config carries
# over unchanged. Rollback is re-pinning 0.2.11; the registration in the
# runner-data volume survives either way.
image: docker.io/gitea/act_runner:0.6.1
depends_on: [dind]
environment:
# The runner reaches its OWN daemon. Never the host's.
@@ -90,6 +122,47 @@ services:
mem_limit: 4g
restart: unless-stopped
# Each extra runner registers itself on first start (own name, own volume —
# the registration lives in /data/.runner, so volumes must never be shared).
runner-2:
<<: *runner
environment: &env2
DOCKER_HOST: tcp://dind:2375
GITEA_INSTANCE_URL: https://app.windygit.com
GITEA_RUNNER_REGISTRATION_TOKEN: ${RUNNER_TOKEN:?set RUNNER_TOKEN}
GITEA_RUNNER_NAME: veron-1-2
CONFIG_FILE: /config.yaml
volumes: [./config.yaml:/config.yaml:ro, runner-data-2:/data]
runner-3:
<<: *runner
environment:
<<: *env2
GITEA_RUNNER_NAME: veron-1-3
volumes: [./config.yaml:/config.yaml:ro, runner-data-3:/data]
runner-4:
<<: *runner
environment:
<<: *env2
GITEA_RUNNER_NAME: veron-1-4
volumes: [./config.yaml:/config.yaml:ro, runner-data-4:/data]
# 5 and 6 added the same day: with ~11 private repos onboarded (windy-chat
# alone queues ~24 jobs per push) four runners left 50+ jobs waiting. The
# CPU ceiling is dind's (12 of 24 cores, G1.5), not the runner count, so more
# runners add concurrency for I/O-bound jobs (npm ci, uv sync) without
# taking more of Grant's workstation.
runner-5:
<<: *runner
environment:
<<: *env2
GITEA_RUNNER_NAME: veron-1-5
volumes: [./config.yaml:/config.yaml:ro, runner-data-5:/data]
runner-6:
<<: *runner
environment:
<<: *env2
GITEA_RUNNER_NAME: veron-1-6
volumes: [./config.yaml:/config.yaml:ro, runner-data-6:/data]
networks:
jobs:
# Untrusted job containers live here. No route to the forge.
@@ -97,5 +170,11 @@ networks:
volumes:
dind-storage:
dind-storage-sysbox:
runner-data:
runner-data-2:
runner-data-3:
runner-data-4:
runner-data-5:
runner-data-6:

51
deploy/runner/egress.sh Executable file
View File

@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# CI egress filter (2026-09-23) — jobs reach the internet, never Grant's network.
#
# Measured before this existed: an ordinary (unprivileged) job container inside
# the CI dind could open SSH, Ollama, and every dev server on Veron
# (192.168.1.73:22/3000/3300/8080/11434) and anything else on the LAN, WireGuard
# or Tailscale. No container escape needed — a malicious npm/pip dependency in
# any first-party repo's CI could walk straight onto the fleet.
#
# All CI traffic leaves through the `windy-git-runner_jobs` bridge (dind NATs
# its job containers onto it). This script, run at boot and after any runner
# compose change, allows on that bridge:
# * traffic between the runners and dind (same bridge)
# * replies (ESTABLISHED/RELATED)
# * DNS (53) — Docker's embedded resolver forwards to the LAN router
# * everything public
# and drops: RFC1918, CGNAT/Tailscale (100.64/10), link-local, and ANY packet
# addressed to the host itself (INPUT), whatever interface IP it targets.
# Idempotent: owned chains are flushed and rebuilt; hooks are added once.
set -euo pipefail
NET=windy-git-runner_jobs
id=$(docker network inspect "$NET" --format '{{.Id}}')
BR="br-${id:0:12}"
ip link show "$BR" >/dev/null
iptables -N WG-CI-EGRESS 2>/dev/null || iptables -F WG-CI-EGRESS
iptables -A WG-CI-EGRESS -o "$BR" -j RETURN
iptables -A WG-CI-EGRESS -m conntrack --ctstate ESTABLISHED,RELATED -j RETURN
iptables -A WG-CI-EGRESS -p udp --dport 53 -j RETURN
iptables -A WG-CI-EGRESS -p tcp --dport 53 -j RETURN
for cidr in 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 169.254.0.0/16; do
iptables -A WG-CI-EGRESS -d "$cidr" -j DROP
done
iptables -A WG-CI-EGRESS -j RETURN
iptables -N WG-CI-INPUT 2>/dev/null || iptables -F WG-CI-INPUT
iptables -A WG-CI-INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j RETURN
iptables -A WG-CI-INPUT -j DROP
# Hooks: remove any stale ones (the bridge name changes if the network is
# recreated), then add exactly one of each at the top.
for chain in DOCKER-USER INPUT; do
target=$([ "$chain" = INPUT ] && echo WG-CI-INPUT || echo WG-CI-EGRESS)
while read -r rule; do
iptables -D $chain ${rule#-A $chain }
done < <(iptables -S "$chain" | grep -- "-j $target" || true)
iptables -I "$chain" 1 -i "$BR" -j "$target"
done
echo "ci egress filter active on $BR ($NET)"

28
deploy/runner/prune.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Keep CI storage bounded (2026-09-23).
#
# Kit 0's 09-01 wipe began with CI `_work` dirs (74 GB) + Docker filling the
# disk. Windy Git's runners have no host `_work` dir — every job runs in a
# container inside the CI-only dind — so the thing that grows here is dind's
# image/volume store (38 GB when this was written, never pruned). This prunes
# ONLY that daemon, over its own socket. It never touches the host's Docker.
#
# In-use images/volumes are never removed, so a running job is safe.
set -euo pipefail
CAP_GB="${CI_STORAGE_CAP_GB:-60}"
D=(docker exec windy-git-runner-dind-1 docker -H tcp://127.0.0.1:2375) # dind listens on TCP only
"${D[@]}" container prune -f --filter until=6h >/dev/null
"${D[@]}" volume prune -af >/dev/null # job workspaces of finished jobs
"${D[@]}" image prune -af --filter until=168h >/dev/null
"${D[@]}" builder prune -af --filter until=168h >/dev/null 2>&1 || true
used_gb=$(du -s --block-size=1G /var/lib/docker/volumes/windy-git-runner_dind-storage | cut -f1)
if (( used_gb > CAP_GB )); then
# Over the cap even after the age-based pass: drop every unused image. The
# next jobs re-pull (the act image is ~2 GB) — slower, never wrong.
"${D[@]}" image prune -af >/dev/null
used_gb=$(du -s --block-size=1G /var/lib/docker/volumes/windy-git-runner_dind-storage | cut -f1)
fi
echo "ci storage ${used_gb}G (cap ${CAP_GB}G)"
(( used_gb <= CAP_GB )) || { echo "STILL OVER CAP"; exit 1; }

View File

@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# Refresh the READ-ONLY CI input cache for windy-pro desktop jobs from the FROZEN
# release clone on Veron. Reads ~/windy-pro-release only; never writes to it.
# Run when the release lane says engines / wheels / the portable bundle changed.
# Linux inputs only: 3 models + requirements-bundle.txt, bundled-portable/linux-x64,
# native/enter-monitor/build. Result is chmod a-w and mounted :ro into dind.
set -euo pipefail
SRC=/home/user1-gpu/windy-pro-release
DST=/home/user1-gpu/ci-inputs/windy-pro
models=$(ls "$SRC/extraResources/model" | grep -E '^windy-(nano|lite|core)-ct2$')
[ "$(wc -w <<<"$models")" = 3 ] || { echo "expected 3 models, got: $models"; exit 1; }
mkdir -p "$DST/extraResources/model" "$DST/bundled-portable" "$DST/native-enter-monitor-build"
chmod -R u+w "$DST"
R="ionice -c3 nice -n 19 rsync -a --delete"
for m in $models; do $R "$SRC/extraResources/model/$m/" "$DST/extraResources/model/$m/"; done
$R "$SRC/extraResources/requirements-bundle.txt" "$DST/extraResources/requirements-bundle.txt"
$R "$SRC/bundled-portable/linux-x64/" "$DST/bundled-portable/linux-x64/"
$R "$SRC/native/enter-monitor/build/" "$DST/native-enter-monitor-build/"
date -u +%FT%TZ > "$DST/.refreshed-from-windy-pro-release"
chmod -R a-w "$DST"
du -sh --apparent-size "$DST"

View File

@@ -0,0 +1,13 @@
[Unit]
Description=Windy Git - CI egress filter (jobs reach the internet, never the LAN/host)
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
# The jobs network exists once the runner compose project is up; retry until it does.
ExecStart=/bin/bash -c 'for i in $(seq 1 60); do /srv/windygit/src/deploy/runner/egress.sh && exit 0; sleep 5; done; exit 1'
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,6 @@
[Unit]
Description=Windy Git - prune CI-only dind storage (bounded, never the host daemon)
[Service]
Type=oneshot
ExecStart=/srv/windygit/src/deploy/runner/prune.sh

View File

@@ -0,0 +1,9 @@
[Unit]
Description=Windy Git - prune CI storage every 6 hours
[Timer]
OnCalendar=*-*-* 00/6:37:00
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,13 @@
[Unit]
Description=Windy Git nightly backup (git bundles + windgit schema -> R2)
After=network-online.target docker.service
[Service]
Type=oneshot
WorkingDirectory=/srv/windygit/src
# The .env holds the R2 credentials. The script refuses to run without them
# rather than reporting a backup that did not happen.
EnvironmentFile=/srv/windygit/src/.env
ExecStart=/bin/bash /srv/windygit/src/scripts/backup.sh
Nice=10
IOSchedulingClass=idle

View File

@@ -0,0 +1,3 @@
[Service]
ExecStart=
ExecStart=/usr/local/bin/windy-job windygit-backup 26h --expect "ok — [0-9]+ repos" --owner 13 -- /bin/bash /srv/windygit/src/scripts/backup.sh

View File

@@ -0,0 +1,12 @@
[Unit]
Description=Nightly Windy Git backup
[Timer]
OnCalendar=*-*-* 04:17:00
# Grant's workstation is not always on at 04:17. Without this a missed window
# is simply skipped and the backup silently never runs.
Persistent=true
RandomizedDelaySec=600
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,3 @@
[Service]
ExecStart=
ExecStart=/usr/local/bin/windy-job windygit-ci-prune 7h --expect "ci storage [0-9]+G" --owner 13 -- /srv/windygit/src/deploy/runner/prune.sh

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Windy Git nightly STATE backup (Postgres + Gitea config + repos -> encrypted restic in R2)
After=network-online.target docker.service
[Service]
Type=oneshot
WorkingDirectory=/srv/windygit/src
# R2 credentials come from the .env; the restic password from /etc/windygit/restic.pass
# (root 600; the same value lives in the lockbox as RESTIC_WINDYGIT_PASSWORD).
EnvironmentFile=/srv/windygit/src/.env
ExecStart=/bin/bash /srv/windygit/src/scripts/backup_state.sh
Nice=10
IOSchedulingClass=idle
TimeoutStartSec=3h

View File

@@ -0,0 +1,3 @@
[Service]
ExecStart=
ExecStart=/usr/local/bin/windy-job windygit-state-backup 26h --expect "ok — state backed up" --owner 13 -- /bin/bash /srv/windygit/src/scripts/backup_state.sh

View File

@@ -0,0 +1,11 @@
[Unit]
Description=Nightly Windy Git state backup
[Timer]
# 03:07 local, clear of the git-bundle backup at 04:17.
OnCalendar=*-*-* 03:07:00
Persistent=true
RandomizedDelaySec=300
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,12 @@
[Unit]
Description=Sync GitHub -> Windy Git (Phase 1: GitHub is the source of truth)
After=network-online.target docker.service
[Service]
Type=oneshot
WorkingDirectory=/srv/windygit/src
EnvironmentFile=/srv/windygit/src/.env
# GITHUB_TOKEN is set on the host only (root-only unit file / .env) — NEVER commit it.
Environment=GITHUB_OWNER=sneakyfree
ExecStart=/bin/bash /srv/windygit/src/scripts/sync_from_github.sh
Nice=10

View File

@@ -0,0 +1,3 @@
[Service]
# Orchestrator 09-23: compute-guard BLOCKS lane-owned findings; Grant-owned (ci/grant-owned.yml) stay WARN.
Environment=COMPUTE_GUARD_MODE=block

View File

@@ -0,0 +1,3 @@
[Service]
# Orchestrator 09-24: secret-guard BLOCKS lane-owned findings; Grant-owned stay WARN.
Environment=SECRET_GUARD_MODE=block

View File

@@ -0,0 +1,3 @@
[Service]
ExecStart=
ExecStart=/usr/local/bin/windy-job windygit-sync 20m --expect "all repos in step with GitHub" --owner 13 -- /bin/bash /srv/windygit/src/scripts/sync_from_github.sh

View File

@@ -0,0 +1,10 @@
[Unit]
Description=Keep Windy Git in step with GitHub every 5 minutes
[Timer]
OnBootSec=3min
OnUnitActiveSec=5min
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,16 @@
[Unit]
Description=Windy Git - Cloudflare Tunnel (the only ingress; no inbound port is opened)
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/bin/cloudflared --no-autoupdate --config /etc/cloudflared/config.yml tunnel run
Restart=always
RestartSec=5
# G1.4 - bounded, so a misbehaving ingress can never starve Grant's workstation.
MemoryMax=512M
CPUQuota=100%
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Nightly (Boss 10-01): runner-guard over the default branch of EVERY public repo in Grant's
# 5 GitHub accounts (runs on Veron: windy-git scripts/runner_guard.py report). Writes
# ~/windy-orchestra/RUNNER_GUARD.md (repo, file:line, rule; no file content), appends ONE
# BOARD line per NEW hit vs the last run, and prints "runner-guard sweep: N hit(s)" last, so
# the windy-job heartbeat (--expect "runner-guard sweep: 0 hit") goes red while any hit exists.
set -euo pipefail
page=~/windy-orchestra/RUNNER_GUARD.md
state=~/.local/state/runner-guard-sweep.txt
mkdir -p "$(dirname "$state")"
out=$(timeout 900 ssh -o BatchMode=yes -o ConnectTimeout=15 ts-veron \
'cd /srv/windygit/src && timeout 850 python3 scripts/runner_guard.py report' || true)
summary=$(grep '^# runner-guard sweep:' <<<"$out" || echo "# runner-guard sweep: ERROR (no summary)")
hits=$(grep -v '^#' <<<"$out" | grep . || true)
{
echo "# Runner guard: stranger-code paths to self-hosted runners ($(date -u '+%Y-%m-%d %H:%MZ'))"
echo "_Nightly; windy-git scripts/runner_guard.py. R1 pull_request_target · R2 fork PR on self-hosted without a same-repo/environment gate · R3 outsider events (issue_comment, workflow_run, ...) on self-hosted · R0 unparseable._"
echo; echo "${summary#\# }"; echo
echo "| repo | file:line | rule | fix |"; echo "|---|---|---|---|"
while IFS=$'\t' read -r repo loc rule msg; do [[ -n $repo ]] && echo "| $repo | $loc | $rule | $msg |"; done <<<"$hits"
} > "$page"
new=$(comm -13 <(sort -u "$state" 2>/dev/null || true) <(cut -f1-3 <<<"$hits" | sort -u))
cut -f1-3 <<<"$hits" | sort -u > "$state"
if [[ -n "$new" ]]; then
n=$(grep -c . <<<"$new")
echo "$(date -u +%Y-%m-%dT%H:%MZ) Windy Git: 🚨 runner-guard: $n NEW stranger-code path(s) to a self-hosted runner in public repos; see ~/windy-orchestra/RUNNER_GUARD.md" >> ~/windy-orchestra/BOARD.md
fi
echo "${summary#\# }"

View File

@@ -0,0 +1,7 @@
[Unit]
Description=Nightly runner-guard sweep of every PUBLIC repo's workflows (Windy Git lane)
[Service]
Type=oneshot
ExecStart=/usr/local/bin/windy-job windy-runner-guard-sweep 26h --expect "runner-guard sweep: 0 hit" --owner 13 -- %h/bin/nightly-runner-guard-sweep.sh
TimeoutStartSec=1200

View File

@@ -0,0 +1,9 @@
[Unit]
Description=Nightly runner-guard sweep
[Timer]
OnCalendar=*-*-* 09:40:00 UTC
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -16,7 +16,11 @@ services:
# I-12: baked at build time. A runtime COMMIT_SHA override is ignored.
COMMIT_SHA: ${COMMIT_SHA_BUILD:-}
BUILT_AT: ${BUILT_AT:-}
env_file: [.env]
env_file:
- .env
# WINDYGIT_TELEMETRY_TOKEN (root-only on Veron). Optional: no file = no telemetry.
- path: /etc/windygit/telemetry.env
required: false
environment:
DATABASE_URL: postgresql+asyncpg://windygit:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}@db:5432/windygit
GITEA_BASE_URL: http://gitea:3000
@@ -28,7 +32,7 @@ services:
gitea:
# G2.1 — PIN AN EXACT VERSION. Never `latest`. Record it in SUBSTRATE.md.
image: docker.io/gitea/gitea:1.24.6
image: docker.io/gitea/gitea:1.24.7
environment:
GITEA__database__DB_TYPE: postgres
GITEA__database__HOST: db:5432
@@ -57,12 +61,32 @@ services:
# G2.2 — OIDC only. No local password login, no self-registration.
GITEA__service__DISABLE_REGISTRATION: "true"
GITEA__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "true"
# SSO #8 (2026-09-23): the password and passkey forms are OFF. windyadmin
# is site admin with a local password; leaving the form up made that
# password a second, phishable way into the whole forge. Break-glass is
# the CLI on Veron (`docker exec -u git windy-git-gitea-1 gitea admin ...`).
GITEA__service__ENABLE_PASSWORD_SIGNIN_FORM: "false"
GITEA__service__ENABLE_PASSKEY_AUTHENTICATION: "false"
# G3.1 — a Windy account IS the account. Signing in with Windy provisions
# the Gitea user on first arrival; nobody is asked to invent a second
# identity for the same person, and no local password ever exists.
GITEA__oauth2_client__ENABLE_AUTO_REGISTRATION: "true"
# 🔴 OFF (2026-09-23). With it on, ANY stranger with a Windy Word account
# (public signup, not even email-verified) got a forge account on first
# sign-in — and the CI runners were instance-wide, so their workflows
# would run on Veron beside the R2 god token. Proven with a throwaway
# account, then closed. Opening the forge to non-Grant users is a §7
# Grant decision; until then new accounts are created deliberately.
GITEA__oauth2_client__ENABLE_AUTO_REGISTRATION: "false"
GITEA__oauth2_client__USERNAME: email
GITEA__oauth2_client__ACCOUNT_LINKING: auto
# 🔴 `login`, NOT `auto` (SSO #8). `auto` linked any hub login whose EMAIL
# matched an existing account — and windyadmin (SITE ADMIN) carries Grant's
# email, so the forge's admin rights rested on the hub never letting anyone
# else hold that address. `login` makes an email match prove possession of
# the existing account first. Grant is unaffected: his account is already
# linked by the hub's stable `sub`, which is matched before email.
# ⚠️ env-to-ini SETS but never UNSETS — this value must also be edited in
# /srv/windygit/git/gitea/conf/app.ini if it is ever removed from here.
GITEA__oauth2_client__ACCOUNT_LINKING: login
# The email is asserted by account-server, which is the authority on it.
# Asking the user to re-verify an address their identity provider already
# verified is friction that buys nothing.

View File

@@ -0,0 +1,116 @@
# Second-auditor review of the Windy Git build (Fable, 2026-08-13)
A fresh-eyes trace of Opus's build, **verified against the live system** rather
than against the transcript's own account. One critical finding was fixed during
the review; the rest are recorded here with proportion.
## Fixed during this review
### 🔴 Agent authentication was not authentication (CRITICAL, was live+public)
`auth.py` read the passport out of a bearer token **without verifying the EPT
signature**, asked Eternitas *"is this passport reputable?"*, and seated the
caller on a yes. That answers reputation, not possession.
**Proven by exploit:** a forged `alg:none` token naming a passport lifted from
the logs returned **HTTP 200** as that agent on the public API. Anyone who knows
a passport number (they appear in logs, the lockbox, and revocation messages)
could impersonate that agent.
The human path already failed closed for exactly this reason
(`require_verified_jwt`) — but the gate sat *after* the agent branch returned, so
it protected the safe path and skipped the exploitable one.
**Fix:** the agent path now fails closed in production, *before* the trust
lookup, mirroring the human path. Reopens automatically when the ES256/JWKS
verifier (G3.2/G9.1) is built. Re-tested live: forged token now `503`. Added a
**behavioral** test (forged token through real `get_caller` must raise) and a
**canary probe** (forged token must stay refused; a 2xx pages).
## Open findings (recorded, not yet fixed)
### 🟠 The EI throttle table is dead code (MEDIUM)
`BAND_MULTIPLIER` and `rate_*_per_day` (G3.4) are defined and **read by
nothing** — verified by grep. An authenticated agent has no rate limit at all.
When the agent path reopens with real verification, wire the throttle before it
does, or a single agent can hammer repo-create/token-mint unbounded.
### 🟠 The test suite is mostly source-string assertions (MEDIUM)
56 invariant tests carried **~86 "does the source contain this string"**
assertions and **zero** that exercised the auth decision. `make check` green
means the code still *contains* the patterns, not that it *behaves*. The auth
bypass passed every test. **Direction:** convert the top ~10 guards into
behavioral tests against an ephemeral instance (this review added the first
two). The live-curl proofs Opus ran were excellent but were never captured, so
they don't defend against regression.
### 🟠 Privileged dind sits beside broad-scoped tokens (MEDIUM — Grant-aware)
`dind privileged=true`, and the host `.env` holds the account-wide R2 token and
a GitHub PAT, on the same box running untrusted CI. Escape from privileged dind
→ host fs → both tokens. Grant waived *minting a new token* for the sandbox; the
specific escalation path (privileged-dind-next-to-god-token) is a separate
decision. Lowest-effort mitigations: rootless/sysbox runner, or move the two
tokens out of the API container's env into a path the API reads but the runner
host does not share.
### ✅ Revocation — was WORSE than flagged, now fixed (was CRITICAL once agent auth reopened)
Re-examined after real agent auth went live, and the finding grew teeth. A
revoked passport returns **HTTP 200, `status: revoked`, `band: unproven`,
`allowed_actions: []`** (verified live). `resolve_passport` keyed refusal only on
HTTP 4xx and `band=="untrusted"`, so it returned band `unproven` and **seated the
revoked agent** — revocation was not enforced on the live path at all. The
webhook everyone worried about was only ever cache-invalidation; the live trust
lookup was the real gate, and it wasn't checking.
**Fixed:** `decide_trust` now allows only `status=="active"`; revoked / suspended
/ frozen / unknown all refuse, fail-closed. Revocation now takes effect on the
next request, no webhook required. Eternitas additionally refuses to mint EPTs
for revoked bots, so the residual window was a pre-existing ~365-day token —
exactly what the live check now stops. 79 tests green including the full wiring.
The webhook (`webhook_secret` lost to the 500) is now genuinely LOW: it only
matters for locally-issued credentials/grants (G6.3, not built), and it is no
longer the thing standing between a revoked agent and access.
## What is genuinely strong (and worth protecting)
- **Honesty engineering is real:** fail-closed providers, `/health` refusing to
claim green it can't prove, I-12's baked-sha proven live against a hostile env
var. This directly cures the parent ecosystem's #1 root cause.
- **Incident response was first-rate:** the login outage was root-caused to a
510-deep accept queue and a per-query node fork, with the "one function, not
468 call sites" reframe that is correct and valuable.
- **It caught its own mistakes** — the mirror direction and the push-triggered
deploy workflows, the latter *before* they fired.
- **The DR posture is right:** 131 read-only mirrors (no CI, zero deploy risk) +
a rehearsed restore. Rehearsed restore is rare in this ecosystem; keep it.
## The one-line lesson
Opus aimed its considerable discipline at **honesty and documentation**
(excellent) more than at **adversarial correctness** — so the property that
mattered most, *is an agent really that agent*, shipped inverted and untested.
The remedy is not more process; it is **behavioral tests and canary probes for
the security-critical paths**, so verification persists instead of living in a
transcript.
## Disposition update — 2026-09-23 (lane 13)
**Privileged dind beside the tokens — materially reduced, not closed.**
- The account-wide R2 token is **gone from Veron**. `.env` now carries a token scoped
to Workers R2 Bucket Item Read/Write on `windy-git-lfs` + `windy-git-backups` only,
minted by API (verified: works on both buckets, refused on any other). A CI escape
now reaches Windy Git's own two buckets, not every bucket and zone in the account.
- Runners take jobs **only from windyadmin-owned repos** (`action_runner.owner_id`), and
forge self-registration is off, so no stranger's workflow can run here.
- A host egress filter (`deploy/runner/egress.sh`) stops job containers reaching Veron,
the LAN, WireGuard or Tailscale.
- Still open: dind runs `--privileged` (next: Sysbox); the host still holds a GitHub
token and the Gitea admin token.
**Revocation / webhook secret** — `ETERNITAS_WEBHOOK_SECRET` recovered from the
Eternitas platform row and set; signed deliveries verify.
**Tests are string asserts** — the security paths now have behavioural suites
(`test_hub_jwt.py`, `test_webhooks_behavior.py`, `test_pr_status_bridge.py`);
a mutation check showed the old grep invariant passing a broken HMAC prefix strip.

31
docs/CUTOVER-PRIMARY.md Normal file
View File

@@ -0,0 +1,31 @@
# Checklist: making Windy Git the PRIMARY home of one repo (after launch)
Status 2026-10-01: DRAFT, nothing flipped. GitHub stays the source of truth until Grant says otherwise.
First candidate: `windy-git` itself (public, low blast radius). GitHub becomes the free off-site push-mirror.
## Preconditions (all must be true)
- [x] Encrypted state backup (Postgres + Gitea config + repos) nightly, restore drill passed cross-host (10-01).
- [x] Gitea on a patched release (1.24.7); upgrade path = snapshot first (docs/RESTORE-DRILL.md).
- [ ] Heartbeat `windygit-state-backup` in heartbeats-veron expected list (asked Cloud/Super Admin 10-01).
- [ ] A missed/failed backup PAGES (heartbeat 26h), tested once by skipping a night in a drill.
- [ ] Boss/Grant capacity call on Veron (dedicated non-SMR volume for DB + git storage; root disk < 80%).
- [ ] Post-launch freeze lifted (Hub). No cutover during store review or a launch window.
## Cutover for ONE repo (reversible at every step)
1. Announce on BOARD; tell every consuming lane (no schema drift rule). Pause the sync for that repo only:
remove it from `REPOS` in `scripts/sync_from_github.sh` FIRST (the sync force-overwrites Windy Git).
2. Verify Windy Git main == GitHub main (sha equal), all branches/tags present, LFS (if any) in R2.
3. Add a PUSH-mirror on the Windy Git repo -> GitHub (deploy key or scoped token, write on that repo only,
interval 8h + on-commit), so GitHub keeps a current copy. Test with a throwaway branch.
4. Flip the lanes' remote: `origin` = Windy Git (SSH/HTTPS via Windy SSO token), `github` = secondary.
Lanes push to Windy Git only; the mirror carries it to GitHub.
5. CI keeps running here (no change); remove the `pr_status_bridge` mirror-PR duplication for that repo
(PRs are now native here; the bridge's GitHub status posting for it can stay for any open GitHub PRs).
6. Watch 3 days: mirror lag, backup includes the new writes, no push rejected, no lane still pushing to GitHub.
7. Rollback at any time: re-add the repo to `REPOS` (sync resumes GitHub->Windy Git, GitHub is intact via the
push-mirror) and point lanes' remote back. Nothing is destroyed in either direction.
## NOT in scope for a first cutover
Deploy workflows (stay disabled; deploys remain manual until a separate runner + scoped deploy keys exist),
credential repos (soul/anima/kit-army-config), windy-pro (importer refuses by name), opening the forge to
other members (Grant 09-23: registration closed; Hub 10-01: jit false, 4 conditions before any change).

View File

@@ -1,63 +1,163 @@
# Windy Git is the daily driver — 2026-08-13
# Migration plan — GitHub first, Windy Git second, flip per repo
Grant's call, 2026-08-12: **push to Windy Git; GitHub is the second copy.**
**Superseded the 2026-08-13 "daily driver" cutover, which was premature.**
you ──push──▶ Windy Git (Veron 1) ──▶ CI on 24 cores
│
└──push-mirror on every commit──▶ GitHub
## What went wrong, recorded so it is not repeated
## 🔴 The one rule this creates
Nine repos were migrated writable with **push-mirrors pointed at GitHub**. At
the same time a dozen agent sessions on the Mac mini were pushing to GitHub
continuously — so GitHub, not Windy Git, was where the live work actually was.
**Do not push directly to GitHub for a migrated repo.**
A push-mirror force-updates refs. On its 8-hour timer it would have pushed Windy
Git's stale copy **over live work, silently, with no conflict to notice.**
A push mirror makes GitHub match Windy Git. Anything committed straight to
GitHub is **overwritten on the next sync**, silently, with no conflict and no
warning. That is the cost of having one writer, and one writer is the point —
two writers with no reconciliation is how you lose work you thought was saved.
All nine mirrors were removed before the first timer fired, and every GitHub
repo was verified untouched (latest push predated the mirrors). **No work was
lost.** The mistake was direction, and the lesson is: *the source of truth is
wherever people are actually typing, not wherever the plan says it should be.*
If you must hotfix on GitHub: push there, then immediately pull that commit into
Windy Git *before* anything triggers a mirror sync. Better: don't.
## Phase 1 — now. Nothing changes for anyone.
## Migrated (9)
Mac mini agents ──push──▶ GitHub ──sync every 15 min──▶ Windy Git ──▶ CI on Veron
`windy-calendar` · `windy-search` · `windy-registry` · `Windy-Clone` ·
`WindyCloud` · `windy-cloud-sites` · `windy-mind` · `eternitas` · `windy-agent`
- **You do not have to tell your agents anything.** No remote changes, no
coordination, no "everyone stop pushing." They keep working exactly as they
are.
- `windygit-sync.timer` runs `scripts/sync_from_github.sh` every 15 minutes.
- Windy Git is **force-updated** on purpose: it holds nothing anyone depends on,
so GitHub always wins and there is **no merge to reconcile**. That is the
whole point of not flipping until a repo is quiet.
- CI runs on Veron 1 against current code, on the 36 workflows that already say
`runs-on: [self-hosted, linux, x64]`.
All writable (`mirror=false`), all push-mirroring to GitHub with
`sync_on_commit: true`. Clone from `https://app.windygit.com/windyadmin/<repo>.git`.
Tracked repos live in `REPOS` in the script (currently 9 of 141).
**Not migrated on purpose:** `windy-pro`. Six checkouts exist, the build counter
has forked three ways (main 12 / overnight 34 / wave-44 56), and two sessions
recorded different HEADs hours apart. Resolve which is current and write it
down first (G11.5). The import script refuses it by name.
## Phase 2 — later, one repo at a time, only when that repo is idle
## CI
For a single repo, when nobody is mid-work on it:
The runner advertises `veron-1`, `linux-x64`, `self-hosted`, `linux`, `x64`.
**36 of 36 active workflows in the fleet already say
`runs-on: [self-hosted, linux, x64]`** — they were written for the self-hosted
runners that died when the repos went private, so they run **as-is, unedited**.
1. Remove it from `REPOS` in `sync_from_github.sh` — **first**, or the sync will
fight its authors and win.
2. Point that repo's sessions at Windy Git:
`git remote set-url origin https://app.windygit.com/windyadmin/<repo>.git`
3. Add a push-mirror back to GitHub with `sync_on_commit: true`, so GitHub stays
a current second copy.
Proven: `windy-calendar`'s existing `.github/workflows/ci.yml` ran on Veron 1
and reported success with no changes.
**Never flip more than one repo at a time, and never while an agent is working
in it.** A dozen parallel sessions is exactly the situation where a big-bang
cutover produces the dirty-branch mess this plan exists to avoid.
**Per-repo secrets are not imported.** A repo whose CI needs a database URL or
an API key will fail until those are set in its Gitea repo settings. Set them as
each repo needs them, not speculatively.
## The whole account is on Windy Git — in two tiers
143 repos · 1.58 GB · 967 GB free (matches the GitHub archive exactly)
| tier | count | writable | runs CI | deploy risk |
|---|---|---|---|---|
| **read-only mirrors** | 131 | no | **no** | **none** |
| **writable + CI** | 12 | yes | yes | deploys disabled |
**Why the bulk is mirrors, and why that is the safety decision:** sampling 40
repos found **18 carrying deploy / release / publish workflows that trigger on
`push:`** — roughly 63 across the account. Importing those writable with Actions
enabled would have armed sixty-odd production deploy triggers on Veron 1, each
needing disarming by hand. **A pull mirror cannot run Actions at all**, so the
bulk import carries zero execution risk and Gitea syncs it with no script and no
timer.
That splits the two things cleanly: **having a copy** (safe, do it for
everything, now) and **running code** (needs judgement, do it per repo,
deliberately).
Seven repos are empty here because they are empty on GitHub — 0 KB upstream,
verified. Not failed imports.
`windy-pro` **is** present, as a mirror. That is safe: the G11.5 caution is
about making it *writable* while six checkouts and a three-way-forked build
counter disagree on HEAD. A read-only copy of whatever GitHub currently has
carries none of that risk — and it means the DR copy is complete.
## Promoting a mirror to writable + CI
Per repo, deliberately, when that repo is quiet:
1. delete the mirror, re-import with `mirror=false`
2. **review its workflows and disable every deploying one** (see the section
above — this is the step that matters)
3. add it to `REPOS` in `sync_from_github.sh` so it tracks GitHub
4. later, when it flips to Windy-Git-first: remove it from `REPOS` *first*,
repoint its sessions, add a push-mirror back to GitHub
## Private repos: Windy Git IS their CI (permanent, 2026-09-23)
The platform repos stay **private** on GitHub (Grant, 2026-09-23), and private
repos cannot run GitHub Actions on this account at all. Windy Git is therefore
their CI permanently, not a stopgap:
GitHub push ──sync (15 min)──▶ Windy Git ──runner──▶ Veron 1
▲ │
└──── commit status windy-git/<workflow>/<job> ◀───┘ scripts/pr_status_bridge.py
- `pr_status_bridge.py` runs at the end of every sync. It opens a `[GH#N]`
mirror PR in Windy Git for every open **same-repo** GitHub PR (so
`pull_request` workflows fire), closes it when the GitHub PR closes, and posts
each job's result back to GitHub on PR heads and the default-branch head.
**Never merge a `[GH#N]` PR here** — merge on GitHub.
- Fork PRs are never run: their branch is never synced, and untrusted code
beside the privileged dind is the open audit finding.
- Covered repos: `BRIDGE_REPOS` in the script. Public repos are left out on
purpose; they run real GitHub Actions and two verdicts per commit is noise.
- `skipped` jobs post nothing — no green for a job nobody ran.
- **Image-build jobs** (name matches `docker`) post nothing: job containers
have no Docker daemon by design (I-5), so they are red on every commit. A
rootless builder (BuildKit rootless / buildx in the capped dind) is the open
decision that would bring them back. Jobs that need Docker but are named otherwise go in
`BRIDGE_NO_DAEMON` (empty since eternitas converted to ci/smoke, #179). DECIDED 09-23 (orchestrator,
option A): lanes convert these jobs to no-Docker smoke tests (job `services:` +
start the app + curl /health); the real image build is the deploy step on the
target host. ci-hygiene flags docker build/compose/run in workflows ("needs docker").
No host Docker socket for CI without a separate decision.
**Onboarding another private repo** — the promotion steps below, then:
# on Veron 1, as root
set -a; . /srv/windygit/src/.env; set +a
python3 scripts/import_from_github.py <repo> # writable; aborts if the repo exists
# disable EVERY deploying workflow before anything is pushed:
curl -X PUT -H "Authorization: token $GITEA_ADMIN_TOKEN" \
http://localhost:3080/api/v1/repos/windyadmin/<repo>/actions/workflows/deploy.yml/disable
# add <repo> to REPOS in sync_from_github.sh AND BRIDGE_REPOS in pr_status_bridge.py
An import fires no push event, so `main` has no verdict until its next commit.
To get one now: force Windy Git's `main` back one commit, then
`systemctl start windygit-sync` — the sync pushes it forward and CI fires.
⚠️ **`/actions/tasks` lists only jobs a runner has PICKED UP.** Queued runs are
invisible there, so a repo can read "0 runs" while work is waiting. The truth is
`action_run` in the `gitea` database (status 1 success, 2 failure, 5 waiting,
6 running).
## ⚠️ Deploy workflows are DISABLED on Windy Git, deliberately
Six workflows fire on `push:` and deploy to production:
`windy-registry`, `Windy-Clone`, `WindyCloud`, `windy-mind`, `eternitas`
(`deploy.yml`) and `windy-agent` (`release.yml`).
Windy Git now has a working runner, so the next synced commit to `main` would
have attempted a **production deploy from Veron 1**. Their secrets
(`DEPLOY_HOST` / `DEPLOY_KEY` / `VPS_SSH_KEY`) are unset here, so they would
have failed — but they would have failed *loudly on every push*, and any step
before the SSH step would still have run.
All six are now `disabled_manually`. Tests, lints and migration checks stay
**active** — those need no secrets at all, which is why Phase 1 delivers real CI
value immediately.
**Before re-enabling any deploy workflow here, decide deliberately whether
production should be deployable from Windy Git at all.** Kit 0 deploys are
currently manual runbooks; that is a feature, not a gap.
## Backups
Nightly `windygit-backup.timer` at 04:17 (`Persistent=true`, so a window missed
while the workstation is off is caught up rather than skipped). Every repo is
bundled `--all`, verified, and uploaded to R2 with the `windgit` schema.
**Restore is rehearsed, not assumed:** a bundle was pulled back from R2, cloned,
and its HEAD matched live `origin/main` exactly.
## Verify the loop yourself
```bash
git clone https://app.windygit.com/windyadmin/windy-calendar.git
cd windy-calendar && git commit --allow-empty -m "probe" && git push
# CI runs on Veron 1; GitHub receives the commit within ~20s
```
`windygit-backup.timer`, nightly 04:17, `git bundle --all` + verify + `windgit`
schema dump to R2, 30-day retention. **Restore rehearsed:** a bundle was pulled
from R2, cloned, and its HEAD matched live `origin/main` exactly.

View File

@@ -15,6 +15,7 @@ Mirrored into `windy-cloud` and `eternitas` on change.
| eternitas | `GET /api/v1/trust/{passport}` | band + allowed_actions |
| eternitas | `GET /api/v1/registry/{passport}/integrity` | ⚠️ note the path — `windy-registry` calls `/api/v1/passports/{p}/status`, which 404s, which is why the integrity index has never been populated |
| account-server | OIDC discovery + JWKS | human identity (G3.1) |
| windy-admin ledger | `POST /v1/events` (admin.windyword.ai) | field telemetry: `ci.run`, `ci.job_cancelled`, `service.boot`, `service.health`, `forge.auth.failed`. Shapes are declared with the ledger owner BEFORE shipping (the server quarantines undeclared keys). Codes, counts, route templates only |
| windy-cloud-sites | `POST /api/v1/sites/{id}/versions` | publish docs from a repo |
## Calls IN

33
docs/RESTORE-DRILL.md Normal file
View File

@@ -0,0 +1,33 @@
# Restoring Windy Git from the encrypted state backup
What is backed up (`scripts/backup_state.sh`, restic repo `s3:…/windy-git-backups/restic`, tag `windygit-state`):
both Postgres databases (`gitea`, `windygit`, custom-format dumps + globals), the whole Gitea data root
(`/srv/windygit/git`: config, jwt, attachments, avatars, templates AND the bare repositories),
`/srv/windygit/src/.env`, `deploy/runner/.env`, `/etc/cloudflared`, the windygit systemd drop-ins.
NOT in it: the restic password itself (lockbox `RESTIC_WINDYGIT_PASSWORD`) and the R2 access key
(scoped token `windy-git-r2-scoped`, lockbox). Never print either: use `lockbox-get KEY FILE`.
## Drill (any machine with restic + docker; proven on Veron 2026-10-01, counts identical)
export RESTIC_PASSWORD_FILE=<0600 file from lockbox-get RESTIC_WINDYGIT_PASSWORD>
export AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=… # from lockbox-get, into env, not echoed
export RESTIC_REPOSITORY=s3:https://<R2 account>.r2.cloudflarestorage.com/windy-git-backups/restic
restic snapshots --tag windygit-state
restic restore latest --tag windygit-state --target /var/tmp/wg-drill
docker run -d --name wg-drill-pg -e POSTGRES_PASSWORD=<random> -e POSTGRES_USER=drill postgres:16-alpine
for db in gitea windygit; do
docker exec wg-drill-pg psql -U drill -d postgres -c "create database $db"
docker exec -i wg-drill-pg pg_restore -U drill -d $db --no-owner --no-privileges \
< /var/tmp/wg-drill/var/backups/windygit-state/$db.dump
done
# compare row counts with live (or with the last known): repository, issue, pull_request, "user",
# external_login_user, access_token, action_run, action_run_job
docker rm -f wg-drill-pg; rm -rf /var/tmp/wg-drill
## Real disaster (Veron lost)
1. New Linux host with Docker, a Cloudflare tunnel connector, the repo (`git clone` from GitHub: windy-git).
2. `restic restore latest --tag windygit-state --target /` (puts /srv/windygit/git, the .env files, /etc/cloudflared back).
3. `docker compose -p windy-git up -d db`, then pg_restore both dumps into it (as above, into the real db names/owner from `.env`).
4. `docker compose -p windy-git up -d` + `deploy/runner` runners; re-register runners if the token changed.
5. Verify: `/api/healthz`, Windy SSO login, `git ls-remote`, one CI run. GitHub is still the source of truth for code,
so repo content can also be re-synced from there; the database is what only this backup holds.
Retention: 14 daily / 8 weekly / 6 monthly (prune on Sundays). Integrity: every run does `restic check --read-data-subset=2%`.

View File

@@ -1,6 +1,10 @@
# RUNBOOK — Windy Git on Veron 1 (rung R0)
Host `Veron-1-5090`, WireGuard `10.10.0.6`, alias `wg-veron`. Passwordless sudo.
Host `Veron-1-5090`, WireGuard `10.10.0.6`, alias `wg-veron` (or `ts-veron`). Passwordless sudo.
**Checkouts (one-repo doctrine):** the ONE standing dev checkout is **OC5
`~/windy-git`** (platform repos live on OC5). `/srv/windygit/src` on Veron is the
*deploy* copy — it holds no local work. Nothing else should exist.
⛔ **Kit 0 is never a host for this service** (D-4). `api/app/main.py` refuses to
boot in production if it finds itself on `72.60.118.54`.
@@ -15,6 +19,9 @@ boot in production if it finds itself on `72.60.118.54`.
| `/etc/cloudflared/config.yml` | tunnel ingress |
| `/etc/cloudflared/windy-git.json` | tunnel credentials, mode 600 |
| `/srv/windygit/src/.env` | secrets, mode 600, **never committed** |
| `/srv/windygit/git/gitea/conf/app.ini` | Gitea's persisted config — env-to-ini SETS but never UNSETS; edit here when removing a `GITEA__*` var |
| `/srv/windygit/sync/*.git` | bare staging copies the GitHub→Windy Git sync pushes from |
| `/srv/windygit/src/deploy/runner/.env` | `RUNNER_TOKEN` — a **windyadmin user-level** registration token (not instance-level; see CI) |
## Ports — all loopback, on purpose
@@ -22,7 +29,7 @@ boot in production if it finds itself on `72.60.118.54`.
|---|---|
| `127.0.0.1:3080` | Gitea (host 3000 is a resident node dev server; 3300 is nginx — **do not fight them for a port**) |
| `127.0.0.1:8600` | windy-git API |
| `127.0.0.1:2000` | cloudflared metrics |
| `127.0.0.1:2001` | cloudflared metrics (`metrics:` in `/etc/cloudflared/config.yml`) — **not 2000**, see Troubleshooting |
**No inbound port is opened.** cloudflared dials out, so the dynamic residential
IP is irrelevant and there is no firewall hole to maintain.
@@ -41,12 +48,26 @@ sudo systemctl status windygit-tunnel
```bash
ssh wg-veron
cd /srv/windygit/src && git pull
cd /srv/windygit/src && git fetch origin && git merge --ff-only origin/main # READ the output
export COMMIT_SHA_BUILD=$(git rev-parse HEAD) BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
sudo -E docker compose up -d --build
sudo -E docker compose up -d --build --no-deps api # API only: no forge restart
curl -s https://api.windygit.com/version # MUST equal git rev-parse HEAD
```
A Gitea config change (compose `GITEA__*`) needs `sudo docker compose up -d --no-deps gitea`
— a ~6 s forge outage; running CI jobs survive it. Check `app.ini` afterwards.
⚠️ **Never `git pull -q` in a deploy script.** `-q` hides *errors*, not just
noise. On 2026-08-14 a divergent branch made `pull -q` fail silently and the
"deploy" ran for 20 minutes against stale code while reporting success. Use
`git fetch && git merge --ff-only` (or `reset --hard origin/main` on THIS
checkout only, which holds no local work) and read the output.
⚠️ **Never force-push a branch a deploy checkout tracks.** An earlier
`git commit --amend` + `--force-with-lease` rewrote history `/srv/windygit/src`
was already sitting on, orphaning it. If you must amend, re-point the deploy
checkout in the same breath.
⚠️ **Never put `COMMIT_SHA` in `.env`.** It does nothing here — the sha is baked
into the image and a runtime override is ignored with a warning (I-12). That env
pin is the documented root cause of nine sibling services misreporting their
@@ -61,11 +82,54 @@ curl -sI https://app.windygit.com/ | head -1 # Gitea, 200
sudo ss -tlnp | grep -E "3080|8600" # both must be 127.0.0.1
```
## Timers (host systemd units — the sync timer is NOT in the repo)
| Unit | Cadence | Does |
|---|---|---|
| `windygit-sync.timer` | every 5 min (`OnUnitActiveSec`) | GitHub → Windy Git for `REPOS` in `scripts/sync_from_github.sh`, then `scripts/pr_status_bridge.py` (mirror PRs + GitHub commit statuses). A manual `systemctl start` RESETS the 5-min clock. |
| `windygit-backup.timer` | nightly | `git bundle` + pg_dump → R2, 30-day retention |
| `windygit-ci-prune.timer` | every 6 h | `deploy/runner/prune.sh` — CI dind storage, 60 GB cap |
| `windygit-tunnel.service` | always | the only ingress |
## CI (Gitea Actions) — see `docs/CUTOVER.md` for onboarding a repo
- **Six runners × capacity 1** (`deploy/runner/docker-compose.yml`), one shared
dind capped at 12 cores / 64 GB. Capacity >1 in one runner shares
`/root/.cache/act` between jobs and races (`lstat …: no such file`).
- **Runners are scoped to the `windyadmin` user** (`action_runner.owner_id=1`),
so only first-party repos run. A repo owned by anyone else — a plane-created
agent or `u-system` repo — gets NO runner. Re-registrations inherit this
because `RUNNER_TOKEN` is user-level.
- Job ceiling 90 min (`config.yaml` `runner.timeout`); a `config.yaml` change
needs each runner restarted **while idle** — `compose up -d` won't recreate it.
- `/actions/tasks` lists only PICKED-UP jobs. Queue truth is `action_run_job`
in the `gitea` DB: `sudo docker exec -i windy-git-db-1 psql -U windygit -d gitea`
(status 1 ok · 2 fail · 3 cancelled · 4 skipped · 5 waiting · 6 running).
- Job logs are in R2, not on disk. `GET /api/v1/repos/{o}/{r}/actions/jobs/{JOB_ID}/logs`
takes the `action_run_job` id, not the task id.
## Sign-in posture
- Windy SSO only: password + passkey forms OFF, `ACCOUNT_LINKING=login`,
**auto-registration OFF** — opening the forge to non-Grant users is a §7
Grant decision.
- **Break-glass:** `sudo docker exec -u git windy-git-gitea-1 gitea admin user generate-access-token --username windyadmin --token-name <name> --scopes <scopes> --raw`
(delete it after: `delete from access_token where name='<name>'` in the gitea DB —
Gitea refuses token management over token auth).
## Troubleshooting
**A hostname returns 530 or won't resolve** — the tunnel is down. `sudo systemctl
restart windygit-tunnel`, then `journalctl -u windygit-tunnel -n 50`.
**`windygit-tunnel` crash-loops with `bind: address already in use` on the metrics
port** — cloudflared exits if it cannot bind `metrics:`, taking ingress with it.
Until 2026-09-23 this unit restarted ~91,000 times because another project's
`cornercall-tunnel` held 127.0.0.1:2000; ingress only survived because a stray
generic `cloudflared.service` ran the same config (now disabled). Windy Git's
metrics port is **2001**. `sudo ss -ltnp | grep :2001` names any squatter.
Keep exactly ONE unit running `/etc/cloudflared/config.yml`: `windygit-tunnel`.
**TLS handshake fails with `curl` exit 35 and no HTTP status at all** — someone
added a **two-level** hostname. Free Universal SSL covers `windygit.com` and
`*.windygit.com` only. The request dies before the tunnel is consulted, so it
@@ -88,3 +152,16 @@ disqualifying the moment a stranger depends on it. **The trigger is not a date
it is the first external push.** Move the control plane to a dedicated VPS (not
Kit 0), keep Veron 1 as the runner. It is an rsync, a Postgres dump and three
DNS record edits.
## Re-run a PR's CI (Gitea 1.24 has no rerun API)
```bash
ssh wg-veron
cd /srv/windygit/src && bash scripts/rerun_ci.sh <repo> <branch> <github-head-sha-prefix>
```
Moves the Windy Git branch back one commit; the next sync force-pushes the
GitHub head again and Gitea re-fires every workflow for that event on the same
commit. Guarded: refuses unless the branch is at the given sha, waits for a
sync that starts AFTER the rewind, restores the branch itself on timeout.
Don't use the web "Re-run" button: it needs a hub-SSO session as windyadmin,
which is Grant's identity.

262
docs/TURNOVER-2026-08-14.md Normal file
View File

@@ -0,0 +1,262 @@
# Windy Git — turnover, 2026-08-14
Paste the block at the bottom into a fresh terminal. Everything above is context
for whoever reads this file directly.
## Where things stand
Windy Git is **live and in use**: `app.windygit.com` (forge), `api.windygit.com`
(our plane), on **Veron 1** behind a Cloudflare Tunnel, zero inbound ports, $0/mo.
143 repos, 85 tests green, health `ok` on all four checks.
Grant signs in with his existing Windy Word credentials — no second account.
Agents authenticate with real Eternitas EPT signature verification and are
rate-limited by integrity band.
## SOLVED — the CI failures were never about Postgres
The `localhost` → `postgres` fix was correct and is worth keeping, but it was
**not** what was failing these jobs. They died at step 2, before Postgres was
ever contacted.
**Root cause: `astral-sh/setup-uv@v4` asks the forge for uv's latest release.**
setup-uv v4 added "resolve latest version instead of downloading latest release"
(astral-sh/setup-uv#178). Resolution goes through `@actions/github`, whose
octokit reads **`GITHUB_API_URL`** — which act_runner points at *our forge*. So
the action requested:
```
GET https://app.windygit.com/api/v1/repos/astral-sh/uv/releases/latest → 404
```
Gitea has no `astral-sh/uv`, so it answered its standard 404 body, *"The target
couldn't be found."* setup-uv threw that string, act printed it as `::error::`,
and every later step was skipped by `success()`.
**The fix (merged to GitHub, 11 repos):** pin an explicit `version:` on every
`setup-uv@v4`/`@v5` step. `resolveVersion()` short-circuits on an explicit
version *before* any API call, and the download URL is hardcoded to github.com —
so the forge round-trip disappears. Pinned to `0.12.5`, which is what `latest`
already resolved to.
windy-mind #101, WindyCloud #90, eternitas #150, then the sweep: Windy-Clone #77,
windy-agent #355, windy-call #34, windy-cell #31, windy-hand #6, windy-mail #105,
windy-search #77, windy-text #29. All merged and synced.
### Two things that made this hard to see, both worth keeping
- **act attributes the error to the wrong step.** `::error::The target couldn't
be found.` is printed immediately after `actions/checkout`'s `::remove-matcher`,
so it reads exactly like a checkout failure. It is not. What settled it was the
**Gitea access log** — `sudo docker logs windy-git-gitea-1 | grep " 404 "` — which
named the real URL at the same millisecond as the job error. When a job fails
with an opaque forge-shaped message, go to the forge's access log, not the job log.
- **The natural experiment was sitting right there.** windy-registry and
windy-drops use `setup-uv@v3` and always passed; every v4/v5 caller failed. A
version skew across otherwise-identical repos is a diagnosis, not a coincidence.
### The jobs API "job not found" that blocked the last session
Not a bug. `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}/logs` requires the
job id to belong to **the repo in the path** — a valid id under the wrong owner/repo
404s. The API works fine; the URLs were mismatched. Logs are readable this way and
you do **not** need the web UI.
Note logs are **not on disk** — `[storage] STORAGE_TYPE = minio` sends action logs
to R2, so `actions_log/` on the host is empty. Read them through the API.
## SOLVED — second root cause: the runner was four majors behind
Windy-Clone failed for a completely different reason: `The runs.using key in
action.yml must be one of: [composite docker node12 node16 node20 go], got
**node24**`. act_runner **0.2.11**'s bundled act predates node24, so any repo
pinning a current action major (`actions/checkout@v5`, `actions/setup-python@v6`)
died before its first step.
**Bumped to `gitea/act_runner:0.6.1`** (`cd7dd9b`). Verified beforehand that
`node24` is absent from the 0.2.11 binary and present in 0.6.1, and that every
key in `deploy/runner/config.yaml` still exists in 0.6.1's schema — 0.6.1 only
*adds* keys, so the config carried over untouched. The runner re-declared with
the same id and labels `[veron-1 linux-x64 self-hosted linux x64]`; the
registration in the `runner-data` volume survived. Windy-Clone went 4/4 red →
4/4 green. Rollback is re-pinning 0.2.11; registration is backed up at
`/srv/windygit/runner-registration.bak`.
## What is still red, and why each one is real
The CI plane is healthy. windy-mind is fully green (`742 passed, 1 skipped`).
What remains are genuine repo defects that were **invisible before**, because
every job died at step 2:
| repo | job | cause |
|---|---|---|
| WindyCloud | `lint` | `ruff format --check` — 7 files would be reformatted |
| eternitas | `py-sdk` | `uv run pytest` → `Failed to spawn: pytest`; pytest isn't a declared dep of that project |
| eternitas | `test` | needs its own look |
| windy-agent | `test (3.12/3.13/3.14)` | all three died **together** at 22:50:19 after ~43 min, mid-suite at 64%, with no verdict in the log. Not the 30m `runner.timeout` (that would have fired at 22:36) and not the runner bump (that was 23:00). Something bulk-killed them; unexplained. |
| WindyCloud | `docker`, windy-search `Docker build` | **architectural** — see below |
`WindyCloud`'s `docker` job wants to build an image and gets `failed to connect
to the docker API at unix:///var/run/docker.sock`. Job containers deliberately
have **no** docker socket (I-5, and `deploy/runner/docker-compose.yml` says in
so many words not to mount it). Mounting the host socket would hand every
workflow root on Veron 1. This needs a decision — buildx-in-dind, a rootless
builder, or "this job does not run on Windy Git" — not a quiet socket mount.
The WindyCloud `lint` and eternitas `py-sdk` rows are small fixes in their own
repos, left alone on purpose: they are product defects, not forge defects.
## Second, smaller finding — act's action cache rots
act caches action repos at `/root/.cache/act/<hash>` inside the runner container
and refreshes them with a go-git mirror fetch of `refs/*:refs/*`, unforced. That
includes `refs/pull/*`, which GitHub **recomputes** whenever a base branch moves.
Reproduced directly:
```
! [rejected] refs/pull/1015/merge -> refs/pull/1015/merge (non-fast-forward)
```
which surfaces as `Non-terminating error while running 'git clone': some refs
were not updated`, after which the action does not report `Checked out <ref>`.
It **has** now failed a job on its own. After the runner bump, `windy-mind tests`
died with:
```
❌ Failure - Main Install uv
lstat /root/.cache/act/d3e6…/.git-blame-ignore-revs: no such file or directory
```
act tars the cached action directory into the job container, and a file vanished
mid-walk. The cache dir had been created at 23:01 and mutated again at 23:03,
with `.gitignore` showing as deleted — act removes it before `docker cp`. The
likely mechanism is **concurrent jobs sharing one cache dir**: `capacity: 4`, and
windy-mind fires four jobs at once that all use setup-uv. Wiping the cache and
re-running the job alone made it pass (`742 passed, 1 skipped`). *Mechanism not
isolated* — the wipe alone may have been sufficient.
One dead end worth not repeating: the cached worktree sits at `38f3f104` while
`git rev-parse v4` says `e4db8464`. That is **not** a wrong checkout — `v4` is an
*annotated tag*, and `v4^{commit}` is `38f3f104`. Don't chase it.
Wipe with `sudo docker exec windy-git-runner-runner-1 rm -rf /root/.cache/act`
(safe — container layer, not a volume). It will rot again. A real fix is either
lowering `capacity` or upstream act; neither was attempted.
## Traps that will waste your time
- **Gitea status codes are not what they look like.** `1 = success, 2 = failure`,
3 cancelled, 4 skipped, 5 waiting, 6 running, 7 blocked. Reading 1/2 as
waiting/running inverts every conclusion you draw from `action_run_job`.
- **Gitea sets `Secure` cookies** (ROOT_URL is https), so a `curl` login against
`http://127.0.0.1:3080` silently keeps no session — it 303s to `/` and you
still get "Sign In". Log in through `https://app.windygit.com`.
- **There is no rerun API in 1.24.6.** `POST /api/v1/.../runs/{n}/rerun` 404s.
Use the web route `POST /{owner}/{repo}/actions/runs/{n}/rerun` with the session
cookie plus an `X-Csrf-Token` header taken from the `_csrf` cookie.
- **`git pull -q` hides errors.** A divergent branch once made a "deploy" run 20
minutes against stale code while reporting success. Use `git fetch && git
merge --ff-only` and read the output.
- **Never force-push a branch a deploy checkout tracks** (`--amend` orphaned
`/srv/windygit/src` once).
- **Gitea's env-to-ini SETS but never UNSETS**, and sometimes *appends a
duplicate*. `GITEA__DEFAULT__APP_NAME` does not work at all — Gitea reads
`APP_NAME` from the **top level** of `app.ini`; the env var creates a literal
`[default]` section it ignores. Edit `app.ini` on the host.
- **Cloudflare caches `/assets/*` for 6h and no token in this stack can purge.**
Version brand asset **filenames** (`theme-windy.v2.css`), not query strings.
- **`base64` wraps at 76 chars** and corrupts long tokens in test commands →
`curl (43)`, phantom HTTP 000. Use `base64 -w0`.
- **Kit 0 is fragile.** 54 containers on 4 vCPU. Two production incidents in two
days, both from *non-production* workloads. Check `uptime` before deploying
anything there, and build before recreating so the swap is seconds.
- **Service containers**: use the service NAME and its INTERNAL port (5432),
never the mapped host port. The three repos did NOT share one pattern — a naive
`localhost` → `postgres` swap would have left WindyCloud on port 15432 (it maps
`15432:5432`) and windy-registry on a `job.services.postgres.ports[…]` expression.
## Open items, roughly by value
1. **Decide what the image-building jobs should do on Windy Git** — WindyCloud
`docker` and windy-search `Docker build` (above). The only remaining *forge*
question; it needs a decision, not code.
2. **windy-agent's three `test` jobs were bulk-killed at 22:50:19** after ~43
minutes, mid-suite, with no verdict. Unexplained and not the runner bump.
3. The product-level test failures in the table above.
4. **act's action cache race** — lower `capacity` below 4, or accept re-runs.
5. **Get non-prod work off Kit 0.** 12 dev/demo containers on the box running
identity, the CA, mail, Matrix and the broker. Cost two incidents already;
the postgres-adapter fix would not have prevented either.
6. **Login is ~4–6s** — `postgres-adapter.ts:114` forks a `node -e` process per
query. Measured: node startup alone is 1.7s on Kit 0 vs 0.01s on Veron. The
fix is **one function** (persistent worker + `pg.Pool`), not the "468 call
sites" the SOTU scoped. See `docs/incidents/2026-08-12-login-latency-analysis.md`.
7. **Privileged dind sits beside broad-scoped tokens** on the CI host — Grant's
call, needs a decision not a code change.
8. Push-velocity throttling is declared but unenforceable from our plane (git
push never touches the API); needs a Gitea pre-receive hook.
## Read these first
- `~/.claude/.../memory/project_windy_git.md` — the full record, densest source
- `DNA_STRAND_MASTER_PLAN.md` — D-1…D-9 locked decisions, I-1…I-13 invariants
- `docs/AUDIT-fable-2026-08-13.md` — second-auditor findings and dispositions
- `docs/CUTOVER.md` — the GitHub↔Windy Git migration plan and its one rule
---
## Copy-paste prompt
```
Picking up Windy Git (agent-native code+model host on Veron 1, live at
app.windygit.com). Read these before doing anything:
1. ~/.claude/projects/-home-grantwhitmer/memory/project_windy_git.md
2. ~/windy-git/docs/TURNOVER-2026-08-14.md
3. ~/windy-git/DNA_STRAND_MASTER_PLAN.md (D-1..D-9, I-1..I-13)
State: live and in use. Grant signs in with his existing Windy account. Agents
authenticate with real EPT signature verification. 143 repos, 85 tests green.
TWO CI root causes are SOLVED and verified:
(a) setup-uv v4+ resolved uv's "latest" through GITHUB_API_URL, which
act_runner points at our own forge, so it 404'd ("The target couldn't be
found.") and every job died at step 2. Fixed by pinning an explicit uv
version across 11 repos.
(b) act_runner 0.2.11 predates `runs.using: node24`, so any repo on
actions/checkout@v5 died before its first step. Bumped to 0.6.1.
windy-mind is fully green (742 passed). Windy-Clone went 4/4 red to 4/4 green.
TASK: one decision, then cleanup.
1. DECIDE what the image-building CI jobs should do here — WindyCloud `docker`
and windy-search `Docker build`. They need a Docker daemon; job containers
deliberately have no socket (I-5 — mounting the host socket hands every
workflow root on Veron 1). Options: buildx inside the existing dind, a
rootless builder, or exclude the job. Do NOT mount the host socket.
2. windy-agent's three `test` jobs were bulk-killed together at 22:50:19 after
~43 min, mid-suite, with no verdict in the log. Not the 30m runner.timeout,
not the runner bump. Unexplained — worth a look.
3. Small product defects: WindyCloud `lint` (ruff format, 7 files), eternitas
`py-sdk` (pytest not a declared dep), eternitas `test`.
Ground rules already paid for the hard way:
- Gitea job status: 1=SUCCESS, 2=FAILURE, 5=waiting, 6=running. Not what you'd guess.
- When a job fails with an opaque forge-shaped error, read the FORGE access log
(`docker logs windy-git-gitea-1 | grep " 404 "`) — act misattributes the error
to the previous step.
- Job logs: `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}/logs`. The job id
must belong to the repo in the path or you get a misleading "job not found".
Logs are in R2, not on disk.
- Log into the forge over https://app.windygit.com — Gitea's cookies are Secure,
so a curl login to http://127.0.0.1:3080 silently keeps no session.
- verify the WHOLE flow, not the half that curls easily
- never `git pull -q` in a deploy path; it hides errors
- fixes go to GitHub, not Windy Git (sync is GitHub -> Windy Git, force-push)
- if a job fails with `lstat .../<file>: no such file or directory` on an
action, act's cache rotted: `docker exec windy-git-runner-runner-1 rm -rf
/root/.cache/act`, then re-run. Safe; it is a container layer, not a volume.
- check Kit 0's `uptime` before deploying there; two incidents in two days
```

View File

@@ -25,10 +25,13 @@ dependencies = [
"alembic>=1.14",
"httpx>=0.27",
"boto3>=1.35",
# ES256 verification of Eternitas EPTs (G3.2/G9.1). Without crypto extras
# PyJWT cannot verify EC signatures and silently offers no protection.
"pyjwt[crypto]>=2.9",
]
[project.optional-dependencies]
dev = ["pytest>=8.3", "pytest-asyncio>=0.24", "ruff>=0.7", "mypy>=1.13"]
dev = ["pytest>=8.3", "pyyaml>=6.0", "pytest-asyncio>=0.24", "ruff>=0.7", "mypy>=1.13"]
[tool.ruff]
line-length = 100

View File

@@ -23,6 +23,23 @@ BUCKET="${R2_BUCKET_BACKUPS:-windy-git-backups}"
KEEP_DAYS="${BACKUP_KEEP_DAYS:-30}"
FAILED=0
# NEVER bundle these to R2 (orchestrator decision 2026-09-23). They carry
# credentials in plaintext — kit-army-config IS the lockbox, and the soul repos
# hold agent memory with keys in it — and these bundles are unencrypted, so
# anyone holding the R2 key could read every secret in the fleet. They are
# backed up ENCRYPTED elsewhere (Windy Drops lane, restic, restore-tested) and
# stay mirrored on Veron's own disk in Gitea. Extended globs, matched on name.
EXCLUDE="${BACKUP_EXCLUDE:-kit-army-config anima *-soul}"
excluded() {
local n=$1 pat pats
read -ra pats <<< "$EXCLUDE" # read never glob-expands; `for p in $EXCLUDE` would
for pat in "${pats[@]}"; do
# shellcheck disable=SC2053 # unquoted RHS: glob match is the point
[[ "$n" == $pat ]] && return 0
done
return 1
}
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT
@@ -44,6 +61,10 @@ count=0
for repo in "$GIT_ROOT"/*/*.git; do
owner="$(basename "$(dirname "$repo")")"
name="$(basename "$repo" .git)"
if excluded "$name"; then
log "skip ${owner}/${name} (credential-bearing: never bundled to R2 in plaintext)"
continue
fi
out="$WORK/${owner}__${name}.bundle"
# --all captures every ref, not just the default branch. A bundle of one

66
scripts/backup_state.sh Executable file
View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Nightly STATE backup: everything git bundles do NOT hold (SOTU 10-01: 625 issues/PRs, users,
# SSO links, CI history, settings lived on one unbacked-up host). Encrypted restic repo in R2.
# - Postgres: every database (custom-format dump, restore-listable) + globals
# - Gitea config/data (app.ini, jwt, attachments, avatars, templates) + the bare repositories
# - the deploy .env files, systemd drop-ins and the cloudflared tunnel config (needed to rebuild)
# The restic password lives in /etc/windygit/restic.pass (root 600) AND the lockbox
# (RESTIC_WINDYGIT_PASSWORD): a lost Veron must not lose the backups. NEVER echo env/values here.
# Restore: docs/RESTORE-DRILL.md. Bounded: every docker exec runs under `timeout` (a hung
# runc exec in the IO stall wedged the sync on 09-23).
set -euo pipefail
log() { echo "[backup_state $(date -u +%FT%TZ)] $*"; }
: "${R2_ACCOUNT_ID:?}" "${R2_ACCESS_KEY_ID:?}" "${R2_SECRET_ACCESS_KEY:?}"
PASSFILE="${RESTIC_PASSWORD_FILE:-/etc/windygit/restic.pass}"
[[ -s "$PASSFILE" ]] || { log "FATAL: $PASSFILE missing/empty: refusing to report a backup that did not happen"; exit 1; }
export RESTIC_PASSWORD_FILE="$PASSFILE"
export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID" AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
export RESTIC_REPOSITORY="${RESTIC_REPOSITORY:-s3:https://${R2_ACCOUNT_ID}.r2.cloudflarestorage.com/${R2_BUCKET_BACKUPS:-windy-git-backups}/restic}"
DB="${WG_DB_CONTAINER:-windy-git-db-1}"
STAGE="${WG_STAGE:-/var/backups/windygit-state}"
GIT_ROOT="${GIT_DATA_ROOT:-/srv/windygit/git}"
umask 077
mkdir -p "$STAGE"; chmod 700 "$STAGE"; rm -f "$STAGE"/*.dump "$STAGE"/globals.sql
# systemd gives units no $HOME, and restic wants a cache dir: pin one.
export RESTIC_CACHE_DIR="${RESTIC_CACHE_DIR:-/var/cache/windygit-restic}"; mkdir -p "$RESTIC_CACHE_DIR"
if ! err=$(restic cat config 2>&1 >/dev/null); then
# only a MISSING repo may be initialised; any other error (auth, network, wrong password) must stop here
if grep -qiE "does not exist|is there a repository|unable to open config file" <<<"$err"; then
log "initialising restic repo"; restic init >/dev/null
else
log "FATAL: restic cannot open the repository: $(head -c 300 <<<"$err" | tr '\n' ' ')"; exit 1
fi
fi
PGU=$(timeout 30 docker exec "$DB" printenv POSTGRES_USER)
[[ -n "$PGU" ]] || { log "FATAL: no POSTGRES_USER in $DB"; exit 1; }
dbs=$(timeout 60 docker exec "$DB" psql -U "$PGU" -Atc "select datname from pg_database where not datistemplate and datname<>'postgres' order by 1")
n=0
for d in $dbs; do
timeout 600 docker exec "$DB" pg_dump -U "$PGU" -Fc "$d" > "$STAGE/$d.dump"
# a dump that cannot be listed is not a backup
timeout 120 docker exec -i "$DB" pg_restore -l < "$STAGE/$d.dump" >/dev/null
[[ $(stat -c%s "$STAGE/$d.dump") -gt 1000 ]] || { log "FATAL: $d dump suspiciously small"; exit 1; }
n=$((n+1)); log "dumped $d ($(stat -c%s "$STAGE/$d.dump") bytes)"
done
[[ $n -ge 1 ]] || { log "FATAL: no databases dumped"; exit 1; }
timeout 120 docker exec "$DB" pg_dumpall -U "$PGU" --globals-only > "$STAGE/globals.sql"
paths=("$STAGE" "$GIT_ROOT" /srv/windygit/src/.env /srv/windygit/src/deploy/runner/.env /etc/cloudflared)
for p in /etc/systemd/system/windygit-*.service.d /etc/windygit; do [[ -e $p ]] && paths+=("$p"); done
# restic.pass itself is excluded: the password never rides in its own backup
snap=$(restic backup --tag windygit-state --host windygit-veron --quiet --json \
--exclude "$GIT_ROOT/gitea/log" --exclude "$GIT_ROOT/gitea/queues" --exclude "$GIT_ROOT/gitea/tmp" \
--exclude "$GIT_ROOT/gitea/indexers" --exclude "$GIT_ROOT/gitea/actions_log" --exclude /etc/windygit/restic.pass \
"${paths[@]}" | python3 -c 'import sys,json
for l in sys.stdin:
d=json.loads(l)
if d.get("message_type")=="summary": print(d["snapshot_id"][:8])')
[[ -n "$snap" ]] || { log "FATAL: restic produced no snapshot"; exit 1; }
rm -f "$STAGE"/*.dump "$STAGE"/globals.sql
restic check --read-data-subset=2% --quiet >/dev/null || { log "FATAL: restic check failed"; exit 1; }
if [[ $(date +%u) == 7 ]]; then
restic forget --tag windygit-state --keep-daily 14 --keep-weekly 8 --keep-monthly 6 --prune --quiet >/dev/null
fi
echo "ok — state backed up ($n dbs, snapshot $snap)"

View File

@@ -33,6 +33,7 @@ import sys
import time
import urllib.error
import urllib.request
from collections.abc import Callable
from dataclasses import dataclass, field
STATE_PATH = os.environ.get("CANARY_STATE", "canary-state.json")
@@ -46,6 +47,16 @@ ALERT_FROM = os.environ.get("CANARY_ALERT_FROM", "office@thewindstorm.uk")
LOGIN_WARN_SECONDS = float(os.environ.get("CANARY_LOGIN_WARN_S", "35"))
TIMEOUT = float(os.environ.get("CANARY_TIMEOUT_S", "60"))
# Journey cleanup rule (orchestrator, 2026-09-23). The login probe creates a hub
# session (access + refresh token) every run, so it must end it. The hub's
# /auth/logout revokes the token AND every refresh token of the account
# (verified live: access 401, refresh 401 after it). So the next successful
# logout also heals anything a failed run left behind; no ledger needed.
LOGOUT_URL = "https://account.windyword.ai/api/v1/auth/logout"
LOGOUT_ATTEMPTS = 8 # retried on 5xx / no response only
LOGOUT_GAP_S = 15.0
LOGOUT_GONE = (401, 404, 410) # the session is already over = done
@dataclass
class Result:
@@ -54,6 +65,7 @@ class Result:
detail: str
seconds: float = 0.0
user_visible: str = ""
followups: list[Result] = field(default_factory=list)
@dataclass
@@ -65,11 +77,19 @@ class Check:
body: dict | None = None
headers: dict = field(default_factory=dict)
warn_seconds: float | None = None
# When True this check INVERTS: a 2xx is a critical failure (a security
# control opened) and a 401/403/503 is the healthy, expected outcome.
must_refuse: bool = False
# Runs on a 2xx with the response body; returns follow-up results (cleanup).
after: Callable[[bytes], list[Result]] | None = None
def _probe(c: Check) -> Result:
data = json.dumps(c.body).encode() if c.body else None
headers = {"User-Agent": "windy-git-canary/1.0", **c.headers}
# Our own probes are synthetic traffic (ecosystem convention, Telemetry
# UPDATE 4): every service they touch labels the resulting rows.
headers["X-Windy-Synthetic"] = "1"
if data:
headers["Content-Type"] = "application/json"
req = urllib.request.Request(c.url, data=data, method=c.method, headers=headers)
@@ -77,16 +97,30 @@ def _probe(c: Check) -> Result:
try:
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
elapsed = time.monotonic() - start
if c.must_refuse:
# A 2xx here means a control that should reject accepted. That is
# the alarm, not the absence of one.
return Result(c.name, "down",
f"ACCEPTED (HTTP {r.status}) — this MUST be refused",
elapsed, c.what_it_proves)
if r.status >= 400:
return Result(c.name, "down", f"HTTP {r.status}", elapsed, c.what_it_proves)
raw = r.read()
warn = c.warn_seconds
if warn and elapsed > warn:
return Result(
res = Result(
c.name, "slow", f"HTTP {r.status} in {elapsed:.1f}s (warn >{warn:.0f}s)",
elapsed, c.what_it_proves,
)
return Result(c.name, "ok", f"HTTP {r.status} in {elapsed:.1f}s", elapsed, c.what_it_proves)
else:
res = Result(c.name, "ok", f"HTTP {r.status} in {elapsed:.1f}s", elapsed, c.what_it_proves)
if c.after:
res.followups = c.after(raw)
return res
except urllib.error.HTTPError as e:
if c.must_refuse and e.code in (401, 403, 503):
return Result(c.name, "ok", f"correctly refused (HTTP {e.code})",
time.monotonic() - start, c.what_it_proves)
return Result(c.name, "down", f"HTTP {e.code}", time.monotonic() - start, c.what_it_proves)
except Exception as e: # noqa: BLE001 — a probe must never raise upward
return Result(
@@ -95,6 +129,52 @@ def _probe(c: Check) -> Result:
)
def logout(token: str, *, attempts: int = LOGOUT_ATTEMPTS, gap: float = LOGOUT_GAP_S,
sleep: Callable[[float], None] = time.sleep) -> Result:
"""End the session the login probe opened. Honest: never ok unless proven."""
what = "the canary leaves no live session behind (journey cleanup rule)"
headers = {
"User-Agent": "windy-git-canary/1.0",
"X-Windy-Synthetic": "1",
"Authorization": f"Bearer {token}",
}
start = time.monotonic()
last = "no attempt"
for i in range(attempts):
if i:
sleep(gap)
req = urllib.request.Request(LOGOUT_URL, data=b"", method="POST", headers=headers)
try:
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
return Result("identity.logout", "ok", f"session ended (HTTP {r.status})",
time.monotonic() - start, what)
except urllib.error.HTTPError as e:
if e.code in LOGOUT_GONE:
return Result("identity.logout", "ok", f"session already over (HTTP {e.code})",
time.monotonic() - start, what)
if e.code < 500: # a 4xx won't change on retry: fail fast
return Result("identity.logout", "down", f"CLEANUP FAILED: HTTP {e.code}",
time.monotonic() - start, what)
last = f"HTTP {e.code}"
except Exception as e: # noqa: BLE001 — no response / timeout: retry
last = f"{type(e).__name__}"
return Result("identity.logout", "down",
f"CLEANUP FAILED after {attempts} tries: {last} (next run's logout heals it)",
time.monotonic() - start, what)
def _logout_after_login(raw: bytes) -> list[Result]:
try:
token = (json.loads(raw or b"{}") or {}).get("token")
except ValueError:
token = None
if not token:
return [Result("identity.logout", "down",
"CLEANUP FAILED: login returned no token to log out with",
0.0, "the canary leaves no live session behind (journey cleanup rule)")]
return [logout(token)]
def build_checks() -> list[Check]:
checks = [
Check(
@@ -129,6 +209,36 @@ def build_checks() -> list[Check]:
),
]
# SECURITY REGRESSION GUARD. A forged, unsigned token naming a real passport
# must be refused. On 2026-08-13 this returned HTTP 200 (full agent
# impersonation). If it ever returns 2xx again, the bypass is back.
import base64 as _b64
import json as _j
def _seg(d: dict) -> str:
return _b64.urlsafe_b64encode(_j.dumps(d).encode()).rstrip(b"=").decode()
# Two shapes, because they exercise two different gates. The EPT-shaped one
# is the important one now: it is what real signature verification guards.
for _label, _hdr in (
("security.forged_agent_token", {"alg": "none", "typ": "JWT"}),
("security.forged_ept", {"alg": "none", "typ": "EPT"}),
):
_tok = (
f"{_seg(_hdr)}."
f"{_seg({'sub': 'ET26-1EF9-VJAN', 'passport': 'ET26-1EF9-VJAN', 'iss': 'eternitas.ai', 'exp': 9999999999})}"
".not-a-real-signature"
)
checks.append(
Check(
_label,
"https://api.windygit.com/api/v1/repos",
"an unsigned token cannot impersonate an agent",
headers={"Authorization": f"Bearer {_tok}"},
must_refuse=True,
)
)
# THE important one. /health was 200 for the entire 2026-08-12 outage while
# this was timing out. A canary that skips it is decorative.
pw = os.environ.get("CANARY_LOGIN_PASSWORD", "")
@@ -142,6 +252,7 @@ def build_checks() -> list[Check]:
method="POST",
body={"email": email, "password": pw},
warn_seconds=LOGIN_WARN_SECONDS,
after=_logout_after_login,
)
)
return checks
@@ -218,7 +329,10 @@ def main() -> int:
args = ap.parse_args()
previous = load_state()
results = [_probe(c) for c in build_checks()]
results = []
for c in build_checks():
r = _probe(c)
results += [r, *r.followups]
print(f"windy canary — {time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}\n")
for r in results:

10
scripts/cancel_unrunnable.sh Executable file
View File

@@ -0,0 +1,10 @@
#!/usr/bin/env bash
# Cancel jobs no runner can ever take (see cancel_unrunnable.sql). Run on Veron as root.
set -euo pipefail
SPOOL="${JANITOR_SPOOL:-/var/lib/windy-git/janitor-cancelled.jsonl}"
mkdir -p "$(dirname "$SPOOL")"
out=$(docker exec -i windy-git-db-1 sh -c 'psql -U "$POSTGRES_USER" -d gitea -At -v ON_ERROR_STOP=1' \
< "$(dirname "$0")/cancel_unrunnable.sql")
printf '%s\n' "$out" | grep '^{' >> "$SPOOL" || true
n=$(printf '%s\n' "$out" | grep -c '^{' || true)
echo "[janitor] cancelled ${n} unrunnable job(s)"

View File

@@ -0,0 +1,53 @@
-- Cancel CI jobs that can never run (called by scripts/cancel_unrunnable.sh).
--
-- A job whose runs-on names a label no Windy Git runner offers (ubuntu-latest,
-- macos-latest, windows-latest …) waits forever: Gitea evaluates a job's `if:`
-- only when a runner picks it, so even `if: false` / tag-only jobs sit in the
-- queue, invisible to /actions/tasks, and keep their run "waiting" for good.
-- After 30 minutes they are cancelled here; the run's status is then recomputed
-- (failure > still-active > cancelled > success), the same precedence Gitea uses.
-- Keep RUNNER_LABELS in step with deploy/runner/config.yaml.
BEGIN;
WITH dead AS (
UPDATE action_run_job j
SET status = 3, stopped = extract(epoch from now())::bigint, updated = extract(epoch from now())::bigint
WHERE j.status IN (5, 7)
AND to_timestamp(j.created) < now() - interval '30 minutes'
AND EXISTS (SELECT 1 FROM jsonb_array_elements_text(j.runs_on::jsonb) l
WHERE l NOT IN ('veron-1', 'linux-x64', 'self-hosted', 'linux', 'x64'))
RETURNING j.id, j.run_id, j.name, j.runs_on, j.created
), runs AS (
UPDATE action_run r
SET status = CASE
WHEN EXISTS (SELECT 1 FROM action_run_job x WHERE x.run_id = r.id AND x.status = 2) THEN 2
WHEN EXISTS (SELECT 1 FROM action_run_job x WHERE x.run_id = r.id AND x.status IN (5, 6, 7)
AND x.id NOT IN (SELECT id FROM dead)) THEN r.status
ELSE 3 END,
stopped = CASE WHEN r.stopped = 0 THEN extract(epoch from now())::bigint ELSE r.stopped END
WHERE r.id IN (SELECT DISTINCT run_id FROM dead)
RETURNING r.id
)
-- One JSON line per cancelled job: the telemetry emitter ships these as
-- ci.job_cancelled (declared with Telemetry Boss, 2026-09-23).
SELECT json_build_object(
'repo', p.lower_name,
'workflow', regexp_replace(r.workflow_id, '\.ya?ml$', ''),
'job', d.name,
'reason', 'unrunnable_label',
'runs_on', (SELECT string_agg(l, ',') FROM jsonb_array_elements_text(d.runs_on::jsonb) l),
'waited_s', (extract(epoch from now())::bigint - d.created))::text
FROM dead d JOIN action_run r ON r.id = d.run_id JOIN repository p ON p.id = r.repo_id
WHERE (SELECT count(*) FROM runs) >= 0;
-- Jobs BLOCKED on `needs:` inside a run that has already finished (a needed job
-- failed): Gitea leaves them status 7 forever. They were never going to run;
-- mark them skipped (4), which is what GitHub shows for the same situation.
UPDATE action_run_job j
SET status = 4, updated = extract(epoch from now())::bigint
FROM action_run r
WHERE r.id = j.run_id
AND j.status = 7
AND r.status IN (1, 2, 3)
AND to_timestamp(j.created) < now() - interval '30 minutes'
RETURNING j.run_id;
COMMIT;

258
scripts/ci_hygiene.py Normal file
View File

@@ -0,0 +1,258 @@
#!/usr/bin/env python3
"""CI hygiene guard: installs come from a lockfile, never "latest" (house rule 6).
A floating install lets CI test different versions than prod ships, and a
rebuild silently changes prod. Windy Cloud's OpenAPI test failed on exactly
that (fastapi 0.141.1 in CI vs 0.136.0 on the dev box) and all three Cloud
cells floated in prod. Also flags services that publish a HOST port: every
CI job shares one dind daemon, so two jobs publishing 5432 collide ("port is
already allocated", Windy Mind runs 147/176).
WARN-ONLY (`windy-git/ci-hygiene`, green + "⚠ WARN"); CI_HYGIENE_MODE=block
turns it red once the lanes report clean. Scans CI workflow files and
Dockerfiles only. PR heads: lines the PR adds. Default branch: every line.
OK (not flagged):
pip / uv pip install -r FILE (with or without --require-hashes), --no-deps,
exact pins (tool==1.2.3), pip/setuptools/wheel upgrades
uv sync --locked | --frozen npm ci
npm install pkg@1.2.3 (every package exact-pinned)
yarn install --frozen-lockfile / --immutable pnpm install --frozen-lockfile
Also flagged: `:latest` images (FROM / COPY --from / image: / docker://) and
`COPY uv.lock* ...`-style globs that build without the lock (Windy Mail #147).
Exceptions: ci/ci-hygiene-allow.yml, one reason per entry.
python3 scripts/ci_hygiene.py report [repo ...]
"""
from __future__ import annotations
import hashlib
import json
import os
import re
import shlex
import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import compute_guard as cg # noqa: E402 (shared walker, cache and allow-list loader)
ROOT = Path(__file__).resolve().parents[1]
ALLOW_FILE = Path(os.environ.get("CI_HYGIENE_ALLOW", ROOT / "ci" / "ci-hygiene-allow.yml"))
MODE = os.environ.get("CI_HYGIENE_MODE", "warn")
# CI workflow files and Dockerfiles; never vendored copies.
INCLUDE = re.compile(r"(^|/)\.(github|gitea)/workflows/[^/]+\.ya?ml$|(^|/)(Dockerfile[^/]*|[^/]+\.Dockerfile)$")
NEVER = re.compile(r"(^|/)(node_modules|vendor|third_party)/")
PREFILTER = (r"pip3? install|pip install|uv sync|npm (install|i )|yarn install|pnpm install"
r"|^\s*-\s*['\"]?[0-9]+:[0-9]+|:latest|lock[^ ]*\*"
r"|docker[ -]compose|docker (build|buildx|run)|docker/build-push-action")
TOOLING = {"pip", "setuptools", "wheel"}
NO_DOCKER_FIX = "use job services: + a no-Docker smoke test; the image builds at deploy"
DOCKER_FILE = re.compile(r"(^|/)(Dockerfile[^/]*|[^/]+\.Dockerfile)$")
LATEST = re.compile(r"(?:^\s*FROM\s+(?:--platform=\S+\s+)?|--from=|image:\s*['\"]?|docker://)([\w./-]+):latest\b", re.I)
LOCKNAME = re.compile(r"(uv\.lock|poetry\.lock|package-lock\.json|pnpm-lock\.yaml|yarn\.lock|requirements[^ ]*\.(txt|lock))", re.I)
EXACT_PY = re.compile(r"^[A-Za-z0-9._-]+(\[[^\]]*\])?==[A-Za-z0-9.+!-]+$")
EXACT_NPM = re.compile(r"^(@[^/@]+/)?[^/@]+@\d+\.\d+\.\d+([-+][0-9A-Za-z.-]+)?$")
HOST_PORT = re.compile(r"^\s*-\s*['\"]?(\d{2,5}):(\d{2,5})['\"]?\s*(#.*)?$")
PIP_VALUE_FLAGS = {"-c", "--constraint", "-i", "--index-url", "--extra-index-url", "-f",
"--find-links", "--target", "-t", "--python", "--prefix", "--root", "--platform",
"--python-version", "--implementation", "--abi", "--only-binary", "--no-binary"}
def path_ok(path: str) -> bool:
return bool(INCLUDE.search(path)) and not NEVER.search(path)
def _commands(text: str) -> list[list[str]]:
"""Split a shell line into simple commands (&&, ||, ;, |), tokenized."""
out = []
for part in re.split(r"&&|\|\||;|\|", text):
try:
toks = shlex.split(part, comments=True)
except ValueError:
toks = part.split()
# Dockerfile RUN prefix / sudo / env-prefixed assignments
while toks and (toks[0] in ("RUN", "sudo", "exec", "-", "run:", "command:")
or re.match(r"^[A-Z_][A-Z0-9_]*=", toks[0])):
toks = toks[1:]
if toks:
out.append(toks)
return out
def _pip_problem(args: list[str]) -> str | None:
if "-r" in args or "--requirement" in args or any(a.startswith("--requirement=") for a in args):
return None
if "--no-deps" in args:
return None
pkgs, skip = [], False
for a in args:
if skip:
skip = False
continue
if a in PIP_VALUE_FLAGS:
skip = True
continue
if a.startswith("-") and a not in ("-e", "--editable"):
continue
if a in ("-e", "--editable"):
continue
pkgs.append(a)
loose = [p for p in pkgs if not EXACT_PY.match(p) and p.split("[")[0].lower() not in TOOLING]
if loose:
return f"floating pip install: {' '.join(loose)[:40]}"
return None
def scan_line(path: str, text: str) -> list[tuple[str, str]]:
if cg.COMMENT.match(text):
return []
hits = []
if "/workflows/" in path and HOST_PORT.match(text):
hits.append(("host port", f"service publishes host port {HOST_PORT.match(text).group(1)} (shared dind)"))
return hits
# Windy Mail #147: a `:latest` build/tool image floats exactly like an
# unpinned package, and `COPY uv.lock* ./` builds WITHOUT the lock when it
# is missing instead of failing.
m = LATEST.search(text)
if m:
hits.append(("floating image", f"{m.group(1)}:latest"))
if DOCKER_FILE.search(path) and re.match(r"^\s*COPY\b", text, re.I):
globbed = [t for t in text.split() if "*" in t and LOCKNAME.search(t)]
if globbed:
hits.append(("optional lock", f"COPY {globbed[0]} (must fail if the lock is missing)"))
# Windy Git jobs get NO Docker daemon (I-5), so a docker build/compose/run
# step in CI can never pass here (orchestrator 09-23, option A). The real
# image build is the deploy step on the target host.
if "/workflows/" in path and re.search(r"uses:\s*['\"]?docker/build-push-action", text):
hits.append(("needs docker", "docker/build-push-action in CI (no Docker daemon on Windy Git; " + NO_DOCKER_FIX + ")"))
for toks in _commands(text):
low = [t.lower() for t in toks]
if "/workflows/" in path and (
low[:2] in (["docker", "build"], ["docker", "buildx"], ["docker", "run"], ["docker", "compose"])
or low[:1] == ["docker-compose"]
):
hits.append(("needs docker", f"{' '.join(low[:2])} in CI (no Docker daemon on Windy Git; " + NO_DOCKER_FIX + ")"))
continue
# pip install / python -m pip install / uv pip install
for i in range(len(low) - 1):
if os.path.basename(low[i]) in ("pip", "pip3") and low[i + 1] == "install":
prob = _pip_problem(toks[i + 2:])
if prob:
hits.append(("floating install", prob))
break
if low[:2] == ["uv", "sync"] and not ({"--locked", "--frozen"} & set(low)):
hits.append(("floating install", "uv sync without --locked/--frozen"))
if low[:1] == ["npm"] and len(low) > 1 and low[1] in ("install", "i", "add"):
pkgs = [t for t in toks[2:] if not t.startswith("-")]
if not pkgs or not all(EXACT_NPM.match(p) for p in pkgs):
hits.append(("floating install", f"npm {low[1]} {' '.join(pkgs)[:30]}".strip() + " (use npm ci)"))
if low[:2] == ["yarn", "install"] and not ({"--frozen-lockfile", "--immutable"} & set(low)):
hits.append(("floating install", "yarn install without --frozen-lockfile"))
if low[:2] == ["pnpm", "install"] and "--frozen-lockfile" not in low:
hits.append(("floating install", "pnpm install without --frozen-lockfile"))
return hits
_DISABLED: dict[str, set[str]] | None = None
def disabled_workflows() -> dict[str, set[str]]:
"""Workflow file names Gitea has DISABLED per repo (lowercased repo name).
Deploy/release workflows are disabled on Windy Git: they run on the target
host, where a Docker daemon really exists, so "needs docker" must not flag
them. One bounded query per process; on any failure nothing is excused
(flag rather than hide).
"""
global _DISABLED
if _DISABLED is not None:
return _DISABLED
_DISABLED = {}
query = ("select coalesce(json_object_agg(r.lower_name, u.config::json->'DisabledWorkflows'), '{}'::json)"
" from repo_unit u join repository r on r.id = u.repo_id"
" where u.type = 10 and u.config like '%DisabledWorkflows%';")
try:
out = subprocess.run(
["docker", "exec", "-i", "windy-git-db-1", "sh", "-c",
'psql -U "$POSTGRES_USER" -d gitea -At -v ON_ERROR_STOP=1'],
input=query, capture_output=True, text=True, check=True, timeout=30,
).stdout.strip()
_DISABLED = {k: set(v or []) for k, v in json.loads(out or "{}").items()}
except (subprocess.SubprocessError, OSError, ValueError):
pass
return _DISABLED
def _runs_here(repo: str, findings):
"""Drop "needs docker" hits in workflows that never run on Windy Git."""
if findings is None:
return None
off = disabled_workflows().get(repo.lower(), set())
return [f for f in findings if not (f.kind == "needs docker" and Path(f.path).name in off)]
def check(repo: str, sha: str, default_branch: str, is_default_head: bool):
return _runs_here(repo, _check(repo, sha, default_branch, is_default_head))
def _check(repo: str, sha: str, default_branch: str, is_default_head: bool):
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir() or not cg.fetched(bare, sha): # pushed after the fetch: next cycle
return None
allow = cg.load_allow(ALLOW_FILE)
rules = hashlib.sha256((PREFILTER + INCLUDE.pattern + EXACT_PY.pattern + EXACT_NPM.pattern).encode()).hexdigest()[:8]
fp = cg._fingerprint(allow) + ":" + rules # hashlib, not hash(): hash() is per-process random
kw = dict(line_fn=scan_line, path_ok=path_ok)
if is_default_head:
return cg.cached_scan(f"hyg-tree:{repo}:{sha}:{fp}",
lambda: cg.scan_tree(repo, bare, sha, allow, prefilter=PREFILTER, **kw))
return cg.cached_scan(f"hyg-pr:{repo}:{sha}:{fp}",
lambda: cg.scan_added(repo, bare, f"refs/heads/{default_branch}", sha, allow, **kw))
def status_for(findings, whole_tree: bool, grant=()):
"""Same contract as compute_guard.status_for: `grant` findings never block."""
scope = "in CI/Dockerfiles" if whole_tree else "added"
if not findings and grant:
g, n = grant[0], len(grant)
desc = f"⚠ WARN (Grant-owned, not blocking): {n} CI hygiene issue{'s' if n > 1 else ''} {scope}, e.g. {g.path}:{g.line} {g.match}"
return "success", desc[:140], g
if not findings:
return "success", f"OK: no floating install or host-port service {scope}", None
f = findings[0]
n = len(findings)
state = "failure" if MODE == "block" else "success"
lead = "BLOCKED" if MODE == "block" else "⚠ WARN (not blocking)"
return state, f"{lead}: {n} CI hygiene issue{'s' if n > 1 else ''} {scope}, e.g. {f.path}:{f.line} {f.match}"[:140], f
def report(repos: list[str]) -> int:
allow = cg.load_allow(ALLOW_FILE)
total = 0
for repo in repos:
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir():
print(f"## {repo}: no sync clone, skipped")
continue
head = cg._git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = cg._git(bare, "rev-parse", head).strip()
fs = _runs_here(repo, cg.scan_tree(repo, bare, sha, allow, line_fn=scan_line, path_ok=path_ok, prefilter=PREFILTER))
total += len(fs)
print(f"## {repo} ({head} {sha[:7]}): {len(fs)} issue(s)")
for f in fs:
print(f" {f.path}:{f.line} [{f.kind}] {f.match}")
print(f"TOTAL {total}")
return 0
if __name__ == "__main__":
if len(sys.argv) >= 2 and sys.argv[1] == "report":
default = os.environ.get("BRIDGE_REPOS", "").split() or sorted(
p.name.removesuffix(".git") for p in cg.WORK.glob("*.git"))
sys.exit(report(sys.argv[2:] or default))
sys.exit(__doc__)

323
scripts/compute_guard.py Normal file
View File

@@ -0,0 +1,323 @@
#!/usr/bin/env python3
"""Compute guard: Windy Mind is the ONLY door to AI compute (Grant, 2026-09-23).
Flags code that talks to an AI provider directly instead of through Windy Mind:
a provider API host, a provider SDK import or dependency, or a raw provider key
name. Direct calls skip Mind's metering, caps and live-model routing, and they
spend whichever key happens to be lying around (the audit found Grant's personal
Max OAuth token inside a platform container).
WARN-ONLY for now: the bridge posts `windy-git/compute-guard` as success with a
"⚠ WARN" description, so nothing turns red. `COMPUTE_GUARD_MODE=block` flips
findings to failure once the repos are clean (orchestrator's call).
- PR heads: only lines the PR ADDS (vs its merge-base with the default branch).
- Default-branch head: the whole tree (the baseline, and what `report` prints).
Exceptions live in ONE file, ci/compute-guard-allow.yml, each with a reason.
Tests, docs, lockfiles, vendored code and CI config are never scanned.
Reads the sync's bare GitHub clones on Veron (no docker exec: IO-stall lesson).
python3 scripts/compute_guard.py report [repo ...] # whole-tree findings on each default branch
"""
from __future__ import annotations
import fnmatch
import hashlib
import json
import os
import re
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
import yaml
ROOT = Path(__file__).resolve().parents[1]
ALLOW_FILE = Path(os.environ.get("COMPUTE_GUARD_ALLOW", ROOT / "ci" / "compute-guard-allow.yml"))
WORK = Path(os.environ.get("SYNC_WORK", "/srv/windygit/sync"))
CACHE = Path(os.environ.get("COMPUTE_GUARD_CACHE", "/var/lib/windy-git/compute-guard-cache.json"))
MODE = os.environ.get("COMPUTE_GUARD_MODE", "warn") # warn | block
HOSTS = [
"api.anthropic.com", "api.openai.com", "api.groq.com",
"generativelanguage.googleapis.com", "api.mistral.ai", "api.perplexity.ai",
"openrouter.ai", "api.together.xyz", "api.together.ai", "api.cerebras.ai",
"api.sambanova.ai", "api.deepseek.com", "api.x.ai", "api.cohere.ai",
"api.cohere.com", "api.fireworks.ai", "api.replicate.com",
"api-inference.huggingface.co",
]
KEYS = [
"ANTHROPIC_API_KEY", "ANTHROPIC_OAUTH_TOKEN", "ANTHROPIC_AUTH_TOKEN",
"OPENAI_API_KEY", "GROQ_API_KEY", "GEMINI_API_KEY", "GOOGLE_GENERATIVE_AI_API_KEY",
"GOOGLE_AI_API_KEY", "MISTRAL_API_KEY", "PERPLEXITY_API_KEY", "PPLX_API_KEY",
"OPENROUTER_API_KEY", "TOGETHER_API_KEY", "CEREBRAS_API_KEY", "SAMBANOVA_API_KEY",
"DEEPSEEK_API_KEY", "XAI_API_KEY", "COHERE_API_KEY", "FIREWORKS_API_KEY",
"REPLICATE_API_TOKEN",
]
PY_SDKS = r"anthropic|openai|groq|mistralai|cohere|google\.generativeai|google\.genai|together|cerebras|litellm"
JS_SDKS = (r"@anthropic-ai/sdk|openai|groq-sdk|@google/generative-ai|@google/genai|@mistralai/mistralai"
r"|cohere-ai|together-ai|@ai-sdk/(?:anthropic|openai|groq|google|mistral)")
RULES: list[tuple[str, re.Pattern]] = [
("provider host", re.compile("|".join(re.escape(h) for h in HOSTS))),
# Grant via Boss 10-01: compute = Windy Mind. A NEW reference to an Ollama port (Veron's :11434) is a
# direct call around Mind's metering/caps. WARN-only, never red, and only for lines a PR ADDS.
("veron ollama", re.compile(r"(?::|%3[aA])11434(?![0-9])")),
("provider key", re.compile(r"\b(?:" + "|".join(KEYS) + r")\b")),
("provider SDK", re.compile(rf"^\s*(?:from|import)\s+(?:{PY_SDKS})(?:\s|\.|$|,)")),
("provider SDK", re.compile(rf"""(?:from\s+|require\(\s*|import\(\s*)['"](?:{JS_SDKS})(?:/[^'"]*)?['"]""")),
# dependency manifests: package.json keys, requirements / pyproject lines
("provider SDK dep", re.compile(rf'''^\s*"(?:{JS_SDKS})"\s*:''')),
("provider SDK dep", re.compile(rf'''^\s*["']?(?:{PY_SDKS.replace(chr(92) + ".", "-")})(?:\[[^\]]*\])?\s*(?:[<>=~!]=?|["',]|$)''')),
]
# Kinds that never block (even in MODE=block) and are only judged on ADDED lines, never the baseline tree.
WARN_ONLY_KINDS = {"veron ollama"}
OLLAMA_MSG = "compute = Windy Mind (endpoint + key); do not call Veron's Ollama directly"
DEP_FILES = re.compile(r"(^|/)(package\.json|requirements[^/]*\.txt|pyproject\.toml|setup\.cfg|Pipfile)$")
# Never scanned: tests, docs, lockfiles, vendored/built code, CI config.
SKIP = re.compile(
r"(^|/)(tests?|__tests__|spec|docs?|node_modules|vendor|dist|build|\.github|\.gitea)/"
r"|(^|/)(test_[^/]*|[^/]*_test\.py|conftest\.py|[^/]*\.(test|spec)\.[cm]?[jt]sx?)$"
r"|\.(md|mdx|rst|txt|lock|snap|svg|png|jpg|pdf)$"
r"|(^|/)(package-lock\.json|pnpm-lock\.yaml|yarn\.lock|uv\.lock|poetry\.lock|Cargo\.lock)$"
)
@dataclass(frozen=True)
class Finding:
path: str
line: int
kind: str
match: str
def load_allow(path: Path = ALLOW_FILE) -> list[dict]:
data = yaml.safe_load(path.read_text()) or {}
entries = data.get("allow") or []
for e in entries: # a reason per entry is the whole point of the file
if not (e.get("repo") and e.get("paths") and str(e.get("reason", "")).strip()):
raise ValueError(f"allow entry needs repo, paths and a reason: {e}")
return entries
def allowed(repo: str, path: str, allow: list[dict], text: str | None = None) -> bool:
"""An entry may carry `matches:` (regexes): then only lines matching one of
them are allowed, so an allowed file can't smuggle in a NEW call (e.g. an
OAuth sign-in endpoint is allowed, an inference endpoint in the same file
still flags). Entries without `matches` cover the whole path."""
for e in allow:
if e["repo"] != repo or not any(fnmatch.fnmatch(path, g) for g in e["paths"]):
continue
pats = e.get("matches")
if not pats:
return True
if text is not None and any(re.search(rx, text) for rx in pats):
return True
return False
COMMENT = re.compile(r"^\s*(?:#|//|/\*|\*|<!--)")
def scan_line(path: str, text: str) -> list[tuple[str, str]]:
# A comment is not a call: "the ANTHROPIC_OAUTH_TOKEN setting was removed"
# (windy-search) must not count, nor a commented-out `# OPENAI_API_KEY=`.
if COMMENT.match(text):
return []
hits = []
for kind, rx in RULES:
if kind == "provider SDK dep" and not DEP_FILES.search(path):
continue
m = rx.search(text)
if m:
hits.append((kind, m.group(0).strip()[:60]))
return hits
def _git(bare: Path, *args: str) -> str:
return subprocess.run(
["git", "--git-dir", str(bare), *args],
capture_output=True, text=True, check=True, timeout=120,
).stdout
def _default_path_ok(path: str) -> bool:
return not SKIP.search(path)
def scan_tree(repo: str, bare: Path, sha: str, allow: list[dict], *, line_fn=None,
path_ok=None, prefilter: str | None = None) -> list[Finding]:
"""Every line in the tree at `sha` (default branch: the baseline)."""
# A cheap prefilter by git, then the real rules in Python.
# Other guards (ci_hygiene) reuse this walker with their own line rules.
line_fn = line_fn or scan_line
path_ok = path_ok or _default_path_ok
pre = prefilter or "|".join([re.escape(h) for h in HOSTS] + KEYS + [
"anthropic", "openai", "groq", "mistral", "generativeai", "genai", "cohere",
"together", "cerebras", "litellm"])
try:
out = _git(bare, "grep", "-nIE", "-e", pre, sha, "--", ".")
except subprocess.CalledProcessError as e:
if e.returncode == 1: # no matches
return []
raise
found = []
for raw in out.splitlines():
# <sha>:<path>:<line>:<text>
try:
_, path, line, text = raw.split(":", 3)
except ValueError:
continue
if not path_ok(path) or allowed(repo, path, allow, text):
continue
for kind, match in line_fn(path, text):
found.append(Finding(path, int(line), kind, match))
return found
def scan_added(repo: str, bare: Path, base_ref: str, sha: str, allow: list[dict], **kw) -> list[Finding]:
"""Only the lines a PR adds, vs its merge-base with the default branch."""
mb = _git(bare, "merge-base", base_ref, sha).strip()
diff = _git(bare, "diff", "-U0", "--no-color", "--no-ext-diff", mb, sha)
return parse_added(repo, diff, allow, **kw)
HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@")
def parse_added(repo: str, diff: str, allow: list[dict], *, line_fn=None, path_ok=None) -> list[Finding]:
line_fn = line_fn or scan_line
path_ok = path_ok or _default_path_ok
found, path, line = [], None, 0
for raw in diff.splitlines():
if raw.startswith("+++ "):
p = raw[4:]
path = None if p == "/dev/null" else p[2:] if p.startswith("b/") else p
continue
m = HUNK.match(raw)
if m:
line = int(m.group(1))
continue
if path is None or raw.startswith("--- "):
continue
if raw.startswith("+"):
if path_ok(path) and not allowed(repo, path, allow, raw[1:]):
for kind, match in line_fn(path, raw[1:]):
found.append(Finding(path, line, kind, match))
line += 1
return found
# ---- cache: a tree scan runs once per (repo, sha, rules+allow) --------------
def _fingerprint(allow: list[dict]) -> str:
return hashlib.sha256(
json.dumps([HOSTS, KEYS, PY_SDKS, JS_SDKS, SKIP.pattern, allow], sort_keys=True).encode()
).hexdigest()[:16]
def cached_scan(key: str, fn) -> list[Finding]:
try:
cache = json.loads(CACHE.read_text())
except (OSError, ValueError):
cache = {}
if key in cache:
return [Finding(**f) for f in cache[key]]
result = fn()
cache[key] = [f.__dict__ for f in result]
if len(cache) > 2000: # keep it small: newest entries win
cache = dict(list(cache.items())[-1000:])
try:
CACHE.parent.mkdir(parents=True, exist_ok=True)
tmp = CACHE.with_suffix(".tmp")
tmp.write_text(json.dumps(cache))
tmp.replace(CACHE)
except OSError:
pass
return result
def fetched(bare: Path, sha: str) -> bool:
"""Is `sha` in the sync clone yet? The bridge learns PR / default heads from
GitHub's API AFTER the sync fetched, so a push in between is simply not here
until the next 5-min cycle. That is a race, not an error: skip quietly."""
try:
_git(bare, "cat-file", "-e", f"{sha}^{{commit}}")
return True
except subprocess.CalledProcessError:
return False
def check(repo: str, sha: str, default_branch: str, is_default_head: bool) -> list[Finding] | None:
"""Findings for one commit, or None when the guard can't run (never a fake OK)."""
bare = WORK / f"{repo}.git"
if not bare.is_dir() or not fetched(bare, sha):
return None
allow = load_allow()
fp = _fingerprint(allow)
if is_default_head:
return cached_scan(f"tree:{repo}:{sha}:{fp}",
lambda: [f for f in scan_tree(repo, bare, sha, allow) if f.kind not in WARN_ONLY_KINDS])
return cached_scan(
f"pr:{repo}:{sha}:{fp}",
lambda: scan_added(repo, bare, f"refs/heads/{default_branch}", sha, allow),
)
def status_for(findings: list[Finding], whole_tree: bool,
grant: list[Finding] = ()) -> tuple[str, str, Finding | None]:
"""(state, description, first finding) for the GitHub commit status.
`findings` = lane-owned (these block in MODE=block); `grant` = findings in
Grant-owned code (ci/grant-owned.yml): always WARN, never red (orchestrator
09-23: his desktop work is never blocked by us)."""
scope = "in tree" if whole_tree else "added"
soft = [f for f in findings if f.kind in WARN_ONLY_KINDS]
findings = [f for f in findings if f.kind not in WARN_ONLY_KINDS]
if not findings and not grant and soft:
f = soft[0]
return "success", f"⚠ WARN: new Veron Ollama ref {f.path}:{f.line}. {OLLAMA_MSG}"[:140], f
if not findings and grant:
g, n = grant[0], len(grant)
desc = (f"⚠ WARN (Grant-owned, not blocking): {n} direct AI-provider use{'s' if n > 1 else ''} "
f"{scope}, e.g. {g.path}:{g.line} {g.match}")
return "success", desc[:140], g
if not findings:
what = "no direct AI-provider use in tree" if whole_tree else "no direct AI-provider use added"
return "success", f"OK: {what} (Windy Mind is the only door)", None
f = findings[0]
n = len(findings)
state = "failure" if MODE == "block" else "success"
lead = "BLOCKED" if MODE == "block" else "⚠ WARN (not blocking)"
desc = f"{lead}: {n} direct AI-provider use{'s' if n > 1 else ''} {scope}, e.g. {f.path}:{f.line} {f.match}"
return state, desc[:140], f
def report(repos: list[str]) -> int:
allow = load_allow()
total = 0
for repo in repos:
bare = WORK / f"{repo}.git"
if not bare.is_dir():
print(f"## {repo}: no sync clone, skipped")
continue
head = _git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = _git(bare, "rev-parse", head).strip()
fs = scan_tree(repo, bare, sha, allow)
total += len(fs)
print(f"## {repo} ({head} {sha[:7]}): {len(fs)} finding(s)")
for f in fs:
print(f" {f.path}:{f.line} [{f.kind}] {f.match}")
print(f"TOTAL {total}")
return 0
if __name__ == "__main__":
if len(sys.argv) >= 2 and sys.argv[1] == "report":
default = os.environ.get("BRIDGE_REPOS", "").split() or sorted(
p.name.removesuffix(".git") for p in WORK.glob("*.git"))
sys.exit(report(sys.argv[2:] or default))
sys.exit(__doc__)

23
scripts/drill_cross_host.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Cross-host restore drill on Windy 0: ONLY lockbox + R2, nothing from Veron. Values never printed.
# Usage: bash drill_cross_host.sh (needs lockbox keys RESTIC_WINDYGIT_PASSWORD, WINDYGIT_R2_ACCESS_KEY_ID, WINDYGIT_R2_SECRET_ACCESS_KEY, WINDYGIT_R2_ENDPOINT)
set -euo pipefail
umask 077; W=$(mktemp -d ~/.cache/wg-xdrill.XXXXXX)
trap 'docker rm -f wg-xdrill-pg >/dev/null 2>&1 || true; rm -rf "$W"' EXIT
lockbox-get RESTIC_WINDYGIT_PASSWORD "$W/pw" >/dev/null
lockbox-get WINDYGIT_R2_ACCESS_KEY_ID "$W/ak" >/dev/null; lockbox-get WINDYGIT_R2_SECRET_ACCESS_KEY "$W/sk" >/dev/null; lockbox-get WINDYGIT_R2_ENDPOINT "$W/ep" >/dev/null
export RESTIC_PASSWORD_FILE="$W/pw" AWS_ACCESS_KEY_ID="$(cat "$W/ak")" AWS_SECRET_ACCESS_KEY="$(cat "$W/sk")"
export RESTIC_REPOSITORY="s3:$(cat "$W/ep")/windy-git-backups/restic"
restic snapshots --tag windygit-state --compact | tail -3
restic restore latest --tag windygit-state --target "$W/r" --include /var/backups/windygit-state --include /srv/windygit/git/gitea/conf --quiet
ls -l "$W/r/var/backups/windygit-state" | awk 'NR>1{print $5, $NF}'
docker run -d --name wg-xdrill-pg -e POSTGRES_PASSWORD="$(python3 -c 'import secrets;print(secrets.token_hex(12))')" -e POSTGRES_USER=drill postgres:16-alpine >/dev/null
for i in $(seq 1 30); do docker exec wg-xdrill-pg pg_isready -U drill >/dev/null 2>&1 && break; sleep 2; done
for db in gitea windygit; do
docker exec wg-xdrill-pg psql -U drill -d postgres -qc "create database $db"
docker exec -i wg-xdrill-pg pg_restore -U drill -d $db --no-owner --no-privileges < "$W/r/var/backups/windygit-state/$db.dump" 2>&1 | grep -v "already exists" | head -2 || true
done
for t in repository issue pull_request '"user"' external_login_user action_run action_run_job; do
echo "$t restored=$(docker exec wg-xdrill-pg psql -U drill -d gitea -Atc "select count(*) from $t")"
done
echo "cross-host drill OK (cleaned up)"

153
scripts/env_names.py Normal file
View File

@@ -0,0 +1,153 @@
#!/usr/bin/env python3
"""env-names: list environment variable NAMES only (Boss ruling 10-01, house rule 10).
env-names <docker container | systemd unit | env file> [--host H] [--sudo] [--hash]
env-names A --compare B [--host H] [--host2 H2]
Prints NAME, set|empty, and value LENGTH. Never a value or fragment. --hash adds sha256[:8]
(compare two places for equality; a hash of a weak value can be guessed, so use it for
real secrets only). --compare prints SAME / DIFFERENT / only-in-A / only-in-B per name
(equality by full-value hash, nothing else shown). Values live in memory only.
Targets: an existing file (dotenv style) | a docker container name | a systemd unit
(Environment= + EnvironmentFile=; --sudo to read root-only files). --host runs the docker /
systemctl / file read over ssh (alias from ~/.ssh/config).
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import shlex
import subprocess
import sys
KV = ("=",)
def run(cmd: list[str], host: str | None, sudo: bool = False) -> tuple[int, str]:
if sudo:
cmd = ["sudo", "-n", *cmd]
if host:
cmd = ["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=10", host, shlex.join(cmd)]
r = subprocess.run(cmd, capture_output=True, text=True, errors="ignore", timeout=60)
return r.returncode, r.stdout
def parse_dotenv(text: str) -> dict[str, str]:
out: dict[str, str] = {}
for raw in text.splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
if line.startswith("export "):
line = line[7:].lstrip()
k, v = line.split("=", 1)
k = k.strip()
v = v.strip()
if len(v) >= 2 and v[0] == v[-1] and v[0] in "\"'":
v = v[1:-1]
if k.replace("_", "").isalnum() and not k[0].isdigit():
out[k] = v
return out
def from_file(path: str, host: str | None, sudo: bool) -> dict[str, str] | None:
if host or sudo:
rc, out = run(["cat", path], host, sudo)
return parse_dotenv(out) if rc == 0 else None
try:
with open(path, errors="ignore") as fh:
return parse_dotenv(fh.read())
except OSError:
return None
def from_docker(name: str, host: str | None, sudo: bool) -> dict[str, str] | None:
rc, out = run(["docker", "inspect", "-f", "{{json .Config.Env}}", name], host, sudo)
if rc != 0 or not out.strip():
return None
try:
items = json.loads(out)
except ValueError:
return None
return {k: v for k, _, v in (i.partition("=") for i in (items or []))}
def from_systemd(unit: str, host: str | None, sudo: bool) -> dict[str, str] | None:
rc, out = run(["systemctl", "show", unit, "-p", "Environment", "-p", "EnvironmentFiles"], host)
if rc != 0 or "LoadState=not-found" in out:
return None
env: dict[str, str] = {}
files: list[str] = []
for line in out.splitlines():
if line.startswith("Environment="):
for tok in shlex.split(line[len("Environment="):]):
k, _, v = tok.partition("=")
env[k] = v
elif line.startswith("EnvironmentFiles="):
f = line[len("EnvironmentFiles="):].split(" (")[0].strip().lstrip("-")
if f:
files.append(f)
for f in files: # later files override earlier, like systemd
d = from_file(f, host, sudo)
if d is None:
print(f"# note: EnvironmentFile {f} unreadable (try --sudo)", file=sys.stderr)
else:
env.update(d)
return env
def load(target: str, host: str | None, sudo: bool) -> dict[str, str] | None:
if (not host and os.path.isfile(target)) or target.startswith(("/", "./", "~")):
return from_file(os.path.expanduser(target), host, sudo)
if target.endswith((".service", ".timer", ".socket")):
return from_systemd(target, host, sudo)
return from_docker(target, host, sudo) or from_systemd(target, host, sudo)
def sh(v: str) -> str:
return hashlib.sha256(v.encode()).hexdigest()
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="env-names", description="env var NAMES only")
ap.add_argument("target")
ap.add_argument("--host")
ap.add_argument("--host2", help="ssh host for the --compare target")
ap.add_argument("--sudo", action="store_true")
ap.add_argument("--hash", action="store_true", help="add sha256[:8] per variable")
ap.add_argument("--compare", metavar="TARGET2")
a = ap.parse_args(argv)
env = load(a.target, a.host, a.sudo)
if env is None:
print(f"error: could not read {a.target!r} (file, docker container or systemd unit)")
return 2
if a.compare:
env2 = load(a.compare, a.host2 or a.host, a.sudo)
if env2 is None:
print(f"error: could not read {a.compare!r}")
return 2
for k in sorted(set(env) | set(env2)):
if k not in env2:
print(f"{k:<40} only-in-A")
elif k not in env:
print(f"{k:<40} only-in-B")
else:
print(f"{k:<40} {'SAME' if sh(env[k]) == sh(env2[k]) else 'DIFFERENT'}"
f" (len {len(env[k])} vs {len(env2[k])})")
return 0
for k in sorted(env):
v = env[k]
extra = f" sha256:{sh(v)[:8]}" if a.hash and v else ""
print(f"{k:<40} {'set ' if v else 'empty'} len={len(v)}{extra}")
print(f"# {len(env)} variable(s); values never printed")
return 0
if __name__ == "__main__":
try:
sys.exit(main())
except Exception as e: # never a traceback
print(f"error: {type(e).__name__}")
sys.exit(2)

161
scripts/guards_report.py Normal file
View File

@@ -0,0 +1,161 @@
#!/usr/bin/env python3
"""Live status of the repo guards (compute-guard + ci-hygiene + secret-guard) as one markdown page.
Scans every bridged repo's DEFAULT branch with both guards and renders what is
left, per repo and owner lane. Findings in code Grant owns (ci/grant-owned.yml:
windy-pro's desktop app and its build jobs) are listed in their OWN section and
do not count against "ready to block": those are proposals for Grant, not a
lane's fix (orchestrator, 09-23).
sudo python3 scripts/guards_report.py > GUARDS_STATUS.md
"""
from __future__ import annotations
import fnmatch
import os
import re
import sys
import time
from pathlib import Path
import yaml
sys.path.insert(0, str(Path(__file__).resolve().parent))
import ci_hygiene as hy # noqa: E402
import compute_guard as cg # noqa: E402
import secret_guard as sgd # noqa: E402
ROOT = Path(__file__).resolve().parents[1]
OWNED = Path(os.environ.get("GRANT_OWNED", ROOT / "ci" / "grant-owned.yml"))
REPOS = os.environ.get("BRIDGE_REPOS", "").split() or [
"windy-chat", "windy-mail", "windy-calendar", "Windy-Clone", "WindyCloud", "windy-search",
"windy-connect", "windy-drops", "windy-code-web", "windy-code", "windy-traveler",
"windy-registry", "eternitas", "windy-translate", "windytranslate-site", "windytraveler-site",
"windy-hand", "windy-cloud-sites", "windy-cloud-domains", "windy-cloud-vps", "windytalk",
"windy-pro", "windy-mind", "windy-git", "windy-inbox"]
# Owner lane per repo = the session name to message (orchestrator routing, 09-23 ~22:45Z:
# hub/account-server/dashboard/site -> "Windy Hub"; windy-calendar only -> "Windy Calender";
# windy-admin/telemetry -> "Windy Admin"). windy-pro here = its server/web side; the desktop
# app is Grant-owned and listed in its own section below.
OWNERS = {
"windy-chat": "Windy Chat", "windy-mail": "Windy Mail", "windy-calendar": "Windy Calender",
"Windy-Clone": "Windy Clone", "WindyCloud": "Windy Cloud", "windy-cloud-sites": "Windy Cloud",
"windy-cloud-domains": "Windy Cloud", "windy-cloud-vps": "Windy Cloud", "windy-search": "Windy Search",
"windy-connect": "Windy Connect", "windy-drops": "Windy Drops", "windy-registry": "Windy Drops",
"windy-code-web": "Windy Code", "windy-code": "Windy Code", "windy-traveler": "Windy Traveler",
"windytraveler-site": "Windy Traveler", "eternitas": "Eternitas", "windy-translate": "Windy Translate",
"windytranslate-site": "Windy Translate", "windy-hand": "Windy Hand", "windytalk": "Windy Talk",
"windy-pro": "Windy Hub", "windy-mind": "WIndy Mind", "windy-git": "Windy Git",
"windy-inbox": "Windy Drops"} # Windy Inbox build lead (09-24)
JOB = re.compile(r"^ ([A-Za-z0-9_-]+):\s*$")
def job_of(text: str, line: int) -> str | None:
"""The workflow job a line belongs to (2-space keys under `jobs:`)."""
in_jobs, job = False, None
for i, raw in enumerate(text.splitlines(), 1):
if raw.startswith("jobs:"):
in_jobs = True
elif in_jobs and JOB.match(raw):
job = JOB.match(raw).group(1)
elif raw and not raw[0].isspace() and not raw.startswith("jobs:"):
in_jobs = False
if i == line:
return job if in_jobs else None
return None
def grant_owned(repo: str, path: str, job: str | None, owned: list[dict]) -> bool:
for e in owned:
if e["repo"] != repo:
continue
if any(fnmatch.fnmatch(path, g) for g in e.get("paths") or []):
return True
if job and job in (e.get("jobs") or {}).get(path, []):
return True
return False
def split_grant(repo: str, sha: str, findings: list) -> tuple[list, list]:
"""(lane-owned, Grant-owned) findings at `sha`, by ci/grant-owned.yml, with
workflow lines attributed to their job exactly as the status page does."""
owned = (yaml.safe_load(OWNED.read_text()) or {}).get("grant_owned") or []
if not any(e["repo"] == repo for e in owned):
return list(findings), []
bare = cg.WORK / f"{repo}.git"
texts: dict[str, str] = {}
lane, grant = [], []
for f in findings:
job = None
if "/workflows/" in f.path:
if f.path not in texts:
texts[f.path] = cg._git(bare, "show", f"{sha}:{f.path}")
job = job_of(texts[f.path], f.line)
(grant if grant_owned(repo, f.path, job, owned) else lane).append(f)
return lane, grant
def scan(repo: str, owned: list[dict]):
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir():
return None
head = cg._git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = cg._git(bare, "rev-parse", head).strip()
out = {"sha": sha, "compute": [], "hygiene": [], "secrets": []}
texts: dict[str, str] = {}
for key, fs in (("compute", cg.check(repo, sha, head, True) or []),
("hygiene", hy.check(repo, sha, head, True) or []),
("secrets", sgd.check(repo, sha, head, True) or [])):
for f in fs:
job = None
if "/workflows/" in f.path:
if f.path not in texts:
texts[f.path] = cg._git(bare, "show", f"{sha}:{f.path}")
job = job_of(texts[f.path], f.line)
out[key].append((f, job, grant_owned(repo, f.path, job, owned)))
return out
def render(results: dict) -> str:
now = time.strftime("%Y-%m-%d %H:%MZ", time.gmtime())
lane = {k: 0 for k in ("compute", "hygiene", "secrets")}
grant = {k: 0 for k in ("compute", "hygiene", "secrets")}
for r in results.values():
for k in lane:
lane[k] += sum(1 for _, _, g in r.get(k, []) if not g)
grant[k] += sum(1 for _, _, g in r.get(k, []) if g)
L = [f"# Repo guards: live status (generated {now}; windy-git scripts/guards_report.py)",
"_Default branches only. WARN-only today; the orchestrator says \"block\" per guard when its LANE column is 0. "
"Grant-owned code (ci/grant-owned.yml) is listed separately and never holds up a block._", "",
"| Guard | Lane-owned findings | Grant-owned (proposals) | Ready to block? |", "|---|---|---|---|",
f"| compute-guard (Mind is the only door) | {lane['compute']} | {grant['compute']} | {'✅ YES' if lane['compute'] == 0 else '❌ not yet'} |",
f"| ci-hygiene (house rule 6) | {lane['hygiene']} | {grant['hygiene']} | {'✅ YES' if lane['hygiene'] == 0 else '❌ not yet'} |",
f"| secret-guard (no credentials in repos; hash only) | {lane['secrets']} | {grant['secrets']} | {'✅ YES' if lane['secrets'] == 0 else '❌ not yet'} |",
"", "## By repo (lane-owned)", "| Repo | owner | head | compute | hygiene | secrets | first items |", "|---|---|---|---|---|---|---|"]
for repo, r in sorted(results.items()):
c = [x for x in r["compute"] if not x[2]]
h = [x for x in r["hygiene"] if not x[2]]
s = [x for x in r.get("secrets", []) if not x[2]]
items = "; ".join(f"`{f.path}:{f.line}` {f.match}" for f, _, _ in (s + c + h)[:3]) or "clean ✅"
L.append(f"| {repo} | {OWNERS.get(repo, '?')} | {r['sha'][:7]} | {len(c)} | {len(h)} | {len(s)} | {items} |")
L += ["", "## Grant-owned (windy-pro desktop app + its build jobs): proposals only, not blocking"]
g = [(repo, f, job) for repo, r in sorted(results.items()) for k in ("compute", "hygiene", "secrets")
for f, job, own in r.get(k, []) if own]
L += [f"- {repo} `{f.path}:{f.line}`{f' (job {job})' if job else ''}: {f.match}" for repo, f, job in g] or ["- none"]
return "\n".join(L) + "\n"
def main() -> int:
owned = (yaml.safe_load(OWNED.read_text()) or {}).get("grant_owned") or []
results = {}
for repo in REPOS:
r = scan(repo, owned)
if r is not None:
results[repo] = r
sys.stdout.write(render(results))
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -43,7 +43,12 @@ import urllib.request
#
# Bulk import belongs on the host anyway: no hairpin through the edge, no
# Cloudflare ~100s proxy ceiling (G4A.5) on a large clone. Run this on Veron 1.
GITEA = os.environ.get("GITEA_BASE_URL", "http://localhost:3080")
#
# 🔴 Deliberately NOT `GITEA_BASE_URL`: the deploy `.env` sets that to
# `http://gitea:3000` for the API container, and sourcing `.env` on the host
# made this script die on DNS *after* a caller had already deleted the mirror it
# was meant to replace (2026-09-23).
GITEA = os.environ.get("IMPORT_GITEA_URL", "http://localhost:3080")
GITEA_TOKEN = os.environ.get("GITEA_ADMIN_TOKEN", "")
GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")
GITHUB_OWNER = os.environ.get("GITHUB_OWNER", "sneakyfree")
@@ -93,6 +98,55 @@ def _api(method: str, path: str, body: dict | None = None) -> tuple[int, dict]:
return e.code, {"message": raw[:300]}
def all_repo_names() -> list[str]:
out = subprocess.run(
["gh", "repo", "list", GITHUB_OWNER, "--limit", "300",
"--json", "name,isArchived"],
capture_output=True, text=True, check=True,
)
return sorted(r["name"] for r in json.loads(out.stdout) if not r["isArchived"])
def import_everything_as_mirrors() -> int:
"""Bulk DR copy: every repo, as a READ-ONLY pull mirror.
Mirrors are the right shape for the bulk, and the reason is safety rather
than tidiness. Sampling 40 repos found **18 carrying deploy / release /
publish workflows that trigger on `push:`** — roughly 63 across the account.
Importing those as writable repos with Actions enabled would arm sixty-odd
production deploy triggers on Veron 1, each of which would then have to be
disarmed by hand.
A pull mirror cannot run Actions at all, so the bulk import carries **zero**
deploy risk, and Gitea does the syncing itself with no script and no timer.
What you get is a complete, current, second copy of the whole account.
Converting one to a writable CI repo is then a deliberate per-repo act:
delete, re-import with `mirror=false`, review its workflows, disable the
deploying ones. That is the moment to make that judgement — not in bulk,
sixty times, by accident.
"""
names = all_repo_names()
existing = 0
done = 0
failed = []
print(f"{len(names)} active repos on GitHub. Importing missing ones as read-only mirrors.\n")
for n in names:
status, _ = _api("GET", f"/repos/{OWNER}/{n}")
if status == 200:
existing += 1
continue
if import_repo(n, mirror=True):
done += 1
else:
failed.append(n)
print(f"\n already present: {existing}")
print(f" newly mirrored: {done}")
if failed:
print(f" FAILED ({len(failed)}): {', '.join(failed[:10])}")
return 1 if failed else 0
def list_candidates() -> None:
"""Private repos whose CI cannot run on GitHub at all."""
out = subprocess.run(
@@ -158,23 +212,29 @@ def main() -> int:
"--mirror", action="store_true",
help="import read-only pull mirrors instead of writable repos. NOTE: mirrors CANNOT run CI.",
)
ap.add_argument(
"--all-as-mirrors", action="store_true",
help="bulk DR copy: every active repo on the account, read-only, no CI, zero deploy risk",
)
args = ap.parse_args()
if args.list_candidates:
list_candidates()
return 0
if args.all_as_mirrors:
return import_everything_as_mirrors()
targets = SAFE_ORDER if args.safe_batch else args.repos
if not targets:
ap.error("name a repo, or pass --safe-batch / --list-candidates")
if "windy-pro" in targets:
sys.exit(
"REFUSING windy-pro. Six checkouts exist, the build counter has forked "
"three ways (main 12 / overnight 34 / wave-44 56), and two sessions "
"recorded different HEADs hours apart. Resolve which is current and "
"write it down BEFORE importing (G11.5)."
)
# G11.5 RESOLVED 2026-09-23 (lane 8c, ~/windy-orchestra/WINDYPRO_CHECKOUTS.md):
# a read-only audit of all 14 windy-pro checkouts on 5 machines found GitHub
# main is canonical (Kit 0 prod and Windy 0 sit exactly on it; the others are
# stale, not divergent). Phase 1 keeps GitHub the source of truth anyway, so a
# writable Windy Git copy is CI only. Its six deploy/release workflows must be
# disabled on import — see docs/CUTOVER.md.
if args.mirror:
print("mirror mode: repos will be read-only and will NOT run CI.\n")

14
scripts/install_secret_tools.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Install the shared hash-only secret tools on THIS machine (Windy 0): secret-scan + env-names.
# Source of truth is this repo (scripts/); re-run after a pull to update.
set -euo pipefail
here=$(cd "$(dirname "$0")" && pwd)
dest="$HOME/.local/share/secret-tools"
mkdir -p "$dest" "$HOME/.local/bin"
cp "$here/secret_shapes.py" "$here/secret_scan.py" "$here/env_names.py" "$here/lockbox_put.py" "$here/lockbox_names.py" "$dest/"
for pair in "secret-scan:secret_scan.py" "env-names:env_names.py" "lockbox-put:lockbox_put.py" "lockbox-names:lockbox_names.py"; do
n=${pair%%:*}; f=${pair##*:}
printf '#!/usr/bin/env bash\nexec python3 "%s/%s" "$@"\n' "$dest" "$f" > "$HOME/.local/bin/$n"
chmod 755 "$HOME/.local/bin/$n"
done
echo "installed secret-scan, env-names, lockbox-put and lockbox-names (shapes from secret_shapes.py, same as secret-guard)"

134
scripts/lockbox_names.py Normal file
View File

@@ -0,0 +1,134 @@
#!/usr/bin/env python3
"""lockbox-names: DISCOVER what the lockbox holds without reading it (Boss rule 10-01).
lockbox-names [REGEX] (case-insensitive; matches the section heading or the label)
Prints one row per entry: SECTION HEADING | LABEL | kind | resolvable
kind env = `KEY=value` line md = `- **`KEY`**: `value`` line
label = a bold prose label (`**Password (X):** ...`) file = secrets/**/*.env key
resolvable yes = `lockbox-get LABEL FILE` returns exactly one value
dup = defined with 2+ different values (lockbox-get refuses)
no = a prose-only label, or not an exact key
NEVER prints a value or any prose after a label. A label or heading that itself looks
like a secret (secret_shapes) is replaced by <secret-shaped>. Reads the COMMITTED lockbox at
origin/main (like lockbox-get; LOCKBOX_REF=<ref> or WORKTREE overrides). Memory only, stdout only.
"""
from __future__ import annotations
import hashlib
import os
import re
import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import secret_shapes as ss # noqa: E402
REPO = os.environ.get("LOCKBOX_REPO", os.path.expanduser("~/kit-army-config"))
REF = os.environ.get("LOCKBOX_REF", "origin/main")
HEAD = re.compile(r"^#{1,6}\s+(.*\S)\s*$")
ENV = re.compile(r"^([A-Z][A-Z0-9_]{2,})=(.*)$")
MD = re.compile(r"^\s*[-*]?\s*\*\*`([A-Za-z0-9_]+)`\*\*\s*:\s*`([^`]+)`")
LABEL = re.compile(r"\*\*([^*`]{2,70}?)\*\*")
def git(*a: str) -> subprocess.CompletedProcess:
return subprocess.run(["git", "-C", REPO, *a], capture_output=True, text=True, errors="ignore")
def read_sources() -> dict[str, str]:
"""{path: text} for ACCESS_LOCKBOX.md and secrets/**/*.env."""
if REF == "WORKTREE":
out = {}
for p in [Path(REPO, "ACCESS_LOCKBOX.md"), *Path(REPO, "secrets").rglob("*.env")]:
if p.is_file():
out[str(p.relative_to(REPO))] = p.read_text(errors="ignore")
return out
if REF.startswith("origin/"):
git("fetch", "-q", "origin", REF.split("/", 1)[1])
names = ["ACCESS_LOCKBOX.md"] + [
p for p in git("ls-tree", "-r", "--name-only", REF, "--", "secrets").stdout.splitlines() if p.endswith(".env")]
out = {}
for n in names:
r = git("show", f"{REF}:{n}")
if r.returncode == 0:
out[n] = r.stdout
return out
def safe(text: str, limit: int = 70) -> str:
text = re.sub(r"\s+", " ", text).strip()
return "<secret-shaped>" if ss.find(text) else text[:limit]
def h(v: str) -> str:
return hashlib.sha256(v.strip().strip('"').strip("'").encode()).hexdigest()[:16]
def collect(src: dict[str, str]):
"""(rows, values) where values[KEY] = {hash,...} for resolvability; nothing printed from it."""
rows, values = [], {}
for path, text in src.items():
section = path
for line in text.splitlines():
m = HEAD.match(line) if path.endswith(".md") else None
if m:
section = safe(m.group(1), 90)
continue
m = ENV.match(line)
if m:
values.setdefault(m.group(1), set()).add(h(m.group(2)))
rows.append((section, m.group(1), "file" if path.startswith("secrets/") else "env"))
continue
m = MD.match(line)
if m:
values.setdefault(m.group(1), set()).add(h(m.group(2)))
rows.append((section, m.group(1), "md"))
continue
if path.endswith(".md"):
mm = LABEL.search(line)
if mm and not mm.group(1).startswith("http"):
rows.append((section, safe(mm.group(1)), "label"))
return rows, values
def resolvable(label: str, kind: str, values) -> str:
if kind not in ("env", "md", "file"):
return "no"
n = len(values.get(label, ()))
return "yes" if n == 1 else "dup" if n > 1 else "no"
def main(argv=None) -> int:
argv = list(sys.argv[1:] if argv is None else argv)
rx = re.compile(argv[0], re.I) if argv else None
src = read_sources()
if not src:
print("lockbox-names: cannot read the lockbox")
return 2
rows, values = collect(src)
seen, n = set(), 0
for section, label, kind in rows:
if rx and not (rx.search(section) or rx.search(label)):
continue
key = (section, label, kind)
if key in seen:
continue
seen.add(key)
n += 1
print(f"{section} | {label} | {kind} | {resolvable(label, kind, values)}")
print(f"# {n} entr{'y' if n == 1 else 'ies'}; names only, values never printed")
return 0
if __name__ == "__main__":
try:
sys.exit(main())
except re.error:
print("lockbox-names: bad regex")
sys.exit(2)
except Exception as e: # never a traceback
print(f"lockbox-names: error: {type(e).__name__}")
sys.exit(2)

141
scripts/lockbox_put.py Normal file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""lockbox-put: add ONE secret to the lockbox by PR, without anyone reading, printing or
grepping the lockbox (Boss rule 10-01; Windy Hub ruling).
lockbox-put KEY FILE [--lane NAME] [--note TEXT]
KEY exact name, ^[A-Z][A-Z0-9_]{2,63}$ . FILE a 0600 file you own (not a symlink) whose
content is the value (one line). Appends ONE line `- **`KEY`**: `<value>`` (the format
lockbox-get reads) under a new heading at the END of ACCESS_LOCKBOX.md in a fresh temp clone,
on a new branch, and opens a kit-army-config PR. Append-only: the diff is verified to be one
file, additions only, before pushing. REFUSES if KEY already exists (a bool computed in
memory; no line, value or location is ever printed). Reviewers see the KEY NAME + lane only
if they look at the diff; the PR body never carries the value. Never echoes the value.
Env (tests): LOCKBOX_PUT_REPO=<clone url/path>, LOCKBOX_PUT_NO_PR=1.
"""
from __future__ import annotations
import argparse
import datetime as dt
import os
import re
import shutil
import stat
import subprocess
import sys
import tempfile
from pathlib import Path
REPO = os.environ.get("LOCKBOX_PUT_REPO", "https://github.com/sneakyfree/kit-army-config.git")
SLUG = "sneakyfree/kit-army-config"
KEY_RE = re.compile(r"^[A-Z][A-Z0-9_]{2,63}$")
VAL_RE = re.compile(r"^[A-Za-z0-9._~+/=:@%,-]{8,512}$") # no backtick, quote, space or newline
def die(msg: str, code: int = 2):
print(f"lockbox-put: {msg}")
sys.exit(code)
def git(cwd: str, *a: str, quiet=True) -> subprocess.CompletedProcess:
# stderr is dropped: git/gh errors can echo URLs; stdout only when we need it.
return subprocess.run(["git", "-C", cwd, *a], capture_output=True, text=True, errors="ignore")
def key_exists(clone: str, key: str) -> bool:
"""True if KEY is already defined anywhere lockbox-get reads. Bool only, nothing printed."""
pat_env = re.compile(r"^" + re.escape(key) + r"=")
pat_md = re.compile(r"^\s*[-*]?\s*\*\*`" + re.escape(key) + r"`\*\*\s*:")
paths = [Path(clone, "ACCESS_LOCKBOX.md")] + [
Path(dp, f) for dp, _d, fs in os.walk(Path(clone, "secrets")) for f in fs if f.endswith(".env")]
for p in paths:
try:
with open(p, errors="ignore") as fh:
for line in fh:
if pat_env.match(line) or pat_md.match(line):
return True
except OSError:
continue
return False
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="lockbox-put")
ap.add_argument("key")
ap.add_argument("file")
ap.add_argument("--lane", default=os.environ.get("LOCKBOX_LANE", "a lane"))
ap.add_argument("--note", default="")
a = ap.parse_args(argv)
if not KEY_RE.match(a.key):
die("KEY must match ^[A-Z][A-Z0-9_]{2,63}$")
try:
st = os.lstat(a.file)
except OSError:
die("FILE not found")
if stat.S_ISLNK(st.st_mode) or not stat.S_ISREG(st.st_mode):
die("FILE must be a regular file (not a symlink)")
if st.st_uid != os.getuid() or (st.st_mode & 0o077):
die("FILE must be owned by you and mode 0600")
with open(a.file) as fh:
value = fh.read().strip()
if not VAL_RE.match(value):
die("value must be one line of 8-512 chars from [A-Za-z0-9._~+/=:@%,-] (no spaces, quotes, backticks)")
note = re.sub(r"[`\n\r]", " ", a.note)[:160]
lane = re.sub(r"[^A-Za-z0-9 ._-]", "", a.lane)[:40]
tmp = tempfile.mkdtemp(prefix="lockbox-put-", dir=str(Path.home() / ".cache") if (Path.home() / ".cache").is_dir() else None)
os.chmod(tmp, 0o700)
clone = os.path.join(tmp, "k")
try:
if subprocess.run(["git", "clone", "-q", "--depth", "1", REPO, clone],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL).returncode != 0:
die("clone failed (details withheld: URLs can carry tokens)")
if key_exists(clone, a.key):
die(f"{a.key} already exists: refusing to overwrite (append-only; pick a new KEY)", 3)
lb = Path(clone, "ACCESS_LOCKBOX.md")
if not lb.is_file():
die("ACCESS_LOCKBOX.md not found in the repo")
today = dt.date.today().isoformat()
stamp = dt.datetime.now(dt.UTC).strftime("%Y%m%d%H%M")
branch = f"lockbox-put/{a.key.lower()}-{stamp}"
with open(lb, "a") as fh:
fh.write(f"\n## 🗝️ {a.key} (added {today} by {lane} via lockbox-put)\n")
fh.write(f"- **`{a.key}`**: `{value}`\n")
if note:
fh.write(f"- **Note:** {note}\n")
git(clone, "checkout", "-q", "-b", branch)
git(clone, "add", "ACCESS_LOCKBOX.md")
ns = git(clone, "diff", "--cached", "--numstat").stdout.split()
# numstat: <added> <deleted> <path>; exactly one file, no deletions
if len(ns) != 3 or ns[1] != "0" or ns[2] != "ACCESS_LOCKBOX.md":
die("diff is not a pure append to ACCESS_LOCKBOX.md: aborting, nothing pushed")
msg = f"lockbox: add {a.key} (via lockbox-put, {lane})\n\nCo-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
if git(clone, "-c", "user.name=lockbox-put", "-c", "user.email=lockbox-put@windy.invalid",
"commit", "-q", "-m", msg).returncode != 0:
die("commit failed")
if git(clone, "push", "-q", "origin", branch).returncode != 0:
die("push failed (details withheld)")
if os.environ.get("LOCKBOX_PUT_NO_PR"):
print(f"ok: pushed branch {branch} ({a.key}); PR skipped")
return 0
body = (f"Adds exactly one key: `{a.key}` (by {lane}). Append-only, one file, no deletions "
f"(verified before push). Review by KEY NAME only; do not paste the value anywhere.\n\n"
f"{note}\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)")
r = subprocess.run(["gh", "pr", "create", "-R", SLUG, "--head", branch, "--base", "main",
"--title", f"lockbox: add {a.key} ({lane})", "--body", body],
capture_output=True, text=True)
if r.returncode != 0:
die("branch pushed but `gh pr create` failed; open the PR for the branch by hand")
print(f"ok: {a.key} added via PR {r.stdout.strip().splitlines()[-1]}")
return 0
finally:
shutil.rmtree(tmp, ignore_errors=True)
if __name__ == "__main__":
try:
sys.exit(main())
except SystemExit:
raise
except Exception as e: # never a traceback: it could carry data
print(f"lockbox-put: error: {type(e).__name__}")
sys.exit(2)

455
scripts/pr_status_bridge.py Executable file
View File

@@ -0,0 +1,455 @@
#!/usr/bin/env python3
"""Give private GitHub repos a CI signal from Windy Git (P1, 2026-09-23).
GitHub Actions cannot run on private repos on this account — not even on
self-hosted runners ([[reference-github-actions-billing-lock]]). The code is
already synced into Windy Git every 15 min and CI runs there, so the only thing
missing is the *signal on GitHub*, where people and agents actually read PRs.
Two jobs, run after every sync:
1. **Mirror open PRs.** The sync carries branches, not PRs, and the workflows
trigger on `pull_request` — so a PR branch alone fires nothing. For each open
same-repo GitHub PR we keep one open Windy Git PR with the same head/base.
Gitea then fires `pull_request` on open and `synchronize` whenever the sync
moves the branch. Windy Git PRs whose GitHub PR closed are closed here too.
Fork PRs are ignored: their head branch is never synced, and untrusted fork
code on this runner is exactly the blast radius the audit warned about.
2. **Post results back** as GitHub commit statuses (context
`windy-git/<workflow>/<job>`) on each PR head and on the default-branch head.
Only posts when a context's state changed, so a 15-min loop doesn't pile
hundreds of identical statuses onto one commit.
Runs ON Veron 1 (localhost Gitea; no Cloudflare hairpin). Needs
GITEA_ADMIN_TOKEN and a GITHUB_TOKEN with `repo` scope. Nothing here executes
repo code, and no secret is handed to any repo.
"""
from __future__ import annotations
import base64
import json
import os
import re
import subprocess
import sys
import time
import urllib.error
import urllib.request
import yaml
GITEA = os.environ.get("BRIDGE_GITEA_URL", "http://localhost:3080").rstrip("/")
PUBLIC = "https://app.windygit.com"
GITEA_TOKEN = os.environ.get("GITEA_ADMIN_TOKEN", "")
GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")
GH_OWNER = os.environ.get("GITHUB_OWNER", "sneakyfree")
WG_OWNER = os.environ.get("WINDYGIT_OWNER", "windyadmin")
# Private repos only. Public repos run real GitHub Actions on veron1's GitHub
# runner; bridging those too would put two competing verdicts on every commit.
REPOS = os.environ.get(
"BRIDGE_REPOS",
"windy-chat windy-mail windy-calendar Windy-Clone WindyCloud windy-search windy-connect"
" windy-drops windy-code-web windy-code windy-traveler windy-registry eternitas"
" windy-translate windytranslate-site windytraveler-site windy-hand"
" windy-cloud-sites windy-cloud-domains windy-cloud-vps windytalk windy-pro windy-mind"
" windy-inbox windy-text windy-call windy-cell windy-hand-site windy-calendar-site",
).split()
# Gitea run status -> GitHub status state. `skipped` is deliberately absent: a
# job skipped by its own `if:` (e.g. substrate-drift's no-secrets path) has no
# verdict, and painting it green would be a claim nobody tested.
STATE = {
"success": "success",
"failure": "failure",
"cancelled": "error",
"running": "pending",
"waiting": "pending",
"blocked": "pending",
}
MIRROR_TAG = "[GH#"
# Image-build jobs cannot pass here BY DESIGN: job containers get no Docker
# daemon (I-5 — the host socket would hand every workflow root on Veron 1).
# Posting them would put a permanent red X on every commit, and a signal that is
# always red trains everyone to ignore red. Not posted until a rootless builder
# exists; that is a decision, recorded in docs/CUTOVER.md, not a failure.
# ...but NOT the no-Docker smoke jobs that replaced them (option A, 09-23):
# windy-search's "Boot smoke (no Docker)" matched plain `docker` and was hidden.
NO_DAEMON_JOB = re.compile(r"(?<!no )(?<!no-)(?<!without )docker", re.IGNORECASE)
# Image-build jobs whose NAME doesn't say docker (orchestrator 09-23, option A:
# each lane converts the job to a no-Docker smoke test; until then it is not
# posted). Format: "repo:workflow/job,...;repo2:...".
NO_DAEMON_NAMED: dict[str, set[str]] = {}
# eternitas:ci/build dropped 09-23: converted to a no-Docker ci/smoke (#179).
for _entry in os.environ.get("BRIDGE_NO_DAEMON", "").split(";"):
if ":" in _entry:
_repo, _jobs = _entry.split(":", 1)
NO_DAEMON_NAMED[_repo.strip()] = {j.strip() for j in _jobs.split(",") if j.strip()}
def needs_daemon(repo: str, wf: str, job: str) -> bool:
"""True for image-build jobs, which cannot run here (no Docker daemon, I-5)."""
return bool(NO_DAEMON_JOB.search(job)) or f"{wf}/{job}" in NO_DAEMON_NAMED.get(repo, ())
# Jobs Grant ruled NON-BLOCKING (GRANT_DECISIONS_2026-09-23): still run on
# Windy Git and visible there, but not posted to GitHub, so they cannot turn a
# commit's combined status red. Format: "repo:workflow/job,workflow/job;repo2:..."
# windy-pro's desktop/installer jobs belong to Grant's desktop side (fixed from
# his Mac mini), not to any lane's merge gate.
NON_BLOCKING: dict[str, set[str]] = {}
for _entry in os.environ.get(
"BRIDGE_NON_BLOCKING", "windy-pro:ci/build-desktop,ci/test-installer,ci/reality-check"
).split(";"):
if ":" in _entry:
_repo, _jobs = _entry.split(":", 1)
NON_BLOCKING[_repo.strip()] = {j.strip() for j in _jobs.split(",") if j.strip()}
# Gitea reads the FIRST of these dirs that has workflow files at a commit (1.24).
WORKFLOW_DIRS = (".gitea/workflows", ".github/workflows")
def workflow_problem(text: str) -> str | None:
"""Why Gitea would drop this workflow file, or None if it looks runnable.
Gitea skips an invalid workflow with one log line and fires no run at all,
so on GitHub the PR just shows nothing, and people wait for CI that is never
coming. These are the shapes we have actually hit, not a full schema.
"""
try:
doc = yaml.safe_load(text)
except yaml.YAMLError as e:
mark = getattr(e, "problem_mark", None)
return f"invalid YAML at line {mark.line + 1}" if mark else "invalid YAML"
if not isinstance(doc, dict):
return "not a YAML mapping"
if "on" not in doc and True not in doc: # YAML 1.1 reads a bare `on` as True
return "no `on:` trigger"
jobs = doc.get("jobs")
if not isinstance(jobs, dict) or not jobs:
return "no `jobs:`"
for name, job in jobs.items():
if not isinstance(job, dict):
return f"job `{name}` is not a mapping"
if "runs-on" not in job and "uses" not in job:
return f"job `{name}` has no `runs-on:`"
return None
def invalid_workflows(repo: str, sha: str) -> dict[str, tuple[str, str]]:
"""{context: (path, problem)} for each workflow file at `sha` that won't run."""
for d in WORKFLOW_DIRS:
st, entries = gitea("GET", f"/repos/{WG_OWNER}/{repo}/contents/{d}?ref={sha}")
if st == 404:
continue
if st != 200:
raise RuntimeError(f"{repo}: Windy Git {d}@{sha[:7]} -> {st}")
files = [e for e in entries or [] if e.get("type") == "file"
and e["name"].endswith((".yml", ".yaml"))]
if not files:
continue
bad = {}
for e in files:
st, f = gitea("GET", f"/repos/{WG_OWNER}/{repo}/contents/{e['path']}?ref={sha}")
if st != 200:
raise RuntimeError(f"{repo}: Windy Git {e['path']}@{sha[:7]} -> {st}")
problem = workflow_problem(base64.b64decode(f["content"]).decode("utf-8", "replace"))
if problem:
stem = re.sub(r"\.ya?ml$", "", e["name"])
bad[f"windy-git/{stem}/workflow"] = (e["path"], problem)
return bad
return {}
def _call(base: str, token_header: str, method: str, path: str, body=None):
req = urllib.request.Request(
base + path,
data=json.dumps(body).encode() if body is not None else None,
method=method,
headers={
"Authorization": token_header,
"Content-Type": "application/json",
"Accept": "application/json",
# urllib's default UA is 403'd as a bot by GitHub's edge and CF.
"User-Agent": "windy-git-pr-bridge/1",
},
)
# Transport errors (TLS handshake timeout, reset) are retried: one GitHub
# blip used to fail the whole sync, flip its heartbeat to ok:false and page
# someone for nothing. HTTP errors are answers, not blips — never retried.
for attempt in range(3):
try:
with urllib.request.urlopen(req, timeout=60) as r:
raw = r.read()
return r.status, (json.loads(raw) if raw else None)
except urllib.error.HTTPError as e:
return e.code, None
except (urllib.error.URLError, TimeoutError, ConnectionError):
if attempt == 2:
raise
time.sleep(2 * (attempt + 1))
raise AssertionError("unreachable")
def gitea(method, path, body=None):
return _call(GITEA + "/api/v1", f"token {GITEA_TOKEN}", method, path, body)
def github(method, path, body=None):
return _call("https://api.github.com", f"Bearer {GITHUB_TOKEN}", method, path, body)
def sync_prs(repo: str) -> list[str]:
"""Mirror open same-repo GitHub PRs into Windy Git. Returns their head shas."""
st, gh_prs = github("GET", f"/repos/{GH_OWNER}/{repo}/pulls?state=open&per_page=100")
if st != 200:
raise RuntimeError(f"{repo}: GitHub PR list -> {st}")
st, wg_prs = gitea("GET", f"/repos/{WG_OWNER}/{repo}/pulls?state=open&limit=50")
if st != 200:
raise RuntimeError(f"{repo}: Windy Git PR list -> {st}")
ours = {p["title"].split("]")[0] + "]": p for p in wg_prs if p["title"].startswith(MIRROR_TAG)}
heads, wanted = [], set()
for pr in gh_prs:
if pr["head"]["repo"] is None or pr["head"]["repo"]["full_name"] != f"{GH_OWNER}/{repo}":
continue # fork PR — never synced, never run here
tag = f"{MIRROR_TAG}{pr['number']}]"
wanted.add(tag)
heads.append(pr["head"]["sha"])
if tag in ours:
cur = (ours[tag].get("base") or {}).get("ref")
if cur is None or cur == pr["base"]["ref"]: # unknown base: never guess, leave it
continue
# Retargeted on GitHub (e.g. a stacked PR moved to main after its
# parent merged). The mirror kept the OLD base, so workflows filtered
# on the base (`pull_request: branches: [main]`) silently stopped
# running: eternitas #167, 09-23. Replace the mirror: an "edited"
# event triggers nothing, a freshly opened PR runs CI at once.
gitea("PATCH", f"/repos/{WG_OWNER}/{repo}/pulls/{ours[tag]['number']}", {"state": "closed"})
print(f" {repo}: GH#{pr['number']} retargeted {cur} -> "
f"{pr['base']['ref']}: replacing its mirror PR")
st, _ = gitea(
"POST",
f"/repos/{WG_OWNER}/{repo}/pulls",
{
"head": pr["head"]["ref"],
"base": pr["base"]["ref"],
"title": f"{tag} {pr['title']}"[:250],
"body": f"Mirror of {pr['html_url']} so CI runs here. Do not merge in Windy Git — "
"GitHub is the source of truth; merge there.",
},
)
print(f" {repo}: opened mirror PR for GH#{pr['number']} -> {st}")
for tag, p in ours.items():
if tag not in wanted:
gitea("PATCH", f"/repos/{WG_OWNER}/{repo}/pulls/{p['number']}", {"state": "closed"})
print(f" {repo}: closed mirror PR {tag} (closed on GitHub)")
return heads
SAFE_NAME = re.compile(r"^[A-Za-z0-9._-]+$")
SAFE_SHA = re.compile(r"^[0-9a-f]{40}$")
def queued_jobs(repo: str, sha: str) -> list[dict]:
"""Jobs at `sha` that are waiting for a runner and that a runner CAN take.
Gitea 1.24's API lists only PICKED-UP jobs (/actions/tasks), so a queued PR
showed nothing on GitHub and people asked whether the push was lost. The
truth is in the gitea DB. Bounded + non-fatal: during the 09-23 IO stall
`docker exec` hung for an hour and must never wedge the bridge again.
Only status 5 (waiting) with labels some live runner has. A `pending` we
post must end in a verdict we will also see, or it sits yellow forever:
blocked jobs (7) often end SKIPPED, and jobs for labels no runner has
(macos-/windows-/ubuntu-latest) are cancelled by the janitor unpicked;
neither ever appears in /actions/tasks.
"""
if not (SAFE_NAME.match(repo) and SAFE_NAME.match(WG_OWNER) and SAFE_SHA.match(sha)):
return []
query = (
"select json_build_object("
" 'jobs', (select coalesce(json_agg(t), '[]'::json) from ("
" select ar.index as run_number, ar.workflow_id, j.name, j.runs_on"
" from action_run_job j join action_run ar on ar.id = j.run_id"
" join repository r on r.id = j.repo_id join \"user\" o on o.id = r.owner_id"
f" where o.lower_name = '{WG_OWNER.lower()}' and r.lower_name = '{repo.lower()}'"
f" and ar.commit_sha = '{sha}' and j.status = 5) t),"
" 'labels', (select coalesce(json_agg(agent_labels), '[]'::json)"
" from action_runner where coalesce(deleted, 0) = 0));"
)
try:
out = subprocess.run(
["docker", "exec", "-i", "windy-git-db-1", "sh", "-c",
'psql -U "$POSTGRES_USER" -d gitea -At -v ON_ERROR_STOP=1'],
input=query, capture_output=True, text=True, check=True, timeout=30,
).stdout.strip()
got = json.loads(out or "{}")
runners = [set(json.loads(x or "[]")) for x in got.get("labels") or []]
return [
j for j in got.get("jobs") or []
if any(set(json.loads(j.get("runs_on") or "[]")) <= r for r in runners)
]
except (subprocess.SubprocessError, OSError, ValueError) as e:
print(f" {repo}: queued-job lookup skipped ({type(e).__name__})")
return []
def post_statuses(repo: str, sha: str) -> None:
# Gitea caps a page at 50 (MAX_RESPONSE_ITEMS) whatever `limit` says, and a
# daily scheduled workflow can push a quiet main's runs off page 1.
runs = []
for page in range(1, 6):
st, body = gitea("GET", f"/repos/{WG_OWNER}/{repo}/actions/tasks?limit=50&page={page}")
if st != 200:
raise RuntimeError(f"{repo}: Windy Git runs -> {st}")
runs += body.get("workflow_runs", [])
if len(body.get("workflow_runs", [])) < 50:
break
latest: dict[str, dict] = {}
for r in runs:
if r["head_sha"] != sha or needs_daemon(repo, r["workflow_id"].removesuffix(".yml"), r["name"]):
continue
if f"{r['workflow_id'].removesuffix('.yml')}/{r['name']}" in NON_BLOCKING.get(repo, ()):
continue
ctx = f"windy-git/{r['workflow_id'].removesuffix('.yml')}/{r['name']}"
if ctx not in latest or r["id"] > latest[ctx]["id"]:
latest[ctx] = r
# Queued jobs: `pending` where nothing newer has been picked up. A re-run
# queued behind an old failure must read pending, not the stale red.
for q in queued_jobs(repo, sha):
wf = q["workflow_id"].removesuffix(".yml")
if needs_daemon(repo, wf, q["name"]):
continue
if f"{wf}/{q['name']}" in NON_BLOCKING.get(repo, ()):
continue
ctx = f"windy-git/{wf}/{q['name']}"
if ctx not in latest or q["run_number"] > latest[ctx]["run_number"]:
latest[ctx] = {"id": 0, "status": "waiting", "run_number": q["run_number"]}
bad = invalid_workflows(repo, sha)
if not (latest or bad):
return
st, existing = github("GET", f"/repos/{GH_OWNER}/{repo}/commits/{sha}/statuses?per_page=100")
current: dict[str, str] = {}
for s in existing or []: # newest first
current.setdefault(s["context"], s["state"])
for ctx, (path, problem) in sorted(bad.items()):
if current.get(ctx) == "error":
continue
st, _ = github(
"POST",
f"/repos/{GH_OWNER}/{repo}/statuses/{sha}",
{
"state": "error",
"context": ctx,
"description": f"Windy Git ignored this workflow, no CI ran: {problem}"[:140],
"target_url": f"{PUBLIC}/{WG_OWNER}/{repo}/src/commit/{sha}/{path}",
},
)
print(f" {repo}@{sha[:7]} {ctx} = error ({problem}) -> {st}")
for ctx, r in sorted(latest.items()):
state = STATE.get(r["status"])
if state is None or current.get(ctx) == state:
continue
st, _ = github(
"POST",
f"/repos/{GH_OWNER}/{repo}/statuses/{sha}",
{
"state": state,
"context": ctx,
"description": f"Windy Git CI on Veron 1: {r['status']}"[:140],
"target_url": f"{PUBLIC}/{WG_OWNER}/{repo}/actions/runs/{r['run_number']}",
},
)
print(f" {repo}@{sha[:7]} {ctx} = {state} -> {st}")
GUARD_CTX = "windy-git/compute-guard"
HYGIENE_CTX = "windy-git/ci-hygiene"
SECRET_CTX = "windy-git/secret-guard"
def post_compute_guard(repo: str, sha: str, default_branch: str, is_default_head: bool) -> None:
"""Windy Mind is the only door to AI compute: flag direct provider use (warn-only)."""
_post_guard("compute_guard", GUARD_CTX, repo, sha, default_branch, is_default_head)
def post_secret_guard(repo: str, sha: str, default_branch: str, is_default_head: bool) -> None:
"""No live credential in a bridged repo (leak hunt 09-24). Findings carry sha256[:8] only."""
_post_guard("secret_guard", SECRET_CTX, repo, sha, default_branch, is_default_head)
def post_ci_hygiene(repo: str, sha: str, default_branch: str, is_default_head: bool) -> None:
"""House rule 6: lockfile-only installs, pinned images, no host-port services (warn-only)."""
_post_guard("ci_hygiene", HYGIENE_CTX, repo, sha, default_branch, is_default_head)
def _post_guard(modname: str, ctx: str, repo: str, sha: str, default_branch: str,
is_default_head: bool) -> None:
"""One code path for every repo-scanning guard. Non-fatal and never a fake OK:
if the guard can't run, nothing is posted."""
try:
import importlib
g = importlib.import_module(modname) # same directory; lazy so the bridge never depends on it
findings = g.check(repo, sha, default_branch, is_default_head)
except Exception as e: # noqa: BLE001 — a guard must never break CI signals
print(f" {repo}@{sha[:7]} {ctx} skipped ({type(e).__name__}: {str(e)[:80]})")
return
if findings is None:
return
try: # Grant-owned code never blocks (orchestrator 09-23); lazy like the guards
import importlib
lane, grant = importlib.import_module("guards_report").split_grant(repo, sha, findings)
except Exception as e: # noqa: BLE001 — can't tell whose code: warn, never block
print(f" {repo}@{sha[:7]} {ctx}: Grant-owned split failed ({type(e).__name__}); WARN only")
lane, grant = [], list(findings)
state, desc, first = g.status_for(lane, whole_tree=is_default_head, grant=grant)
st, existing = github("GET", f"/repos/{GH_OWNER}/{repo}/commits/{sha}/statuses?per_page=100")
for s in existing or []: # newest first: compare the latest guard status only
if s["context"] == ctx:
if (s["state"], s.get("description")) == (state, desc):
return
break
url = (f"{PUBLIC}/{WG_OWNER}/{repo}/src/commit/{sha}/{first.path}#L{first.line}"
if first else f"{PUBLIC}/{WG_OWNER}/{repo}/src/commit/{sha}")
st, _ = github("POST", f"/repos/{GH_OWNER}/{repo}/statuses/{sha}",
{"state": state, "context": ctx, "description": desc, "target_url": url})
print(f" {repo}@{sha[:7]} {ctx} = {state} ({len(findings)} finding(s)) -> {st}")
def main() -> int:
if not (GITEA_TOKEN and GITHUB_TOKEN):
sys.exit("GITEA_ADMIN_TOKEN and GITHUB_TOKEN are required")
failed = 0
for repo in REPOS:
try:
shas = sync_prs(repo)
default_branch, default_head = "main", None
st, br = github("GET", f"/repos/{GH_OWNER}/{repo}")
if st == 200:
default_branch = br["default_branch"]
st, b = github("GET", f"/repos/{GH_OWNER}/{repo}/branches/{default_branch}")
if st == 200:
default_head = b["commit"]["sha"]
shas.append(default_head)
for sha in dict.fromkeys(shas):
post_statuses(repo, sha)
post_compute_guard(repo, sha, default_branch, sha == default_head)
post_ci_hygiene(repo, sha, default_branch, sha == default_head)
post_secret_guard(repo, sha, default_branch, sha == default_head)
except Exception as e: # one repo's failure must not hide the others'
print(f" FAILED {repo}: {e}")
failed = 1
return failed
if __name__ == "__main__":
sys.exit(main())

24
scripts/promote_to_ci.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env bash
# promote_to_ci.sh <repo> [workflow-to-disable ...] — pull mirror -> writable CI repo.
# Run ON Veron as root. See docs/CUTOVER.md "Onboarding another private repo".
#
# ⚠️ It DELETES the mirror before importing (Gitea's migrate refuses an existing
# name). If the import then fails, the Windy Git copy is gone until you re-run —
# GitHub and the nightly R2 bundles still hold everything, but check first that
# scripts/import_from_github.py will accept the repo. (2026-09-23: windy-pro was
# deleted this way while the importer still refused it by name.)
set -euo pipefail
set -a; . /srv/windygit/src/.env; set +a
export IMPORT_GITEA_URL=http://localhost:3080
A=http://localhost:3080/api/v1; H="Authorization: token $GITEA_ADMIN_TOKEN"; r=$1; shift
info=$(curl -s -H "$H" $A/repos/windyadmin/$r)
m=$(echo "$info" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("mirror"))')
if [ "$m" = True ]; then
curl -sf -o /dev/null -X DELETE -H "$H" $A/repos/windyadmin/$r
(cd /srv/windygit/src && python3 scripts/import_from_github.py "$r" | tail -1)
elif [ "$m" = False ]; then echo "$r already writable"; else echo "$r absent -> importing"; (cd /srv/windygit/src && python3 scripts/import_from_github.py "$r" | tail -1); fi
db=$(curl -s -H "$H" $A/repos/windyadmin/$r | python3 -c 'import json,sys;print(json.load(sys.stdin).get("default_branch","main"))')
for i in $(seq 1 120); do curl -sf -o /dev/null -H "$H" $A/repos/windyadmin/$r/branches/$db && break; sleep 5; done
for w in "$@"; do printf " disable %s: " "$w"; curl -s -o /dev/null -w '%{http_code}\n' -X PUT -H "$H" $A/repos/windyadmin/$r/actions/workflows/$w/disable; done
curl -s -H "$H" $A/repos/windyadmin/$r/actions/workflows | python3 -c 'import json,sys,os;print(" "+os.environ.get("R",""),[(w["path"].split("/")[-1],w["state"]) for w in json.load(sys.stdin).get("workflows",[])])'
echo " default=$db"

View File

@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Weekly: every PUBLIC repo in Grant's GitHub accounts, full history (every object,
reachable or not, incl. PR refs), for secret-shaped strings. Leak hunt 09-24: a
real bot token sat in a public test fixture for five months.
Output is hash + location only, never a value (house rule 10). Known fakes are
excused by ci/secret-guard-allow.yml (same file as secret-guard; the repo NAME is
matched across accounts). Runs on Veron as user1-gpu (gh is logged in there):
python3 scripts/public_secret_scan.py [--out FILE] [--owners a,b,...]
"""
from __future__ import annotations
import argparse
import fnmatch
import json
import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import secret_shapes as ss # noqa: E402
OWNERS = ["sneakyfree", "VERONTECH", "Windstorm-Institute", "Windstorm-Labs", "Public-Streamer"]
WORK = Path.home() / "leakscan" / "public"
MAX_BLOB = 20_000_000
def _run(*a: str) -> subprocess.CompletedProcess:
return subprocess.run(a, capture_output=True)
def blob_hits(bare: Path) -> dict[str, list[tuple[str, str]]]:
"""{blob: [(kind, hash8)]} over every blob object in the repo."""
chk = _run("git", "-C", str(bare), "cat-file", "--batch-all-objects",
"--batch-check=%(objectname) %(objecttype) %(objectsize)")
blobs = [p[0] for p in (ln.split() for ln in chk.stdout.decode().splitlines())
if len(p) == 3 and p[1] == "blob" and int(p[2]) < MAX_BLOB]
if not blobs:
return {}
import threading
p = subprocess.Popen(["git", "-C", str(bare), "cat-file", "--batch"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
def feed() -> None: # separate thread: writing everything first deadlocks (both pipes fill)
p.stdin.write(("\n".join(blobs) + "\n").encode())
p.stdin.close()
threading.Thread(target=feed, daemon=True).start()
out: dict[str, list[tuple[str, str]]] = {}
for _ in blobs:
hdr = p.stdout.readline().split()
data = p.stdout.read(int(hdr[2]))
p.stdout.read(1)
found = ss.find(data.decode("utf-8", "ignore"))
if found:
out[hdr[0].decode()] = found
p.wait()
return out
def locate(bare: Path, blob: str) -> dict:
lg = _run("git", "-C", str(bare), "log", "--all", "--format=@@%H %cI", "--name-only",
f"--find-object={blob}").stdout.decode()
commits, paths = [], set()
for line in lg.splitlines():
if line.startswith("@@"):
commits.append(line[2:].split())
elif line.strip():
paths.add(line.strip())
head = _run("git", "-C", str(bare), "ls-tree", "-r", "HEAD").stdout.decode()
first = commits[-1] if commits else ["unreachable", "-"]
return {"paths": sorted(paths), "first_commit": first[0][:10], "first_date": first[1],
"in_head": blob in head}
def excused(repo: str, kind: str, h: str, paths: list[str], allow: dict) -> bool:
a = allow.get(repo) or {"hashes": set(), "paths": []}
if kind in {"private key block"}:
return bool(paths) and all(any(kind in k and fnmatch.fnmatch(p, g) for g, k in a["paths"]) for p in paths)
return h in a["hashes"]
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--out", default=str(WORK / "latest.json"))
ap.add_argument("--owners", default=",".join(OWNERS))
args = ap.parse_args()
import secret_guard as sg
allow = sg.load_allow()
WORK.mkdir(parents=True, exist_ok=True)
rows, scanned, errors = [], [], []
for owner in args.owners.split(","):
lst = _run("gh", "repo", "list", owner, "--limit", "1000", "--visibility", "public", "--json", "name")
for r in json.loads(lst.stdout or b"[]"):
name = r["name"]
bare = WORK / owner / f"{name}.git"
if bare.exists():
c = _run("git", "-C", str(bare), "remote", "update", "--prune")
else:
bare.parent.mkdir(parents=True, exist_ok=True)
c = _run("gh", "repo", "clone", f"{owner}/{name}", str(bare), "--", "--mirror", "-q")
if c.returncode:
errors.append(f"{owner}/{name}")
continue
scanned.append(f"{owner}/{name}")
for blob, found in blob_hits(bare).items():
loc = locate(bare, blob)
for kind, h in sorted(set(found)):
rows.append({"repo": f"{owner}/{name}", "kind": kind, "hash8": h, **loc,
"excused": excused(name, kind, h, loc["paths"], allow)})
Path(args.out).write_text(json.dumps({"scanned": scanned, "errors": errors, "rows": rows}, indent=1))
new = [r for r in rows if not r["excused"]]
print(f"scanned {len(scanned)} public repos, {len(errors)} errors, {len(rows)} secret-shaped, {len(new)} NOT excused")
return 0
if __name__ == "__main__":
sys.exit(main())

52
scripts/rerun_ci.sh Executable file
View File

@@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Re-run a PR's (or branch's) CI on Windy Git. Runs ON Veron 1.
#
# bash scripts/rerun_ci.sh <repo> <branch> <sha-prefix>
#
# Gitea 1.24 has NO rerun API; the web button needs a hub-SSO session as
# windyadmin, which is Grant's identity, so we don't use it. Instead: move the
# Windy Git branch back one commit, let the next sync force-push the GitHub head
# again, and Gitea fires an ordinary push / pull_request_sync event on the SAME
# commit. Every workflow on that event re-runs, not only the failed one.
#
# Safety: refuses unless the branch is exactly at <sha-prefix> (GitHub's head),
# never rewinds while a sync is running (a run already past this repo would
# not push it back), waits for a sync that STARTS after the rewind, and if the
# branch is not verifiably back at <sha-prefix> by the deadline, restores it
# itself, so Windy Git is never left behind GitHub.
set -euo pipefail
repo="${1:?repo}"; branch="${2:?branch}"; want="${3:?sha prefix}"
# wg-q lives in the INVOKING user's ~/bin; under sudo, ~ is /root.
WGQ="${WGQ:-$(getent passwd "${SUDO_USER:-$USER}" | cut -d: -f6)/bin/wg-q}"
# Gitea stores repositories LOWERCASED on disk (WindyCloud -> windycloud.git).
G="sudo docker exec -u git windy-git-gitea-1 git -C /data/git/repositories/windyadmin/${repo,,}.git"
head=$($G rev-parse "refs/heads/${branch}")
[[ "$head" == "$want"* ]] || { echo "refusing: ${branch} is at ${head:0:7}, not ${want}"; exit 1; }
parent=$($G rev-parse "${head}^")
# NOT `systemctl is-active`: the sync is Type=oneshot, which reads "activating"
# (exit 3) for its whole run, so is-active says "idle" mid-run.
busy() { case "$(systemctl show windygit-sync -p ActiveState --value)" in
activating|active|deactivating|reloading) return 0;; esac; return 1; }
while busy; do sleep 5; done
$G update-ref "refs/heads/${branch}" "$parent" "$head"
mark=$(awk '{print int($1*1000000)}' /proc/uptime)
echo "rewound ${repo}:${branch} ${head:0:7} -> ${parent:0:7}"
deadline=$(( $(date +%s) + 900 ))
until [ "$(systemctl show windygit-sync -p ExecMainStartTimestampMonotonic --value)" -gt "$mark" ] \
&& [ "$($G rev-parse "refs/heads/${branch}")" = "$head" ]; do
if [ "$(date +%s)" -ge "$deadline" ]; then
$G update-ref "refs/heads/${branch}" "$head" "$($G rev-parse "refs/heads/${branch}")" || true
echo "TIMEOUT: restored ${branch} to ${head:0:7} by hand; NO new run fired"; exit 1
fi
sleep 10
done
echo "restored by sync: ${branch} = ${head:0:7}"
sleep 5
"$WGQ" <<SQL
select ar.index, ar.workflow_id, ar.event, ar.status, to_char(to_timestamp(ar.created),'HH24:MI:SS')
from action_run ar join repository r on r.id = ar.repo_id
where r.name = '${repo}' and ar.commit_sha = '${head}' order by ar.id desc limit 6;
SQL

209
scripts/runner_guard.py Normal file
View File

@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""Runner guard: no workflow may let a STRANGER's code reach a self-hosted runner (Boss 10-01).
Our self-hosted GitHub runners run on Veron as `github-runner`, which is in the docker group
(= root on Veron). Until ephemeral containerised runners exist (after launch), the cheap
guard is to refuse the workflow shapes that hand a self-hosted runner to outsiders:
R1 pull_request_target : runs with secrets/write token in the BASE repo context
R2 pull_request on self-hosted : fork PRs run their own code, unless the job is gated to
same-repo heads (`if:` on head.repo.full_name == github.repository
or head.repo.fork == false) or an `environment:`
R3 issue_comment / workflow_run / issues / discussion* / pull_request_review* / fork / watch
: anyone can fire these; never on self-hosted without an environment gate
(push, tags, schedule, workflow_dispatch, repository_dispatch, workflow_call: writers only, fine)
A job counts as self-hosted when its runs-on names `self-hosted`, or is an expression we
can't resolve (conservative). Findings carry file:line, never file content beyond that.
python3 scripts/runner_guard.py lint FILE... # local files
python3 scripts/runner_guard.py report [--owners a,b] # default branch of every PUBLIC repo
python3 scripts/runner_guard.py pr [--owners a,b] [--post] # open PRs on public repos: changed workflows
"""
from __future__ import annotations
import argparse
import base64
import json
import os
import subprocess
import sys
import yaml
OWNERS = ["sneakyfree", "VERONTECH", "Windstorm-Institute", "Windstorm-Labs", "Public-Streamer"]
CTX = "windy-git/runner-guard"
OUTSIDE = {"issue_comment", "workflow_run", "issues", "discussion", "discussion_comment",
"pull_request_review", "pull_request_review_comment", "fork", "watch"}
SAME_REPO_GATES = ("head.repo.full_name == github.repository", "github.repository == github.event.pull_request.head.repo.full_name",
"head.repo.fork == false", "!github.event.pull_request.head.repo.fork")
def _node_map(node):
"""{key: (value_node, line)} for a YAML mapping node."""
if not isinstance(node, yaml.MappingNode):
return {}
return {k.value: (v, k.start_mark.line + 1) for k, v in node.value if isinstance(k, yaml.ScalarNode)}
def _triggers(on_node) -> dict[str, int]:
"""{event: line}."""
if isinstance(on_node, yaml.ScalarNode):
return {on_node.value: on_node.start_mark.line + 1}
if isinstance(on_node, yaml.SequenceNode):
return {n.value: n.start_mark.line + 1 for n in on_node.value if isinstance(n, yaml.ScalarNode)}
return {k: line for k, (_v, line) in _node_map(on_node).items()}
def _self_hosted(runs_on) -> bool:
if runs_on is None:
return False
text = yaml.serialize(runs_on) if isinstance(runs_on, yaml.Node) else str(runs_on)
return "self-hosted" in text or "${{" in text
def lint_text(path: str, text: str) -> list[tuple[str, int, str, str]]:
"""[(path, line, rule, message)]. Unparseable YAML is a finding (it cannot be reviewed)."""
try:
root = yaml.compose(text)
except yaml.YAMLError as e:
line = getattr(getattr(e, "problem_mark", None), "line", 0) + 1
return [(path, line, "R0", "workflow YAML does not parse; cannot be checked")]
top = _node_map(root)
if "on" not in top or "jobs" not in top:
return []
trig = _triggers(top["on"][0])
jobs = _node_map(top["jobs"][0])
out = []
if "pull_request_target" in trig:
out.append((path, trig["pull_request_target"], "R1",
"pull_request_target runs fork code with base-repo secrets; not allowed"))
for name, (jnode, jline) in jobs.items():
j = _node_map(jnode)
ro = j.get("runs-on", (None, jline))
if not _self_hosted(ro[0]):
continue
has_env = "environment" in j
cond = j["if"][0].value if "if" in j and isinstance(j["if"][0], yaml.ScalarNode) else ""
same_repo = any(g in cond.replace(" ", " ") for g in SAME_REPO_GATES)
if "pull_request" in trig and not (has_env or same_repo):
out.append((path, ro[1], "R2", f"job '{name}' runs fork PR code on a self-hosted runner "
"(gate it: if: github.event.pull_request.head.repo.full_name == github.repository, or an environment)"))
for ev in sorted(OUTSIDE & trig.keys()):
if not has_env:
out.append((path, trig[ev], "R3", f"'{ev}' can be fired by anyone and job '{name}' is self-hosted "
"without an environment gate"))
return out
# ---------------------------------------------------------------- GitHub side
def gh(*args: str, check=True) -> str:
r = subprocess.run(["gh", "api", *args], capture_output=True, text=True, timeout=60)
if check and r.returncode != 0:
raise RuntimeError(f"gh api {args[0]} failed")
return r.stdout
def public_repos(owners) -> list[tuple[str, str]]:
out = []
for o in owners:
txt = gh(f"users/{o}/repos?per_page=100&type=owner", "--paginate",
"--jq", '.[]|select(.private==false and .archived==false)|.full_name+" "+.default_branch', check=False)
out += [tuple(row.split()) for row in txt.splitlines() if row.strip()]
return out
def workflows_at(full: str, ref: str) -> list[tuple[str, str]]:
txt = gh(f"repos/{full}/contents/.github/workflows?ref={ref}", "--jq",
'.[]|select(.type=="file")|.path', check=False)
files = [p for p in txt.splitlines() if p.endswith((".yml", ".yaml"))]
return [(p, file_at(full, p, ref)) for p in files]
def file_at(full: str, path: str, ref: str) -> str:
raw = gh(f"repos/{full}/contents/{path}?ref={ref}", "--jq", ".content", check=False).strip()
return base64.b64decode(raw).decode("utf-8", "replace") if raw else ""
def cmd_report(owners) -> int:
hits = 0
repos = public_repos(owners)
for full, branch in repos:
for path, text in workflows_at(full, branch):
for p, line, rule, msg in lint_text(path, text):
hits += 1
print(f"{full}\t{p}:{line}\t{rule}\t{msg}")
print(f"# runner-guard sweep: {hits} hit(s) in {len(repos)} public repos")
return 1 if hits else 0
STATE = os.environ.get("RUNNER_GUARD_STATE", "/var/lib/windy-git/runner-guard-posted.json")
def cmd_pr(owners, post: bool) -> int:
# Post each (repo, sha, state, description) ONCE: the sync runs every 5 min and GitHub caps
# statuses per sha+context at 1000.
try:
with open(STATE) as fh:
posted = set(json.load(fh))
except (OSError, ValueError):
posted = set()
seen = set()
for full, _branch in public_repos(owners):
prs = gh(f"repos/{full}/pulls?state=open&per_page=50", "--jq",
'.[]|(.number|tostring)+" "+.head.sha+" "+.head.repo.full_name', check=False)
for line in prs.splitlines():
num, sha, head_repo = line.split(" ", 2)
files = gh(f"repos/{full}/pulls/{num}/files?per_page=100", "--jq",
'.[]|select(.status!="removed")|.filename', check=False).split()
wf = [f for f in files if f.startswith(".github/workflows/") and f.endswith((".yml", ".yaml"))]
# a fork's own content is read from the head repo at the head sha
src = head_repo if head_repo and head_repo != "null" else full
found = [h for f in wf for h in lint_text(f, file_at(src, f, sha))]
if found:
p, ln, rule, msg = found[0]
state, desc = "failure", f"BLOCKED: {rule} {p}:{ln}: {msg}"[:140]
else:
state, desc = "success", ("OK: no workflow changes" if not wf else
"OK: no stranger-code path to a self-hosted runner")
key = f"{full}@{sha}:{state}:{desc}"
seen.add(key)
if key in posted:
continue
print(f"{full}#{num}@{sha[:7]} {state} {desc}")
if post:
gh(f"repos/{full}/statuses/{sha}", "-f", f"state={state}", "-f", f"context={CTX}",
"-f", f"description={desc}", check=False)
posted.add(key)
if post: # keep only keys for PRs still open, so the file never grows without bound
os.makedirs(os.path.dirname(STATE), exist_ok=True)
with open(STATE, "w") as fh:
json.dump(sorted(posted & seen), fh)
return 0
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="runner_guard")
sub = ap.add_subparsers(dest="cmd", required=True)
lint_p = sub.add_parser("lint")
lint_p.add_argument("files", nargs="+")
rep = sub.add_parser("report")
rep.add_argument("--owners", default=",".join(OWNERS))
prp = sub.add_parser("pr")
prp.add_argument("--owners", default="sneakyfree") # self-hosted runners exist only there
prp.add_argument("--post", action="store_true")
a = ap.parse_args(argv)
if a.cmd == "lint":
hits = []
for f in a.files:
with open(f, errors="replace") as fh:
hits += lint_text(f, fh.read())
for p_, ln, rule, msg in hits:
print(f"{p_}:{ln}\t{rule}\t{msg}")
return 1 if hits else 0
owners = a.owners.split(",")
return cmd_report(owners) if a.cmd == "report" else cmd_pr(owners, a.post)
if __name__ == "__main__":
sys.exit(main())

131
scripts/secret_guard.py Normal file
View File

@@ -0,0 +1,131 @@
#!/usr/bin/env python3
"""Secret guard: no live credential lands in a bridged repo (leak hunt 09-24).
Same walker, cache and GitHub posting as compute_guard / ci_hygiene
(`windy-git/secret-guard`), but over EVERY text file, and a finding carries only
"<kind> #<sha256[:8]>", never the value (house rule 10). Known fakes are allowed
BY HASH in ci/secret-guard-allow.yml (repo + hashes + reason).
sudo python3 scripts/secret_guard.py report [repo ...]
"""
from __future__ import annotations
import fnmatch
import hashlib
import os
import re
import sys
from pathlib import Path
import yaml
sys.path.insert(0, str(Path(__file__).resolve().parent))
import compute_guard as cg # noqa: E402 (shared walker, cache)
import secret_shapes as ss # noqa: E402
ROOT = Path(__file__).resolve().parents[1]
ALLOW_FILE = Path(os.environ.get("SECRET_GUARD_ALLOW", ROOT / "ci" / "secret-guard-allow.yml"))
MODE = os.environ.get("SECRET_GUARD_MODE", "warn")
# Kinds that only WARN (rolled out warn-first); empty = every kind blocks in block mode.
WARN_KINDS = {k for k in os.environ.get("SECRET_GUARD_WARN_KINDS", "").split(",") if k}
NEVER = re.compile(r"(^|/)(node_modules|vendor|third_party)/")
def path_ok(path: str) -> bool:
return not NEVER.search(path)
# A private-key match is only its BEGIN line, so its hash is the same everywhere:
# those are allowed by PATH (entries with `paths` + `kinds`), everything else by HASH.
PATH_ONLY_KINDS = {"private key block"}
def load_allow(path: Path = ALLOW_FILE) -> dict[str, dict]:
"""{repo: {"hashes": {hash8}, "paths": [(glob, {kind})]}}; every entry needs a reason."""
data = yaml.safe_load(path.read_text()) if path.exists() else {}
out: dict[str, dict] = {}
for e in (data or {}).get("allow") or []:
if not (e.get("repo") and (e.get("hashes") or (e.get("paths") and e.get("kinds")))
and str(e.get("reason", "")).strip()):
raise ValueError(f"allow entry needs repo, hashes (or paths + kinds) and a reason: {e}")
if e.get("paths") and not set(e["kinds"]) <= PATH_ONLY_KINDS:
raise ValueError(f"path allows are only for {sorted(PATH_ONLY_KINDS)}: {e}")
r = out.setdefault(e["repo"], {"hashes": set(), "paths": []})
r["hashes"].update(str(h) for h in e.get("hashes") or [])
r["paths"] += [(g, set(e["kinds"])) for g in e.get("paths") or []]
return out
def scan_line(path: str, text: str) -> list[tuple[str, str]]:
return [(kind, f"{kind} #{h}") for kind, h in ss.find(text)]
def _drop_allowed(repo: str, findings, allow: dict[str, dict]):
a = allow.get(repo) or {"hashes": set(), "paths": []}
def ok(f) -> bool:
if f.kind in PATH_ONLY_KINDS:
return any(f.kind in kinds and fnmatch.fnmatch(f.path, g) for g, kinds in a["paths"])
return f.match.rsplit("#", 1)[-1] in a["hashes"]
return [f for f in findings if not ok(f)]
def check(repo: str, sha: str, default_branch: str, is_default_head: bool):
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir() or not cg.fetched(bare, sha): # pushed after the fetch: next cycle
return None
allow = load_allow()
rules = hashlib.sha256(("|".join(rx.pattern for _, rx in ss.PATTERNS) + ss.PREFILTER).encode()).hexdigest()[:8]
kw = dict(line_fn=scan_line, path_ok=path_ok)
if is_default_head:
fs = cg.cached_scan(f"sec-tree:{repo}:{sha}:{rules}",
lambda: cg.scan_tree(repo, bare, sha, [], prefilter=ss.PREFILTER, **kw))
else:
fs = cg.cached_scan(f"sec-pr:{repo}:{sha}:{rules}",
lambda: cg.scan_added(repo, bare, f"refs/heads/{default_branch}", sha, [], **kw))
return _drop_allowed(repo, fs, allow)
def status_for(findings, whole_tree: bool, grant=()):
"""Same contract as the other guards. `grant` findings never block."""
scope = "in tree" if whole_tree else "added"
if not findings and grant:
g, n = grant[0], len(grant)
return "success", f"⚠ WARN (Grant-owned, not blocking): {n} secret-shaped string{'s' if n > 1 else ''} {scope}, e.g. {g.path}:{g.line} {g.match}"[:140], g
if not findings:
return "success", f"OK: no secret-shaped strings {scope}", None
f, n = findings[0], len(findings)
soft = MODE != "block" or all(x.kind in WARN_KINDS for x in findings)
state = "success" if soft else "failure"
lead = "⚠ WARN (not blocking)" if soft else "BLOCKED"
if not soft:
f = next(x for x in findings if x.kind not in WARN_KINDS)
return state, f"{lead}: {n} secret-shaped string{'s' if n > 1 else ''} {scope}, e.g. {f.path}:{f.line} {f.match}"[:140], f
def report(repos: list[str]) -> int:
allow = load_allow()
total = 0
for repo in repos:
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir():
continue
head = cg._git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = cg._git(bare, "rev-parse", head).strip()
fs = _drop_allowed(repo, cg.scan_tree(repo, bare, sha, [], line_fn=scan_line, path_ok=path_ok,
prefilter=ss.PREFILTER), allow)
total += len(fs)
print(f"## {repo} ({head} {sha[:7]}): {len(fs)} finding(s)")
for f in fs:
print(f" {f.path}:{f.line} {f.match}")
print(f"TOTAL {total}")
return 0
if __name__ == "__main__":
if len(sys.argv) >= 2 and sys.argv[1] == "report":
default = os.environ.get("BRIDGE_REPOS", "").split() or sorted(
p.name.removesuffix(".git") for p in cg.WORK.glob("*.git"))
sys.exit(report(sys.argv[2:] or default))
sys.exit(__doc__)

210
scripts/secret_scan.py Normal file
View File

@@ -0,0 +1,210 @@
#!/usr/bin/env python3
"""secret-scan: hash-only secret finder. Boss ruling 10-01 after three lanes printed secrets
into their own transcripts while hunting secrets (house rule 10).
secret-scan <path> [--history] [--repo <git url or path>] [--no-lockbox]
Reports `file:line` (and the commit with --history), WHICH lockbox entry matched (the KEY
NAME only) or which secret SHAPE matched (twilio, zai, aws, ...), plus a sha256[:8] of the
token for allow-listing. It NEVER prints, logs or writes a value or any fragment of one
(no context line, no masking). The lockbox is loaded in memory only. stdout only.
Exit 0 = clean, 1 = findings, 2 = usage/error.
"""
from __future__ import annotations
import argparse
import hashlib
import os
import re
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import secret_shapes as ss # noqa: E402 (the SAME shapes as secret-guard)
HOME = Path.home()
LOCKBOX_PATHS = [p for p in os.environ.get("SECRET_SCAN_LOCKBOX_PATHS", "").split(":") if p] or [
str(HOME / "kit-army-config" / "secrets"), str(HOME / "kit-army-config" / "ACCESS_LOCKBOX.md")]
# Extra shapes that are scan-only (not in the blocking guard): label, regex.
EXTRA = [("zai key", re.compile(r"(?<![0-9a-f])[0-9a-f]{32}\.[A-Za-z0-9]{16}(?![A-Za-z0-9])"))]
SKIP_DIRS = {".git", "node_modules", "vendor", "third_party", "__pycache__", ".venv"}
MAX_BYTES = 5_000_000
RUN = re.compile(r"[A-Za-z0-9][A-Za-z0-9_\-]{15,199}")
UUID = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$")
NAME = re.compile(r"[\s>*`|-]*([A-Za-z][A-Za-z0-9_]{2,60})\s*[=:|]")
def h16(t: str) -> str:
return hashlib.sha256(t.encode()).hexdigest()[:16]
def cands(line: str):
"""Token candidates: runs >=16 chars with a digit and a letter; git shas/uuids excluded."""
for chunk in re.split(r"[^A-Za-z0-9_\-.]+", line):
parts = [chunk, *chunk.split(".")] if "." in chunk else [chunk]
for p in parts:
for m in RUN.finditer(p):
t = m.group(0)
if not (re.search(r"\d", t) and re.search(r"[A-Za-z]", t)):
continue
if re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", t) or UUID.match(t.lower()):
continue
yield t
def load_lockbox() -> dict[str, set[str]]:
"""{hash16: {key-name labels}}; values never leave this dict."""
out: dict[str, set[str]] = {}
files: list[str] = []
for p in LOCKBOX_PATHS:
if os.path.isdir(p):
for root, _d, fs in os.walk(p):
files += [os.path.join(root, f) for f in fs]
elif os.path.isfile(p):
files.append(p)
for f in files:
try:
with open(f, errors="ignore") as fh:
for line in fh:
m = NAME.match(line)
label = m.group(1) if m else "?"
for t in cands(line):
out.setdefault(h16(t), set()).add(label)
except OSError:
continue
return out
def scan_line(line: str, lockbox: dict[str, set[str]]) -> list[tuple[str, str]]:
"""[(label, hash8)]: `shape:<kind>` and/or `lockbox:<NAMES>`. No value escapes."""
res: list[tuple[str, str]] = []
for kind, h in ss.find(line):
res.append((f"shape:{kind}", h))
for kind, rx in EXTRA:
for m in rx.finditer(line):
res.append((f"shape:{kind}", ss.h8(m.group(0))))
for t in cands(line):
k = h16(t)
if k in lockbox:
names = sorted(n for n in lockbox[k])
res.append(("lockbox:" + ",".join(names)[:70], k[:8]))
return sorted(set(res))
def is_text(path: Path) -> bool:
try:
with open(path, "rb") as fh:
return b"\0" not in fh.read(4096)
except OSError:
return False
def scan_tree(root: Path, lockbox):
files = [root] if root.is_file() else [
Path(dp) / f for dp, dn, fn in os.walk(root) for f in fn
if not set(Path(dp).relative_to(root).parts) & SKIP_DIRS]
for p in sorted(files):
try:
if p.stat().st_size > MAX_BYTES or not is_text(p):
continue
with open(p, errors="ignore") as fh:
for n, line in enumerate(fh, 1):
for label, h in scan_line(line, lockbox):
yield (str(p), n, None, label, h)
except OSError:
continue
def git(repo: str, *a: str) -> subprocess.Popen:
return subprocess.Popen(["git", "--git-dir", repo, *a], stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, text=True, errors="ignore")
def scan_history(gitdir: str, lockbox):
"""Every ADDED line on every ref (incl. PR refs). Oldest commit per (hash, file, line)."""
seen: dict[tuple, str] = {}
p = git(gitdir, "log", "--all", "-p", "-U0", "--no-color", "--format=@@C %h", "-a")
commit = path = None
ln = 0
for row in p.stdout: # type: ignore[union-attr]
if row.startswith("@@C "):
commit = row[4:].strip()
elif row.startswith("+++ "):
path = row[6:].strip() if row.startswith("+++ b/") else None
elif row.startswith("@@ "):
m = re.search(r"\+(\d+)", row)
ln = int(m.group(1)) - 1 if m else 0
elif row.startswith("+") and path:
ln += 1
for label, h in scan_line(row[1:], lockbox):
seen[(label, h, path, ln)] = commit or "?"
p.wait()
for (label, h, path, ln), c in sorted(seen.items(), key=lambda x: (x[0][2], x[0][3])):
yield (path, ln, c, label, h)
def resolve_gitdir(p: Path) -> str | None:
for cand in (p / ".git", p):
if (cand / "HEAD").exists() and ((cand / "objects").exists()):
return str(cand)
return None
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="secret-scan", description=__doc__.split("\n\n")[1] if __doc__ else "")
ap.add_argument("path", nargs="?", help="file, directory, or git repo (with --history)")
ap.add_argument("--history", action="store_true", help="scan every added line in all git history")
ap.add_argument("--repo", help="git URL or path to scan (mirror-cloned to a temp dir, then deleted)")
ap.add_argument("--no-lockbox", action="store_true", help="shapes only")
a = ap.parse_args(argv)
if not (a.path or a.repo):
ap.print_usage()
return 2
lockbox = {} if a.no_lockbox else load_lockbox()
print(f"# secret-scan: {len(lockbox)} lockbox tokens in memory, values never printed", flush=True)
tmp = None
findings = 0
try:
if a.repo:
tmp = tempfile.mkdtemp(prefix="secret-scan-", dir=str(HOME / ".cache") if (HOME / ".cache").is_dir() else None)
os.chmod(tmp, 0o700)
target = os.path.join(tmp, "r.git")
rc = subprocess.run(["git", "clone", "-q", "--mirror", a.repo, target],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL).returncode
if rc != 0:
print("error: clone failed (details withheld: URLs can carry tokens)")
return 2
a.history = True
gitdir = target
elif a.history:
gitdir = resolve_gitdir(Path(a.path))
if not gitdir:
print("error: --history needs a git repo path")
return 2
if a.history:
for path, ln, c, label, h in scan_history(gitdir, lockbox):
findings += 1
print(f"{path}:{ln} commit={c} {label} #{h}")
else:
for path, ln, _c, label, h in scan_tree(Path(a.path), lockbox):
findings += 1
print(f"{path}:{ln} {label} #{h}")
finally:
if tmp:
shutil.rmtree(tmp, ignore_errors=True)
print(f"# {findings} finding(s)")
return 1 if findings else 0
if __name__ == "__main__":
try:
sys.exit(main())
except KeyboardInterrupt:
sys.exit(130)
except Exception as e: # never a traceback: it could carry data
print(f"error: {type(e).__name__}")
sys.exit(2)

46
scripts/secret_shapes.py Normal file
View File

@@ -0,0 +1,46 @@
"""Secret-shaped strings, shared by secret_guard (bridged repos) and
public_secret_scan (weekly, every public repo). A finding NEVER carries the value:
only its kind and sha256[:8] (house rule 10). Leak hunt 09-24: @Windy_0_bot's
token sat in a public repo's test fixture for five months."""
from __future__ import annotations
import hashlib
import re
# (kind, regex). Order matters only for readability; each match is reported once.
PATTERNS: list[tuple[str, re.Pattern[str]]] = [
("telegram bot token", re.compile(r"(?<![0-9])[0-9]{8,10}:[A-Za-z0-9_-]{35}(?![A-Za-z0-9_-])")),
("github token", re.compile(r"\b(?:gh[pousr]_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{82})\b")),
("aws access key", re.compile(r"\b(?:AKIA|ASIA)[0-9A-Z]{16}\b")),
("slack token", re.compile(r"\bxox[abprs]-[A-Za-z0-9-]{10,}")),
("anthropic key", re.compile(r"\bsk-ant-[A-Za-z0-9_-]{20,}")),
("openai key", re.compile(r"\bsk-(?:proj-|svcacct-)?(?!ant-)[A-Za-z0-9_-]{32,}")),
("stripe live key", re.compile(r"\b[rs]k_live_[A-Za-z0-9]{20,}")),
("google api key", re.compile(r"\bAIza[0-9A-Za-z_-]{35}(?![0-9A-Za-z_-])")),
# Twilio (Windy Text 10-01: a live auth token sat in test files for months). An auth token
# is a bare 32-hex with no prefix, so it is only caught when ASSIGNED to a secret-ish name.
("twilio sid/api key", re.compile(r"\b(?:AC|SK)[0-9a-f]{32}\b")),
("32-hex secret assignment", re.compile(
r"(?i)\b[a-z0-9_.-]*(?:token|secret|key|password)[a-z0-9_.-]*[\"']?\s*[:=]\s*[\"']?(?P<v>(?<![0-9a-f])[0-9a-f]{32}(?![0-9a-f]))")),
("pypi token", re.compile(r"\bpypi-AgE[A-Za-z0-9_-]{50,}")),
("private key block", re.compile(r"-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED )?PRIVATE KEY-----")),
]
# git grep -E (POSIX ERE) prefilter: cheap superset of PATTERNS.
PREFILTER = ("[0-9]{8,10}:[A-Za-z0-9_-]{35}|gh[pousr]_[A-Za-z0-9]{36}|github_pat_|(AKIA|ASIA)[0-9A-Z]{16}"
"|xox[abprs]-|sk-ant-|sk-[A-Za-z0-9_-]{32}|sk-proj-|[rs]k_live_|AIza[0-9A-Za-z_-]{35}"
"|-----BEGIN [A-Z ]*PRIVATE KEY-----|(AC|SK)[0-9a-f]{32}|[0-9a-fA-F]{32}|pypi-AgE")
def h8(value: str | bytes) -> str:
return hashlib.sha256(value.encode() if isinstance(value, str) else value).hexdigest()[:8]
def find(text: str) -> list[tuple[str, str]]:
"""[(kind, hash8)] for every secret-shaped string in `text`. Values never leave."""
out = []
for kind, rx in PATTERNS:
for m in rx.finditer(text):
out.append((kind, h8(m.group('v') if 'v' in rx.groupindex else m.group(0))))
return out

107
scripts/sync_from_github.sh Executable file
View File

@@ -0,0 +1,107 @@
#!/usr/bin/env bash
# Phase 1 sync: GitHub is the source of truth, Windy Git follows.
#
# ── Why this direction, and why the other one was wrong ────────────────────
#
# On 2026-08-13 nine repos were migrated writable with push-mirrors pointed AT
# GitHub. That was premature: a dozen agent sessions on the Mac mini are pushing
# to GitHub continuously, so GitHub — not Windy Git — is where the current work
# actually lives. A push-mirror force-updates refs, so on its 8-hour timer it
# would have pushed Windy Git's stale copy over live work, silently, with no
# conflict to notice. The mirrors were removed before the first timer fired.
#
# This script is the correct Phase 1: pull from GitHub, push into Windy Git.
#
# Mac mini agents ──push──▶ GitHub ──this script──▶ Windy Git ──▶ CI on Veron
#
# **It requires nothing from anyone.** No remote changes, no coordination, no
# "everybody stop pushing for a minute." Agents keep working exactly as they are
# and CI starts running on 24 cores.
#
# Windy Git is force-updated on purpose. In Phase 1 it holds nothing anyone
# depends on, so GitHub always wins and there is no merge to reconcile — which
# is the entire point of not flipping direction until a repo is quiet.
#
# Phase 2, per repo, only when that repo is idle: point its agents at Windy Git,
# drop it from REPOS here, and add a push-mirror back to GitHub. One repo at a
# time. Never a big-bang cutover across a dozen live sessions.
set -uo pipefail
: "${GITHUB_TOKEN:?GITHUB_TOKEN required}"
: "${GITEA_ADMIN_TOKEN:?GITEA_ADMIN_TOKEN required}"
GH_OWNER="${GITHUB_OWNER:-sneakyfree}"
WG="${WG_HOST:-app.windygit.com}"
WG_OWNER="${WINDYGIT_OWNER:-windyadmin}"
WORK="${SYNC_WORK:-/srv/windygit/sync}"
FAILED=0
# Repos Windy Git tracks FROM GitHub. Remove a repo from this list at the moment
# it flips to Windy-Git-first, or the sync will fight its authors and win.
REPOS="${SYNC_REPOS:-windy-calendar windy-search windy-registry Windy-Clone WindyCloud windy-cloud-sites windy-mind eternitas windy-agent windy-git windy-chat windy-mail windy-connect windy-drops windy-code-web windy-code windy-traveler windy-translate windytranslate-site windytraveler-site windy-hand windy-cloud-domains windy-cloud-vps windytalk windy-pro windy-inbox windy-text windy-call windy-cell windy-hand-site windy-calendar-site}"
# Repos whose TAGS must not reach Windy Git. A tag push fires `on: push: tags`
# workflows; windy-pro's build-electron is a matrix over ubuntu/macos/windows-
# latest, labels no runner here has, so every leg would queue forever (and
# queued jobs are invisible in /actions/tasks). Releases are built elsewhere.
NO_TAGS="${SYNC_NO_TAGS:-windy-pro}"
# `archive/*` branches never reach Windy Git (negative refspec, git >= 2.29).
# They are off-machine safety copies of unpushed work (one-repo doctrine), not
# work in progress: GitHub holds them, and CI time on them is waste.
mkdir -p "$WORK"
log() { printf '[sync %s] %s\n' "$(date -u +%H:%M:%SZ)" "$*"; }
for r in $REPOS; do
bare="$WORK/${r}.git"
if [[ ! -d "$bare" ]]; then
git clone --quiet --bare "https://x-access-token:${GITHUB_TOKEN}@github.com/${GH_OWNER}/${r}.git" "$bare" 2>/dev/null \
|| { log "FAILED initial clone of $r"; FAILED=1; continue; }
fi
# +refs/heads/* — branches only, deliberately.
#
# `--mirror` would also carry refs/pull/* (GitHub's read-only PR refs, which
# Gitea rejects) and every remote-tracking ref, turning a working sync into a
# wall of errors that hides the one that matters.
if ! git --git-dir="$bare" fetch --quiet --prune origin '+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*' 2>/dev/null; then
log "FAILED fetch $r"; FAILED=1; continue
fi
before="$(git --git-dir="$bare" rev-parse HEAD 2>/dev/null || echo none)"
if git --git-dir="$bare" push --quiet --force \
"https://${WG_OWNER}:${GITEA_ADMIN_TOKEN}@${WG}/${WG_OWNER}/${r}.git" \
'+refs/heads/*:refs/heads/*' '^refs/heads/archive/*' $([[ " $NO_TAGS " == *" $r "* ]] || echo '+refs/tags/*:refs/tags/*') 2>/dev/null; then
log "$r ok (${before:0:7})"
else
log "FAILED push $r -> windy git"; FAILED=1
fi
done
# Jobs that name labels no runner has (ubuntu/macos/windows-latest) would wait
# forever and invisibly; cancel them after 30 min. Never fails the sync.
# Both DB steps go through `docker exec`, which hangs outright while the host
# is in an IO stall (09-23: data2 SMR cliff wedged this sync for 10+ min and
# stopped mirroring + the bridge for every lane). They are optional; mirroring
# and the bridge are not. Bound them so a stuck exec costs one step, not the run.
timeout -k 10 120 bash "$(dirname "$0")/cancel_unrunnable.sh" || log "janitor failed or timed out (non-fatal)"
# Private repos can't run GitHub Actions; mirror their open PRs here so CI
# fires, and post the verdicts back to GitHub as commit statuses.
if ! python3 "$(dirname "$0")/pr_status_bridge.py"; then
log "FAILED pr status bridge"; FAILED=1
fi
# Runner guard (Boss 10-01): PUBLIC sneakyfree repos have self-hosted GitHub runners on Veron.
# A PR that changes a workflow so a stranger's code could reach one gets a red
# windy-git/runner-guard status. Each status is posted once; never fails the sync.
timeout -k 10 120 python3 "$(dirname "$0")/runner_guard.py" pr --post || log "runner-guard failed or timed out (non-fatal)"
# CI telemetry -> admin.windyword.ai (shapes declared with Windy Telemetry 40).
# Sends nothing until WINDYGIT_TELEMETRY_TOKEN is set; never fails the sync.
timeout -k 10 180 python3 "$(dirname "$0")/telemetry_emit.py" || log "telemetry emit failed or timed out (non-fatal)"
[[ "$FAILED" -ne 0 ]] && { log "COMPLETED WITH FAILURES"; exit 1; }
log "all repos in step with GitHub"

Some files were not shown because too many files have changed in this diff Show More