38 Commits

Author SHA1 Message Date
ef96051d6f Merge pull request #2 from sneakyfree/runner-guard
All checks were successful
check / gate (push) Successful in 14s
canary / probe (push) Successful in 22s
runner-guard: no stranger code on self-hosted runners (PR check + nightly sweep)
2026-10-01 05:29:07 -04:00
Kit OC5
a0dc6f8568 runner-guard: block workflows that hand a self-hosted runner to strangers (Boss 10-01)
R1 pull_request_target; R2 fork pull_request on self-hosted without a same-repo/environment
gate; R3 outsider events (issue_comment, workflow_run, ...) on self-hosted; R0 unparseable.
PR mode in the 5-min sync posts windy-git/runner-guard on open PRs of public sneakyfree repos
that change a workflow (each status once). Nightly sweep (Windy 0 timer) over every public
repo: page + BOARD line on new hits + red windy-job heartbeat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 05:28:37 -04:00
Kit OC5
51e0b9d30b bridge + sync: onboard windy-calendar-site (static, no workflows; guards only)
All checks were successful
check / gate (push) Successful in 28s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 05:04:49 -04:00
Kit OC5
51777b0ad0 bridge + sync: onboard windy-hand-site (static site, no workflows yet; guards only)
All checks were successful
check / gate (push) Successful in 13s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 04:50:15 -04:00
Kit OC5
cbcb4c206b docs: CUTOVER-PRIMARY.md checklist (draft, nothing flipped)
All checks were successful
check / gate (push) Successful in 45s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 04:03:09 -04:00
Kit OC5
2c62e2834a secret-guard allow: windy-agent #412 synthetic Z.ai fixture (hash not in lockbox)
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:56:48 -04:00
3a9b7ecab7 Merge pull request #1 from sneakyfree/veron-ollama-warn
All checks were successful
check / gate (push) Successful in 22s
canary / probe (push) Successful in 8s
compute-guard: WARN on new Veron Ollama (:11434) references
2026-10-01 03:36:32 -04:00
Kit OC5
cd0400c371 compute-guard: WARN (never red) on NEW references to Veron Ollama :11434
Grant via Boss 10-01: compute = Windy Mind (endpoint + key); do not call Veron Ollama directly.
Judged on lines a PR adds only (not the baseline tree), never blocks even in MODE=block,
windy-mind (the compute door) allowed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:36:11 -04:00
Kit OC5
4e7ab667ed backup_state: pin restic cache dir (systemd has no HOME); init only on a MISSING repo, log other errors
All checks were successful
check / gate (push) Successful in 38s
canary / probe (push) Successful in 10s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:23:41 -04:00
Kit OC5
f10db19316 drill_cross_host.sh: read the R2 pair + endpoint from the lockbox (drill PASSED 10-01)
All checks were successful
check / gate (push) Successful in 18s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 03:19:56 -04:00
Kit OC5
3503894a1e state backup: systemd units (NOT enabled) + cross-host drill script
All checks were successful
check / gate (push) Successful in 23s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:59:19 -04:00
Kit OC5
fbab5c4669 secret-guard/secret-scan: PyPI API token shape (pypi-AgE macaroon), WARN-first
All checks were successful
check / gate (push) Successful in 20s
canary / probe (push) Successful in 13s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:47:53 -04:00
Kit OC5
1b552c0882 docs: RESTORE-DRILL.md (state backup + restore procedure)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:46:53 -04:00
Kit OC5
8ebc18c3bc gitea 1.24.6 -> 1.24.7 (security: LFS auth bypass, symlink bypass, OAuth2 missed return)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:45:28 -04:00
Kit OC5
9566826ce9 lockbox-put: append-only lockbox PR tool (no one reads/greps the lockbox); installer updated
All checks were successful
check / gate (push) Successful in 28s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:40:31 -04:00
Kit OC5
160a3d69ba backup_state.sh: encrypted restic backup of Postgres + Gitea state to R2 (SOTU 10-01 gap: DB was not backed up)
All checks were successful
check / gate (push) Successful in 23s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 02:35:09 -04:00
Kit OC5
7e28a23c22 secret-scan + env-names: shared hash-only tools (Boss 10-01: lanes printed secrets while hunting them)
All checks were successful
check / gate (push) Successful in 9s
canary / probe (push) Successful in 5s
secret-scan reports file:line/commit + lockbox KEY NAME or shape, never a value or fragment;
env-names lists variable names/length/hash only. Same shapes as secret-guard. Seeded-fake tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 01:17:30 -04:00
Kit OC5
1da4d39c0b bridge + sync: onboard windy-text, windy-call, windy-cell (telephony in scope, GitHub Actions dead since 08-14; deploy.yml disabled)
All checks were successful
check / gate (push) Successful in 14s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 01:00:48 -04:00
Kit OC5
53fbcfe78f secret-guard allow: windy-code VS Code public aiKey, windytalk redaction fixture (hash not in lockbox)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 00:59:59 -04:00
Kit OC5
1c552e94de secret-guard: Twilio shapes (SID/API key, 32-hex secret assignment) + per-kind warn mode
Windy Text 10-01: a live Twilio auth token sat in bridged-repo tests. Auth tokens are bare
32-hex, so they are matched only when assigned to a name containing token/secret/key/password;
hash is of the value alone. SECRET_GUARD_WARN_KINDS lets a new shape warn before it blocks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 00:59:05 -04:00
Kit OC5
8690c4f8d3 secret-guard allow: 5 windy-agent #395 FAKE fixtures (weekly scan 09-28; verified never in the lockbox)
All checks were successful
check / gate (push) Successful in 17s
canary / probe (push) Successful in 5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 06:13:22 -04:00
Kit OC5
313f41b49c deploy: secret-guard BLOCK drop-in (orchestrator 09-24)
All checks were successful
check / gate (push) Successful in 8s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:08:11 -04:00
Kit OC5
ea02ade0cb weekly public secret scan (all 5 GitHub accounts, full history, hash-only)
All checks were successful
check / gate (push) Successful in 13s
canary / probe (push) Successful in 4s
scripts/public_secret_scan.py: every PUBLIC repo, every object (incl.
unreachable + PR refs), secret_shapes patterns, excused by the secret-guard
allow list. Output JSON with hash8 + location only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:05:16 -04:00
Kit OC5
497222094f secret-guard: triaged allow list (fakes by hash, test PEMs by path)
Private-key matches are only the BEGIN line (same hash everywhere), so they
are allowed by path+kind; everything else by hash. Real revoked tokens
(1354fc9b, d49dc2ba) are pinned by a test to never be allowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:04:13 -04:00
Kit OC5
d28da26000 secret-guard: no live credentials in bridged repos (hash-only findings)
All checks were successful
check / gate (push) Successful in 22s
New guard windy-git/secret-guard (warn-only) over EVERY text file: Telegram,
GitHub, AWS, Slack, Anthropic, OpenAI, Stripe live, Google API keys and
private-key blocks (scripts/secret_shapes.py, shared with the weekly public
scan). A finding carries "<kind> #<sha256[:8]>", never the value (house rule
10). Known fakes allowed BY HASH (ci/secret-guard-allow.yml). GUARDS_STATUS
gets a secrets column. Leak hunt 09-24: @Windy_0_bot token in a public fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 03:01:59 -04:00
Kit OC5
04c857654a rerun_ci.sh: find wg-q in the invoking user home under sudo
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 5s
Under sudo ~ is /root, so the final run listing failed (windy-inbox #14).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 02:38:25 -04:00
Kit OC5
32512a32fc bridge windy-inbox (new private repo; Windy Drops is build lead)
All checks were successful
check / gate (push) Successful in 17s
canary / probe (push) Successful in 4s
sync REPOS + BRIDGE_REPOS + guards page (owner Windy Drops). Imported
writable into Gitea first (repo 164).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:51:09 -04:00
Kit OC5
b52e6b4784 bridge: post no-Docker smoke jobs (name regex skipped "no Docker")
All checks were successful
check / gate (push) Successful in 9s
canary / probe (push) Successful in 5s
windy-search #96 "Boot smoke (no Docker)" passed but was hidden by the
image-build name filter. Negative lookbehind for no/no-/without.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:11:55 -04:00
Kit OC5
6440c7d5f4 deploy: compute-guard BLOCK drop-in (live on Veron since 09-24 00:0xZ)
All checks were successful
check / gate (push) Successful in 12s
canary / probe (push) Successful in 7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:05:34 -04:00
Kit OC5
17acae6af6 guards: Grant-owned findings never block (compute-guard + ci-hygiene)
status_for(lane, whole_tree, grant=...): only lane-owned findings fail in
MODE=block; Grant-owned (ci/grant-owned.yml) post WARN. The bridge splits via
guards_report.split_grant; if the split cannot run it WARNs (never blocks).
Orchestrator 09-23: block compute-guard for lane-owned paths only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 20:04:01 -04:00
Kit OC5
790d794900 bridge: drop eternitas ci/build from BRIDGE_NO_DAEMON (converted to ci/smoke, #179)
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:51:26 -04:00
Kit OC5
1b8cebbfe1 branding: friendly invite-only page instead of Gitea's raw Registration is disabled
A Windy account with no forge account (registration CLOSED at launch, Grant
09-23) lands on /user/link_account. Override shows invite-only + signed-in name
+ link to the dashboard; other cases = Gitea 1.24.6 template verbatim. No change
to who can register. Orchestrator ask 09-23.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:49:34 -04:00
Kit OC5
83fbf1f992 guards_report: chat lane is now the Windy Chat session
All checks were successful
check / gate (push) Successful in 10s
canary / probe (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:12:40 -04:00
Kit OC5
9b5334fee7 test: allow-list entries must be line-scoped (replaces is-empty check)
f71a5aa pushed with this test red (tail hid pytest rc); fixed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:11:53 -04:00
Kit OC5
f71a5aab39 ci-hygiene allow: windy-pro disabled deploy job (needs-docker false positive)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:11:20 -04:00
Kit OC5
f219437282 ci-hygiene report: same disabled-workflow filter as check()
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:10:35 -04:00
Kit OC5
b15584d33a ci-hygiene: needs-docker skips workflows disabled on Windy Git
deploy/release workflows run on the target host (real daemon) and are
disabled here (repo_unit DisabledWorkflows); one bounded query per process,
flag everything if it fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:10:11 -04:00
Kit OC5
0a57d96f2e bridge + ci-hygiene: Docker-needing CI jobs (option A)
- bridge: BRIDGE_NO_DAEMON names image-build jobs whose name lacks docker
  (default eternitas:ci/build); never posted, like the docker-named ones.
- ci-hygiene: flag docker build/buildx/run/compose, docker-compose and
  docker/build-push-action in workflow steps ("needs docker") with the fix:
  job services: + a no-Docker smoke test; the image builds at deploy.
- test_guards_report: owner column (14ed23a broke it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:09:09 -04:00
40 changed files with 2127 additions and 41 deletions

View File

@@ -83,8 +83,17 @@ def test_warn_mode_never_turns_red(monkeypatch):
assert state == "success" and desc.startswith("⚠ WARN (not blocking): 1 CI hygiene issue in CI/Dockerfiles")
def test_allow_file_loads_and_is_empty_today():
assert hy.cg.load_allow(hy.ALLOW_FILE) == []
def test_allow_file_is_line_scoped_exceptions_only():
"""Every exception is line-scoped (`matches`), so an allowed file can't hide a
NEW floating install or docker step. Today: windy-pro's if:false deploy job."""
allow = hy.cg.load_allow(hy.ALLOW_FILE)
assert [(e["repo"], e["paths"]) for e in allow] == [("windy-pro", [".github/workflows/ci.yml"])]
assert all(e.get("matches") for e in allow)
ok = " run: docker build -f account-server/Dockerfile -t windy-pro:${{ github.sha }} ."
new = " run: docker build -t windy-pro-api ."
assert hy.cg.allowed("windy-pro", WF, allow, ok)
assert not hy.cg.allowed("windy-pro", WF, allow, new)
assert not hy.cg.allowed("windy-chat", WF, allow, ok)
@pytest.mark.parametrize("path, text, want", [
@@ -116,3 +125,40 @@ def test_optional_lock_globs_are_flagged(text):
])
def test_pinned_images_and_real_locks_pass(path, text):
assert hy.scan_line(path, text) == []
@pytest.mark.parametrize("text", [
" - run: docker compose -f docker-compose.yml -f docker-compose.ci.yml build", # eternitas ci/build
" run: docker build -t windy-mail .",
" - run: docker-compose up -d",
" run: docker buildx build --load .",
" - uses: docker/build-push-action@v6",
])
def test_docker_in_ci_is_flagged_with_the_fix(text):
hits = hy.scan_line(WF, text)
assert [k for k, _ in hits] == ["needs docker"]
assert "no-Docker smoke test" in hits[0][1]
@pytest.mark.parametrize("path, text", [
("Dockerfile", "RUN docker build ."), # not a workflow
(WF, " run: ssh host 'docker compose up -d'"), # remote host has a daemon
(WF, " # docker compose build"), # comment
(WF, " run: echo docker build"),
])
def test_docker_not_flagged_outside_ci_steps(path, text):
assert [k for k, _ in hy.scan_line(path, text) if k == "needs docker"] == []
def test_needs_docker_skips_workflows_disabled_on_windy_git(monkeypatch):
"""deploy.yml runs on the target host (a real daemon); Gitea has it disabled here."""
F = hy.cg.Finding
monkeypatch.setattr(hy, "_DISABLED", {"eternitas": {"deploy.yml"}})
got = hy._runs_here("Eternitas", [
F(".github/workflows/deploy.yml", 70, "needs docker", "docker compose in CI"),
F(".github/workflows/ci.yml", 176, "needs docker", "docker compose in CI"),
F(".github/workflows/deploy.yml", 12, "floating install", "npm install"),
])
assert [(f.path.rsplit("/", 1)[1], f.kind) for f in got] == [
("ci.yml", "needs docker"), ("deploy.yml", "floating install")]
assert hy._runs_here("eternitas", None) is None

View File

@@ -220,3 +220,33 @@ def test_scoped_allow_in_a_real_diff():
"""
fs = cg.parse_added("windy-pro", diff, ALLOW)
assert [(f.line, f.match) for f in fs] == [(3, "openrouter.ai")]
def test_block_mode_never_blocks_grant_owned(monkeypatch):
monkeypatch.setattr(cg, "MODE", "block")
g = cg.Finding("src/client/desktop/x.js", 9, "provider host", "api.openai.com")
state, desc, f = cg.status_for([], whole_tree=True, grant=[g])
assert state == "success" and desc.startswith("⚠ WARN (Grant-owned, not blocking): 1") and f is g
lane = cg.Finding("a.py", 3, "provider host", "x")
assert cg.status_for([lane], whole_tree=True, grant=[g])[0] == "failure"
def test_veron_ollama_warns_on_added_lines_and_never_blocks(monkeypatch):
hits = cg.scan_line("app/llm.py", 'OLLAMA = "http://192.168.1.73:11434/api/generate"')
assert [k for k, _ in hits] == ["veron ollama"]
assert cg.scan_line("app/llm.py", 'port = 114345') == [] # not the port
assert cg.scan_line("app/llm.py", "# was http://x:11434 (removed)") == [] # a comment is not a call
f = cg.Finding("app/llm.py", 7, "veron ollama", ":11434")
monkeypatch.setattr(cg, "MODE", "block")
state, desc, _ = cg.status_for([f], whole_tree=False)
assert state == "success" and desc.startswith("⚠ WARN: new Veron Ollama ref app/llm.py:7")
assert "Windy Mind" in desc and len(desc) <= 140
hard = cg.Finding("app/llm.py", 1, "provider host", "api.openai.com")
assert cg.status_for([f, hard], whole_tree=False)[0] == "failure" # a real violation still blocks
def test_ollama_in_added_pr_lines_only():
diff = ("+++ b/svc/client.py\n@@ -0,0 +1,2 @@\n+import httpx\n"
"+URL = 'http://veron:11434/api/chat'\n")
got = cg.parse_added("some-repo", diff, [])
assert [(f.kind, f.line) for f in got] == [("veron ollama", 2)]

View File

@@ -57,10 +57,21 @@ def test_render_splits_lane_and_grant_counts():
md = gr.render(res)
assert "| ci-hygiene (house rule 6) | 1 | 1 | ❌ not yet |" in md
assert "| compute-guard (Mind is the only door) | 0 | 0 | ✅ YES |" in md
assert "| windy-git | bbbbbbb | 0 | 0 | clean ✅ |" in md
assert "| windy-git | Windy Git | bbbbbbb | 0 | 0 | 0 | clean ✅ |" in md
assert "| windy-pro | Windy Hub | aaaaaaa |" in md # owner = session to message
assert "(job reality-check)" in md
def test_windy_pro_root_env_example_is_grant_owned_but_not_the_account_servers():
assert gr.grant_owned("windy-pro", ".env.example", None, OWNED)
assert not gr.grant_owned("windy-pro", "account-server/.env.example", None, OWNED)
def test_split_grant_sends_desktop_code_to_grant(monkeypatch):
F = gr.cg.Finding
fs = [F("src/client/desktop/main.js", 3, "provider host", "x"),
F("account-server/src/llm.ts", 5, "provider host", "y")]
lane, grant = gr.split_grant("windy-pro", "a" * 40, fs)
assert [f.path for f in grant] == ["src/client/desktop/main.js"]
assert [f.path for f in lane] == ["account-server/src/llm.ts"]
assert gr.split_grant("windy-chat", "a" * 40, fs) == (fs, [])

View File

@@ -0,0 +1,77 @@
"""lockbox-put against a LOCAL fake lockbox repo only (never the real one)."""
from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
FAKE = "Zk3vQ8mT1pLw7Xn2Rb5Hd9Yc" # synthetic
def sh(*a, cwd=None):
return subprocess.run(a, cwd=cwd, check=True, capture_output=True, text=True)
def make_remote(tmp_path):
work = tmp_path / "seed"
work.mkdir()
sh("git", "init", "-q", "-b", "main", cwd=work)
(work / "ACCESS_LOCKBOX.md").write_text("# LOCKBOX\n\n- **`EXISTING_KEY`**: `abcdefgh12345678`\nOLD_ENV_KEY=whatever123456\n")
sh("git", "add", "-A", cwd=work)
sh("git", "-c", "user.name=t", "-c", "user.email=t@t", "commit", "-qm", "seed", cwd=work)
bare = tmp_path / "remote.git"
sh("git", "clone", "-q", "--bare", str(work), str(bare))
return bare
def put(tmp_path, bare, key, content, mode=0o600):
f = tmp_path / "val"
f.write_text(content)
os.chmod(f, mode)
e = {**os.environ, "LOCKBOX_PUT_REPO": str(bare), "LOCKBOX_PUT_NO_PR": "1", "HOME": str(tmp_path / "h")}
(tmp_path / "h" / ".cache").mkdir(parents=True, exist_ok=True)
r = subprocess.run([sys.executable, str(ROOT / "scripts" / "lockbox_put.py"), key, str(f), "--lane", "test", "--note", "n"],
capture_output=True, text=True, env=e)
return r.returncode, r.stdout + r.stderr
def test_appends_one_key_by_branch_and_never_echoes_value(tmp_path):
bare = make_remote(tmp_path)
rc, out = put(tmp_path, bare, "NEW_TEST_KEY", FAKE)
assert rc == 0 and "pushed branch lockbox-put/new_test_key-" in out
assert FAKE not in out and FAKE[:6] not in out
br = [b.strip() for b in sh("git", "branch", "--list", "lockbox-put/*", cwd=bare).stdout.splitlines()]
assert len(br) == 1
diff = sh("git", "diff", "--numstat", f"main..{br[0]}", cwd=bare).stdout.split()
assert diff[1] == "0" and diff[2] == "ACCESS_LOCKBOX.md" # additions only
content = sh("git", "show", f"{br[0]}:ACCESS_LOCKBOX.md", cwd=bare).stdout
assert f"- **`NEW_TEST_KEY`**: `{FAKE}`" in content and "EXISTING_KEY" in content
# main is untouched
assert FAKE not in sh("git", "show", "main:ACCESS_LOCKBOX.md", cwd=bare).stdout
def test_refuses_existing_key_both_formats_and_bad_input(tmp_path):
bare = make_remote(tmp_path)
for k in ("EXISTING_KEY", "OLD_ENV_KEY"):
rc, out = put(tmp_path, bare, k, FAKE)
assert rc == 3 and "already exists" in out and FAKE not in out
assert put(tmp_path, bare, "lower_case", FAKE)[0] == 2
assert put(tmp_path, bare, "OK_KEY_1", FAKE, mode=0o644)[0] == 2 # not 0600
assert put(tmp_path, bare, "OK_KEY_2", "has space `tick`")[0] == 2 # unsafe value
assert not sh("git", "branch", "--list", "lockbox-put/*", cwd=bare).stdout.strip() # nothing pushed
def test_symlink_refused(tmp_path):
bare = make_remote(tmp_path)
real = tmp_path / "real"
real.write_text(FAKE)
os.chmod(real, 0o600)
link = tmp_path / "link"
link.symlink_to(real)
e = {**os.environ, "LOCKBOX_PUT_REPO": str(bare), "LOCKBOX_PUT_NO_PR": "1"}
r = subprocess.run([sys.executable, str(ROOT / "scripts" / "lockbox_put.py"), "SYM_KEY", str(link)],
capture_output=True, text=True, env=e)
assert r.returncode == 2 and FAKE not in r.stdout + r.stderr

View File

@@ -351,10 +351,12 @@ class _Guard:
return self.findings
@staticmethod
def status_for(findings, whole_tree):
def status_for(findings, whole_tree, grant=()):
if not findings and grant:
return "success", f"GRANT-WARN {len(grant)}", grant[0]
if not findings:
return "success", "OK: clean", None
return "success", f"WARN {len(findings)}", findings[0]
return "failure", f"BLOCK {len(findings)}", findings[0]
class _F:
@@ -366,12 +368,12 @@ def test_guard_posts_warn_with_a_link_to_the_first_finding(fake, monkeypatch):
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
bridge.post_compute_guard("windy-chat", SHA, "main", False)
assert [(p["context"], p["state"], p["description"]) for p in f.posted] == [
("windy-git/compute-guard", "success", "WARN 1")]
("windy-git/compute-guard", "failure", "BLOCK 1")]
assert f.posted[0]["target_url"].endswith(f"/src/commit/{SHA}/app/llm.py#L7")
def test_guard_same_status_is_not_reposted(fake, monkeypatch):
f = fake(statuses=[{"context": "windy-git/compute-guard", "state": "success", "description": "WARN 1"}])
f = fake(statuses=[{"context": "windy-git/compute-guard", "state": "failure", "description": "BLOCK 1"}])
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
bridge.post_compute_guard("windy-chat", SHA, "main", False)
assert f.posted == []
@@ -385,11 +387,11 @@ def test_guard_that_cannot_run_posts_nothing(fake, monkeypatch):
def test_ci_hygiene_posts_under_its_own_context(fake, monkeypatch):
f = fake(statuses=[{"context": "windy-git/compute-guard", "state": "success", "description": "WARN 1"}])
f = fake(statuses=[{"context": "windy-git/compute-guard", "state": "failure", "description": "BLOCK 1"}])
monkeypatch.setitem(sys.modules, "ci_hygiene", _Guard([_F()]))
bridge.post_ci_hygiene("windy-chat", SHA, "main", True)
# the compute-guard status with the same description must not suppress it
assert [(p["context"], p["description"]) for p in f.posted] == [("windy-git/ci-hygiene", "WARN 1")]
assert [(p["context"], p["description"]) for p in f.posted] == [("windy-git/ci-hygiene", "BLOCK 1")]
def test_retargeted_pr_gets_a_fresh_mirror_on_the_new_base(fake):
@@ -411,3 +413,51 @@ def test_unchanged_base_leaves_the_mirror_alone(fake):
f = fake(gh_prs=gh, wg_prs=[{"number": 3, "title": "[GH#5] t", "base": {"ref": "main"}}])
bridge.sync_prs("windy-chat")
assert f.closed == [] and f.opened == []
def test_named_no_daemon_job_is_not_posted_for_that_repo_only(fake, monkeypatch):
"""eternitas ci/build needs Docker but its name doesn't say so (option A, 09-23)."""
monkeypatch.setattr(bridge, "NO_DAEMON_NAMED", {"eternitas": {"ci/build"}})
runs = [_run(1, "ci.yml", "build", "failure"), _run(2, "ci.yml", "test", "success")]
f = fake(runs=runs)
bridge.post_statuses("eternitas", SHA)
assert [p["context"] for p in f.posted] == ["windy-git/ci/test"]
f2 = fake(runs=runs)
bridge.post_statuses("windy-chat", SHA)
assert sorted(p["context"] for p in f2.posted) == ["windy-git/ci/build", "windy-git/ci/test"]
def test_default_no_daemon_named_is_empty():
"""eternitas converted its ci/build to a no-Docker ci/smoke (#179); nothing left."""
assert bridge.NO_DAEMON_NAMED == {}
def test_grant_owned_findings_never_block(fake, monkeypatch):
"""Orchestrator 09-23: compute-guard blocks lane-owned code only."""
f = fake()
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
monkeypatch.setitem(sys.modules, "guards_report",
type("GR", (), {"split_grant": staticmethod(lambda r, s, fs: ([], list(fs)))}))
bridge.post_compute_guard("windy-pro", SHA, "main", True)
assert [(p["state"], p["description"]) for p in f.posted] == [("success", "GRANT-WARN 1")]
def test_failed_grant_split_warns_instead_of_blocking(fake, monkeypatch):
def boom(*a):
raise RuntimeError("no bare clone")
f = fake()
monkeypatch.setitem(sys.modules, "compute_guard", _Guard([_F()]))
monkeypatch.setitem(sys.modules, "guards_report", type("GR", (), {"split_grant": staticmethod(boom)}))
bridge.post_compute_guard("windy-pro", SHA, "main", True)
assert [p["state"] for p in f.posted] == ["success"]
@pytest.mark.parametrize("name, hidden", [
("Docker Build", True), ("docker-build", True), ("Docker build", True), ("docker", True),
("Boot smoke (no Docker)", False), ("smoke (no-docker)", False), ("boot without Docker", False),
("smoke", False),
])
def test_no_docker_smoke_jobs_are_posted(name, hidden):
"""windy-search #96: its replacement job says "no Docker" and was hidden."""
assert bridge.needs_daemon("windy-search", "ci", name) is hidden

View File

@@ -0,0 +1,70 @@
"""runner-guard: workflow shapes that hand a self-hosted runner to strangers."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
_spec = importlib.util.spec_from_file_location("runner_guard", ROOT / "scripts" / "runner_guard.py")
rg = importlib.util.module_from_spec(_spec)
sys.modules["runner_guard"] = rg
_spec.loader.exec_module(rg)
def rules(text):
return [(r, ln) for _p, ln, r, _m in rg.lint_text("w.yml", text)]
def test_pull_request_target_always_fails():
assert rules("on: pull_request_target\njobs:\n a:\n runs-on: ubuntu-latest\n steps: []\n")[0][0] == "R1"
def test_fork_pr_on_self_hosted_fails_and_points_at_runs_on():
wf = "on:\n pull_request:\njobs:\n t:\n runs-on: [self-hosted, linux, x64]\n steps: []\n"
assert rules(wf) == [("R2", 5)]
def test_same_repo_gate_or_environment_passes():
gated = ("on: [pull_request]\njobs:\n t:\n if: github.event.pull_request.head.repo.full_name == github.repository\n"
" runs-on: [self-hosted]\n steps: []\n")
env = "on: [pull_request]\njobs:\n t:\n environment: ci\n runs-on: self-hosted\n steps: []\n"
assert rules(gated) == [] and rules(env) == []
def test_outsider_events_on_self_hosted_fail_but_writer_events_pass():
wf = "on:\n issue_comment:\n workflow_run:\n workflows: [x]\njobs:\n t:\n runs-on: self-hosted\n steps: []\n"
assert sorted(r for r, _ in rules(wf)) == ["R3", "R3"]
ok = "on:\n push:\n tags: ['v*']\n workflow_dispatch:\n schedule:\n - cron: '0 3 * * *'\njobs:\n t:\n runs-on: self-hosted\n steps: []\n"
assert rules(ok) == []
def test_expression_runs_on_is_treated_as_self_hosted_and_hosted_runner_is_fine():
expr = "on: pull_request\njobs:\n t:\n runs-on: ${{ matrix.os }}\n steps: []\n"
hosted = "on: pull_request\njobs:\n t:\n runs-on: ubuntu-latest\n steps: []\n"
assert rules(expr) == [("R2", 4)] and rules(hosted) == []
def test_broken_yaml_is_a_finding_and_non_workflows_are_ignored():
assert rules("on: [push\njobs: {")[0][0] == "R0"
assert rules("name: just a file\n") == []
def test_pr_mode_posts_each_status_once(monkeypatch, tmp_path):
calls = []
monkeypatch.setattr(rg, "STATE", str(tmp_path / "s.json"))
monkeypatch.setattr(rg, "public_repos", lambda owners: [("o/r", "main")])
monkeypatch.setattr(rg, "file_at", lambda *a: "on: push\\njobs:\\n t:\\n runs-on: self-hosted\\n steps: []\\n")
def fake_gh(*args, check=True):
if args[0].startswith("repos/o/r/pulls?"):
return "7 abc123 o/r\\n"
if args[0].endswith("/files?per_page=100"):
return ".github/workflows/ci.yml\\n"
calls.append(args[0])
return ""
monkeypatch.setattr(rg, "gh", fake_gh)
rg.cmd_pr(["o"], post=True)
rg.cmd_pr(["o"], post=True)
assert calls == ["repos/o/r/statuses/abc123"]

View File

@@ -0,0 +1,156 @@
"""Secret guard: shapes, hash-only findings, allow by hash."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "scripts"))
_spec = importlib.util.spec_from_file_location("secret_guard", ROOT / "scripts" / "secret_guard.py")
sg = importlib.util.module_from_spec(_spec)
sys.modules["secret_guard"] = sg
_spec.loader.exec_module(sg)
ss = sg.ss
# Synthetic shapes only: none of these is a real credential.
TG = "1234567890:" + "A" * 35
CASES = [
("telegram bot token", f"TELEGRAM_BOT_TOKEN={TG}"),
("github token", "token = 'ghp_" + "a1" * 18 + "'"),
("aws access key", "aws_access_key_id = AKIA" + "ABCDEFGHIJKLMNOP"),
("slack token", "xoxb-" + "1234567890-abcdefghij"),
("anthropic key", "ANTHROPIC_API_KEY=sk-ant-" + "x" * 30),
("openai key", "OPENAI_API_KEY=sk-proj-" + "y" * 40),
("stripe live key", "STRIPE=sk_live_" + "z" * 24),
("google api key", "key=AIza" + "B" * 35),
("private key block", "-----BEGIN OPENSSH PRIVATE KEY-----"),
]
@pytest.mark.parametrize("kind, text", CASES)
def test_each_shape_is_found_and_only_its_hash_is_kept(kind, text):
hits = sg.scan_line("app.py", text)
assert [k for k, _ in hits] == [kind]
match = hits[0][1]
assert match.startswith(f"{kind} #") and len(match.rsplit("#", 1)[1]) == 8
# house rule 10: the value itself must never appear in a finding
secret = text.split("=", 1)[-1].strip(" '")
assert secret not in match
@pytest.mark.parametrize("text", [
"sha512-" + "Q" * 86 + "==", # lockfile integrity
"version: 12345678:abc", # short, not a token
"sk-ant-short", # too short
"re_test_register_sends_verification", # windy-pro's fake Resend key
"ANTHROPIC_API_KEY=", # a name, not a value
])
def test_non_secrets_are_not_flagged(text):
assert sg.scan_line("x", text) == []
def test_anthropic_key_is_not_double_counted_as_openai():
assert [k for k, _ in sg.scan_line("x", "sk-ant-" + "q" * 40)] == ["anthropic key"]
def test_allow_is_by_hash_only():
F = sg.cg.Finding
fake = F("tests/t.py", 3, "telegram bot token", f"telegram bot token #{ss.h8(TG)}")
real = F("tests/t.py", 9, "telegram bot token", "telegram bot token #deadbeef")
allow = {"windy-chat": {"hashes": {ss.h8(TG)}, "paths": []}}
assert sg._drop_allowed("windy-chat", [fake, real], allow) == [real]
assert sg._drop_allowed("windy-mail", [fake], allow) == [fake]
def test_private_key_blocks_are_allowed_by_path_never_by_hash():
F = sg.cg.Finding
hdr = "private key block #" + ss.h8("-----BEGIN PRIVATE KEY-----")
test_key = F("tests/keys/test.pem", 1, "private key block", hdr)
prod_key = F("deploy/prod.pem", 1, "private key block", hdr)
allow = {"r": {"hashes": {hdr.rsplit("#", 1)[1]}, "paths": [("tests/keys/*", {"private key block"})]}}
assert sg._drop_allowed("r", [test_key, prod_key], allow) == [prod_key]
def test_path_allow_cannot_cover_real_token_kinds(tmp_path):
bad = tmp_path / "a.yml"
bad.write_text("allow:\n - repo: r\n paths: [tests/*]\n kinds: [telegram bot token]\n reason: no\n")
with pytest.raises(ValueError):
sg.load_allow(bad)
def test_shipped_allow_file_never_excuses_the_real_leaked_tokens():
a = sg.load_allow()
every = set().union(*(v["hashes"] for v in a.values())) if a else set()
assert not {"1354fc9b", "d49dc2ba"} & every # real (now revoked) credentials: remove, never allow
def test_allow_file_loads_and_needs_reasons(tmp_path):
assert isinstance(sg.load_allow(), dict)
bad = tmp_path / "a.yml"
bad.write_text("allow:\n - repo: r\n hashes: [abcd1234]\n")
with pytest.raises(ValueError):
sg.load_allow(bad)
def test_block_and_warn(monkeypatch):
f = sg.cg.Finding("a.py", 1, "github token", "github token #abcd1234")
monkeypatch.setattr(sg, "MODE", "block")
assert sg.status_for([f], False)[0] == "failure"
assert sg.status_for([], False, grant=[f])[0] == "success"
monkeypatch.setattr(sg, "MODE", "warn")
state, desc, _ = sg.status_for([f], True)
assert state == "success" and desc.startswith("⚠ WARN (not blocking): 1 secret-shaped string in tree")
def test_public_scan_excuses_by_hash_and_by_path():
spec = importlib.util.spec_from_file_location("public_secret_scan", ROOT / "scripts" / "public_secret_scan.py")
ps = importlib.util.module_from_spec(spec)
spec.loader.exec_module(ps)
allow = {"windy-agent": {"hashes": {"aaaa1111"}, "paths": [("tests/keys/*", {"private key block"})]}}
assert ps.excused("windy-agent", "openai key", "aaaa1111", ["tests/x.py"], allow)
assert not ps.excused("windy-agent", "telegram bot token", "1354fc9b", ["tests/test_log_redaction.py"], allow)
assert ps.excused("windy-agent", "private key block", "ffff0000", ["tests/keys/a.pem"], allow)
# a PEM header anywhere outside the allowed paths still counts
assert not ps.excused("windy-agent", "private key block", "ffff0000", ["tests/keys/a.pem", "deploy/k.pem"], allow)
assert not ps.excused("other", "openai key", "aaaa1111", ["x"], allow)
HEX32 = "0123456789abcdef" * 2 # synthetic
def test_twilio_shapes_hash_only_and_no_md5_noise():
kinds = lambda t: [k for k, _ in ss.find(t)] # noqa: E731
assert kinds(f'TWILIO_AUTH_TOKEN = "{HEX32}"') == ["32-hex secret assignment"]
assert kinds(f"auth_token: {HEX32}") == ["32-hex secret assignment"]
assert kinds("AC" + HEX32) == ["twilio sid/api key"]
assert kinds("SK" + HEX32) == ["twilio sid/api key"]
# plain md5 / uuid-without-dashes / a 64-hex sha256 are NOT secrets by shape
assert kinds(f"md5 = {HEX32}") == []
assert kinds(f"checksum_key = {HEX32}{HEX32}") == []
assert kinds(f"name = 'x{HEX32}'") == []
# the hash is of the value alone, so renaming the variable keeps the same allow hash
a = ss.find(f"A_TOKEN={HEX32}")[0][1]
b = ss.find(f"OTHER_SECRET: '{HEX32}'")[0][1]
assert a == b == ss.h8(HEX32)
assert HEX32 not in repr(ss.find(f"A_TOKEN={HEX32}"))
def test_warn_kinds_do_not_block(monkeypatch):
f = sg.cg.Finding("a.py", 1, "32-hex secret assignment", "32-hex secret assignment #abcd1234")
monkeypatch.setattr(sg, "MODE", "block")
monkeypatch.setattr(sg, "WARN_KINDS", {"32-hex secret assignment"})
assert sg.status_for([f], True)[0] == "success"
monkeypatch.setattr(sg, "WARN_KINDS", set())
assert sg.status_for([f], True)[0] == "failure"
def test_pypi_token_shape_hash_only():
tok = "pypi-AgE" + "Ab1_-" * 20 # synthetic
got = ss.find(f"password = {tok}")
assert [k for k, _ in got] == ["pypi token"] and got[0][1] == ss.h8(tok)
assert tok not in repr(got)
assert ss.find("pypi-AgE-too-short") == []

View File

@@ -0,0 +1,106 @@
"""secret-scan + env-names: findings carry label/location/hash, NEVER a value or fragment."""
from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
SCRIPTS = ROOT / "scripts"
# Synthetic, random-looking, never real. FAKE_LB is in the fake lockbox; TWI matches a shape.
FAKE_LB = "k7Xv2QpLm9RtZw4HnB8dYc3S"
TWI = "9f3a7c1e5b2d48806a1f4e7d2c9b0835"
def run(script, *args, env=None):
e = {**os.environ, **(env or {})}
r = subprocess.run([sys.executable, str(SCRIPTS / script), *args], capture_output=True, text=True, env=e)
return r.returncode, r.stdout + r.stderr
def no_fragment(out: str, value: str, n: int = 6):
assert value not in out
for i in range(len(value) - n + 1):
assert value[i:i + n] not in out, f"fragment of the value leaked at {i}"
def seeded(tmp_path):
lb = tmp_path / "lockbox.md"
lb.write_text(f"FAKE_VENDOR_API_KEY={FAKE_LB}\nNOTE: nothing here\n")
repo = tmp_path / "repo"
repo.mkdir()
g = lambda *a: subprocess.run(["git", "-C", str(repo), *a], check=True, capture_output=True) # noqa: E731
g("init", "-q", "-b", "main")
g("config", "user.email", "t@t")
g("config", "user.name", "t")
(repo / "app.py").write_text(f'KEY = "{FAKE_LB}"\nTWILIO_AUTH_TOKEN = "{TWI}"\n')
g("add", "-A")
g("commit", "-qm", "add secrets")
(repo / "app.py").write_text("KEY = None\n") # removed from HEAD, still in history
g("commit", "-qam", "remove")
return lb, repo
def test_tree_scan_labels_locations_no_value(tmp_path):
lb, repo = seeded(tmp_path)
(repo / "live.py").write_text(f'x = "{FAKE_LB}"\n')
rc, out = run("secret_scan.py", str(repo / "live.py"), env={"SECRET_SCAN_LOCKBOX_PATHS": str(lb)})
assert rc == 1
assert "live.py:1" in out and "lockbox:FAKE_VENDOR_API_KEY" in out
no_fragment(out, FAKE_LB)
def test_history_finds_removed_secret_with_commit(tmp_path):
lb, repo = seeded(tmp_path)
rc, out = run("secret_scan.py", str(repo), "--history", env={"SECRET_SCAN_LOCKBOX_PATHS": str(lb)})
assert rc == 1
assert "app.py:1" in out and "commit=" in out and "lockbox:FAKE_VENDOR_API_KEY" in out
assert "app.py:2" in out and "32-hex secret assignment" in out
no_fragment(out, FAKE_LB)
no_fragment(out, TWI)
def test_repo_flag_clones_scans_and_cleans_up(tmp_path):
lb, repo = seeded(tmp_path)
cache = tmp_path / "home"
(cache / ".cache").mkdir(parents=True)
rc, out = run("secret_scan.py", "--repo", str(repo),
env={"SECRET_SCAN_LOCKBOX_PATHS": str(lb), "HOME": str(cache)})
assert rc == 1 and "app.py:1" in out
no_fragment(out, FAKE_LB)
assert not [p for p in (cache / ".cache").iterdir() if p.name.startswith("secret-scan-")]
def test_clean_tree_and_bad_input(tmp_path):
(tmp_path / "ok.txt").write_text("hello world\n")
rc, out = run("secret_scan.py", str(tmp_path / "ok.txt"), "--no-lockbox")
assert rc == 0 and "0 finding(s)" in out
rc, out = run("secret_scan.py", str(tmp_path), "--history", "--no-lockbox")
assert rc == 2 and "needs a git repo" in out
def test_env_names_never_prints_values(tmp_path):
a = tmp_path / "a.env"
b = tmp_path / "b.env"
a.write_text(f"# c\nexport DB_PASSWORD={FAKE_LB}\nTOKEN=\"{TWI}\"\nEMPTY=\nONLY_A=1\n")
b.write_text(f"DB_PASSWORD={FAKE_LB}\nTOKEN=different-value-here\nONLY_B=2\n")
rc, out = run("env_names.py", str(a), "--hash")
assert rc == 0 and "DB_PASSWORD" in out and "set" in out and "empty" in out and "len=24" in out
assert "sha256:" in out
no_fragment(out, FAKE_LB)
no_fragment(out, TWI, 7)
rc, out = run("env_names.py", str(a), "--compare", str(b))
assert "DB_PASSWORD" in out and "SAME" in out and "DIFFERENT" in out
assert "only-in-A" in out and "only-in-B" in out
no_fragment(out, FAKE_LB)
rc, out = run("env_names.py", str(tmp_path / "missing.env"))
assert rc == 2
def test_shapes_are_the_guards_shapes():
sys.path.insert(0, str(SCRIPTS))
import secret_scan as sc
import secret_shapes as ss
assert sc.ss is ss # one source of shapes: a new guard shape is automatically a scan shape

View File

@@ -2,4 +2,12 @@
# a host port. House rule 6 (09-23): installs come from a lockfile. Every entry
# is an exception and MUST say why. Paths are fnmatch globs from the repo root.
# Owner: Windy Git lane (13); changes go through the orchestrator.
allow: []
allow:
- repo: windy-pro
paths: [".github/workflows/ci.yml"]
# ONLY the old deploy job's two docker lines. That job is `if: false`
# (CD boundary, 2026-07), and the compose line runs ON windyword.ai inside
# the ssh string. Any other docker step in ci.yml still flags.
matches: ['docker build -f account-server/Dockerfile -t windy-pro:', 'docker compose down && docker compose up -d --build']
reason: "needs-docker false positive: the deploy job is if: false and its compose runs on the remote host over ssh. Added with the needs-docker rule (orchestrator option A, 09-23)."

View File

@@ -45,3 +45,8 @@ allow:
- repo: windy-git
paths: ["scripts/compute_guard.py", "ci/compute-guard-allow.yml"]
reason: "The guard's own pattern list and this file."
- repo: windy-mind
paths: ["*"]
matches: [':11434']
reason: "Windy Mind IS the compute door (endpoint + key); it may call Ollama. Only the Ollama port is allowed here, any provider host/SDK in Mind still flags."

62
ci/secret-guard-allow.yml Normal file
View File

@@ -0,0 +1,62 @@
# Secret guard allow-list: KNOWN FAKE values that look like secrets (test
# fixtures, docs). Allowed BY HASH (sha256[:8] of the value), so a real secret
# in the same file still flags. Private-key blocks (the match is only the BEGIN
# line, same hash everywhere) are allowed by PATH + kind instead.
# Every entry MUST say why. Owner: Windy Git lane (13); changes via the orchestrator.
# Triage 09-24 (values never printed): each hash checked against every version of
# the lockbox; fakes judged by impossible length for the kind (real Anthropic keys
# ~108 chars, OpenAI 51 or 160+), fake-words, or identity with upstream public
# fixtures. NOT allowed, remove instead: 1354fc9b (old @Windy_0_bot token) and
# d49dc2ba (old Anthropic key), both real and revoked, in public windy-agent.
allow:
- repo: windy-code
hashes: [ac9265e5, 46eb1235]
reason: "Upstream microsoft/vscode terminalEnvironment.test.ts fixtures (identical hash upstream; public)."
- repo: windy-code
paths: ["build/azure-pipelines/common/publish.ts"]
kinds: [private key block]
reason: "Upstream VS Code build script (PEM header string in code, not a key)."
- repo: windy-agent
hashes: [a9235a6d, dd6a2baa, 02c362d8, a6f6ff79, f7503b21, d0c94833, 4ab092e3, e3aa1eb8, 833382ee]
reason: "Redaction/sanitizer test fixtures; lengths impossible for real Anthropic/OpenAI keys; never in the lockbox."
- repo: windy-agent
hashes: [89bd408f, b8c94b72, bf23cdf5, d1d85dfe, e3e07f06]
reason: "09-24 #395 replacement fixtures (each contains FAKE; token-SHAPED on purpose so redaction tests prove real tokens are scrubbed); verified by Windy Agent sha-for-sha against every lockbox version: never real. Weekly scan 09-28."
- repo: windy-agent
paths: ["tests/test_agent_keys.py"]
kinds: [private key block]
reason: "Test-generated key material for agent-key tests."
- repo: windy-mind
hashes: [f8a630b2]
reason: "Provider test fixture (27 chars; a real Anthropic key is ~108)."
- repo: windy-pro
hashes: [756de8d8, 7828319d, 1a5d44a2]
reason: ".env.production.example placeholder + crash-summary test fixtures (AWS doc EXAMPLE key shape, short fake Slack token)."
- repo: windy-pro
paths: ["account-server/docs/oauth-providers.md"]
kinds: [private key block]
reason: "Docs show the PEM header format; no key material."
- repo: windytalk
hashes: [baf8656a]
reason: "Diagnostics redaction test fixture (fake-word in value)."
- repo: eternitas
paths: ["tests/golden_vectors/**", "tests/test_soul_vault_key_separation.py"]
kinds: [private key block]
reason: "Test vectors and throwaway keys for signature/vault tests."
- repo: windy-drops
paths: ["tools/conformance/test-keys/*"]
kinds: [private key block]
reason: "Conformance-suite test keys (named test-private.pem)."
- repo: windy-git
paths: ["api/tests/test_secret_guard.py"]
kinds: [private key block]
reason: "The guard's own test uses a PEM header string as a sample."
- repo: windy-code
hashes: ["23f32607"]
reason: "VS Code OSS extensions' package.json aiKey: Microsoft's public telemetry (App Insights) key, shipped in every VS Code build; not a Windy credential."
- repo: windytalk
hashes: ["c4189d79"]
reason: "apps/desktop/test/diagnostics.test.ts redaction fixture (hexSecret beside a fake sk-ant token); hash checked against the lockbox 10-01: not present."
- repo: windy-agent
hashes: ["84e0c0ea"]
reason: "tests/test_log_redaction.py:56 Z.ai redaction fixture REPLACED by windy-agent #412 with a synthetic value; hash checked against the lockbox 10-01: not present."

View File

@@ -0,0 +1,61 @@
{{/* Windy Git override of Gitea 1.24.6 templates/user/auth/link_account.tmpl.
Forge registration is CLOSED at launch (Grant, 09-23), so a Windy account
with no forge account used to land on Gitea's raw "Registration is disabled"
error. That case now gets a plain invite-only page; every other case is
Gitea's template unchanged (re-diff it on Gitea upgrades). No change to who
can register. */}}
{{if and .DisableRegistration (not .user_exists)}}
{{template "base/head" .}}
<div role="main" aria-label="Windy Git is invite-only" class="page-content user link-account">
<div class="ui middle very relaxed page grid">
<div class="column tw-my-5">
<div class="tw-flex tw-flex-col tw-gap-4 tw-max-w-2xl tw-m-auto">
<h4 class="ui top attached header center">Windy Git is invite-only while we launch</h4>
<div class="ui attached segment">
<p>You're signed in with your Windy account{{if or .user_name .email}} as <strong>{{or .user_name .email}}</strong>{{end}}. Nothing is wrong with it: Windy Git just isn't open to every account yet.</p>
<p>We'll let you know when it opens.</p>
<a class="ui primary button" href="https://app.windyword.ai/dashboard">Back to your dashboard</a>
</div>
</div>
</div>
</div>
</div>
{{template "base/footer" .}}
{{else}}
{{template "base/head" .}}
<div role="main" aria-label="{{.Title}}" class="page-content user link-account">
<overflow-menu class="ui secondary pointing tabular top attached borderless menu secondary-nav">
<div class="overflow-menu-items tw-justify-center">
<!-- TODO handle .ShowRegistrationButton once other login bugs are fixed -->
{{if not .AllowOnlyInternalRegistration}}
<a class="item {{if not .user_exists}}active{{end}}"
data-tab="auth-link-signup-tab">
{{ctx.Locale.Tr "auth.oauth_signup_tab"}}
</a>
{{end}}
<a class="item {{if .user_exists}}active{{end}}"
data-tab="auth-link-signin-tab">
{{ctx.Locale.Tr "auth.oauth_signin_tab"}}
</a>
</div>
</overflow-menu>
<div class="ui middle very relaxed page grid">
<div class="column tw-my-5">
{{/* these styles are quite tricky but it needs to be the same as the signin page */}}
<div class="ui tab {{if not .user_exists}}active{{end}}" data-tab="auth-link-signup-tab">
<div class="tw-flex tw-flex-col tw-gap-4 tw-max-w-2xl tw-m-auto">
{{if .AutoRegistrationFailedPrompt}}<div class="ui message">{{.AutoRegistrationFailedPrompt}}</div>{{end}}
{{template "user/auth/signup_inner" .}}
</div>
</div>
<div class="ui tab {{if .user_exists}}active{{end}}" data-tab="auth-link-signin-tab">
<div class="tw-flex tw-flex-col tw-gap-4 tw-max-w-2xl tw-m-auto">
{{template "user/auth/signin_inner" .}}
</div>
</div>
</div>
</div>
</div>
{{template "base/footer" .}}
{{end}}

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Windy Git nightly STATE backup (Postgres + Gitea config + repos -> encrypted restic in R2)
After=network-online.target docker.service
[Service]
Type=oneshot
WorkingDirectory=/srv/windygit/src
# R2 credentials come from the .env; the restic password from /etc/windygit/restic.pass
# (root 600; the same value lives in the lockbox as RESTIC_WINDYGIT_PASSWORD).
EnvironmentFile=/srv/windygit/src/.env
ExecStart=/bin/bash /srv/windygit/src/scripts/backup_state.sh
Nice=10
IOSchedulingClass=idle
TimeoutStartSec=3h

View File

@@ -0,0 +1,3 @@
[Service]
ExecStart=
ExecStart=/usr/local/bin/windy-job windygit-state-backup 26h --expect "ok — state backed up" --owner 13 -- /bin/bash /srv/windygit/src/scripts/backup_state.sh

View File

@@ -0,0 +1,11 @@
[Unit]
Description=Nightly Windy Git state backup
[Timer]
# 03:07 local, clear of the git-bundle backup at 04:17.
OnCalendar=*-*-* 03:07:00
Persistent=true
RandomizedDelaySec=300
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,3 @@
[Service]
# Orchestrator 09-23: compute-guard BLOCKS lane-owned findings; Grant-owned (ci/grant-owned.yml) stay WARN.
Environment=COMPUTE_GUARD_MODE=block

View File

@@ -0,0 +1,3 @@
[Service]
# Orchestrator 09-24: secret-guard BLOCKS lane-owned findings; Grant-owned stay WARN.
Environment=SECRET_GUARD_MODE=block

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Nightly (Boss 10-01): runner-guard over the default branch of EVERY public repo in Grant's
# 5 GitHub accounts (runs on Veron: windy-git scripts/runner_guard.py report). Writes
# ~/windy-orchestra/RUNNER_GUARD.md (repo, file:line, rule; no file content), appends ONE
# BOARD line per NEW hit vs the last run, and prints "runner-guard sweep: N hit(s)" last, so
# the windy-job heartbeat (--expect "runner-guard sweep: 0 hit") goes red while any hit exists.
set -euo pipefail
page=~/windy-orchestra/RUNNER_GUARD.md
state=~/.local/state/runner-guard-sweep.txt
mkdir -p "$(dirname "$state")"
out=$(timeout 900 ssh -o BatchMode=yes -o ConnectTimeout=15 ts-veron \
'cd /srv/windygit/src && timeout 850 python3 scripts/runner_guard.py report' || true)
summary=$(grep '^# runner-guard sweep:' <<<"$out" || echo "# runner-guard sweep: ERROR (no summary)")
hits=$(grep -v '^#' <<<"$out" | grep . || true)
{
echo "# Runner guard: stranger-code paths to self-hosted runners ($(date -u '+%Y-%m-%d %H:%MZ'))"
echo "_Nightly; windy-git scripts/runner_guard.py. R1 pull_request_target · R2 fork PR on self-hosted without a same-repo/environment gate · R3 outsider events (issue_comment, workflow_run, ...) on self-hosted · R0 unparseable._"
echo; echo "${summary#\# }"; echo
echo "| repo | file:line | rule | fix |"; echo "|---|---|---|---|"
while IFS=$'\t' read -r repo loc rule msg; do [[ -n $repo ]] && echo "| $repo | $loc | $rule | $msg |"; done <<<"$hits"
} > "$page"
new=$(comm -13 <(sort -u "$state" 2>/dev/null || true) <(cut -f1-3 <<<"$hits" | sort -u))
cut -f1-3 <<<"$hits" | sort -u > "$state"
if [[ -n "$new" ]]; then
n=$(grep -c . <<<"$new")
echo "$(date -u +%Y-%m-%dT%H:%MZ) Windy Git: 🚨 runner-guard: $n NEW stranger-code path(s) to a self-hosted runner in public repos; see ~/windy-orchestra/RUNNER_GUARD.md" >> ~/windy-orchestra/BOARD.md
fi
echo "${summary#\# }"

View File

@@ -0,0 +1,7 @@
[Unit]
Description=Nightly runner-guard sweep of every PUBLIC repo's workflows (Windy Git lane)
[Service]
Type=oneshot
ExecStart=/usr/local/bin/windy-job windy-runner-guard-sweep 26h --expect "runner-guard sweep: 0 hit" --owner 13 -- %h/bin/nightly-runner-guard-sweep.sh
TimeoutStartSec=1200

View File

@@ -0,0 +1,9 @@
[Unit]
Description=Nightly runner-guard sweep
[Timer]
OnCalendar=*-*-* 09:40:00 UTC
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -32,7 +32,7 @@ services:
gitea:
# G2.1 — PIN AN EXACT VERSION. Never `latest`. Record it in SUBSTRATE.md.
image: docker.io/gitea/gitea:1.24.6
image: docker.io/gitea/gitea:1.24.7
environment:
GITEA__database__DB_TYPE: postgres
GITEA__database__HOST: db:5432

31
docs/CUTOVER-PRIMARY.md Normal file
View File

@@ -0,0 +1,31 @@
# Checklist: making Windy Git the PRIMARY home of one repo (after launch)
Status 2026-10-01: DRAFT, nothing flipped. GitHub stays the source of truth until Grant says otherwise.
First candidate: `windy-git` itself (public, low blast radius). GitHub becomes the free off-site push-mirror.
## Preconditions (all must be true)
- [x] Encrypted state backup (Postgres + Gitea config + repos) nightly, restore drill passed cross-host (10-01).
- [x] Gitea on a patched release (1.24.7); upgrade path = snapshot first (docs/RESTORE-DRILL.md).
- [ ] Heartbeat `windygit-state-backup` in heartbeats-veron expected list (asked Cloud/Super Admin 10-01).
- [ ] A missed/failed backup PAGES (heartbeat 26h), tested once by skipping a night in a drill.
- [ ] Boss/Grant capacity call on Veron (dedicated non-SMR volume for DB + git storage; root disk < 80%).
- [ ] Post-launch freeze lifted (Hub). No cutover during store review or a launch window.
## Cutover for ONE repo (reversible at every step)
1. Announce on BOARD; tell every consuming lane (no schema drift rule). Pause the sync for that repo only:
remove it from `REPOS` in `scripts/sync_from_github.sh` FIRST (the sync force-overwrites Windy Git).
2. Verify Windy Git main == GitHub main (sha equal), all branches/tags present, LFS (if any) in R2.
3. Add a PUSH-mirror on the Windy Git repo -> GitHub (deploy key or scoped token, write on that repo only,
interval 8h + on-commit), so GitHub keeps a current copy. Test with a throwaway branch.
4. Flip the lanes' remote: `origin` = Windy Git (SSH/HTTPS via Windy SSO token), `github` = secondary.
Lanes push to Windy Git only; the mirror carries it to GitHub.
5. CI keeps running here (no change); remove the `pr_status_bridge` mirror-PR duplication for that repo
(PRs are now native here; the bridge's GitHub status posting for it can stay for any open GitHub PRs).
6. Watch 3 days: mirror lag, backup includes the new writes, no push rejected, no lane still pushing to GitHub.
7. Rollback at any time: re-add the repo to `REPOS` (sync resumes GitHub->Windy Git, GitHub is intact via the
push-mirror) and point lanes' remote back. Nothing is destroyed in either direction.
## NOT in scope for a first cutover
Deploy workflows (stay disabled; deploys remain manual until a separate runner + scoped deploy keys exist),
credential repos (soul/anima/kit-army-config), windy-pro (importer refuses by name), opening the forge to
other members (Grant 09-23: registration closed; Hub 10-01: jit false, 4 conditions before any change).

View File

@@ -110,7 +110,12 @@ their CI permanently, not a stopgap:
- **Image-build jobs** (name matches `docker`) post nothing: job containers
have no Docker daemon by design (I-5), so they are red on every commit. A
rootless builder (BuildKit rootless / buildx in the capped dind) is the open
decision that would bring them back.
decision that would bring them back. Jobs that need Docker but are named otherwise go in
`BRIDGE_NO_DAEMON` (empty since eternitas converted to ci/smoke, #179). DECIDED 09-23 (orchestrator,
option A): lanes convert these jobs to no-Docker smoke tests (job `services:` +
start the app + curl /health); the real image build is the deploy step on the
target host. ci-hygiene flags docker build/compose/run in workflows ("needs docker").
No host Docker socket for CI without a separate decision.
**Onboarding another private repo** — the promotion steps below, then:

33
docs/RESTORE-DRILL.md Normal file
View File

@@ -0,0 +1,33 @@
# Restoring Windy Git from the encrypted state backup
What is backed up (`scripts/backup_state.sh`, restic repo `s3:…/windy-git-backups/restic`, tag `windygit-state`):
both Postgres databases (`gitea`, `windygit`, custom-format dumps + globals), the whole Gitea data root
(`/srv/windygit/git`: config, jwt, attachments, avatars, templates AND the bare repositories),
`/srv/windygit/src/.env`, `deploy/runner/.env`, `/etc/cloudflared`, the windygit systemd drop-ins.
NOT in it: the restic password itself (lockbox `RESTIC_WINDYGIT_PASSWORD`) and the R2 access key
(scoped token `windy-git-r2-scoped`, lockbox). Never print either: use `lockbox-get KEY FILE`.
## Drill (any machine with restic + docker; proven on Veron 2026-10-01, counts identical)
export RESTIC_PASSWORD_FILE=<0600 file from lockbox-get RESTIC_WINDYGIT_PASSWORD>
export AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=… # from lockbox-get, into env, not echoed
export RESTIC_REPOSITORY=s3:https://<R2 account>.r2.cloudflarestorage.com/windy-git-backups/restic
restic snapshots --tag windygit-state
restic restore latest --tag windygit-state --target /var/tmp/wg-drill
docker run -d --name wg-drill-pg -e POSTGRES_PASSWORD=<random> -e POSTGRES_USER=drill postgres:16-alpine
for db in gitea windygit; do
docker exec wg-drill-pg psql -U drill -d postgres -c "create database $db"
docker exec -i wg-drill-pg pg_restore -U drill -d $db --no-owner --no-privileges \
< /var/tmp/wg-drill/var/backups/windygit-state/$db.dump
done
# compare row counts with live (or with the last known): repository, issue, pull_request, "user",
# external_login_user, access_token, action_run, action_run_job
docker rm -f wg-drill-pg; rm -rf /var/tmp/wg-drill
## Real disaster (Veron lost)
1. New Linux host with Docker, a Cloudflare tunnel connector, the repo (`git clone` from GitHub: windy-git).
2. `restic restore latest --tag windygit-state --target /` (puts /srv/windygit/git, the .env files, /etc/cloudflared back).
3. `docker compose -p windy-git up -d db`, then pg_restore both dumps into it (as above, into the real db names/owner from `.env`).
4. `docker compose -p windy-git up -d` + `deploy/runner` runners; re-register runners if the token changed.
5. Verify: `/api/healthz`, Windy SSO login, `git ls-remote`, one CI run. GitHub is still the source of truth for code,
so repo content can also be re-synced from there; the database is what only this backup holds.
Retention: 14 daily / 8 weekly / 6 monthly (prune on Sundays). Integrity: every run does `restic check --read-data-subset=2%`.

66
scripts/backup_state.sh Executable file
View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Nightly STATE backup: everything git bundles do NOT hold (SOTU 10-01: 625 issues/PRs, users,
# SSO links, CI history, settings lived on one unbacked-up host). Encrypted restic repo in R2.
# - Postgres: every database (custom-format dump, restore-listable) + globals
# - Gitea config/data (app.ini, jwt, attachments, avatars, templates) + the bare repositories
# - the deploy .env files, systemd drop-ins and the cloudflared tunnel config (needed to rebuild)
# The restic password lives in /etc/windygit/restic.pass (root 600) AND the lockbox
# (RESTIC_WINDYGIT_PASSWORD): a lost Veron must not lose the backups. NEVER echo env/values here.
# Restore: docs/RESTORE-DRILL.md. Bounded: every docker exec runs under `timeout` (a hung
# runc exec in the IO stall wedged the sync on 09-23).
set -euo pipefail
log() { echo "[backup_state $(date -u +%FT%TZ)] $*"; }
: "${R2_ACCOUNT_ID:?}" "${R2_ACCESS_KEY_ID:?}" "${R2_SECRET_ACCESS_KEY:?}"
PASSFILE="${RESTIC_PASSWORD_FILE:-/etc/windygit/restic.pass}"
[[ -s "$PASSFILE" ]] || { log "FATAL: $PASSFILE missing/empty: refusing to report a backup that did not happen"; exit 1; }
export RESTIC_PASSWORD_FILE="$PASSFILE"
export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID" AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
export RESTIC_REPOSITORY="${RESTIC_REPOSITORY:-s3:https://${R2_ACCOUNT_ID}.r2.cloudflarestorage.com/${R2_BUCKET_BACKUPS:-windy-git-backups}/restic}"
DB="${WG_DB_CONTAINER:-windy-git-db-1}"
STAGE="${WG_STAGE:-/var/backups/windygit-state}"
GIT_ROOT="${GIT_DATA_ROOT:-/srv/windygit/git}"
umask 077
mkdir -p "$STAGE"; chmod 700 "$STAGE"; rm -f "$STAGE"/*.dump "$STAGE"/globals.sql
# systemd gives units no $HOME, and restic wants a cache dir: pin one.
export RESTIC_CACHE_DIR="${RESTIC_CACHE_DIR:-/var/cache/windygit-restic}"; mkdir -p "$RESTIC_CACHE_DIR"
if ! err=$(restic cat config 2>&1 >/dev/null); then
# only a MISSING repo may be initialised; any other error (auth, network, wrong password) must stop here
if grep -qiE "does not exist|is there a repository|unable to open config file" <<<"$err"; then
log "initialising restic repo"; restic init >/dev/null
else
log "FATAL: restic cannot open the repository: $(head -c 300 <<<"$err" | tr '\n' ' ')"; exit 1
fi
fi
PGU=$(timeout 30 docker exec "$DB" printenv POSTGRES_USER)
[[ -n "$PGU" ]] || { log "FATAL: no POSTGRES_USER in $DB"; exit 1; }
dbs=$(timeout 60 docker exec "$DB" psql -U "$PGU" -Atc "select datname from pg_database where not datistemplate and datname<>'postgres' order by 1")
n=0
for d in $dbs; do
timeout 600 docker exec "$DB" pg_dump -U "$PGU" -Fc "$d" > "$STAGE/$d.dump"
# a dump that cannot be listed is not a backup
timeout 120 docker exec -i "$DB" pg_restore -l < "$STAGE/$d.dump" >/dev/null
[[ $(stat -c%s "$STAGE/$d.dump") -gt 1000 ]] || { log "FATAL: $d dump suspiciously small"; exit 1; }
n=$((n+1)); log "dumped $d ($(stat -c%s "$STAGE/$d.dump") bytes)"
done
[[ $n -ge 1 ]] || { log "FATAL: no databases dumped"; exit 1; }
timeout 120 docker exec "$DB" pg_dumpall -U "$PGU" --globals-only > "$STAGE/globals.sql"
paths=("$STAGE" "$GIT_ROOT" /srv/windygit/src/.env /srv/windygit/src/deploy/runner/.env /etc/cloudflared)
for p in /etc/systemd/system/windygit-*.service.d /etc/windygit; do [[ -e $p ]] && paths+=("$p"); done
# restic.pass itself is excluded: the password never rides in its own backup
snap=$(restic backup --tag windygit-state --host windygit-veron --quiet --json \
--exclude "$GIT_ROOT/gitea/log" --exclude "$GIT_ROOT/gitea/queues" --exclude "$GIT_ROOT/gitea/tmp" \
--exclude "$GIT_ROOT/gitea/indexers" --exclude "$GIT_ROOT/gitea/actions_log" --exclude /etc/windygit/restic.pass \
"${paths[@]}" | python3 -c 'import sys,json
for l in sys.stdin:
d=json.loads(l)
if d.get("message_type")=="summary": print(d["snapshot_id"][:8])')
[[ -n "$snap" ]] || { log "FATAL: restic produced no snapshot"; exit 1; }
rm -f "$STAGE"/*.dump "$STAGE"/globals.sql
restic check --read-data-subset=2% --quiet >/dev/null || { log "FATAL: restic check failed"; exit 1; }
if [[ $(date +%u) == 7 ]]; then
restic forget --tag windygit-state --keep-daily 14 --keep-weekly 8 --keep-monthly 6 --prune --quiet >/dev/null
fi
echo "ok — state backed up ($n dbs, snapshot $snap)"

View File

@@ -28,9 +28,11 @@ Exceptions: ci/ci-hygiene-allow.yml, one reason per entry.
from __future__ import annotations
import hashlib
import json
import os
import re
import shlex
import subprocess
import sys
from pathlib import Path
@@ -45,9 +47,11 @@ MODE = os.environ.get("CI_HYGIENE_MODE", "warn")
INCLUDE = re.compile(r"(^|/)\.(github|gitea)/workflows/[^/]+\.ya?ml$|(^|/)(Dockerfile[^/]*|[^/]+\.Dockerfile)$")
NEVER = re.compile(r"(^|/)(node_modules|vendor|third_party)/")
PREFILTER = (r"pip3? install|pip install|uv sync|npm (install|i )|yarn install|pnpm install"
r"|^\s*-\s*['\"]?[0-9]+:[0-9]+|:latest|lock[^ ]*\*")
r"|^\s*-\s*['\"]?[0-9]+:[0-9]+|:latest|lock[^ ]*\*"
r"|docker[ -]compose|docker (build|buildx|run)|docker/build-push-action")
TOOLING = {"pip", "setuptools", "wheel"}
NO_DOCKER_FIX = "use job services: + a no-Docker smoke test; the image builds at deploy"
DOCKER_FILE = re.compile(r"(^|/)(Dockerfile[^/]*|[^/]+\.Dockerfile)$")
LATEST = re.compile(r"(?:^\s*FROM\s+(?:--platform=\S+\s+)?|--from=|image:\s*['\"]?|docker://)([\w./-]+):latest\b", re.I)
LOCKNAME = re.compile(r"(uv\.lock|poetry\.lock|package-lock\.json|pnpm-lock\.yaml|yarn\.lock|requirements[^ ]*\.(txt|lock))", re.I)
@@ -121,8 +125,19 @@ def scan_line(path: str, text: str) -> list[tuple[str, str]]:
globbed = [t for t in text.split() if "*" in t and LOCKNAME.search(t)]
if globbed:
hits.append(("optional lock", f"COPY {globbed[0]} (must fail if the lock is missing)"))
# Windy Git jobs get NO Docker daemon (I-5), so a docker build/compose/run
# step in CI can never pass here (orchestrator 09-23, option A). The real
# image build is the deploy step on the target host.
if "/workflows/" in path and re.search(r"uses:\s*['\"]?docker/build-push-action", text):
hits.append(("needs docker", "docker/build-push-action in CI (no Docker daemon on Windy Git; " + NO_DOCKER_FIX + ")"))
for toks in _commands(text):
low = [t.lower() for t in toks]
if "/workflows/" in path and (
low[:2] in (["docker", "build"], ["docker", "buildx"], ["docker", "run"], ["docker", "compose"])
or low[:1] == ["docker-compose"]
):
hits.append(("needs docker", f"{' '.join(low[:2])} in CI (no Docker daemon on Windy Git; " + NO_DOCKER_FIX + ")"))
continue
# pip install / python -m pip install / uv pip install
for i in range(len(low) - 1):
if os.path.basename(low[i]) in ("pip", "pip3") and low[i + 1] == "install":
@@ -143,7 +158,49 @@ def scan_line(path: str, text: str) -> list[tuple[str, str]]:
return hits
_DISABLED: dict[str, set[str]] | None = None
def disabled_workflows() -> dict[str, set[str]]:
"""Workflow file names Gitea has DISABLED per repo (lowercased repo name).
Deploy/release workflows are disabled on Windy Git: they run on the target
host, where a Docker daemon really exists, so "needs docker" must not flag
them. One bounded query per process; on any failure nothing is excused
(flag rather than hide).
"""
global _DISABLED
if _DISABLED is not None:
return _DISABLED
_DISABLED = {}
query = ("select coalesce(json_object_agg(r.lower_name, u.config::json->'DisabledWorkflows'), '{}'::json)"
" from repo_unit u join repository r on r.id = u.repo_id"
" where u.type = 10 and u.config like '%DisabledWorkflows%';")
try:
out = subprocess.run(
["docker", "exec", "-i", "windy-git-db-1", "sh", "-c",
'psql -U "$POSTGRES_USER" -d gitea -At -v ON_ERROR_STOP=1'],
input=query, capture_output=True, text=True, check=True, timeout=30,
).stdout.strip()
_DISABLED = {k: set(v or []) for k, v in json.loads(out or "{}").items()}
except (subprocess.SubprocessError, OSError, ValueError):
pass
return _DISABLED
def _runs_here(repo: str, findings):
"""Drop "needs docker" hits in workflows that never run on Windy Git."""
if findings is None:
return None
off = disabled_workflows().get(repo.lower(), set())
return [f for f in findings if not (f.kind == "needs docker" and Path(f.path).name in off)]
def check(repo: str, sha: str, default_branch: str, is_default_head: bool):
return _runs_here(repo, _check(repo, sha, default_branch, is_default_head))
def _check(repo: str, sha: str, default_branch: str, is_default_head: bool):
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir() or not cg.fetched(bare, sha): # pushed after the fetch: next cycle
return None
@@ -158,8 +215,13 @@ def check(repo: str, sha: str, default_branch: str, is_default_head: bool):
lambda: cg.scan_added(repo, bare, f"refs/heads/{default_branch}", sha, allow, **kw))
def status_for(findings, whole_tree: bool):
def status_for(findings, whole_tree: bool, grant=()):
"""Same contract as compute_guard.status_for: `grant` findings never block."""
scope = "in CI/Dockerfiles" if whole_tree else "added"
if not findings and grant:
g, n = grant[0], len(grant)
desc = f"⚠ WARN (Grant-owned, not blocking): {n} CI hygiene issue{'s' if n > 1 else ''} {scope}, e.g. {g.path}:{g.line} {g.match}"
return "success", desc[:140], g
if not findings:
return "success", f"OK: no floating install or host-port service {scope}", None
f = findings[0]
@@ -179,7 +241,7 @@ def report(repos: list[str]) -> int:
continue
head = cg._git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = cg._git(bare, "rev-parse", head).strip()
fs = cg.scan_tree(repo, bare, sha, allow, line_fn=scan_line, path_ok=path_ok, prefilter=PREFILTER)
fs = _runs_here(repo, cg.scan_tree(repo, bare, sha, allow, line_fn=scan_line, path_ok=path_ok, prefilter=PREFILTER))
total += len(fs)
print(f"## {repo} ({head} {sha[:7]}): {len(fs)} issue(s)")
for f in fs:

View File

@@ -63,6 +63,9 @@ JS_SDKS = (r"@anthropic-ai/sdk|openai|groq-sdk|@google/generative-ai|@google/gen
RULES: list[tuple[str, re.Pattern]] = [
("provider host", re.compile("|".join(re.escape(h) for h in HOSTS))),
# Grant via Boss 10-01: compute = Windy Mind. A NEW reference to an Ollama port (Veron's :11434) is a
# direct call around Mind's metering/caps. WARN-only, never red, and only for lines a PR ADDS.
("veron ollama", re.compile(r"(?::|%3[aA])11434(?![0-9])")),
("provider key", re.compile(r"\b(?:" + "|".join(KEYS) + r")\b")),
("provider SDK", re.compile(rf"^\s*(?:from|import)\s+(?:{PY_SDKS})(?:\s|\.|$|,)")),
("provider SDK", re.compile(rf"""(?:from\s+|require\(\s*|import\(\s*)['"](?:{JS_SDKS})(?:/[^'"]*)?['"]""")),
@@ -70,6 +73,9 @@ RULES: list[tuple[str, re.Pattern]] = [
("provider SDK dep", re.compile(rf'''^\s*"(?:{JS_SDKS})"\s*:''')),
("provider SDK dep", re.compile(rf'''^\s*["']?(?:{PY_SDKS.replace(chr(92) + ".", "-")})(?:\[[^\]]*\])?\s*(?:[<>=~!]=?|["',]|$)''')),
]
# Kinds that never block (even in MODE=block) and are only judged on ADDED lines, never the baseline tree.
WARN_ONLY_KINDS = {"veron ollama"}
OLLAMA_MSG = "compute = Windy Mind (endpoint + key); do not call Veron's Ollama directly"
DEP_FILES = re.compile(r"(^|/)(package\.json|requirements[^/]*\.txt|pyproject\.toml|setup\.cfg|Pipfile)$")
# Never scanned: tests, docs, lockfiles, vendored/built code, CI config.
@@ -253,16 +259,32 @@ def check(repo: str, sha: str, default_branch: str, is_default_head: bool) -> li
allow = load_allow()
fp = _fingerprint(allow)
if is_default_head:
return cached_scan(f"tree:{repo}:{sha}:{fp}", lambda: scan_tree(repo, bare, sha, allow))
return cached_scan(f"tree:{repo}:{sha}:{fp}",
lambda: [f for f in scan_tree(repo, bare, sha, allow) if f.kind not in WARN_ONLY_KINDS])
return cached_scan(
f"pr:{repo}:{sha}:{fp}",
lambda: scan_added(repo, bare, f"refs/heads/{default_branch}", sha, allow),
)
def status_for(findings: list[Finding], whole_tree: bool) -> tuple[str, str, Finding | None]:
"""(state, description, first finding) for the GitHub commit status."""
def status_for(findings: list[Finding], whole_tree: bool,
grant: list[Finding] = ()) -> tuple[str, str, Finding | None]:
"""(state, description, first finding) for the GitHub commit status.
`findings` = lane-owned (these block in MODE=block); `grant` = findings in
Grant-owned code (ci/grant-owned.yml): always WARN, never red (orchestrator
09-23: his desktop work is never blocked by us)."""
scope = "in tree" if whole_tree else "added"
soft = [f for f in findings if f.kind in WARN_ONLY_KINDS]
findings = [f for f in findings if f.kind not in WARN_ONLY_KINDS]
if not findings and not grant and soft:
f = soft[0]
return "success", f"⚠ WARN: new Veron Ollama ref {f.path}:{f.line}. {OLLAMA_MSG}"[:140], f
if not findings and grant:
g, n = grant[0], len(grant)
desc = (f"⚠ WARN (Grant-owned, not blocking): {n} direct AI-provider use{'s' if n > 1 else ''} "
f"{scope}, e.g. {g.path}:{g.line} {g.match}")
return "success", desc[:140], g
if not findings:
what = "no direct AI-provider use in tree" if whole_tree else "no direct AI-provider use added"
return "success", f"OK: {what} (Windy Mind is the only door)", None

23
scripts/drill_cross_host.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Cross-host restore drill on Windy 0: ONLY lockbox + R2, nothing from Veron. Values never printed.
# Usage: bash drill_cross_host.sh (needs lockbox keys RESTIC_WINDYGIT_PASSWORD, WINDYGIT_R2_ACCESS_KEY_ID, WINDYGIT_R2_SECRET_ACCESS_KEY, WINDYGIT_R2_ENDPOINT)
set -euo pipefail
umask 077; W=$(mktemp -d ~/.cache/wg-xdrill.XXXXXX)
trap 'docker rm -f wg-xdrill-pg >/dev/null 2>&1 || true; rm -rf "$W"' EXIT
lockbox-get RESTIC_WINDYGIT_PASSWORD "$W/pw" >/dev/null
lockbox-get WINDYGIT_R2_ACCESS_KEY_ID "$W/ak" >/dev/null; lockbox-get WINDYGIT_R2_SECRET_ACCESS_KEY "$W/sk" >/dev/null; lockbox-get WINDYGIT_R2_ENDPOINT "$W/ep" >/dev/null
export RESTIC_PASSWORD_FILE="$W/pw" AWS_ACCESS_KEY_ID="$(cat "$W/ak")" AWS_SECRET_ACCESS_KEY="$(cat "$W/sk")"
export RESTIC_REPOSITORY="s3:$(cat "$W/ep")/windy-git-backups/restic"
restic snapshots --tag windygit-state --compact | tail -3
restic restore latest --tag windygit-state --target "$W/r" --include /var/backups/windygit-state --include /srv/windygit/git/gitea/conf --quiet
ls -l "$W/r/var/backups/windygit-state" | awk 'NR>1{print $5, $NF}'
docker run -d --name wg-xdrill-pg -e POSTGRES_PASSWORD="$(python3 -c 'import secrets;print(secrets.token_hex(12))')" -e POSTGRES_USER=drill postgres:16-alpine >/dev/null
for i in $(seq 1 30); do docker exec wg-xdrill-pg pg_isready -U drill >/dev/null 2>&1 && break; sleep 2; done
for db in gitea windygit; do
docker exec wg-xdrill-pg psql -U drill -d postgres -qc "create database $db"
docker exec -i wg-xdrill-pg pg_restore -U drill -d $db --no-owner --no-privileges < "$W/r/var/backups/windygit-state/$db.dump" 2>&1 | grep -v "already exists" | head -2 || true
done
for t in repository issue pull_request '"user"' external_login_user action_run action_run_job; do
echo "$t restored=$(docker exec wg-xdrill-pg psql -U drill -d gitea -Atc "select count(*) from $t")"
done
echo "cross-host drill OK (cleaned up)"

153
scripts/env_names.py Normal file
View File

@@ -0,0 +1,153 @@
#!/usr/bin/env python3
"""env-names: list environment variable NAMES only (Boss ruling 10-01, house rule 10).
env-names <docker container | systemd unit | env file> [--host H] [--sudo] [--hash]
env-names A --compare B [--host H] [--host2 H2]
Prints NAME, set|empty, and value LENGTH. Never a value or fragment. --hash adds sha256[:8]
(compare two places for equality; a hash of a weak value can be guessed, so use it for
real secrets only). --compare prints SAME / DIFFERENT / only-in-A / only-in-B per name
(equality by full-value hash, nothing else shown). Values live in memory only.
Targets: an existing file (dotenv style) | a docker container name | a systemd unit
(Environment= + EnvironmentFile=; --sudo to read root-only files). --host runs the docker /
systemctl / file read over ssh (alias from ~/.ssh/config).
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import shlex
import subprocess
import sys
KV = ("=",)
def run(cmd: list[str], host: str | None, sudo: bool = False) -> tuple[int, str]:
if sudo:
cmd = ["sudo", "-n", *cmd]
if host:
cmd = ["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=10", host, shlex.join(cmd)]
r = subprocess.run(cmd, capture_output=True, text=True, errors="ignore", timeout=60)
return r.returncode, r.stdout
def parse_dotenv(text: str) -> dict[str, str]:
out: dict[str, str] = {}
for raw in text.splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
if line.startswith("export "):
line = line[7:].lstrip()
k, v = line.split("=", 1)
k = k.strip()
v = v.strip()
if len(v) >= 2 and v[0] == v[-1] and v[0] in "\"'":
v = v[1:-1]
if k.replace("_", "").isalnum() and not k[0].isdigit():
out[k] = v
return out
def from_file(path: str, host: str | None, sudo: bool) -> dict[str, str] | None:
if host or sudo:
rc, out = run(["cat", path], host, sudo)
return parse_dotenv(out) if rc == 0 else None
try:
with open(path, errors="ignore") as fh:
return parse_dotenv(fh.read())
except OSError:
return None
def from_docker(name: str, host: str | None, sudo: bool) -> dict[str, str] | None:
rc, out = run(["docker", "inspect", "-f", "{{json .Config.Env}}", name], host, sudo)
if rc != 0 or not out.strip():
return None
try:
items = json.loads(out)
except ValueError:
return None
return {k: v for k, _, v in (i.partition("=") for i in (items or []))}
def from_systemd(unit: str, host: str | None, sudo: bool) -> dict[str, str] | None:
rc, out = run(["systemctl", "show", unit, "-p", "Environment", "-p", "EnvironmentFiles"], host)
if rc != 0 or "LoadState=not-found" in out:
return None
env: dict[str, str] = {}
files: list[str] = []
for line in out.splitlines():
if line.startswith("Environment="):
for tok in shlex.split(line[len("Environment="):]):
k, _, v = tok.partition("=")
env[k] = v
elif line.startswith("EnvironmentFiles="):
f = line[len("EnvironmentFiles="):].split(" (")[0].strip().lstrip("-")
if f:
files.append(f)
for f in files: # later files override earlier, like systemd
d = from_file(f, host, sudo)
if d is None:
print(f"# note: EnvironmentFile {f} unreadable (try --sudo)", file=sys.stderr)
else:
env.update(d)
return env
def load(target: str, host: str | None, sudo: bool) -> dict[str, str] | None:
if (not host and os.path.isfile(target)) or target.startswith(("/", "./", "~")):
return from_file(os.path.expanduser(target), host, sudo)
if target.endswith((".service", ".timer", ".socket")):
return from_systemd(target, host, sudo)
return from_docker(target, host, sudo) or from_systemd(target, host, sudo)
def sh(v: str) -> str:
return hashlib.sha256(v.encode()).hexdigest()
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="env-names", description="env var NAMES only")
ap.add_argument("target")
ap.add_argument("--host")
ap.add_argument("--host2", help="ssh host for the --compare target")
ap.add_argument("--sudo", action="store_true")
ap.add_argument("--hash", action="store_true", help="add sha256[:8] per variable")
ap.add_argument("--compare", metavar="TARGET2")
a = ap.parse_args(argv)
env = load(a.target, a.host, a.sudo)
if env is None:
print(f"error: could not read {a.target!r} (file, docker container or systemd unit)")
return 2
if a.compare:
env2 = load(a.compare, a.host2 or a.host, a.sudo)
if env2 is None:
print(f"error: could not read {a.compare!r}")
return 2
for k in sorted(set(env) | set(env2)):
if k not in env2:
print(f"{k:<40} only-in-A")
elif k not in env:
print(f"{k:<40} only-in-B")
else:
print(f"{k:<40} {'SAME' if sh(env[k]) == sh(env2[k]) else 'DIFFERENT'}"
f" (len {len(env[k])} vs {len(env2[k])})")
return 0
for k in sorted(env):
v = env[k]
extra = f" sha256:{sh(v)[:8]}" if a.hash and v else ""
print(f"{k:<40} {'set ' if v else 'empty'} len={len(v)}{extra}")
print(f"# {len(env)} variable(s); values never printed")
return 0
if __name__ == "__main__":
try:
sys.exit(main())
except Exception as e: # never a traceback
print(f"error: {type(e).__name__}")
sys.exit(2)

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Live status of the repo guards (compute-guard + ci-hygiene) as one markdown page.
"""Live status of the repo guards (compute-guard + ci-hygiene + secret-guard) as one markdown page.
Scans every bridged repo's DEFAULT branch with both guards and renders what is
left, per repo and owner lane. Findings in code Grant owns (ci/grant-owned.yml:
@@ -24,6 +24,7 @@ import yaml
sys.path.insert(0, str(Path(__file__).resolve().parent))
import ci_hygiene as hy # noqa: E402
import compute_guard as cg # noqa: E402
import secret_guard as sgd # noqa: E402
ROOT = Path(__file__).resolve().parents[1]
OWNED = Path(os.environ.get("GRANT_OWNED", ROOT / "ci" / "grant-owned.yml"))
@@ -32,20 +33,21 @@ REPOS = os.environ.get("BRIDGE_REPOS", "").split() or [
"windy-connect", "windy-drops", "windy-code-web", "windy-code", "windy-traveler",
"windy-registry", "eternitas", "windy-translate", "windytranslate-site", "windytraveler-site",
"windy-hand", "windy-cloud-sites", "windy-cloud-domains", "windy-cloud-vps", "windytalk",
"windy-pro", "windy-mind", "windy-git"]
"windy-pro", "windy-mind", "windy-git", "windy-inbox"]
# Owner lane per repo = the session name to message (orchestrator routing, 09-23 ~22:45Z:
# hub/account-server/dashboard/site -> "Windy Hub"; windy-calendar only -> "Windy Calender";
# windy-admin/telemetry -> "Windy Admin"). windy-pro here = its server/web side; the desktop
# app is Grant-owned and listed in its own section below.
OWNERS = {
"windy-chat": "grantwhitmer-ca", "windy-mail": "Windy Mail", "windy-calendar": "Windy Calender",
"windy-chat": "Windy Chat", "windy-mail": "Windy Mail", "windy-calendar": "Windy Calender",
"Windy-Clone": "Windy Clone", "WindyCloud": "Windy Cloud", "windy-cloud-sites": "Windy Cloud",
"windy-cloud-domains": "Windy Cloud", "windy-cloud-vps": "Windy Cloud", "windy-search": "Windy Search",
"windy-connect": "Windy Connect", "windy-drops": "Windy Drops", "windy-registry": "Windy Drops",
"windy-code-web": "Windy Code", "windy-code": "Windy Code", "windy-traveler": "Windy Traveler",
"windytraveler-site": "Windy Traveler", "eternitas": "Eternitas", "windy-translate": "Windy Translate",
"windytranslate-site": "Windy Translate", "windy-hand": "Windy Hand", "windytalk": "Windy Talk",
"windy-pro": "Windy Hub", "windy-mind": "WIndy Mind", "windy-git": "Windy Git"}
"windy-pro": "Windy Hub", "windy-mind": "WIndy Mind", "windy-git": "Windy Git",
"windy-inbox": "Windy Drops"} # Windy Inbox build lead (09-24)
JOB = re.compile(r"^ ([A-Za-z0-9_-]+):\s*$")
@@ -75,16 +77,36 @@ def grant_owned(repo: str, path: str, job: str | None, owned: list[dict]) -> boo
return False
def split_grant(repo: str, sha: str, findings: list) -> tuple[list, list]:
"""(lane-owned, Grant-owned) findings at `sha`, by ci/grant-owned.yml, with
workflow lines attributed to their job exactly as the status page does."""
owned = (yaml.safe_load(OWNED.read_text()) or {}).get("grant_owned") or []
if not any(e["repo"] == repo for e in owned):
return list(findings), []
bare = cg.WORK / f"{repo}.git"
texts: dict[str, str] = {}
lane, grant = [], []
for f in findings:
job = None
if "/workflows/" in f.path:
if f.path not in texts:
texts[f.path] = cg._git(bare, "show", f"{sha}:{f.path}")
job = job_of(texts[f.path], f.line)
(grant if grant_owned(repo, f.path, job, owned) else lane).append(f)
return lane, grant
def scan(repo: str, owned: list[dict]):
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir():
return None
head = cg._git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = cg._git(bare, "rev-parse", head).strip()
out = {"sha": sha, "compute": [], "hygiene": []}
out = {"sha": sha, "compute": [], "hygiene": [], "secrets": []}
texts: dict[str, str] = {}
for key, fs in (("compute", cg.check(repo, sha, head, True) or []),
("hygiene", hy.check(repo, sha, head, True) or [])):
("hygiene", hy.check(repo, sha, head, True) or []),
("secrets", sgd.check(repo, sha, head, True) or [])):
for f in fs:
job = None
if "/workflows/" in f.path:
@@ -97,27 +119,29 @@ def scan(repo: str, owned: list[dict]):
def render(results: dict) -> str:
now = time.strftime("%Y-%m-%d %H:%MZ", time.gmtime())
lane = {k: 0 for k in ("compute", "hygiene")}
grant = {k: 0 for k in ("compute", "hygiene")}
lane = {k: 0 for k in ("compute", "hygiene", "secrets")}
grant = {k: 0 for k in ("compute", "hygiene", "secrets")}
for r in results.values():
for k in lane:
lane[k] += sum(1 for _, _, g in r[k] if not g)
grant[k] += sum(1 for _, _, g in r[k] if g)
lane[k] += sum(1 for _, _, g in r.get(k, []) if not g)
grant[k] += sum(1 for _, _, g in r.get(k, []) if g)
L = [f"# Repo guards: live status (generated {now}; windy-git scripts/guards_report.py)",
"_Default branches only. WARN-only today; the orchestrator says \"block\" per guard when its LANE column is 0. "
"Grant-owned code (ci/grant-owned.yml) is listed separately and never holds up a block._", "",
"| Guard | Lane-owned findings | Grant-owned (proposals) | Ready to block? |", "|---|---|---|---|",
f"| compute-guard (Mind is the only door) | {lane['compute']} | {grant['compute']} | {'✅ YES' if lane['compute'] == 0 else '❌ not yet'} |",
f"| ci-hygiene (house rule 6) | {lane['hygiene']} | {grant['hygiene']} | {'✅ YES' if lane['hygiene'] == 0 else '❌ not yet'} |",
"", "## By repo (lane-owned)", "| Repo | owner | head | compute | hygiene | first items |", "|---|---|---|---|---|---|"]
f"| secret-guard (no credentials in repos; hash only) | {lane['secrets']} | {grant['secrets']} | {'✅ YES' if lane['secrets'] == 0 else '❌ not yet'} |",
"", "## By repo (lane-owned)", "| Repo | owner | head | compute | hygiene | secrets | first items |", "|---|---|---|---|---|---|---|"]
for repo, r in sorted(results.items()):
c = [x for x in r["compute"] if not x[2]]
h = [x for x in r["hygiene"] if not x[2]]
items = "; ".join(f"`{f.path}:{f.line}` {f.match}" for f, _, _ in (c + h)[:3]) or "clean ✅"
L.append(f"| {repo} | {OWNERS.get(repo, '?')} | {r['sha'][:7]} | {len(c)} | {len(h)} | {items} |")
s = [x for x in r.get("secrets", []) if not x[2]]
items = "; ".join(f"`{f.path}:{f.line}` {f.match}" for f, _, _ in (s + c + h)[:3]) or "clean ✅"
L.append(f"| {repo} | {OWNERS.get(repo, '?')} | {r['sha'][:7]} | {len(c)} | {len(h)} | {len(s)} | {items} |")
L += ["", "## Grant-owned (windy-pro desktop app + its build jobs): proposals only, not blocking"]
g = [(repo, f, job) for repo, r in sorted(results.items()) for k in ("compute", "hygiene")
for f, job, own in r[k] if own]
g = [(repo, f, job) for repo, r in sorted(results.items()) for k in ("compute", "hygiene", "secrets")
for f, job, own in r.get(k, []) if own]
L += [f"- {repo} `{f.path}:{f.line}`{f' (job {job})' if job else ''}: {f.match}" for repo, f, job in g] or ["- none"]
return "\n".join(L) + "\n"

14
scripts/install_secret_tools.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Install the shared hash-only secret tools on THIS machine (Windy 0): secret-scan + env-names.
# Source of truth is this repo (scripts/); re-run after a pull to update.
set -euo pipefail
here=$(cd "$(dirname "$0")" && pwd)
dest="$HOME/.local/share/secret-tools"
mkdir -p "$dest" "$HOME/.local/bin"
cp "$here/secret_shapes.py" "$here/secret_scan.py" "$here/env_names.py" "$here/lockbox_put.py" "$dest/"
for pair in "secret-scan:secret_scan.py" "env-names:env_names.py" "lockbox-put:lockbox_put.py"; do
n=${pair%%:*}; f=${pair##*:}
printf '#!/usr/bin/env bash\nexec python3 "%s/%s" "$@"\n' "$dest" "$f" > "$HOME/.local/bin/$n"
chmod 755 "$HOME/.local/bin/$n"
done
echo "installed secret-scan, env-names and lockbox-put (shapes from secret_shapes.py, same as secret-guard)"

141
scripts/lockbox_put.py Normal file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""lockbox-put: add ONE secret to the lockbox by PR, without anyone reading, printing or
grepping the lockbox (Boss rule 10-01; Windy Hub ruling).
lockbox-put KEY FILE [--lane NAME] [--note TEXT]
KEY exact name, ^[A-Z][A-Z0-9_]{2,63}$ . FILE a 0600 file you own (not a symlink) whose
content is the value (one line). Appends ONE line `- **`KEY`**: `<value>`` (the format
lockbox-get reads) under a new heading at the END of ACCESS_LOCKBOX.md in a fresh temp clone,
on a new branch, and opens a kit-army-config PR. Append-only: the diff is verified to be one
file, additions only, before pushing. REFUSES if KEY already exists (a bool computed in
memory; no line, value or location is ever printed). Reviewers see the KEY NAME + lane only
if they look at the diff; the PR body never carries the value. Never echoes the value.
Env (tests): LOCKBOX_PUT_REPO=<clone url/path>, LOCKBOX_PUT_NO_PR=1.
"""
from __future__ import annotations
import argparse
import datetime as dt
import os
import re
import shutil
import stat
import subprocess
import sys
import tempfile
from pathlib import Path
REPO = os.environ.get("LOCKBOX_PUT_REPO", "https://github.com/sneakyfree/kit-army-config.git")
SLUG = "sneakyfree/kit-army-config"
KEY_RE = re.compile(r"^[A-Z][A-Z0-9_]{2,63}$")
VAL_RE = re.compile(r"^[A-Za-z0-9._~+/=:@%,-]{8,512}$") # no backtick, quote, space or newline
def die(msg: str, code: int = 2):
print(f"lockbox-put: {msg}")
sys.exit(code)
def git(cwd: str, *a: str, quiet=True) -> subprocess.CompletedProcess:
# stderr is dropped: git/gh errors can echo URLs; stdout only when we need it.
return subprocess.run(["git", "-C", cwd, *a], capture_output=True, text=True, errors="ignore")
def key_exists(clone: str, key: str) -> bool:
"""True if KEY is already defined anywhere lockbox-get reads. Bool only, nothing printed."""
pat_env = re.compile(r"^" + re.escape(key) + r"=")
pat_md = re.compile(r"^\s*[-*]?\s*\*\*`" + re.escape(key) + r"`\*\*\s*:")
paths = [Path(clone, "ACCESS_LOCKBOX.md")] + [
Path(dp, f) for dp, _d, fs in os.walk(Path(clone, "secrets")) for f in fs if f.endswith(".env")]
for p in paths:
try:
with open(p, errors="ignore") as fh:
for line in fh:
if pat_env.match(line) or pat_md.match(line):
return True
except OSError:
continue
return False
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="lockbox-put")
ap.add_argument("key")
ap.add_argument("file")
ap.add_argument("--lane", default=os.environ.get("LOCKBOX_LANE", "a lane"))
ap.add_argument("--note", default="")
a = ap.parse_args(argv)
if not KEY_RE.match(a.key):
die("KEY must match ^[A-Z][A-Z0-9_]{2,63}$")
try:
st = os.lstat(a.file)
except OSError:
die("FILE not found")
if stat.S_ISLNK(st.st_mode) or not stat.S_ISREG(st.st_mode):
die("FILE must be a regular file (not a symlink)")
if st.st_uid != os.getuid() or (st.st_mode & 0o077):
die("FILE must be owned by you and mode 0600")
with open(a.file) as fh:
value = fh.read().strip()
if not VAL_RE.match(value):
die("value must be one line of 8-512 chars from [A-Za-z0-9._~+/=:@%,-] (no spaces, quotes, backticks)")
note = re.sub(r"[`\n\r]", " ", a.note)[:160]
lane = re.sub(r"[^A-Za-z0-9 ._-]", "", a.lane)[:40]
tmp = tempfile.mkdtemp(prefix="lockbox-put-", dir=str(Path.home() / ".cache") if (Path.home() / ".cache").is_dir() else None)
os.chmod(tmp, 0o700)
clone = os.path.join(tmp, "k")
try:
if subprocess.run(["git", "clone", "-q", "--depth", "1", REPO, clone],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL).returncode != 0:
die("clone failed (details withheld: URLs can carry tokens)")
if key_exists(clone, a.key):
die(f"{a.key} already exists: refusing to overwrite (append-only; pick a new KEY)", 3)
lb = Path(clone, "ACCESS_LOCKBOX.md")
if not lb.is_file():
die("ACCESS_LOCKBOX.md not found in the repo")
today = dt.date.today().isoformat()
stamp = dt.datetime.now(dt.UTC).strftime("%Y%m%d%H%M")
branch = f"lockbox-put/{a.key.lower()}-{stamp}"
with open(lb, "a") as fh:
fh.write(f"\n## 🗝️ {a.key} (added {today} by {lane} via lockbox-put)\n")
fh.write(f"- **`{a.key}`**: `{value}`\n")
if note:
fh.write(f"- **Note:** {note}\n")
git(clone, "checkout", "-q", "-b", branch)
git(clone, "add", "ACCESS_LOCKBOX.md")
ns = git(clone, "diff", "--cached", "--numstat").stdout.split()
# numstat: <added> <deleted> <path>; exactly one file, no deletions
if len(ns) != 3 or ns[1] != "0" or ns[2] != "ACCESS_LOCKBOX.md":
die("diff is not a pure append to ACCESS_LOCKBOX.md: aborting, nothing pushed")
msg = f"lockbox: add {a.key} (via lockbox-put, {lane})\n\nCo-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
if git(clone, "-c", "user.name=lockbox-put", "-c", "user.email=lockbox-put@windy.invalid",
"commit", "-q", "-m", msg).returncode != 0:
die("commit failed")
if git(clone, "push", "-q", "origin", branch).returncode != 0:
die("push failed (details withheld)")
if os.environ.get("LOCKBOX_PUT_NO_PR"):
print(f"ok: pushed branch {branch} ({a.key}); PR skipped")
return 0
body = (f"Adds exactly one key: `{a.key}` (by {lane}). Append-only, one file, no deletions "
f"(verified before push). Review by KEY NAME only; do not paste the value anywhere.\n\n"
f"{note}\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)")
r = subprocess.run(["gh", "pr", "create", "-R", SLUG, "--head", branch, "--base", "main",
"--title", f"lockbox: add {a.key} ({lane})", "--body", body],
capture_output=True, text=True)
if r.returncode != 0:
die("branch pushed but `gh pr create` failed; open the PR for the branch by hand")
print(f"ok: {a.key} added via PR {r.stdout.strip().splitlines()[-1]}")
return 0
finally:
shutil.rmtree(tmp, ignore_errors=True)
if __name__ == "__main__":
try:
sys.exit(main())
except SystemExit:
raise
except Exception as e: # never a traceback: it could carry data
print(f"lockbox-put: error: {type(e).__name__}")
sys.exit(2)

View File

@@ -53,7 +53,8 @@ REPOS = os.environ.get(
"windy-chat windy-mail windy-calendar Windy-Clone WindyCloud windy-search windy-connect"
" windy-drops windy-code-web windy-code windy-traveler windy-registry eternitas"
" windy-translate windytranslate-site windytraveler-site windy-hand"
" windy-cloud-sites windy-cloud-domains windy-cloud-vps windytalk windy-pro windy-mind",
" windy-cloud-sites windy-cloud-domains windy-cloud-vps windytalk windy-pro windy-mind"
" windy-inbox windy-text windy-call windy-cell windy-hand-site windy-calendar-site",
).split()
# Gitea run status -> GitHub status state. `skipped` is deliberately absent: a
@@ -74,7 +75,23 @@ MIRROR_TAG = "[GH#"
# Posting them would put a permanent red X on every commit, and a signal that is
# always red trains everyone to ignore red. Not posted until a rootless builder
# exists; that is a decision, recorded in docs/CUTOVER.md, not a failure.
NO_DAEMON_JOB = re.compile(r"docker", re.IGNORECASE)
# ...but NOT the no-Docker smoke jobs that replaced them (option A, 09-23):
# windy-search's "Boot smoke (no Docker)" matched plain `docker` and was hidden.
NO_DAEMON_JOB = re.compile(r"(?<!no )(?<!no-)(?<!without )docker", re.IGNORECASE)
# Image-build jobs whose NAME doesn't say docker (orchestrator 09-23, option A:
# each lane converts the job to a no-Docker smoke test; until then it is not
# posted). Format: "repo:workflow/job,...;repo2:...".
NO_DAEMON_NAMED: dict[str, set[str]] = {}
# eternitas:ci/build dropped 09-23: converted to a no-Docker ci/smoke (#179).
for _entry in os.environ.get("BRIDGE_NO_DAEMON", "").split(";"):
if ":" in _entry:
_repo, _jobs = _entry.split(":", 1)
NO_DAEMON_NAMED[_repo.strip()] = {j.strip() for j in _jobs.split(",") if j.strip()}
def needs_daemon(repo: str, wf: str, job: str) -> bool:
"""True for image-build jobs, which cannot run here (no Docker daemon, I-5)."""
return bool(NO_DAEMON_JOB.search(job)) or f"{wf}/{job}" in NO_DAEMON_NAMED.get(repo, ())
# Jobs Grant ruled NON-BLOCKING (GRANT_DECISIONS_2026-09-23): still run on
# Windy Git and visible there, but not posted to GitHub, so they cannot turn a
@@ -294,7 +311,7 @@ def post_statuses(repo: str, sha: str) -> None:
break
latest: dict[str, dict] = {}
for r in runs:
if r["head_sha"] != sha or NO_DAEMON_JOB.search(r["name"]):
if r["head_sha"] != sha or needs_daemon(repo, r["workflow_id"].removesuffix(".yml"), r["name"]):
continue
if f"{r['workflow_id'].removesuffix('.yml')}/{r['name']}" in NON_BLOCKING.get(repo, ()):
continue
@@ -304,9 +321,9 @@ def post_statuses(repo: str, sha: str) -> None:
# Queued jobs: `pending` where nothing newer has been picked up. A re-run
# queued behind an old failure must read pending, not the stale red.
for q in queued_jobs(repo, sha):
if NO_DAEMON_JOB.search(q["name"]):
continue
wf = q["workflow_id"].removesuffix(".yml")
if needs_daemon(repo, wf, q["name"]):
continue
if f"{wf}/{q['name']}" in NON_BLOCKING.get(repo, ()):
continue
ctx = f"windy-git/{wf}/{q['name']}"
@@ -355,6 +372,7 @@ def post_statuses(repo: str, sha: str) -> None:
GUARD_CTX = "windy-git/compute-guard"
HYGIENE_CTX = "windy-git/ci-hygiene"
SECRET_CTX = "windy-git/secret-guard"
def post_compute_guard(repo: str, sha: str, default_branch: str, is_default_head: bool) -> None:
@@ -362,6 +380,11 @@ def post_compute_guard(repo: str, sha: str, default_branch: str, is_default_head
_post_guard("compute_guard", GUARD_CTX, repo, sha, default_branch, is_default_head)
def post_secret_guard(repo: str, sha: str, default_branch: str, is_default_head: bool) -> None:
"""No live credential in a bridged repo (leak hunt 09-24). Findings carry sha256[:8] only."""
_post_guard("secret_guard", SECRET_CTX, repo, sha, default_branch, is_default_head)
def post_ci_hygiene(repo: str, sha: str, default_branch: str, is_default_head: bool) -> None:
"""House rule 6: lockfile-only installs, pinned images, no host-port services (warn-only)."""
_post_guard("ci_hygiene", HYGIENE_CTX, repo, sha, default_branch, is_default_head)
@@ -381,7 +404,14 @@ def _post_guard(modname: str, ctx: str, repo: str, sha: str, default_branch: str
return
if findings is None:
return
state, desc, first = g.status_for(findings, whole_tree=is_default_head)
try: # Grant-owned code never blocks (orchestrator 09-23); lazy like the guards
import importlib
lane, grant = importlib.import_module("guards_report").split_grant(repo, sha, findings)
except Exception as e: # noqa: BLE001 — can't tell whose code: warn, never block
print(f" {repo}@{sha[:7]} {ctx}: Grant-owned split failed ({type(e).__name__}); WARN only")
lane, grant = [], list(findings)
state, desc, first = g.status_for(lane, whole_tree=is_default_head, grant=grant)
st, existing = github("GET", f"/repos/{GH_OWNER}/{repo}/commits/{sha}/statuses?per_page=100")
for s in existing or []: # newest first: compare the latest guard status only
if s["context"] == ctx:
@@ -414,6 +444,7 @@ def main() -> int:
post_statuses(repo, sha)
post_compute_guard(repo, sha, default_branch, sha == default_head)
post_ci_hygiene(repo, sha, default_branch, sha == default_head)
post_secret_guard(repo, sha, default_branch, sha == default_head)
except Exception as e: # one repo's failure must not hide the others'
print(f" FAILED {repo}: {e}")
failed = 1

View File

@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Weekly: every PUBLIC repo in Grant's GitHub accounts, full history (every object,
reachable or not, incl. PR refs), for secret-shaped strings. Leak hunt 09-24: a
real bot token sat in a public test fixture for five months.
Output is hash + location only, never a value (house rule 10). Known fakes are
excused by ci/secret-guard-allow.yml (same file as secret-guard; the repo NAME is
matched across accounts). Runs on Veron as user1-gpu (gh is logged in there):
python3 scripts/public_secret_scan.py [--out FILE] [--owners a,b,...]
"""
from __future__ import annotations
import argparse
import fnmatch
import json
import subprocess
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import secret_shapes as ss # noqa: E402
OWNERS = ["sneakyfree", "VERONTECH", "Windstorm-Institute", "Windstorm-Labs", "Public-Streamer"]
WORK = Path.home() / "leakscan" / "public"
MAX_BLOB = 20_000_000
def _run(*a: str) -> subprocess.CompletedProcess:
return subprocess.run(a, capture_output=True)
def blob_hits(bare: Path) -> dict[str, list[tuple[str, str]]]:
"""{blob: [(kind, hash8)]} over every blob object in the repo."""
chk = _run("git", "-C", str(bare), "cat-file", "--batch-all-objects",
"--batch-check=%(objectname) %(objecttype) %(objectsize)")
blobs = [p[0] for p in (ln.split() for ln in chk.stdout.decode().splitlines())
if len(p) == 3 and p[1] == "blob" and int(p[2]) < MAX_BLOB]
if not blobs:
return {}
import threading
p = subprocess.Popen(["git", "-C", str(bare), "cat-file", "--batch"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
def feed() -> None: # separate thread: writing everything first deadlocks (both pipes fill)
p.stdin.write(("\n".join(blobs) + "\n").encode())
p.stdin.close()
threading.Thread(target=feed, daemon=True).start()
out: dict[str, list[tuple[str, str]]] = {}
for _ in blobs:
hdr = p.stdout.readline().split()
data = p.stdout.read(int(hdr[2]))
p.stdout.read(1)
found = ss.find(data.decode("utf-8", "ignore"))
if found:
out[hdr[0].decode()] = found
p.wait()
return out
def locate(bare: Path, blob: str) -> dict:
lg = _run("git", "-C", str(bare), "log", "--all", "--format=@@%H %cI", "--name-only",
f"--find-object={blob}").stdout.decode()
commits, paths = [], set()
for line in lg.splitlines():
if line.startswith("@@"):
commits.append(line[2:].split())
elif line.strip():
paths.add(line.strip())
head = _run("git", "-C", str(bare), "ls-tree", "-r", "HEAD").stdout.decode()
first = commits[-1] if commits else ["unreachable", "-"]
return {"paths": sorted(paths), "first_commit": first[0][:10], "first_date": first[1],
"in_head": blob in head}
def excused(repo: str, kind: str, h: str, paths: list[str], allow: dict) -> bool:
a = allow.get(repo) or {"hashes": set(), "paths": []}
if kind in {"private key block"}:
return bool(paths) and all(any(kind in k and fnmatch.fnmatch(p, g) for g, k in a["paths"]) for p in paths)
return h in a["hashes"]
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--out", default=str(WORK / "latest.json"))
ap.add_argument("--owners", default=",".join(OWNERS))
args = ap.parse_args()
import secret_guard as sg
allow = sg.load_allow()
WORK.mkdir(parents=True, exist_ok=True)
rows, scanned, errors = [], [], []
for owner in args.owners.split(","):
lst = _run("gh", "repo", "list", owner, "--limit", "1000", "--visibility", "public", "--json", "name")
for r in json.loads(lst.stdout or b"[]"):
name = r["name"]
bare = WORK / owner / f"{name}.git"
if bare.exists():
c = _run("git", "-C", str(bare), "remote", "update", "--prune")
else:
bare.parent.mkdir(parents=True, exist_ok=True)
c = _run("gh", "repo", "clone", f"{owner}/{name}", str(bare), "--", "--mirror", "-q")
if c.returncode:
errors.append(f"{owner}/{name}")
continue
scanned.append(f"{owner}/{name}")
for blob, found in blob_hits(bare).items():
loc = locate(bare, blob)
for kind, h in sorted(set(found)):
rows.append({"repo": f"{owner}/{name}", "kind": kind, "hash8": h, **loc,
"excused": excused(name, kind, h, loc["paths"], allow)})
Path(args.out).write_text(json.dumps({"scanned": scanned, "errors": errors, "rows": rows}, indent=1))
new = [r for r in rows if not r["excused"]]
print(f"scanned {len(scanned)} public repos, {len(errors)} errors, {len(rows)} secret-shaped, {len(new)} NOT excused")
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -16,6 +16,8 @@
# itself, so Windy Git is never left behind GitHub.
set -euo pipefail
repo="${1:?repo}"; branch="${2:?branch}"; want="${3:?sha prefix}"
# wg-q lives in the INVOKING user's ~/bin; under sudo, ~ is /root.
WGQ="${WGQ:-$(getent passwd "${SUDO_USER:-$USER}" | cut -d: -f6)/bin/wg-q}"
# Gitea stores repositories LOWERCASED on disk (WindyCloud -> windycloud.git).
G="sudo docker exec -u git windy-git-gitea-1 git -C /data/git/repositories/windyadmin/${repo,,}.git"
@@ -43,7 +45,7 @@ until [ "$(systemctl show windygit-sync -p ExecMainStartTimestampMonotonic --val
done
echo "restored by sync: ${branch} = ${head:0:7}"
sleep 5
~/bin/wg-q <<SQL
"$WGQ" <<SQL
select ar.index, ar.workflow_id, ar.event, ar.status, to_char(to_timestamp(ar.created),'HH24:MI:SS')
from action_run ar join repository r on r.id = ar.repo_id
where r.name = '${repo}' and ar.commit_sha = '${head}' order by ar.id desc limit 6;

209
scripts/runner_guard.py Normal file
View File

@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""Runner guard: no workflow may let a STRANGER's code reach a self-hosted runner (Boss 10-01).
Our self-hosted GitHub runners run on Veron as `github-runner`, which is in the docker group
(= root on Veron). Until ephemeral containerised runners exist (after launch), the cheap
guard is to refuse the workflow shapes that hand a self-hosted runner to outsiders:
R1 pull_request_target : runs with secrets/write token in the BASE repo context
R2 pull_request on self-hosted : fork PRs run their own code, unless the job is gated to
same-repo heads (`if:` on head.repo.full_name == github.repository
or head.repo.fork == false) or an `environment:`
R3 issue_comment / workflow_run / issues / discussion* / pull_request_review* / fork / watch
: anyone can fire these; never on self-hosted without an environment gate
(push, tags, schedule, workflow_dispatch, repository_dispatch, workflow_call: writers only, fine)
A job counts as self-hosted when its runs-on names `self-hosted`, or is an expression we
can't resolve (conservative). Findings carry file:line, never file content beyond that.
python3 scripts/runner_guard.py lint FILE... # local files
python3 scripts/runner_guard.py report [--owners a,b] # default branch of every PUBLIC repo
python3 scripts/runner_guard.py pr [--owners a,b] [--post] # open PRs on public repos: changed workflows
"""
from __future__ import annotations
import argparse
import base64
import json
import os
import subprocess
import sys
import yaml
OWNERS = ["sneakyfree", "VERONTECH", "Windstorm-Institute", "Windstorm-Labs", "Public-Streamer"]
CTX = "windy-git/runner-guard"
OUTSIDE = {"issue_comment", "workflow_run", "issues", "discussion", "discussion_comment",
"pull_request_review", "pull_request_review_comment", "fork", "watch"}
SAME_REPO_GATES = ("head.repo.full_name == github.repository", "github.repository == github.event.pull_request.head.repo.full_name",
"head.repo.fork == false", "!github.event.pull_request.head.repo.fork")
def _node_map(node):
"""{key: (value_node, line)} for a YAML mapping node."""
if not isinstance(node, yaml.MappingNode):
return {}
return {k.value: (v, k.start_mark.line + 1) for k, v in node.value if isinstance(k, yaml.ScalarNode)}
def _triggers(on_node) -> dict[str, int]:
"""{event: line}."""
if isinstance(on_node, yaml.ScalarNode):
return {on_node.value: on_node.start_mark.line + 1}
if isinstance(on_node, yaml.SequenceNode):
return {n.value: n.start_mark.line + 1 for n in on_node.value if isinstance(n, yaml.ScalarNode)}
return {k: line for k, (_v, line) in _node_map(on_node).items()}
def _self_hosted(runs_on) -> bool:
if runs_on is None:
return False
text = yaml.serialize(runs_on) if isinstance(runs_on, yaml.Node) else str(runs_on)
return "self-hosted" in text or "${{" in text
def lint_text(path: str, text: str) -> list[tuple[str, int, str, str]]:
"""[(path, line, rule, message)]. Unparseable YAML is a finding (it cannot be reviewed)."""
try:
root = yaml.compose(text)
except yaml.YAMLError as e:
line = getattr(getattr(e, "problem_mark", None), "line", 0) + 1
return [(path, line, "R0", "workflow YAML does not parse; cannot be checked")]
top = _node_map(root)
if "on" not in top or "jobs" not in top:
return []
trig = _triggers(top["on"][0])
jobs = _node_map(top["jobs"][0])
out = []
if "pull_request_target" in trig:
out.append((path, trig["pull_request_target"], "R1",
"pull_request_target runs fork code with base-repo secrets; not allowed"))
for name, (jnode, jline) in jobs.items():
j = _node_map(jnode)
ro = j.get("runs-on", (None, jline))
if not _self_hosted(ro[0]):
continue
has_env = "environment" in j
cond = j["if"][0].value if "if" in j and isinstance(j["if"][0], yaml.ScalarNode) else ""
same_repo = any(g in cond.replace(" ", " ") for g in SAME_REPO_GATES)
if "pull_request" in trig and not (has_env or same_repo):
out.append((path, ro[1], "R2", f"job '{name}' runs fork PR code on a self-hosted runner "
"(gate it: if: github.event.pull_request.head.repo.full_name == github.repository, or an environment)"))
for ev in sorted(OUTSIDE & trig.keys()):
if not has_env:
out.append((path, trig[ev], "R3", f"'{ev}' can be fired by anyone and job '{name}' is self-hosted "
"without an environment gate"))
return out
# ---------------------------------------------------------------- GitHub side
def gh(*args: str, check=True) -> str:
r = subprocess.run(["gh", "api", *args], capture_output=True, text=True, timeout=60)
if check and r.returncode != 0:
raise RuntimeError(f"gh api {args[0]} failed")
return r.stdout
def public_repos(owners) -> list[tuple[str, str]]:
out = []
for o in owners:
txt = gh(f"users/{o}/repos?per_page=100&type=owner", "--paginate",
"--jq", '.[]|select(.private==false and .archived==false)|.full_name+" "+.default_branch', check=False)
out += [tuple(row.split()) for row in txt.splitlines() if row.strip()]
return out
def workflows_at(full: str, ref: str) -> list[tuple[str, str]]:
txt = gh(f"repos/{full}/contents/.github/workflows?ref={ref}", "--jq",
'.[]|select(.type=="file")|.path', check=False)
files = [p for p in txt.splitlines() if p.endswith((".yml", ".yaml"))]
return [(p, file_at(full, p, ref)) for p in files]
def file_at(full: str, path: str, ref: str) -> str:
raw = gh(f"repos/{full}/contents/{path}?ref={ref}", "--jq", ".content", check=False).strip()
return base64.b64decode(raw).decode("utf-8", "replace") if raw else ""
def cmd_report(owners) -> int:
hits = 0
repos = public_repos(owners)
for full, branch in repos:
for path, text in workflows_at(full, branch):
for p, line, rule, msg in lint_text(path, text):
hits += 1
print(f"{full}\t{p}:{line}\t{rule}\t{msg}")
print(f"# runner-guard sweep: {hits} hit(s) in {len(repos)} public repos")
return 1 if hits else 0
STATE = os.environ.get("RUNNER_GUARD_STATE", "/var/lib/windy-git/runner-guard-posted.json")
def cmd_pr(owners, post: bool) -> int:
# Post each (repo, sha, state, description) ONCE: the sync runs every 5 min and GitHub caps
# statuses per sha+context at 1000.
try:
with open(STATE) as fh:
posted = set(json.load(fh))
except (OSError, ValueError):
posted = set()
seen = set()
for full, _branch in public_repos(owners):
prs = gh(f"repos/{full}/pulls?state=open&per_page=50", "--jq",
'.[]|(.number|tostring)+" "+.head.sha+" "+.head.repo.full_name', check=False)
for line in prs.splitlines():
num, sha, head_repo = line.split(" ", 2)
files = gh(f"repos/{full}/pulls/{num}/files?per_page=100", "--jq",
'.[]|select(.status!="removed")|.filename', check=False).split()
wf = [f for f in files if f.startswith(".github/workflows/") and f.endswith((".yml", ".yaml"))]
# a fork's own content is read from the head repo at the head sha
src = head_repo if head_repo and head_repo != "null" else full
found = [h for f in wf for h in lint_text(f, file_at(src, f, sha))]
if found:
p, ln, rule, msg = found[0]
state, desc = "failure", f"BLOCKED: {rule} {p}:{ln}: {msg}"[:140]
else:
state, desc = "success", ("OK: no workflow changes" if not wf else
"OK: no stranger-code path to a self-hosted runner")
key = f"{full}@{sha}:{state}:{desc}"
seen.add(key)
if key in posted:
continue
print(f"{full}#{num}@{sha[:7]} {state} {desc}")
if post:
gh(f"repos/{full}/statuses/{sha}", "-f", f"state={state}", "-f", f"context={CTX}",
"-f", f"description={desc}", check=False)
posted.add(key)
if post: # keep only keys for PRs still open, so the file never grows without bound
os.makedirs(os.path.dirname(STATE), exist_ok=True)
with open(STATE, "w") as fh:
json.dump(sorted(posted & seen), fh)
return 0
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="runner_guard")
sub = ap.add_subparsers(dest="cmd", required=True)
lint_p = sub.add_parser("lint")
lint_p.add_argument("files", nargs="+")
rep = sub.add_parser("report")
rep.add_argument("--owners", default=",".join(OWNERS))
prp = sub.add_parser("pr")
prp.add_argument("--owners", default="sneakyfree") # self-hosted runners exist only there
prp.add_argument("--post", action="store_true")
a = ap.parse_args(argv)
if a.cmd == "lint":
hits = []
for f in a.files:
with open(f, errors="replace") as fh:
hits += lint_text(f, fh.read())
for p_, ln, rule, msg in hits:
print(f"{p_}:{ln}\t{rule}\t{msg}")
return 1 if hits else 0
owners = a.owners.split(",")
return cmd_report(owners) if a.cmd == "report" else cmd_pr(owners, a.post)
if __name__ == "__main__":
sys.exit(main())

131
scripts/secret_guard.py Normal file
View File

@@ -0,0 +1,131 @@
#!/usr/bin/env python3
"""Secret guard: no live credential lands in a bridged repo (leak hunt 09-24).
Same walker, cache and GitHub posting as compute_guard / ci_hygiene
(`windy-git/secret-guard`), but over EVERY text file, and a finding carries only
"<kind> #<sha256[:8]>", never the value (house rule 10). Known fakes are allowed
BY HASH in ci/secret-guard-allow.yml (repo + hashes + reason).
sudo python3 scripts/secret_guard.py report [repo ...]
"""
from __future__ import annotations
import fnmatch
import hashlib
import os
import re
import sys
from pathlib import Path
import yaml
sys.path.insert(0, str(Path(__file__).resolve().parent))
import compute_guard as cg # noqa: E402 (shared walker, cache)
import secret_shapes as ss # noqa: E402
ROOT = Path(__file__).resolve().parents[1]
ALLOW_FILE = Path(os.environ.get("SECRET_GUARD_ALLOW", ROOT / "ci" / "secret-guard-allow.yml"))
MODE = os.environ.get("SECRET_GUARD_MODE", "warn")
# Kinds that only WARN (rolled out warn-first); empty = every kind blocks in block mode.
WARN_KINDS = {k for k in os.environ.get("SECRET_GUARD_WARN_KINDS", "").split(",") if k}
NEVER = re.compile(r"(^|/)(node_modules|vendor|third_party)/")
def path_ok(path: str) -> bool:
return not NEVER.search(path)
# A private-key match is only its BEGIN line, so its hash is the same everywhere:
# those are allowed by PATH (entries with `paths` + `kinds`), everything else by HASH.
PATH_ONLY_KINDS = {"private key block"}
def load_allow(path: Path = ALLOW_FILE) -> dict[str, dict]:
"""{repo: {"hashes": {hash8}, "paths": [(glob, {kind})]}}; every entry needs a reason."""
data = yaml.safe_load(path.read_text()) if path.exists() else {}
out: dict[str, dict] = {}
for e in (data or {}).get("allow") or []:
if not (e.get("repo") and (e.get("hashes") or (e.get("paths") and e.get("kinds")))
and str(e.get("reason", "")).strip()):
raise ValueError(f"allow entry needs repo, hashes (or paths + kinds) and a reason: {e}")
if e.get("paths") and not set(e["kinds"]) <= PATH_ONLY_KINDS:
raise ValueError(f"path allows are only for {sorted(PATH_ONLY_KINDS)}: {e}")
r = out.setdefault(e["repo"], {"hashes": set(), "paths": []})
r["hashes"].update(str(h) for h in e.get("hashes") or [])
r["paths"] += [(g, set(e["kinds"])) for g in e.get("paths") or []]
return out
def scan_line(path: str, text: str) -> list[tuple[str, str]]:
return [(kind, f"{kind} #{h}") for kind, h in ss.find(text)]
def _drop_allowed(repo: str, findings, allow: dict[str, dict]):
a = allow.get(repo) or {"hashes": set(), "paths": []}
def ok(f) -> bool:
if f.kind in PATH_ONLY_KINDS:
return any(f.kind in kinds and fnmatch.fnmatch(f.path, g) for g, kinds in a["paths"])
return f.match.rsplit("#", 1)[-1] in a["hashes"]
return [f for f in findings if not ok(f)]
def check(repo: str, sha: str, default_branch: str, is_default_head: bool):
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir() or not cg.fetched(bare, sha): # pushed after the fetch: next cycle
return None
allow = load_allow()
rules = hashlib.sha256(("|".join(rx.pattern for _, rx in ss.PATTERNS) + ss.PREFILTER).encode()).hexdigest()[:8]
kw = dict(line_fn=scan_line, path_ok=path_ok)
if is_default_head:
fs = cg.cached_scan(f"sec-tree:{repo}:{sha}:{rules}",
lambda: cg.scan_tree(repo, bare, sha, [], prefilter=ss.PREFILTER, **kw))
else:
fs = cg.cached_scan(f"sec-pr:{repo}:{sha}:{rules}",
lambda: cg.scan_added(repo, bare, f"refs/heads/{default_branch}", sha, [], **kw))
return _drop_allowed(repo, fs, allow)
def status_for(findings, whole_tree: bool, grant=()):
"""Same contract as the other guards. `grant` findings never block."""
scope = "in tree" if whole_tree else "added"
if not findings and grant:
g, n = grant[0], len(grant)
return "success", f"⚠ WARN (Grant-owned, not blocking): {n} secret-shaped string{'s' if n > 1 else ''} {scope}, e.g. {g.path}:{g.line} {g.match}"[:140], g
if not findings:
return "success", f"OK: no secret-shaped strings {scope}", None
f, n = findings[0], len(findings)
soft = MODE != "block" or all(x.kind in WARN_KINDS for x in findings)
state = "success" if soft else "failure"
lead = "⚠ WARN (not blocking)" if soft else "BLOCKED"
if not soft:
f = next(x for x in findings if x.kind not in WARN_KINDS)
return state, f"{lead}: {n} secret-shaped string{'s' if n > 1 else ''} {scope}, e.g. {f.path}:{f.line} {f.match}"[:140], f
def report(repos: list[str]) -> int:
allow = load_allow()
total = 0
for repo in repos:
bare = cg.WORK / f"{repo}.git"
if not bare.is_dir():
continue
head = cg._git(bare, "symbolic-ref", "--short", "HEAD").strip()
sha = cg._git(bare, "rev-parse", head).strip()
fs = _drop_allowed(repo, cg.scan_tree(repo, bare, sha, [], line_fn=scan_line, path_ok=path_ok,
prefilter=ss.PREFILTER), allow)
total += len(fs)
print(f"## {repo} ({head} {sha[:7]}): {len(fs)} finding(s)")
for f in fs:
print(f" {f.path}:{f.line} {f.match}")
print(f"TOTAL {total}")
return 0
if __name__ == "__main__":
if len(sys.argv) >= 2 and sys.argv[1] == "report":
default = os.environ.get("BRIDGE_REPOS", "").split() or sorted(
p.name.removesuffix(".git") for p in cg.WORK.glob("*.git"))
sys.exit(report(sys.argv[2:] or default))
sys.exit(__doc__)

210
scripts/secret_scan.py Normal file
View File

@@ -0,0 +1,210 @@
#!/usr/bin/env python3
"""secret-scan: hash-only secret finder. Boss ruling 10-01 after three lanes printed secrets
into their own transcripts while hunting secrets (house rule 10).
secret-scan <path> [--history] [--repo <git url or path>] [--no-lockbox]
Reports `file:line` (and the commit with --history), WHICH lockbox entry matched (the KEY
NAME only) or which secret SHAPE matched (twilio, zai, aws, ...), plus a sha256[:8] of the
token for allow-listing. It NEVER prints, logs or writes a value or any fragment of one
(no context line, no masking). The lockbox is loaded in memory only. stdout only.
Exit 0 = clean, 1 = findings, 2 = usage/error.
"""
from __future__ import annotations
import argparse
import hashlib
import os
import re
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import secret_shapes as ss # noqa: E402 (the SAME shapes as secret-guard)
HOME = Path.home()
LOCKBOX_PATHS = [p for p in os.environ.get("SECRET_SCAN_LOCKBOX_PATHS", "").split(":") if p] or [
str(HOME / "kit-army-config" / "secrets"), str(HOME / "kit-army-config" / "ACCESS_LOCKBOX.md")]
# Extra shapes that are scan-only (not in the blocking guard): label, regex.
EXTRA = [("zai key", re.compile(r"(?<![0-9a-f])[0-9a-f]{32}\.[A-Za-z0-9]{16}(?![A-Za-z0-9])"))]
SKIP_DIRS = {".git", "node_modules", "vendor", "third_party", "__pycache__", ".venv"}
MAX_BYTES = 5_000_000
RUN = re.compile(r"[A-Za-z0-9][A-Za-z0-9_\-]{15,199}")
UUID = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$")
NAME = re.compile(r"[\s>*`|-]*([A-Za-z][A-Za-z0-9_]{2,60})\s*[=:|]")
def h16(t: str) -> str:
return hashlib.sha256(t.encode()).hexdigest()[:16]
def cands(line: str):
"""Token candidates: runs >=16 chars with a digit and a letter; git shas/uuids excluded."""
for chunk in re.split(r"[^A-Za-z0-9_\-.]+", line):
parts = [chunk, *chunk.split(".")] if "." in chunk else [chunk]
for p in parts:
for m in RUN.finditer(p):
t = m.group(0)
if not (re.search(r"\d", t) and re.search(r"[A-Za-z]", t)):
continue
if re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", t) or UUID.match(t.lower()):
continue
yield t
def load_lockbox() -> dict[str, set[str]]:
"""{hash16: {key-name labels}}; values never leave this dict."""
out: dict[str, set[str]] = {}
files: list[str] = []
for p in LOCKBOX_PATHS:
if os.path.isdir(p):
for root, _d, fs in os.walk(p):
files += [os.path.join(root, f) for f in fs]
elif os.path.isfile(p):
files.append(p)
for f in files:
try:
with open(f, errors="ignore") as fh:
for line in fh:
m = NAME.match(line)
label = m.group(1) if m else "?"
for t in cands(line):
out.setdefault(h16(t), set()).add(label)
except OSError:
continue
return out
def scan_line(line: str, lockbox: dict[str, set[str]]) -> list[tuple[str, str]]:
"""[(label, hash8)]: `shape:<kind>` and/or `lockbox:<NAMES>`. No value escapes."""
res: list[tuple[str, str]] = []
for kind, h in ss.find(line):
res.append((f"shape:{kind}", h))
for kind, rx in EXTRA:
for m in rx.finditer(line):
res.append((f"shape:{kind}", ss.h8(m.group(0))))
for t in cands(line):
k = h16(t)
if k in lockbox:
names = sorted(n for n in lockbox[k])
res.append(("lockbox:" + ",".join(names)[:70], k[:8]))
return sorted(set(res))
def is_text(path: Path) -> bool:
try:
with open(path, "rb") as fh:
return b"\0" not in fh.read(4096)
except OSError:
return False
def scan_tree(root: Path, lockbox):
files = [root] if root.is_file() else [
Path(dp) / f for dp, dn, fn in os.walk(root) for f in fn
if not set(Path(dp).relative_to(root).parts) & SKIP_DIRS]
for p in sorted(files):
try:
if p.stat().st_size > MAX_BYTES or not is_text(p):
continue
with open(p, errors="ignore") as fh:
for n, line in enumerate(fh, 1):
for label, h in scan_line(line, lockbox):
yield (str(p), n, None, label, h)
except OSError:
continue
def git(repo: str, *a: str) -> subprocess.Popen:
return subprocess.Popen(["git", "--git-dir", repo, *a], stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, text=True, errors="ignore")
def scan_history(gitdir: str, lockbox):
"""Every ADDED line on every ref (incl. PR refs). Oldest commit per (hash, file, line)."""
seen: dict[tuple, str] = {}
p = git(gitdir, "log", "--all", "-p", "-U0", "--no-color", "--format=@@C %h", "-a")
commit = path = None
ln = 0
for row in p.stdout: # type: ignore[union-attr]
if row.startswith("@@C "):
commit = row[4:].strip()
elif row.startswith("+++ "):
path = row[6:].strip() if row.startswith("+++ b/") else None
elif row.startswith("@@ "):
m = re.search(r"\+(\d+)", row)
ln = int(m.group(1)) - 1 if m else 0
elif row.startswith("+") and path:
ln += 1
for label, h in scan_line(row[1:], lockbox):
seen[(label, h, path, ln)] = commit or "?"
p.wait()
for (label, h, path, ln), c in sorted(seen.items(), key=lambda x: (x[0][2], x[0][3])):
yield (path, ln, c, label, h)
def resolve_gitdir(p: Path) -> str | None:
for cand in (p / ".git", p):
if (cand / "HEAD").exists() and ((cand / "objects").exists()):
return str(cand)
return None
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="secret-scan", description=__doc__.split("\n\n")[1] if __doc__ else "")
ap.add_argument("path", nargs="?", help="file, directory, or git repo (with --history)")
ap.add_argument("--history", action="store_true", help="scan every added line in all git history")
ap.add_argument("--repo", help="git URL or path to scan (mirror-cloned to a temp dir, then deleted)")
ap.add_argument("--no-lockbox", action="store_true", help="shapes only")
a = ap.parse_args(argv)
if not (a.path or a.repo):
ap.print_usage()
return 2
lockbox = {} if a.no_lockbox else load_lockbox()
print(f"# secret-scan: {len(lockbox)} lockbox tokens in memory, values never printed", flush=True)
tmp = None
findings = 0
try:
if a.repo:
tmp = tempfile.mkdtemp(prefix="secret-scan-", dir=str(HOME / ".cache") if (HOME / ".cache").is_dir() else None)
os.chmod(tmp, 0o700)
target = os.path.join(tmp, "r.git")
rc = subprocess.run(["git", "clone", "-q", "--mirror", a.repo, target],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL).returncode
if rc != 0:
print("error: clone failed (details withheld: URLs can carry tokens)")
return 2
a.history = True
gitdir = target
elif a.history:
gitdir = resolve_gitdir(Path(a.path))
if not gitdir:
print("error: --history needs a git repo path")
return 2
if a.history:
for path, ln, c, label, h in scan_history(gitdir, lockbox):
findings += 1
print(f"{path}:{ln} commit={c} {label} #{h}")
else:
for path, ln, _c, label, h in scan_tree(Path(a.path), lockbox):
findings += 1
print(f"{path}:{ln} {label} #{h}")
finally:
if tmp:
shutil.rmtree(tmp, ignore_errors=True)
print(f"# {findings} finding(s)")
return 1 if findings else 0
if __name__ == "__main__":
try:
sys.exit(main())
except KeyboardInterrupt:
sys.exit(130)
except Exception as e: # never a traceback: it could carry data
print(f"error: {type(e).__name__}")
sys.exit(2)

46
scripts/secret_shapes.py Normal file
View File

@@ -0,0 +1,46 @@
"""Secret-shaped strings, shared by secret_guard (bridged repos) and
public_secret_scan (weekly, every public repo). A finding NEVER carries the value:
only its kind and sha256[:8] (house rule 10). Leak hunt 09-24: @Windy_0_bot's
token sat in a public repo's test fixture for five months."""
from __future__ import annotations
import hashlib
import re
# (kind, regex). Order matters only for readability; each match is reported once.
PATTERNS: list[tuple[str, re.Pattern[str]]] = [
("telegram bot token", re.compile(r"(?<![0-9])[0-9]{8,10}:[A-Za-z0-9_-]{35}(?![A-Za-z0-9_-])")),
("github token", re.compile(r"\b(?:gh[pousr]_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{82})\b")),
("aws access key", re.compile(r"\b(?:AKIA|ASIA)[0-9A-Z]{16}\b")),
("slack token", re.compile(r"\bxox[abprs]-[A-Za-z0-9-]{10,}")),
("anthropic key", re.compile(r"\bsk-ant-[A-Za-z0-9_-]{20,}")),
("openai key", re.compile(r"\bsk-(?:proj-|svcacct-)?(?!ant-)[A-Za-z0-9_-]{32,}")),
("stripe live key", re.compile(r"\b[rs]k_live_[A-Za-z0-9]{20,}")),
("google api key", re.compile(r"\bAIza[0-9A-Za-z_-]{35}(?![0-9A-Za-z_-])")),
# Twilio (Windy Text 10-01: a live auth token sat in test files for months). An auth token
# is a bare 32-hex with no prefix, so it is only caught when ASSIGNED to a secret-ish name.
("twilio sid/api key", re.compile(r"\b(?:AC|SK)[0-9a-f]{32}\b")),
("32-hex secret assignment", re.compile(
r"(?i)\b[a-z0-9_.-]*(?:token|secret|key|password)[a-z0-9_.-]*[\"']?\s*[:=]\s*[\"']?(?P<v>(?<![0-9a-f])[0-9a-f]{32}(?![0-9a-f]))")),
("pypi token", re.compile(r"\bpypi-AgE[A-Za-z0-9_-]{50,}")),
("private key block", re.compile(r"-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED )?PRIVATE KEY-----")),
]
# git grep -E (POSIX ERE) prefilter: cheap superset of PATTERNS.
PREFILTER = ("[0-9]{8,10}:[A-Za-z0-9_-]{35}|gh[pousr]_[A-Za-z0-9]{36}|github_pat_|(AKIA|ASIA)[0-9A-Z]{16}"
"|xox[abprs]-|sk-ant-|sk-[A-Za-z0-9_-]{32}|sk-proj-|[rs]k_live_|AIza[0-9A-Za-z_-]{35}"
"|-----BEGIN [A-Z ]*PRIVATE KEY-----|(AC|SK)[0-9a-f]{32}|[0-9a-fA-F]{32}|pypi-AgE")
def h8(value: str | bytes) -> str:
return hashlib.sha256(value.encode() if isinstance(value, str) else value).hexdigest()[:8]
def find(text: str) -> list[tuple[str, str]]:
"""[(kind, hash8)] for every secret-shaped string in `text`. Values never leave."""
out = []
for kind, rx in PATTERNS:
for m in rx.finditer(text):
out.append((kind, h8(m.group('v') if 'v' in rx.groupindex else m.group(0))))
return out

View File

@@ -38,7 +38,7 @@ FAILED=0
# Repos Windy Git tracks FROM GitHub. Remove a repo from this list at the moment
# it flips to Windy-Git-first, or the sync will fight its authors and win.
REPOS="${SYNC_REPOS:-windy-calendar windy-search windy-registry Windy-Clone WindyCloud windy-cloud-sites windy-mind eternitas windy-agent windy-git windy-chat windy-mail windy-connect windy-drops windy-code-web windy-code windy-traveler windy-translate windytranslate-site windytraveler-site windy-hand windy-cloud-domains windy-cloud-vps windytalk windy-pro}"
REPOS="${SYNC_REPOS:-windy-calendar windy-search windy-registry Windy-Clone WindyCloud windy-cloud-sites windy-mind eternitas windy-agent windy-git windy-chat windy-mail windy-connect windy-drops windy-code-web windy-code windy-traveler windy-translate windytranslate-site windytraveler-site windy-hand windy-cloud-domains windy-cloud-vps windytalk windy-pro windy-inbox windy-text windy-call windy-cell windy-hand-site windy-calendar-site}"
# Repos whose TAGS must not reach Windy Git. A tag push fires `on: push: tags`
# workflows; windy-pro's build-electron is a matrix over ubuntu/macos/windows-
@@ -94,6 +94,11 @@ if ! python3 "$(dirname "$0")/pr_status_bridge.py"; then
log "FAILED pr status bridge"; FAILED=1
fi
# Runner guard (Boss 10-01): PUBLIC sneakyfree repos have self-hosted GitHub runners on Veron.
# A PR that changes a workflow so a stranger's code could reach one gets a red
# windy-git/runner-guard status. Each status is posted once; never fails the sync.
timeout -k 10 120 python3 "$(dirname "$0")/runner_guard.py" pr --post || log "runner-guard failed or timed out (non-fatal)"
# CI telemetry -> admin.windyword.ai (shapes declared with Windy Telemetry 40).
# Sends nothing until WINDYGIT_TELEMETRY_TOKEN is set; never fails the sync.
timeout -k 10 180 python3 "$(dirname "$0")/telemetry_emit.py" || log "telemetry emit failed or timed out (non-fatal)"