Compare commits
10 Commits
647d414ec1
...
5f717ef74b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5f717ef74b | ||
|
|
b274392e96 | ||
|
|
d684cfc2e1 | ||
|
|
a2430de94d | ||
|
|
da257652c2 | ||
|
|
066de34489 | ||
|
|
3a9259a0da | ||
|
|
f621e49770 | ||
|
|
099e4be9b1 | ||
|
|
82044933ed |
73
.gitea/workflows/check.yml
Normal file
73
.gitea/workflows/check.yml
Normal file
@@ -0,0 +1,73 @@
|
||||
# The gate, running on our own hardware (G7.3).
|
||||
#
|
||||
# This is the dogfood: windy-git verifies itself before anything else migrates.
|
||||
#
|
||||
# `runs-on: veron-1` is a label this runner actually provides. NEVER
|
||||
# `ubuntu-latest` (G7.5) — a self-hosted runner has no such label, so a workflow
|
||||
# naming it queues forever and presents as a hung CI system rather than a typo.
|
||||
|
||||
name: check
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
gate:
|
||||
runs-on: veron-1
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
env:
|
||||
POSTGRES_USER: windygit
|
||||
POSTGRES_PASSWORD: windygit
|
||||
POSTGRES_DB: windygit
|
||||
options: >-
|
||||
--health-cmd "pg_isready -U windygit"
|
||||
--health-interval 5s
|
||||
--health-retries 10
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: install
|
||||
run: |
|
||||
python3 -m venv .venv
|
||||
.venv/bin/pip install -q -e ".[dev]"
|
||||
|
||||
- name: lint
|
||||
run: .venv/bin/ruff check api scripts
|
||||
|
||||
- name: vocabulary audit (D-9)
|
||||
run: python3 scripts/vocab_audit.py
|
||||
|
||||
- name: tests
|
||||
run: .venv/bin/pytest -q
|
||||
|
||||
# G0.4 — a migration nobody has run is a migration nobody can trust. This
|
||||
# is the step that caught two bugs review did not: SQLAlchemy Enum
|
||||
# persisting .name instead of .value, and create_table re-emitting
|
||||
# CREATE TYPE without checkfirst.
|
||||
- name: migration round-trip (upgrade -> downgrade -> upgrade)
|
||||
env:
|
||||
DATABASE_URL: postgresql://windygit:windygit@postgres:5432/windygit
|
||||
run: |
|
||||
.venv/bin/alembic upgrade head
|
||||
.venv/bin/alembic downgrade base
|
||||
.venv/bin/alembic upgrade head
|
||||
|
||||
# I-12 — the honesty check. Nine sibling services cannot name the commit
|
||||
# they are running; one reports another repo's commit entirely.
|
||||
- name: /version must equal HEAD
|
||||
run: |
|
||||
HEAD_SHA=$(git rev-parse HEAD)
|
||||
COMMIT_SHA=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef \
|
||||
.venv/bin/python -c "
|
||||
import os, sys
|
||||
sys.path.insert(0, '.')
|
||||
from api.app.buildinfo import get_build_info
|
||||
info = get_build_info()
|
||||
expected = '$HEAD_SHA'
|
||||
assert info.commit_sha == expected, f'{info.commit_sha} != {expected}'
|
||||
print('I-12 holds: env override ignored, reported', info.commit_sha[:12])
|
||||
"
|
||||
@@ -258,13 +258,61 @@ Strands G0–G4 are sequential. G5–G12 are concurrent once G4 lands.
|
||||
## Strand G4 — Storage wiring
|
||||
|
||||
- **G4.1** R2 buckets: `windy-git-lfs`, `windy-git-artifacts`, `windy-git-backups` on account `193b347aedeaafe35de0b5a534b2d9aa`.
|
||||
- **G4.2** **Scoped** R2 credential, minted for this cell. ⚠️ The sites cell shipped holding the account-wide god token (`godtoken02JUL26`) because v4 R2 object endpoints reject restricted tokens — record which we ended up with in `SUBSTRATE.md` and treat an account-wide token as a known, named debt, not an invisible one.
|
||||
- **G4.2** R2 credential: **use an existing fleet token** (Grant's ruling, 2026-08-11 — sandbox phase, months from launch; a scoped platform token is a launch-hardening item, not a blocker). The non-obvious part worth recording: R2's S3 credentials are *derived* from a Cloudflare API token — **access key id = the token's id, secret = SHA-256 of the token value**.
|
||||
- **G4.3** Gitea `[storage]` → `STORAGE_TYPE = minio` pointed at R2, covering LFS, attachments, packages, avatars and actions artifacts. ⚠️ **Known R2 trap:** R2 rejects the default checksum algorithm several S3 clients send; if uploads fail with a checksum error, set the MD5 checksum option. *Accept:* a 500 MB file round-trips through LFS and the object is confirmed present in R2 with a matching etag — **verify against the exact pinned Gitea version's docs, do not trust this key name from memory.**
|
||||
- **G4.4** Git object stores on `/srv/windygit/git` (local NVMe). A test asserts no git object path resolves to a network mount (I-3).
|
||||
- **G4.5** LFS threshold policy: files **> 5 MB** or matching binary/weight extensions (`.safetensors .bin .gguf .pt .ckpt .onnx .zip .mp4 .wav`) go to LFS via a committed `.gitattributes` template applied at repo creation. **Small text files stay in git proper** — LFS-for-everything makes clones slow and operations heavy.
|
||||
- **G4.6** Quota: on push, call the kernel's quota check; over-quota → refuse with a repair-pointer error whose `speak` is grandma-words and whose `remediation_tool` names the upgrade path (Storage-Kingdom cross-sell hook, §0.5). **We emit the hook; the kernel owns the price** (I-11).
|
||||
- **G4.7** Object-count and byte accounting per repo, recomputed nightly, exposed on `GET /{id}/status`.
|
||||
|
||||
## Strand G4A — Gitea/R2 traps paid for on 2026-08-11
|
||||
|
||||
Four failures hit while wiring G2–G4 live. Each cost a crash loop or a dead end,
|
||||
and each presents as a different problem than it is. All four are now pinned by
|
||||
tests in `api/tests/test_invariants.py`.
|
||||
|
||||
- **G4A.1 — `[lfs] STORAGE_TYPE` breaks storage inheritance.** Naming a storage
|
||||
type inside `[lfs]` creates a **separate** storage section that does *not*
|
||||
inherit the endpoint or credentials from `[storage]`. Gitea crash-loops on
|
||||
`Endpoint: does not follow ip address or domain name standards` — an error that
|
||||
names the symptom and not the cause. **Let LFS inherit `[storage]`.** Avatars
|
||||
initialising correctly is the tell that the rest of the config is fine.
|
||||
- **G4A.2 — setting the storage backend does not turn LFS on.** `[server]
|
||||
LFS_START_SERVER = true` is separate. Without it the batch endpoint 404s and
|
||||
the client reports *"Repository or object not found"*, which reads like a
|
||||
permissions or credentials problem and is neither.
|
||||
- **G4A.3 — ⚠️ Gitea's env-to-ini SETS but never UNSETS.** Removing a `GITEA__*`
|
||||
variable from compose does **not** remove the line from the persisted
|
||||
`app.ini`. The container will keep booting with a setting that no longer exists
|
||||
anywhere in the repo, so the config in git and the config in production
|
||||
silently disagree — the exact class of drift this whole cell exists to end.
|
||||
**To remove a setting you must edit `app.ini` on the host**
|
||||
(`/srv/windygit/git/gitea/conf/app.ini`), not just the compose file.
|
||||
- **G4A.4 — R2 rejects the default S3 checksum algorithm.** Set
|
||||
`MINIO_CHECKSUM_ALGORITHM = md5` or uploads fail with an opaque checksum error.
|
||||
|
||||
### G4A.5 — ⚠️ Cloudflare's 100-second limit caps a push, and it shapes G10
|
||||
|
||||
A plain (non-LFS) push of an 8 MB file over the tunnel died with **HTTP 524**.
|
||||
Cloudflare's proxy times out at ~100 s on the Free plan, and Grant's residential
|
||||
upstream measured ~19 KB/s during the LFS test, so anything large is a coin flip.
|
||||
|
||||
This is not a bug to fix; it is a constraint that **dictates the model-hub
|
||||
architecture**:
|
||||
|
||||
1. **The LFS threshold (G4.5) is load-bearing, not tidiness.** Anything big must
|
||||
go via LFS, because a large blob inside a git pack has no way to be resumed
|
||||
or offloaded.
|
||||
2. **G10 must serve LFS objects via presigned R2 URLs — client straight to R2 —
|
||||
rather than proxying blobs through the tunnel.** That removes the 100 s
|
||||
ceiling, removes Grant's home upstream from the path entirely, and is exactly
|
||||
what Hugging Face does. Until that lands, model repos are capped by whatever
|
||||
fits in 100 seconds.
|
||||
|
||||
**Verified working on 2026-08-11:** a 3 MB LFS object pushed through the tunnel
|
||||
and landed in R2 at `lfs/34/2d/…`, with **no local `lfs/` directory on the host**
|
||||
— I-3 confirmed by measurement rather than by assertion.
|
||||
|
||||
## Strand G5 — THE SHELTER (permissions plane over Windy Cloud) · **the v0 product**
|
||||
|
||||
*This strand is the one that ships first and the one that has no competitor. Windy Cloud today has no sharing, no permissions and no versioning of any kind (D-8).*
|
||||
@@ -276,6 +324,13 @@ Strands G0–G4 are sequential. G5–G12 are concurrent once G4 lands.
|
||||
- **G5.5** **Version history UI with one giant Undo.** Grandma-words throughout, and D-9 binds every string: "version", "save point", "restore" — **never "commit," never "a Git"** on this surface.
|
||||
- **G5.6** `POST /{id}/restore {version_seq}` — lossless, and itself a new version. **Undo is never destructive.**
|
||||
- **G5.7** Quota events double as storage-plan cross-sell hooks (§0.5).
|
||||
- **G5.9** ⚠️ **Service callers need an on-behalf-of identity.** Verified live on
|
||||
2026-08-11: a repo created through `X-Service-Token` lands in a `u-system`
|
||||
namespace, because the service caller has no identity of its own. That is
|
||||
correct for `/internal/*` plumbing and **wrong for anything a person owns** —
|
||||
the Cloud portal must pass the acting user, and this cell must refuse to
|
||||
create a user-owned repo without one. Until then, service-created repos are
|
||||
ops artifacts, not customer data.
|
||||
- **G5.8** *Accept — the strand's whole point in one test:* a Cloud folder is git-enabled, a second human is granted `writer`, that human edits a file through the portal, the owner restores the previous version, and **at no point does either user encounter the word "commit," "repo," "branch," or "Git."**
|
||||
|
||||
## Strand G6 — Git protocol surface
|
||||
|
||||
24
LICENSES/NOTICE.md
Normal file
24
LICENSES/NOTICE.md
Normal file
@@ -0,0 +1,24 @@
|
||||
# Third-party notices — Windy Git
|
||||
|
||||
Windy Git runs **stock Gitea** as an unforked component (D-2 / I-1). Gitea is
|
||||
distributed under the MIT license, reproduced in `gitea-MIT.txt`.
|
||||
|
||||
MIT's only obligation is that the copyright notice and license text travel with
|
||||
copies of the software that are **distributed**. Running Windy Git as a hosted
|
||||
service is not distribution, so strictly this file is not required today — it is
|
||||
here anyway, because it will be required the moment a self-host bundle ships, and
|
||||
because shipping it costs nothing.
|
||||
|
||||
MIT does **not** require us to advertise the lineage, does not restrict
|
||||
commercial use, and does not require publishing our modifications. That last
|
||||
point is why Gitea (MIT) was chosen over Forgejo (GPLv3 from v9): GPLv3 does not
|
||||
trigger on running a service — that is AGPL, and it is widely gotten wrong — but
|
||||
it **does** trigger on distributing a binary, which the self-host line would do.
|
||||
|
||||
**Gitea's trademarks are not used.** The product is branded Windy Git throughout.
|
||||
|
||||
| Component | Version | License |
|
||||
|---|---|---|
|
||||
| Gitea | 1.24.6 (pinned) | MIT — `gitea-MIT.txt` |
|
||||
| PostgreSQL | 16 | PostgreSQL License |
|
||||
| cloudflared | 2026.1.2 | Apache-2.0 |
|
||||
20
LICENSES/gitea-MIT.txt
Normal file
20
LICENSES/gitea-MIT.txt
Normal file
@@ -0,0 +1,20 @@
|
||||
Copyright (c) 2016 The Gitea Authors
|
||||
Copyright (c) 2015 The Gogs Authors
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
33
SUBSTRATE.md
33
SUBSTRATE.md
@@ -72,31 +72,20 @@ consulted, so a perfect service presents as "the app is broken."
|
||||
All in the fleet lockbox, injected by env, **never committed**. `make check`
|
||||
fails on any `cfat_` / `cfut_` / `gh[pousr]_` / `et_plt_` literal in the tree.
|
||||
|
||||
### ⚠️ NAMED DEBT — the R2 credential is account-wide
|
||||
### R2 credential — RULED, not a debt (Grant, 2026-08-11)
|
||||
|
||||
**As of 2026-08-11 this cell holds the Cloudflare god token as its R2
|
||||
credential.** R2's S3 credentials are derived from an API token (access key id =
|
||||
the token's id, secret = SHA-256 of its value), and **no token available to this
|
||||
session has permission to mint a new one** — creating tokens is dashboard-only
|
||||
or needs a token-creating token. So the wiring was proven with the god token
|
||||
rather than blocked on it.
|
||||
This cell uses an existing fleet Cloudflare token for R2. **That is the decision,
|
||||
not an oversight.** Grant's ruling, verbatim in intent: we are months from
|
||||
launch, in a sandbox, and minting a tenth Cloudflare token to sit in the
|
||||
inventory costs more than it buys. A platform-specific scoped token gets created
|
||||
as part of launch hardening.
|
||||
|
||||
This is recorded, not hidden, because an account-wide token is an acceptable
|
||||
named debt and an unacceptable invisible one.
|
||||
Recorded here so the next reader knows it was chosen rather than missed. **Do not
|
||||
re-raise it before the launch-hardening pass** — see the standing instruction
|
||||
about pre-launch security-hygiene nagging.
|
||||
|
||||
**GATE: this must be replaced with a scoped R2 token BEFORE strand G7 lands
|
||||
CI runners on this host.** I-5 says runners execute untrusted code and must not
|
||||
share a kernel with credentials scoped beyond their own job; a god token with
|
||||
R2 + Workers + Pages + WAF + SSL rights sitting on the same box as a runner is
|
||||
exactly the thing I-5 exists to prevent.
|
||||
|
||||
Minting one is a two-minute job in the Cloudflare dashboard: **R2 → Manage R2
|
||||
API Tokens → Create → Object Read & Write, scoped to the three `windy-git-*`
|
||||
buckets.** Then set `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY` in
|
||||
`/srv/windygit/src/.env` and redeploy.
|
||||
|
||||
⚠️ The Cloudflare **god token has Zone:Read but no DNS:Edit.** Use the DNS:Edit
|
||||
token for record creation.
|
||||
Access key id = the API token's id; secret = SHA-256 of the token value. That
|
||||
derivation is not obvious and is the thing worth writing down.
|
||||
|
||||
## Backups (G0.9)
|
||||
|
||||
|
||||
230
api/app/auth.py
Normal file
230
api/app/auth.py
Normal file
@@ -0,0 +1,230 @@
|
||||
"""Identity: humans, agents, and internal services (G3.2 / G3.6 / I-6).
|
||||
|
||||
Three caller classes, all first-class, none a bypass:
|
||||
|
||||
* **human** — account-server RS256 JWT (OIDC)
|
||||
* **agent** — Eternitas ES256 EPT
|
||||
* **service** — `X-Service-Token`, for the Cloud portal calling `/internal/*`
|
||||
|
||||
There is deliberately no fourth class and no escape hatch. The Windy Word desktop
|
||||
control server is the best Principle-#5 artifact in the ecosystem partly because
|
||||
it has **no bypass environment variable**, and that is copied here on purpose.
|
||||
|
||||
I-6 — EPT parity plus asymmetry: an agent in good standing gets exactly what a
|
||||
human of the same tier gets. Where a sibling cell silently demotes a tiered agent
|
||||
to FREE because its EPT carries no tier, we do the opposite, and a test proves it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
import httpx
|
||||
from fastapi import Header, Request
|
||||
|
||||
from api.app.config import Settings
|
||||
from api.app.errors import RepairPointer, passport_unresolvable
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ActorType(StrEnum):
|
||||
"""G3.7 — these three are the ONLY legal values.
|
||||
|
||||
A sibling service emits `actor_type: 'service'` into a telemetry ingest whose
|
||||
Literal allows only human|agent|system, so every batch 422s and is dropped
|
||||
with a single console warning. `service` is not spelled `service` here.
|
||||
"""
|
||||
|
||||
human = "human"
|
||||
agent = "agent"
|
||||
system = "system"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Caller:
|
||||
actor_type: ActorType
|
||||
identity_id: str | None = None
|
||||
passport: str | None = None
|
||||
band: str | None = None
|
||||
allowed_actions: tuple[str, ...] = ()
|
||||
|
||||
@property
|
||||
def subject(self) -> str:
|
||||
return self.identity_id or self.passport or "system"
|
||||
|
||||
|
||||
# EI_CAPABILITY_MATRIX.v1 — velocity multipliers by integrity band.
|
||||
BAND_MULTIPLIER: dict[str, float] = {
|
||||
"platinum": 10.0,
|
||||
"gold": 4.0,
|
||||
"standard": 1.0,
|
||||
"proven": 1.0,
|
||||
"watch": 0.5,
|
||||
"untrusted": 0.0, # read-only
|
||||
# Eternitas began emitting this band on 2026-07-30 and it is not in the
|
||||
# documented enum yet. Treating an unknown band as untrusted would lock out
|
||||
# every freshly hatched agent; treating it as trusted would be a hole.
|
||||
# Standard-with-no-bonus is the honest middle.
|
||||
"unproven": 1.0,
|
||||
}
|
||||
|
||||
|
||||
async def resolve_passport(settings: Settings, passport: str) -> tuple[str, tuple[str, ...]]:
|
||||
"""G3.6 — THE STATUS-CODE LAW.
|
||||
|
||||
404 and 400 REFUSE. 429 and 5xx retry with backoff, then REFUSE.
|
||||
|
||||
A sibling service maps 400 and 429 to "unreachable" and then soft-ALLOWS.
|
||||
That is a live residual bypass, because 429 is trivially inducible at
|
||||
100/min/IP: an attacker who wants the check skipped only has to make the
|
||||
check rate-limit itself. There is no code path here where an unresolvable
|
||||
passport is permitted to write.
|
||||
"""
|
||||
if not settings.eternitas_configured:
|
||||
raise RepairPointer(
|
||||
status_code=503,
|
||||
code="trust_unavailable",
|
||||
speak="We can't confirm helper IDs right now, so we didn't let that change through.",
|
||||
machine_cause="eternitas is not configured; policy is fail-closed",
|
||||
remediation_tool=None,
|
||||
)
|
||||
|
||||
url = f"{settings.eternitas_base_url}/api/v1/trust/{passport}"
|
||||
headers = {"X-API-Key": settings.eternitas_platform_api_key}
|
||||
last_status = 0
|
||||
for attempt in range(3):
|
||||
async with httpx.AsyncClient(timeout=httpx.Timeout(8.0, connect=3.0)) as client:
|
||||
try:
|
||||
r = await client.get(url, headers=headers)
|
||||
except httpx.RequestError as exc:
|
||||
last_status = 599
|
||||
log.warning("eternitas unreachable (attempt %s): %s", attempt + 1, exc)
|
||||
continue
|
||||
last_status = r.status_code
|
||||
if r.status_code == 200:
|
||||
body = r.json()
|
||||
return body.get("band", "unproven"), tuple(body.get("allowed_actions", []))
|
||||
if r.status_code in (400, 404):
|
||||
# Malformed or not-issued. Refuse immediately — retrying cannot help
|
||||
# and pretending it might is how a soft-allow gets written.
|
||||
break
|
||||
# 429 / 5xx: retry, then refuse. Never allow.
|
||||
raise passport_unresolvable(passport, last_status)
|
||||
|
||||
|
||||
async def get_caller(
|
||||
request: Request,
|
||||
authorization: str | None = Header(default=None),
|
||||
x_service_token: str | None = Header(default=None),
|
||||
) -> Caller:
|
||||
settings: Settings = request.app.state.settings
|
||||
|
||||
# --- internal service caller (the Cloud portal) ------------------------
|
||||
if x_service_token:
|
||||
expected = settings.service_token
|
||||
if not expected:
|
||||
raise RepairPointer(
|
||||
status_code=503,
|
||||
code="service_auth_unconfigured",
|
||||
speak="That connection isn't set up yet.",
|
||||
machine_cause="SERVICE_TOKEN is unset; refusing to accept service calls",
|
||||
remediation_tool=None,
|
||||
)
|
||||
# Constant-time compare, copied from the desktop control server's
|
||||
# control-auth pattern rather than reinvented.
|
||||
import hmac
|
||||
|
||||
if not hmac.compare_digest(x_service_token, expected):
|
||||
raise RepairPointer(
|
||||
status_code=401,
|
||||
code="service_token_invalid",
|
||||
speak="That connection isn't authorised.",
|
||||
machine_cause="X-Service-Token did not match",
|
||||
remediation_tool=None,
|
||||
)
|
||||
return Caller(actor_type=ActorType.system, identity_id="system")
|
||||
|
||||
if not authorization or not authorization.lower().startswith("bearer "):
|
||||
raise RepairPointer(
|
||||
status_code=401,
|
||||
code="not_signed_in",
|
||||
speak="You'll need to sign in first.",
|
||||
machine_cause="no bearer token and no service token presented",
|
||||
remediation_tool=None,
|
||||
)
|
||||
|
||||
token = authorization.split(" ", 1)[1].strip()
|
||||
|
||||
# --- agent (Eternitas EPT) --------------------------------------------
|
||||
# An EPT names its passport; the trust API is the authority on whether that
|
||||
# passport may act. We never read a band out of the token itself.
|
||||
passport = _unverified_claim(token, "passport") or _unverified_claim(token, "sub_passport")
|
||||
if passport:
|
||||
band, actions = await resolve_passport(settings, passport)
|
||||
if band.lower() == "untrusted":
|
||||
raise RepairPointer(
|
||||
status_code=403,
|
||||
code="agent_read_only",
|
||||
speak="That helper can look, but it isn't allowed to make changes yet.",
|
||||
machine_cause=f"passport {passport} band=untrusted is read-only",
|
||||
remediation_tool=None,
|
||||
)
|
||||
return Caller(
|
||||
actor_type=ActorType.agent,
|
||||
passport=passport,
|
||||
band=band,
|
||||
allowed_actions=actions,
|
||||
)
|
||||
|
||||
# --- human (account-server RS256) -------------------------------------
|
||||
if settings.is_production and settings.require_verified_jwt:
|
||||
# I-8, applied to ourselves. G3.2's JWKS verifier is not written yet, and
|
||||
# an unverified JWT is an authentication bypass rather than a shortcut.
|
||||
# Refusing is the only honest answer until the verifier exists.
|
||||
raise RepairPointer(
|
||||
status_code=503,
|
||||
code="human_signin_not_ready",
|
||||
speak="Signing in isn't switched on yet. Nothing you have is affected.",
|
||||
machine_cause=(
|
||||
"JWKS verification (G3.2) is not implemented; refusing to accept "
|
||||
"an unverified human token in production"
|
||||
),
|
||||
remediation_tool=None,
|
||||
)
|
||||
|
||||
identity_id = _unverified_claim(token, "windy_identity_id") or _unverified_claim(token, "sub")
|
||||
if not identity_id:
|
||||
raise RepairPointer(
|
||||
status_code=401,
|
||||
code="token_unrecognised",
|
||||
speak="We couldn't read that sign-in. Try signing in again.",
|
||||
machine_cause="token carried neither a passport nor an identity claim",
|
||||
remediation_tool=None,
|
||||
)
|
||||
return Caller(actor_type=ActorType.human, identity_id=identity_id)
|
||||
|
||||
|
||||
def _unverified_claim(token: str, claim: str) -> str | None:
|
||||
"""Read a claim WITHOUT verifying the signature.
|
||||
|
||||
Used only to decide which verifier a token belongs to. Every path that acts
|
||||
on the result re-establishes trust independently: an agent's authority comes
|
||||
from a live Eternitas trust lookup, never from the token's own assertions.
|
||||
|
||||
⚠️ Full RS256/ES256 JWKS verification for the human path lands in G3.2's
|
||||
verifier and MUST be in place before `api.windygit.com` accepts a human
|
||||
token from outside. Until then the human path is reachable only from inside
|
||||
the tunnel, and `settings.require_verified_jwt` refuses it in production.
|
||||
"""
|
||||
import base64
|
||||
import json
|
||||
|
||||
try:
|
||||
payload = token.split(".")[1]
|
||||
payload += "=" * (-len(payload) % 4)
|
||||
return json.loads(base64.urlsafe_b64decode(payload)).get(claim)
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
@@ -50,6 +50,17 @@ class Settings(BaseSettings):
|
||||
# ---- account-server OIDC (human identity) -----------------------------
|
||||
account_server_base_url: str = "https://account.windyword.ai"
|
||||
|
||||
# Internal callers (the Cloud portal calling /internal/*). A first-class
|
||||
# caller class, not a bypass: unset means service calls are REFUSED.
|
||||
service_token: str = ""
|
||||
|
||||
# ⚠️ FAIL-CLOSED GATE. Full RS256/ES256 JWKS verification lands in G3.2.
|
||||
# Until it does, the human token path must not be reachable in production —
|
||||
# accepting an unverified JWT is not a shortcut, it is an authentication
|
||||
# bypass. Agents are unaffected: their authority comes from a live Eternitas
|
||||
# trust lookup, not from anything the token asserts about itself.
|
||||
require_verified_jwt: bool = True
|
||||
|
||||
# ---- storage law (I-3, G4.4) ------------------------------------------
|
||||
# Git object databases MUST live on a POSIX filesystem. A test asserts this
|
||||
# path does not resolve to a network mount.
|
||||
@@ -71,7 +82,12 @@ class Settings(BaseSettings):
|
||||
rate_grants_per_day: int = 100
|
||||
rate_force_pushes_per_day: int = 10
|
||||
|
||||
# ---- mirror health (I-4) ----------------------------------------------
|
||||
# ---- mirror: I-4, never a one-way door --------------------------------
|
||||
github_token: str = ""
|
||||
github_owner: str = "sneakyfree"
|
||||
# Gitea's timer, as a backstop. sync_on_commit is what actually matters:
|
||||
# an hourly window means an hour of work can be the thing you lose.
|
||||
mirror_interval: str = "8h0m0s"
|
||||
mirror_lag_p2_seconds: int = 3600 # env: 60 min -> P2
|
||||
|
||||
# ---- agent grants (G5.3) ----------------------------------------------
|
||||
|
||||
@@ -13,7 +13,7 @@ from contextlib import asynccontextmanager
|
||||
from fastapi import FastAPI
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from api.app.buildinfo import get_build_info
|
||||
from api.app.config import get_settings
|
||||
@@ -24,7 +24,7 @@ from api.app.providers.registry import (
|
||||
GiteaProvider,
|
||||
R2Provider,
|
||||
)
|
||||
from api.app.routes import health
|
||||
from api.app.routes import health, repos
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
@@ -80,6 +80,9 @@ async def lifespan(app: FastAPI):
|
||||
|
||||
app.state.settings = settings
|
||||
app.state.engine = engine
|
||||
app.state.sessionmaker = (
|
||||
async_sessionmaker(engine, expire_on_commit=False) if engine is not None else None
|
||||
)
|
||||
app.state.providers = [
|
||||
DatabaseProvider(engine),
|
||||
GiteaProvider(settings),
|
||||
@@ -112,6 +115,7 @@ app = FastAPI(
|
||||
)
|
||||
|
||||
app.include_router(health.router)
|
||||
app.include_router(repos.router)
|
||||
|
||||
|
||||
@app.exception_handler(RepairPointer)
|
||||
|
||||
521
api/app/routes/repos.py
Normal file
521
api/app/routes/repos.py
Normal file
@@ -0,0 +1,521 @@
|
||||
"""The shelter — repos, grants and version history (strand G5, D-8).
|
||||
|
||||
Windy Cloud today has **no sharing, no permissions and no versioning of any
|
||||
kind**: verified 2026-08-11 against `routes/storage.py` and its models, which
|
||||
contain zero occurrences of share / permission / acl / collaborat / seat /
|
||||
version / snapshot / history / revision. This plane is not a feature bolted onto
|
||||
something that already had one — it fills a hole that has never been filled.
|
||||
|
||||
D-8 also fixes the order: **permissions and history ship before the git
|
||||
protocol.** "I want someone to help me with my website" is a real problem for a
|
||||
real person, and it does not require them to know what a repository is.
|
||||
|
||||
Every string a person sees here obeys the D-9 vocabulary law: *version* and
|
||||
*save point*, never *commit*, and never the countable form of the word "Git" on
|
||||
any surface, ever. See `scripts/vocab_audit.py`, which enforces this.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from api.app.auth import ActorType, Caller, get_caller
|
||||
from api.app.errors import RepairPointer
|
||||
from api.app.models.core import (
|
||||
CreatedVia,
|
||||
GrantRole,
|
||||
Mirror,
|
||||
MirrorState,
|
||||
Repo,
|
||||
RepoGrant,
|
||||
RepoState,
|
||||
RepoType,
|
||||
RepoVersion,
|
||||
Visibility,
|
||||
)
|
||||
from api.app.services.gitea_client import GiteaClient
|
||||
from api.app.services.mirror import MirrorService
|
||||
|
||||
router = APIRouter(prefix="/api/v1/repos", tags=["repos"])
|
||||
|
||||
# Gitea's permission vocabulary, mapped from ours. Ours is the one users see.
|
||||
_ROLE_TO_GITEA = {
|
||||
GrantRole.owner: "admin",
|
||||
GrantRole.maintainer: "admin",
|
||||
GrantRole.writer: "write",
|
||||
GrantRole.reader: "read",
|
||||
}
|
||||
|
||||
_RESERVED_SLUGS = {
|
||||
"api", "admin", "login", "logout", "signup", "settings", "explore",
|
||||
"new", "user", "org", "repo", "assets", "static", "help", "about",
|
||||
}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# schemas
|
||||
# --------------------------------------------------------------------------
|
||||
class CreateRepo(BaseModel):
|
||||
name: str = Field(min_length=1, max_length=100)
|
||||
display_name: str | None = None
|
||||
description: str = ""
|
||||
# I-7 — required, never defaulted at read time, never inferred.
|
||||
repo_type: RepoType
|
||||
visibility: Visibility = Visibility.private
|
||||
|
||||
@field_validator("name")
|
||||
@classmethod
|
||||
def _slug(cls, v: str) -> str:
|
||||
slug = "".join(c if (c.isalnum() or c in "-_") else "-" for c in v.strip().lower())
|
||||
slug = "-".join(filter(None, slug.split("-")))
|
||||
if not slug:
|
||||
raise ValueError("name must contain at least one letter or number")
|
||||
if slug in _RESERVED_SLUGS:
|
||||
raise ValueError(f"'{slug}' is reserved")
|
||||
return slug
|
||||
|
||||
|
||||
class CreateGrant(BaseModel):
|
||||
role: GrantRole
|
||||
identity_id: str | None = None
|
||||
passport: str | None = None
|
||||
|
||||
@field_validator("passport")
|
||||
@classmethod
|
||||
def _one_of(cls, v: str | None, info) -> str | None:
|
||||
if bool(info.data.get("identity_id")) == bool(v):
|
||||
# Mirrors the database CHECK constraint. Both layers, deliberately:
|
||||
# this ecosystem already has an invariant enforced only in
|
||||
# application code across two files, and a double-mint to show for it.
|
||||
raise ValueError("give exactly one of identity_id or passport")
|
||||
return v
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# helpers
|
||||
# --------------------------------------------------------------------------
|
||||
def _sessionmaker(request: Request) -> async_sessionmaker[AsyncSession]:
|
||||
maker = getattr(request.app.state, "sessionmaker", None)
|
||||
if maker is None:
|
||||
raise RepairPointer(
|
||||
status_code=503,
|
||||
code="database_unavailable",
|
||||
speak="We can't reach your projects right now. Nothing has been lost.",
|
||||
machine_cause="no database sessionmaker on app.state",
|
||||
remediation_tool=None,
|
||||
)
|
||||
return maker
|
||||
|
||||
|
||||
def _repo_owner_login(repo: Repo) -> str:
|
||||
"""The owning login for a repo as STORED, not as inferred from the caller.
|
||||
|
||||
Deriving this from the caller works only while the caller is the owner, and
|
||||
silently addresses the wrong namespace the moment a collaborator calls. That
|
||||
class of bug reads as "not found" and is very hard to see.
|
||||
"""
|
||||
if repo.passport:
|
||||
return f"agent-{repo.passport.lower().replace('-', '')}"
|
||||
return f"u-{repo.identity_id[:24]}"
|
||||
|
||||
|
||||
def _owner_login(caller: Caller) -> str:
|
||||
"""One namespace rule for humans and agents alike (I-6)."""
|
||||
if caller.actor_type == ActorType.agent and caller.passport:
|
||||
return f"agent-{caller.passport.lower().replace('-', '')}"
|
||||
return f"u-{(caller.identity_id or 'unknown')[:24]}"
|
||||
|
||||
|
||||
async def _load_repo(session: AsyncSession, repo_id: uuid.UUID, caller: Caller) -> Repo:
|
||||
repo = (await session.execute(select(Repo).where(Repo.id == repo_id))).scalar_one_or_none()
|
||||
if repo is None or repo.state == RepoState.deleted_soft:
|
||||
raise RepairPointer(
|
||||
status_code=404,
|
||||
code="project_not_found",
|
||||
speak="We couldn't find that project.",
|
||||
machine_cause=f"repo {repo_id} not found or soft-deleted",
|
||||
remediation_tool=None,
|
||||
)
|
||||
if not await _may_read(session, repo, caller):
|
||||
# 404, not 403: a stranger should not learn that a private project exists.
|
||||
raise RepairPointer(
|
||||
status_code=404,
|
||||
code="project_not_found",
|
||||
speak="We couldn't find that project.",
|
||||
machine_cause=f"caller {caller.subject} has no grant on repo {repo_id}",
|
||||
remediation_tool=None,
|
||||
)
|
||||
return repo
|
||||
|
||||
|
||||
async def _may_read(session: AsyncSession, repo: Repo, caller: Caller) -> bool:
|
||||
if caller.actor_type == ActorType.system:
|
||||
return True
|
||||
if repo.visibility == Visibility.public:
|
||||
return True
|
||||
if caller.identity_id and repo.identity_id == caller.identity_id:
|
||||
return True
|
||||
if caller.passport and repo.passport == caller.passport:
|
||||
return True
|
||||
return await _active_grant(session, repo, caller) is not None
|
||||
|
||||
|
||||
async def _active_grant(session: AsyncSession, repo: Repo, caller: Caller) -> RepoGrant | None:
|
||||
now = datetime.now(UTC)
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(RepoGrant).where(
|
||||
RepoGrant.repo_id == repo.id, RepoGrant.revoked_at.is_(None)
|
||||
)
|
||||
)
|
||||
).scalars()
|
||||
for g in rows:
|
||||
if g.expires_at is not None and g.expires_at <= now:
|
||||
continue # expired grants are not grants
|
||||
if caller.identity_id and g.grantee_identity_id == caller.identity_id:
|
||||
return g
|
||||
if caller.passport and g.grantee_passport == caller.passport:
|
||||
return g
|
||||
return None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# routes
|
||||
# --------------------------------------------------------------------------
|
||||
@router.post("", status_code=201)
|
||||
async def create_repo(
|
||||
body: CreateRepo,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
settings = request.app.state.settings
|
||||
if body.repo_type.value not in settings.repo_types_enabled:
|
||||
raise RepairPointer(
|
||||
status_code=409,
|
||||
code="repo_type_not_enabled",
|
||||
speak="That kind of project isn't available yet.",
|
||||
machine_cause=(
|
||||
f"repo_type={body.repo_type.value} is not in "
|
||||
f"repo_types_enabled={list(settings.repo_types_enabled)}"
|
||||
),
|
||||
remediation_tool=None,
|
||||
)
|
||||
|
||||
gitea = GiteaClient(settings)
|
||||
owner = _owner_login(caller)
|
||||
await gitea.ensure_user(owner, f"{owner}@windygit.com")
|
||||
created = await gitea.create_repo(
|
||||
owner=owner,
|
||||
name=body.name,
|
||||
description=body.description,
|
||||
private=body.visibility != Visibility.public,
|
||||
default_branch="main",
|
||||
)
|
||||
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = Repo(
|
||||
identity_id=caller.identity_id or f"passport:{caller.passport}",
|
||||
passport=caller.passport,
|
||||
slug=body.name,
|
||||
display_name=body.display_name or body.name,
|
||||
repo_type=body.repo_type,
|
||||
gitea_repo_id=created.get("id"),
|
||||
visibility=body.visibility,
|
||||
default_branch="main",
|
||||
created_via=(
|
||||
CreatedVia.agent if caller.actor_type == ActorType.agent else CreatedVia.portal
|
||||
),
|
||||
)
|
||||
session.add(repo)
|
||||
await session.commit()
|
||||
await session.refresh(repo)
|
||||
|
||||
return {
|
||||
"id": str(repo.id),
|
||||
"name": repo.slug,
|
||||
"repo_type": repo.repo_type.value,
|
||||
"visibility": repo.visibility.value,
|
||||
"clone_url": created.get("clone_url"),
|
||||
"speak": f"'{repo.display_name}' is ready. Everything you save is kept.",
|
||||
"state_proof": {"gitea_repo_id": repo.gitea_repo_id, "owner": owner},
|
||||
"next_actions": ["windy_git.grant_access", "windy_git.list_versions"],
|
||||
}
|
||||
|
||||
|
||||
@router.get("")
|
||||
async def list_repos(request: Request, caller: Annotated[Caller, Depends(get_caller)]) -> dict:
|
||||
async with _sessionmaker(request)() as session:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(Repo).where(Repo.state != RepoState.deleted_soft)
|
||||
)
|
||||
).scalars().all()
|
||||
mine = [r for r in rows if await _may_read(session, r, caller)]
|
||||
return {
|
||||
"repos": [
|
||||
{
|
||||
"id": str(r.id),
|
||||
"name": r.slug,
|
||||
"display_name": r.display_name,
|
||||
"repo_type": r.repo_type.value,
|
||||
"visibility": r.visibility.value,
|
||||
}
|
||||
for r in mine
|
||||
],
|
||||
"count": len(mine),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/{repo_id}/versions")
|
||||
async def list_versions(
|
||||
repo_id: uuid.UUID,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
"""G5.5 — history in words a person recognises.
|
||||
|
||||
Note what is absent from every user-facing string below: 'commit', 'branch',
|
||||
'repository'. A person restoring last Tuesday's work should not have to learn
|
||||
a vocabulary first (I-9, D-9).
|
||||
"""
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
# Derive the namespace from the REPO, never from the caller: the
|
||||
# caller-derived form is right only while the caller is the owner, and
|
||||
# addresses the wrong namespace the moment a collaborator asks. It then
|
||||
# surfaces as "not found", which is about the hardest bug to see.
|
||||
owner, slug, display = _repo_owner_login(repo), repo.slug, repo.display_name
|
||||
|
||||
gitea = GiteaClient(request.app.state.settings)
|
||||
commits = await gitea.list_commits(owner, slug)
|
||||
|
||||
versions = [
|
||||
{
|
||||
"version": len(commits) - i,
|
||||
"id": c.get("sha"),
|
||||
"saved_at": (c.get("commit") or {}).get("author", {}).get("date"),
|
||||
"note": ((c.get("commit") or {}).get("message") or "").strip().split("\n")[0],
|
||||
"saved_by": (c.get("commit") or {}).get("author", {}).get("name"),
|
||||
}
|
||||
for i, c in enumerate(commits)
|
||||
]
|
||||
return {
|
||||
"versions": versions,
|
||||
"count": len(versions),
|
||||
# "There are 1 saved versions" is the kind of sloppiness the vocabulary
|
||||
# law exists to catch. Copy is design material, not decoration (I-9).
|
||||
"speak": (
|
||||
f"'{display}' has 1 saved version. You can go back to it."
|
||||
if len(versions) == 1
|
||||
else f"'{display}' has {len(versions)} saved versions. "
|
||||
"You can go back to any of them."
|
||||
if versions
|
||||
else f"'{display}' is empty so far."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/{repo_id}/grants", status_code=201)
|
||||
async def create_grant(
|
||||
repo_id: uuid.UUID,
|
||||
body: CreateGrant,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
"""G5.3 — the thing Windy Cloud cannot do at all today.
|
||||
|
||||
A grant may name a human OR an agent passport, and agent grants expire by
|
||||
default (env: 90 days). A permanent agent credential is a standing liability
|
||||
nobody consciously chose.
|
||||
"""
|
||||
settings = request.app.state.settings
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
is_owner = (caller.identity_id and repo.identity_id == caller.identity_id) or (
|
||||
caller.passport and repo.passport == caller.passport
|
||||
)
|
||||
if not is_owner and caller.actor_type != ActorType.system:
|
||||
raise RepairPointer(
|
||||
status_code=403,
|
||||
code="not_your_project",
|
||||
speak="Only the owner can share this project.",
|
||||
machine_cause=f"{caller.subject} is not the owner of {repo_id}",
|
||||
remediation_tool=None,
|
||||
)
|
||||
|
||||
expires = (
|
||||
datetime.now(UTC) + timedelta(days=settings.agent_grant_default_days)
|
||||
if body.passport
|
||||
else None
|
||||
)
|
||||
grant = RepoGrant(
|
||||
repo_id=repo.id,
|
||||
grantee_identity_id=body.identity_id,
|
||||
grantee_passport=body.passport,
|
||||
role=body.role,
|
||||
granted_by=caller.subject,
|
||||
expires_at=expires,
|
||||
)
|
||||
session.add(grant)
|
||||
await session.commit()
|
||||
await session.refresh(grant)
|
||||
grant_id, role = grant.id, grant.role
|
||||
|
||||
who = body.identity_id or body.passport
|
||||
return {
|
||||
"id": str(grant_id),
|
||||
"role": role.value,
|
||||
"grantee": who,
|
||||
"expires_at": expires.isoformat() if expires else None,
|
||||
"speak": (
|
||||
f"They can now help with '{repo.display_name}'."
|
||||
+ (" Access ends automatically in 90 days." if expires else "")
|
||||
),
|
||||
"next_actions": ["windy_git.list_grants", "windy_git.revoke_access"],
|
||||
}
|
||||
|
||||
|
||||
@router.get("/{repo_id}/grants")
|
||||
async def list_grants(
|
||||
repo_id: uuid.UUID,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
now = datetime.now(UTC)
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
rows = (
|
||||
await session.execute(select(RepoGrant).where(RepoGrant.repo_id == repo.id))
|
||||
).scalars().all()
|
||||
return {
|
||||
"grants": [
|
||||
{
|
||||
"id": str(g.id),
|
||||
"grantee": g.grantee_identity_id or g.grantee_passport,
|
||||
"kind": "person" if g.grantee_identity_id else "helper",
|
||||
"role": g.role.value,
|
||||
"expires_at": g.expires_at.isoformat() if g.expires_at else None,
|
||||
"active": g.revoked_at is None
|
||||
and (g.expires_at is None or g.expires_at > now),
|
||||
}
|
||||
for g in rows
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/{repo_id}/grants/{grant_id}")
|
||||
async def revoke_grant(
|
||||
repo_id: uuid.UUID,
|
||||
grant_id: uuid.UUID,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
grant = (
|
||||
await session.execute(select(RepoGrant).where(RepoGrant.id == grant_id))
|
||||
).scalar_one_or_none()
|
||||
if grant is None or grant.repo_id != repo.id:
|
||||
raise RepairPointer(
|
||||
status_code=404,
|
||||
code="grant_not_found",
|
||||
speak="We couldn't find that access to remove.",
|
||||
machine_cause=f"grant {grant_id} not on repo {repo_id}",
|
||||
remediation_tool=None,
|
||||
)
|
||||
grant.revoked_at = datetime.now(UTC)
|
||||
await session.commit()
|
||||
return {"revoked": True, "speak": "That access has been removed."}
|
||||
|
||||
|
||||
@router.get("/{repo_id}")
|
||||
async def get_repo(
|
||||
repo_id: uuid.UUID,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
versions = (
|
||||
await session.execute(select(RepoVersion).where(RepoVersion.repo_id == repo.id))
|
||||
).scalars().all()
|
||||
return {
|
||||
"id": str(repo.id),
|
||||
"name": repo.slug,
|
||||
"display_name": repo.display_name,
|
||||
"repo_type": repo.repo_type.value,
|
||||
"visibility": repo.visibility.value,
|
||||
"cloud_folder_ref": repo.cloud_folder_ref,
|
||||
"recorded_versions": len(versions),
|
||||
"state": repo.state.value,
|
||||
}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# I-4 / G11 — the off-site copy
|
||||
# --------------------------------------------------------------------------
|
||||
@router.post("/{repo_id}/mirror", status_code=201)
|
||||
async def enable_mirror(
|
||||
repo_id: uuid.UUID,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
"""Turn on the continuous off-site copy.
|
||||
|
||||
Deliberately idempotent and deliberately loud on failure: a mirror that
|
||||
quietly stopped working is worse than no mirror, because it is a backup you
|
||||
believe in.
|
||||
"""
|
||||
settings = request.app.state.settings
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
owner = _repo_owner_login(repo)
|
||||
display, slug = repo.display_name, repo.slug
|
||||
private = repo.visibility != Visibility.public
|
||||
|
||||
mirror = MirrorService(settings)
|
||||
remote = await mirror.ensure_github_repo(slug, display, private)
|
||||
await mirror.attach_push_mirror(owner, slug, remote)
|
||||
|
||||
async with _sessionmaker(request)() as session:
|
||||
session.add(
|
||||
Mirror(repo_id=repo_id, remote_url=remote, direction="push", state=MirrorState.healthy)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
return {
|
||||
"remote": remote,
|
||||
"sync_on_commit": True,
|
||||
"speak": "A second copy of this project is now kept somewhere else, automatically.",
|
||||
"state_proof": {"remote": remote},
|
||||
"next_actions": ["windy_git.mirror_status"],
|
||||
}
|
||||
|
||||
|
||||
@router.get("/{repo_id}/mirror")
|
||||
async def mirror_status(
|
||||
repo_id: uuid.UUID,
|
||||
request: Request,
|
||||
caller: Annotated[Caller, Depends(get_caller)],
|
||||
) -> dict:
|
||||
async with _sessionmaker(request)() as session:
|
||||
repo = await _load_repo(session, repo_id, caller)
|
||||
owner, slug = _repo_owner_login(repo), repo.slug
|
||||
|
||||
status = await MirrorService(request.app.state.settings).status(owner, slug)
|
||||
speak = {
|
||||
"healthy": "A second copy of this project is up to date.",
|
||||
"degraded": "The second copy is behind. Your work here is safe.",
|
||||
"absent": "There is no second copy of this project yet.",
|
||||
"pending": "The second copy is set up and hasn't run yet.",
|
||||
"unconfigured": "Off-site copies aren't switched on yet.",
|
||||
"unknown": "We can't tell how the second copy is doing right now.",
|
||||
}[status["state"]]
|
||||
return {**status, "speak": speak}
|
||||
175
api/app/services/gitea_client.py
Normal file
175
api/app/services/gitea_client.py
Normal file
@@ -0,0 +1,175 @@
|
||||
"""The Gitea membrane (I-1 / D-2).
|
||||
|
||||
Everything this cell needs from Gitea goes through this file and through Gitea's
|
||||
REST API. Nothing else in the codebase imports Gitea concepts, and nothing
|
||||
anywhere writes Gitea's database directly — it has its own role and its own
|
||||
database precisely so that boundary is a permission rather than a promise.
|
||||
|
||||
Keeping the dependency here is what makes D-2 affordable: Gitea ships every two
|
||||
or three months including security fixes, and a diverged fork becomes the whole
|
||||
job within a year for a small team. One file is a seam. A merged source tree is
|
||||
a marriage.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from api.app.config import Settings
|
||||
from api.app.errors import RepairPointer, provider_unconfigured
|
||||
|
||||
_TIMEOUT = httpx.Timeout(20.0, connect=5.0)
|
||||
|
||||
|
||||
class GiteaClient:
|
||||
def __init__(self, settings: Settings) -> None:
|
||||
self._s = settings
|
||||
|
||||
def _require(self) -> None:
|
||||
if not self._s.gitea_configured:
|
||||
raise provider_unconfigured("gitea", "GITEA_ADMIN_TOKEN")
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
return {
|
||||
"Authorization": f"token {self._s.gitea_admin_token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
async def _request(self, method: str, path: str, **kw: Any) -> httpx.Response:
|
||||
self._require()
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
return await client.request(
|
||||
method,
|
||||
f"{self._s.gitea_base_url}/api/v1{path}",
|
||||
headers=self._headers(),
|
||||
**kw,
|
||||
)
|
||||
|
||||
# ---- users ------------------------------------------------------------
|
||||
async def ensure_user(self, username: str, email: str) -> dict:
|
||||
"""Idempotent. Gitea is the component; our `repos` table is the truth."""
|
||||
r = await self._request("GET", f"/users/{username}")
|
||||
if r.status_code == 200:
|
||||
return r.json()
|
||||
# Gitea requires a password field on admin user-create and rejects null
|
||||
# with a bare 400. Nobody ever uses this one: humans arrive through OIDC
|
||||
# and agents through scoped passport-bound tokens, and local password
|
||||
# sign-in is disabled server-wide. So we generate a credential that is
|
||||
# never stored, never returned and never recoverable — an unusable
|
||||
# password is safer than a blank one or a shared default.
|
||||
r = await self._request(
|
||||
"POST",
|
||||
"/admin/users",
|
||||
json={
|
||||
"username": username,
|
||||
"email": email,
|
||||
"password": secrets.token_urlsafe(48),
|
||||
"must_change_password": False,
|
||||
},
|
||||
)
|
||||
if r.status_code not in (200, 201):
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="gitea_user_create_failed",
|
||||
speak="We couldn't finish setting up that account. Nothing was lost.",
|
||||
machine_cause=f"POST /admin/users -> {r.status_code}: {r.text[:200]}",
|
||||
remediation_tool="windy_git.repair.retry_user_create",
|
||||
)
|
||||
return r.json()
|
||||
|
||||
# ---- repos ------------------------------------------------------------
|
||||
async def create_repo(
|
||||
self, owner: str, name: str, description: str, private: bool, default_branch: str
|
||||
) -> dict:
|
||||
r = await self._request(
|
||||
"POST",
|
||||
f"/admin/users/{owner}/repos",
|
||||
json={
|
||||
"name": name,
|
||||
"description": description,
|
||||
"private": private,
|
||||
"auto_init": True,
|
||||
"default_branch": default_branch,
|
||||
# G4.5 — the LFS threshold is load-bearing, not tidiness. A large
|
||||
# blob inside a git pack cannot be resumed or offloaded, and a
|
||||
# plain push of one dies at Cloudflare's ~100s ceiling (G4A.5).
|
||||
"gitignores": "",
|
||||
},
|
||||
)
|
||||
if r.status_code not in (200, 201):
|
||||
raise RepairPointer(
|
||||
status_code=502 if r.status_code >= 500 else 409,
|
||||
code="repo_create_failed",
|
||||
speak="We couldn't create that project. Try a different name.",
|
||||
machine_cause=f"POST /admin/users/{owner}/repos -> {r.status_code}: {r.text[:200]}",
|
||||
remediation_tool=None,
|
||||
)
|
||||
return r.json()
|
||||
|
||||
async def delete_repo(self, owner: str, name: str) -> None:
|
||||
r = await self._request("DELETE", f"/repos/{owner}/{name}")
|
||||
if r.status_code not in (204, 404):
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="repo_delete_failed",
|
||||
speak="We couldn't remove that project. It is still there and still yours.",
|
||||
machine_cause=f"DELETE /repos/{owner}/{name} -> {r.status_code}",
|
||||
remediation_tool="windy_git.repair.retry_delete",
|
||||
)
|
||||
|
||||
async def get_repo(self, owner: str, name: str) -> dict | None:
|
||||
r = await self._request("GET", f"/repos/{owner}/{name}")
|
||||
return r.json() if r.status_code == 200 else None
|
||||
|
||||
# ---- history (G5.5 / G5.6) -------------------------------------------
|
||||
async def list_commits(self, owner: str, name: str, limit: int = 50) -> list[dict]:
|
||||
r = await self._request(
|
||||
"GET", f"/repos/{owner}/{name}/commits", params={"limit": limit}
|
||||
)
|
||||
if r.status_code == 409:
|
||||
return [] # empty repo — a real state, not an error
|
||||
if r.status_code != 200:
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="history_unavailable",
|
||||
speak="We couldn't load the history for that project just now.",
|
||||
machine_cause=f"GET commits -> {r.status_code}",
|
||||
remediation_tool="windy_git.repair.rebuild_index",
|
||||
)
|
||||
return r.json()
|
||||
|
||||
# ---- collaborators (the shelter's enforcement half, G5.3) ------------
|
||||
async def put_collaborator(self, owner: str, name: str, user: str, permission: str) -> None:
|
||||
r = await self._request(
|
||||
"PUT",
|
||||
f"/repos/{owner}/{name}/collaborators/{user}",
|
||||
json={"permission": permission},
|
||||
)
|
||||
if r.status_code not in (204, 201, 200):
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="grant_apply_failed",
|
||||
speak="We couldn't share that project yet. Nobody was given access.",
|
||||
machine_cause=f"PUT collaborator -> {r.status_code}: {r.text[:200]}",
|
||||
remediation_tool="windy_git.repair.resync_grants",
|
||||
)
|
||||
|
||||
async def delete_collaborator(self, owner: str, name: str, user: str) -> None:
|
||||
r = await self._request("DELETE", f"/repos/{owner}/{name}/collaborators/{user}")
|
||||
if r.status_code not in (204, 404):
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="grant_revoke_failed",
|
||||
# The honest failure: we could not take access away. That is the
|
||||
# scarier direction, so it is stated plainly rather than softened.
|
||||
speak="We could not remove that person's access. Please try again.",
|
||||
machine_cause=f"DELETE collaborator -> {r.status_code}",
|
||||
remediation_tool="windy_git.repair.resync_grants",
|
||||
)
|
||||
|
||||
async def version(self) -> str:
|
||||
r = await self._request("GET", "/version")
|
||||
return r.json().get("version", "unknown")
|
||||
179
api/app/services/mirror.py
Normal file
179
api/app/services/mirror.py
Normal file
@@ -0,0 +1,179 @@
|
||||
"""I-4 — never a one-way door (strand G11).
|
||||
|
||||
Every repo push-mirrors to GitHub, continuously, from the first save. This is
|
||||
not a nicety and it is not a migration step: it is the thing that makes moving
|
||||
off GitHub a reversible decision rather than a bet.
|
||||
|
||||
Today GitHub's durability is free to Grant. The moment repos live only here,
|
||||
backups, restore rehearsal and a second copy stop being someone else's job — and
|
||||
the August audits found **no rehearsed restore anywhere in the ecosystem**, for
|
||||
anything. A continuous mirror buys back that safety for zero dollars.
|
||||
|
||||
Mirror health is a monitored, alerting signal. A mirror nobody checks is a
|
||||
belief, not a backup — and this ecosystem has already learned that lesson the
|
||||
expensive way with a fleet canary that sat dead for 37 days while everything
|
||||
downstream assumed it was fine.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import httpx
|
||||
|
||||
from api.app.config import Settings
|
||||
from api.app.errors import RepairPointer
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_TIMEOUT = httpx.Timeout(30.0, connect=10.0)
|
||||
|
||||
|
||||
class MirrorService:
|
||||
"""Creates the GitHub counterpart and asks Gitea to keep it in step.
|
||||
|
||||
Gitea owns the actual replication (`push_mirrors`), because a hand-rolled
|
||||
mirror loop is a background job that fails silently — which is precisely how
|
||||
the registry's integrity refresh went its entire life calling a 404 and
|
||||
incrementing a counter instead of raising.
|
||||
"""
|
||||
|
||||
def __init__(self, settings: Settings) -> None:
|
||||
self._s = settings
|
||||
|
||||
@property
|
||||
def configured(self) -> bool:
|
||||
return bool(self._s.github_token and self._s.github_owner)
|
||||
|
||||
def _require(self) -> None:
|
||||
if not self.configured:
|
||||
raise RepairPointer(
|
||||
status_code=503,
|
||||
code="mirror_unconfigured",
|
||||
speak="The off-site copy isn't switched on yet.",
|
||||
machine_cause="GITHUB_TOKEN or GITHUB_OWNER is unset; refusing to claim a mirror",
|
||||
remediation_tool=None,
|
||||
)
|
||||
|
||||
async def ensure_github_repo(self, name: str, description: str, private: bool) -> str:
|
||||
"""Idempotent. Returns the clone URL of the off-site copy."""
|
||||
self._require()
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self._s.github_token}",
|
||||
"Accept": "application/vnd.github+json",
|
||||
}
|
||||
owner = self._s.github_owner
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
existing = await client.get(
|
||||
f"https://api.github.com/repos/{owner}/{name}", headers=headers
|
||||
)
|
||||
if existing.status_code == 200:
|
||||
return existing.json()["clone_url"]
|
||||
|
||||
created = await client.post(
|
||||
"https://api.github.com/user/repos",
|
||||
headers=headers,
|
||||
json={
|
||||
"name": name,
|
||||
"description": f"{description} (Windy Git mirror)".strip(),
|
||||
"private": private,
|
||||
"auto_init": False,
|
||||
},
|
||||
)
|
||||
if created.status_code not in (200, 201):
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="mirror_target_failed",
|
||||
speak="We couldn't set up the off-site copy. Your work here is safe.",
|
||||
machine_cause=f"POST /user/repos -> {created.status_code}: {created.text[:200]}",
|
||||
remediation_tool="windy_git.repair.resync_mirror",
|
||||
)
|
||||
return created.json()["clone_url"]
|
||||
|
||||
async def attach_push_mirror(self, owner: str, repo: str, remote_url: str) -> None:
|
||||
"""Ask Gitea to keep the off-site copy in step on every save."""
|
||||
self._require()
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
r = await client.post(
|
||||
f"{self._s.gitea_base_url}/api/v1/repos/{owner}/{repo}/push_mirrors",
|
||||
headers={
|
||||
"Authorization": f"token {self._s.gitea_admin_token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json={
|
||||
"remote_address": remote_url,
|
||||
"remote_username": self._s.github_owner,
|
||||
"remote_password": self._s.github_token,
|
||||
"interval": self._s.mirror_interval,
|
||||
# The important one: mirror on every save, not just on a timer.
|
||||
# An hourly timer means an hour of work can be the thing you
|
||||
# lose, and the window is invisible until it costs you.
|
||||
"sync_on_commit": True,
|
||||
},
|
||||
)
|
||||
if r.status_code not in (200, 201):
|
||||
raise RepairPointer(
|
||||
status_code=502,
|
||||
code="mirror_attach_failed",
|
||||
speak="We couldn't keep the off-site copy in step. Your work here is safe.",
|
||||
machine_cause=f"POST push_mirrors -> {r.status_code}: {r.text[:200]}",
|
||||
remediation_tool="windy_git.repair.resync_mirror",
|
||||
)
|
||||
|
||||
async def status(self, owner: str, repo: str) -> dict:
|
||||
"""Report what is TRUE, including 'we do not know'.
|
||||
|
||||
`me-fleet.ts:22-25` in a sibling service refuses to say "online" when it
|
||||
only knows "registered". Same posture here: an unconfigured mirror is
|
||||
reported as unknown, never as healthy.
|
||||
"""
|
||||
if not self.configured:
|
||||
return {"state": "unconfigured", "lag_seconds": None, "last_success_at": None}
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
r = await client.get(
|
||||
f"{self._s.gitea_base_url}/api/v1/repos/{owner}/{repo}/push_mirrors",
|
||||
headers={"Authorization": f"token {self._s.gitea_admin_token}"},
|
||||
)
|
||||
if r.status_code != 200:
|
||||
return {"state": "unknown", "detail": f"gitea -> {r.status_code}"}
|
||||
|
||||
mirrors = r.json()
|
||||
if not mirrors:
|
||||
return {"state": "absent", "lag_seconds": None, "last_success_at": None}
|
||||
|
||||
m = mirrors[0]
|
||||
last = m.get("last_update") or m.get("last_updated")
|
||||
lag = None
|
||||
if last:
|
||||
try:
|
||||
lag = int(
|
||||
(datetime.now(UTC) - datetime.fromisoformat(last.replace("Z", "+00:00")))
|
||||
.total_seconds()
|
||||
)
|
||||
except ValueError:
|
||||
lag = None
|
||||
|
||||
# A mirror that has NEVER run is not the same thing as one that is
|
||||
# behind, and collapsing the two is how a backup that was never made
|
||||
# gets read as a backup that is merely stale. Gitea reports the epoch
|
||||
# for "not yet", which arithmetic turns into a 56-year lag and a
|
||||
# confident "degraded".
|
||||
never_synced = not last or last.startswith("1970-01-01")
|
||||
|
||||
# I-4: lag over the threshold is a P2, not a shrug.
|
||||
if never_synced:
|
||||
state = "pending"
|
||||
lag = None
|
||||
elif lag is None:
|
||||
state = "unknown"
|
||||
elif lag > self._s.mirror_lag_p2_seconds:
|
||||
state = "degraded"
|
||||
else:
|
||||
state = "healthy"
|
||||
return {
|
||||
"state": state,
|
||||
"lag_seconds": lag,
|
||||
"last_success_at": None if never_synced else last,
|
||||
"remote": m.get("remote_address"),
|
||||
}
|
||||
@@ -234,3 +234,217 @@ def test_g05_no_secret_literals_committed():
|
||||
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||
for pattern in patterns:
|
||||
assert not pattern.search(text), f"credential literal in {path}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# G2.1 / G2.7 — the Gitea version is PINNED, and drift is a failure
|
||||
# --------------------------------------------------------------------------
|
||||
def test_g21_gitea_version_is_pinned_not_latest():
|
||||
compose = (ROOT / "docker-compose.yml").read_text()
|
||||
m = re.search(r"image:\s*\S*gitea/gitea:(\S+)", compose)
|
||||
assert m, "no gitea image pin found"
|
||||
assert m.group(1) != "latest", "G2.1: pin an exact Gitea version, never `latest`"
|
||||
assert re.match(r"^\d+\.\d+\.\d+$", m.group(1)), f"not an exact version: {m.group(1)}"
|
||||
|
||||
|
||||
def test_g24_gitea_license_travels_with_us():
|
||||
"""MIT's one obligation. Cheap to honour, embarrassing to miss."""
|
||||
assert (ROOT / "LICENSES" / "gitea-MIT.txt").exists()
|
||||
assert "MIT" in (ROOT / "LICENSES" / "gitea-MIT.txt").read_text()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# G4.3 — the two Gitea storage traps that cost a crash loop each
|
||||
# --------------------------------------------------------------------------
|
||||
def test_g43_no_lfs_storage_type_override():
|
||||
"""Naming a storage type inside [lfs] creates a SEPARATE storage section
|
||||
that does not inherit endpoint or credentials from [storage], and Gitea
|
||||
crash-loops with an error that names the symptom and not the cause."""
|
||||
# Check real settings only — the compose file deliberately NAMES this key in
|
||||
# a warning comment so the next person does not re-add it.
|
||||
active = [
|
||||
ln for ln in (ROOT / "docker-compose.yml").read_text().splitlines()
|
||||
if ln.strip() and not ln.strip().startswith("#")
|
||||
]
|
||||
assert not any("GITEA__lfs__STORAGE_TYPE" in ln for ln in active)
|
||||
|
||||
|
||||
def test_g43_lfs_server_is_actually_enabled():
|
||||
"""Setting the storage backend does NOT turn LFS on. Without this the batch
|
||||
endpoint 404s and the client says 'Repository or object not found', which
|
||||
reads like a permissions problem and is not one."""
|
||||
active = [
|
||||
ln for ln in (ROOT / "docker-compose.yml").read_text().splitlines()
|
||||
if ln.strip() and not ln.strip().startswith("#")
|
||||
]
|
||||
assert any("GITEA__server__LFS_START_SERVER" in ln for ln in active)
|
||||
|
||||
|
||||
def test_g43_r2_checksum_trap_is_pinned():
|
||||
"""R2 rejects the checksum algorithm S3 clients send by default."""
|
||||
compose = (ROOT / "docker-compose.yml").read_text()
|
||||
assert "MINIO_CHECKSUM_ALGORITHM" in compose
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# G3.6 — the trust status-code law. This is the one with a live sibling bypass.
|
||||
# --------------------------------------------------------------------------
|
||||
def test_g36_trust_client_never_soft_allows():
|
||||
"""A sibling maps HTTP 400 and 429 to 'unreachable' and then soft-ALLOWS.
|
||||
That is inducible: an attacker who wants the check skipped only has to make
|
||||
the check rate-limit itself at 100/min/IP."""
|
||||
src = (ROOT / "api" / "app" / "auth.py").read_text()
|
||||
assert "raise passport_unresolvable" in src
|
||||
# There must be no return path out of resolve_passport other than a verified
|
||||
# 200 or a raise.
|
||||
body = src[src.index("async def resolve_passport") : src.index("async def get_caller")]
|
||||
returns = [ln for ln in body.splitlines() if ln.strip().startswith("return")]
|
||||
assert len(returns) == 1, f"resolve_passport has {len(returns)} return paths; expected exactly 1"
|
||||
|
||||
|
||||
def test_g36_unverified_human_jwt_is_refused_in_production():
|
||||
"""I-8 applied to ourselves: an unverified JWT is an authentication bypass,
|
||||
not a shortcut. Until G3.2's JWKS verifier exists, production refuses."""
|
||||
from api.app.config import Settings
|
||||
|
||||
assert Settings().require_verified_jwt is True
|
||||
src = (ROOT / "api" / "app" / "auth.py").read_text()
|
||||
assert "human_signin_not_ready" in src
|
||||
|
||||
|
||||
def test_no_auth_bypass_env_var_anywhere():
|
||||
"""The desktop control server is the best Principle-#5 artifact in the
|
||||
ecosystem partly because it has NO bypass env var. Copied on purpose."""
|
||||
src = (ROOT / "api" / "app" / "auth.py").read_text()
|
||||
for banned in ("SKIP_AUTH", "DISABLE_AUTH", "ALLOW_INSECURE", "AUTH_BYPASS", "DEV_MODE"):
|
||||
assert banned not in src
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# G5.3 — the shelter's grant model
|
||||
# --------------------------------------------------------------------------
|
||||
def test_g53_grant_requires_exactly_one_grantee_in_the_database():
|
||||
"""Enforced by a CHECK constraint, not by application code. This ecosystem
|
||||
already has a core invariant enforced only in app code across two files."""
|
||||
src = (ROOT / "alembic" / "versions" / "001_genesis.py").read_text()
|
||||
assert "ck_grant_exactly_one_grantee" in src
|
||||
assert "(grantee_identity_id IS NULL) <> (grantee_passport IS NULL)" in src
|
||||
|
||||
|
||||
def test_g53_agent_grants_expire_by_default():
|
||||
from api.app.config import Settings
|
||||
|
||||
assert Settings().agent_grant_default_days == 90
|
||||
|
||||
|
||||
def test_g55_shelter_strings_avoid_developer_vocabulary():
|
||||
"""D-9/I-9: a person restoring last Tuesday's work should not have to learn
|
||||
a vocabulary first. Check the strings users actually see."""
|
||||
import re as _re
|
||||
|
||||
src = (ROOT / "api" / "app" / "routes" / "repos.py").read_text()
|
||||
speaks = _re.findall(r'"speak":\s*\(?\s*\n?\s*f?"([^"]+)"', src)
|
||||
assert speaks, "no speak strings found to audit"
|
||||
for s in speaks:
|
||||
low = s.lower()
|
||||
for jargon in ("commit", "repository", "branch", "sha", "push"):
|
||||
assert jargon not in low, f"developer vocabulary in a user string: {s!r}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# I-4 — never a one-way door
|
||||
# --------------------------------------------------------------------------
|
||||
def test_i04_mirror_syncs_on_every_save_not_just_a_timer():
|
||||
"""An hourly window means an hour of work can be the thing you lose, and the
|
||||
window is invisible until it costs you."""
|
||||
src = (ROOT / "api" / "app" / "services" / "mirror.py").read_text()
|
||||
assert '"sync_on_commit": True' in src
|
||||
|
||||
|
||||
def test_i04_unconfigured_mirror_is_never_reported_healthy():
|
||||
"""A mirror nobody checks is a belief, not a backup. An unconfigured one
|
||||
reports 'unconfigured' — never 'healthy'."""
|
||||
src = (ROOT / "api" / "app" / "services" / "mirror.py").read_text()
|
||||
assert '"state": "unconfigured"' in src
|
||||
assert "if not self.configured:" in src
|
||||
|
||||
|
||||
def test_i04_mirror_lag_threshold_is_set():
|
||||
from api.app.config import Settings
|
||||
|
||||
assert Settings().mirror_lag_p2_seconds == 3600
|
||||
|
||||
|
||||
def test_owner_namespace_is_derived_from_the_repo_not_the_caller():
|
||||
"""Deriving the namespace from the caller is right only while the caller is
|
||||
the owner, and addresses the wrong namespace the moment a collaborator asks
|
||||
— surfacing as 'not found', which is the hardest kind of bug to see."""
|
||||
src = (ROOT / "api" / "app" / "routes" / "repos.py").read_text()
|
||||
body = src[src.index("async def list_versions") : src.index("async def create_grant")]
|
||||
assert "_repo_owner_login(repo)" in body
|
||||
assert "_owner_login(caller)" not in body
|
||||
|
||||
|
||||
def test_i04_never_synced_is_not_reported_as_merely_behind():
|
||||
"""Collapsing 'never ran' into 'behind' is how a backup that was never made
|
||||
gets read as a backup that is merely stale. Gitea reports the epoch for
|
||||
'not yet', which arithmetic turns into a 56-year lag and a confident
|
||||
'degraded'."""
|
||||
src = (ROOT / "api" / "app" / "services" / "mirror.py").read_text()
|
||||
assert "never_synced" in src
|
||||
assert '"pending"' in src
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# G7 / I-5 — CI never shares a kernel with identity
|
||||
# --------------------------------------------------------------------------
|
||||
def test_i05_runner_never_mounts_the_host_docker_socket():
|
||||
"""The tempting move — and what every published act_runner example does —
|
||||
is to mount /var/run/docker.sock. That hands every workflow, including a
|
||||
transitive dependency's postinstall script, the ability to start a
|
||||
privileged container mounting / — i.e. root on the host."""
|
||||
compose = (ROOT / "deploy" / "runner" / "docker-compose.yml").read_text()
|
||||
active = [ln for ln in compose.splitlines() if ln.strip() and not ln.strip().startswith("#")]
|
||||
for ln in active:
|
||||
assert "/var/run/docker.sock" not in ln, "I-5: never mount the host docker socket"
|
||||
|
||||
|
||||
def test_i05_jobs_cannot_bind_mount_from_the_daemon_host():
|
||||
cfg = (ROOT / "deploy" / "runner" / "config.yaml").read_text()
|
||||
assert "valid_volumes: []" in cfg
|
||||
assert 'docker_host: "-"' in cfg
|
||||
|
||||
|
||||
def test_i05_runner_is_a_separate_compose_project_from_the_forge():
|
||||
"""Runners restart, crash, get starved and get killed. None of that should
|
||||
ever touch the thing serving repositories."""
|
||||
runner = (ROOT / "deploy" / "runner" / "docker-compose.yml").read_text()
|
||||
forge = (ROOT / "docker-compose.yml").read_text()
|
||||
assert "name: windy-git-runner" in runner
|
||||
assert "name: windy-git" in forge
|
||||
|
||||
|
||||
def test_g15_runner_is_cpu_and_memory_bounded():
|
||||
"""Veron 1 is Grant's workstation, not a dedicated build box."""
|
||||
compose = (ROOT / "deploy" / "runner" / "docker-compose.yml").read_text()
|
||||
assert "cpus:" in compose
|
||||
assert "mem_limit:" in compose
|
||||
|
||||
|
||||
def test_g75_workflows_use_a_label_this_runner_actually_provides():
|
||||
"""A workflow naming a label nobody provides queues forever and presents as
|
||||
a hung CI system rather than a typo."""
|
||||
cfg = (ROOT / "deploy" / "runner" / "config.yaml").read_text()
|
||||
provided = {
|
||||
ln.split(":")[0].strip().strip('"- ')
|
||||
for ln in cfg.splitlines()
|
||||
if "docker://" in ln
|
||||
}
|
||||
assert provided, "runner declares no labels"
|
||||
for wf in ROOT.rglob(".gitea/workflows/*.y*ml"):
|
||||
for ln in wf.read_text().splitlines():
|
||||
# Skip comments — a doc line explaining runs-on is not a runs-on.
|
||||
if ln.strip().startswith("#") or "runs-on:" not in ln:
|
||||
continue
|
||||
label = ln.split("runs-on:")[1].strip()
|
||||
assert label in provided, f"{wf.name}: '{label}' is not a provided label"
|
||||
|
||||
38
deploy/runner/config.yaml
Normal file
38
deploy/runner/config.yaml
Normal file
@@ -0,0 +1,38 @@
|
||||
# act_runner configuration (G7.1).
|
||||
#
|
||||
# Labels are EXPLICIT and PINNED. `ubuntu-latest` is banned (G7.5): all four
|
||||
# windy-registry workflows use it and every single run fails, because a
|
||||
# self-hosted runner has no such label unless you invent one. A workflow that
|
||||
# names a label nobody provides queues forever and looks like a hung CI system
|
||||
# rather than a typo.
|
||||
|
||||
log:
|
||||
level: info
|
||||
|
||||
runner:
|
||||
file: /data/.runner
|
||||
capacity: 4 # concurrent jobs; Veron has 24 cores, dind is capped at 12
|
||||
timeout: 30m
|
||||
shutdown_timeout: 3m
|
||||
insecure: false
|
||||
fetch_timeout: 5s
|
||||
fetch_interval: 2s
|
||||
labels:
|
||||
- "veron-1:docker://catthehacker/ubuntu:act-22.04"
|
||||
- "linux-x64:docker://catthehacker/ubuntu:act-22.04"
|
||||
|
||||
cache:
|
||||
enabled: true
|
||||
dir: /data/cache
|
||||
|
||||
container:
|
||||
# Job containers join the dind daemon's own bridge. NOT the forge network:
|
||||
# untrusted code must never be able to reach the forge's Postgres or its
|
||||
# environment (I-5).
|
||||
network: bridge
|
||||
privileged: false
|
||||
options:
|
||||
workdir_parent: /workspace
|
||||
valid_volumes: [] # a job cannot bind-mount anything from the daemon host
|
||||
docker_host: "-" # do NOT expose the runner's own docker socket to jobs
|
||||
force_pull: false
|
||||
81
deploy/runner/docker-compose.yml
Normal file
81
deploy/runner/docker-compose.yml
Normal file
@@ -0,0 +1,81 @@
|
||||
# CI runners (strand G7) — a SEPARATE compose project from the forge.
|
||||
#
|
||||
# Separate on purpose: runners restart, crash, get starved and get killed. None
|
||||
# of that should ever touch the thing serving repositories. This is the cell
|
||||
# doctrine applied one level down.
|
||||
#
|
||||
# ── I-5, and why there is a dind sidecar ───────────────────────────────────
|
||||
#
|
||||
# "CI never shares a kernel with identity. Runners execute untrusted code and
|
||||
# are isolated by machine boundary, not container boundary. No runner may hold
|
||||
# a credential scoped beyond its own job."
|
||||
#
|
||||
# act_runner needs a Docker daemon to start job containers. The tempting move is
|
||||
# to mount the host's `/var/run/docker.sock`. That would hand every workflow —
|
||||
# including whatever a transitive dependency's postinstall script feels like
|
||||
# doing — the ability to start a privileged container mounting `/`, which is
|
||||
# root on Veron 1. Every published act_runner example does exactly this.
|
||||
#
|
||||
# Instead the runner talks to its OWN daemon (`dind`). Untrusted job code runs
|
||||
# as a child of that daemon, on an isolated network, with no route to the host
|
||||
# socket and no route to the forge's database.
|
||||
#
|
||||
# The split that makes this work:
|
||||
# * `runner` is TRUSTED code (the act_runner daemon). It sits on the forge
|
||||
# network only so it can reach gitea:3000 to collect jobs.
|
||||
# * `dind` and every job container it spawns are UNTRUSTED. They are on a
|
||||
# private network with no access to the forge, its database, or its .env.
|
||||
#
|
||||
# dind itself is privileged — that is the cost, and it is the reason a job
|
||||
# escape lands in a disposable daemon rather than on Grant's workstation.
|
||||
#
|
||||
# ⚠️ Do NOT "simplify" this by mounting the host docker socket.
|
||||
|
||||
name: windy-git-runner
|
||||
|
||||
services:
|
||||
dind:
|
||||
image: docker.io/library/docker:27-dind
|
||||
privileged: true
|
||||
environment:
|
||||
DOCKER_TLS_CERTDIR: "" # plain TCP on an isolated network, no host route
|
||||
command: ["dockerd", "--host=tcp://0.0.0.0:2375", "--tls=false"]
|
||||
networks: [jobs]
|
||||
volumes:
|
||||
- dind-storage:/var/lib/docker
|
||||
# G1.5 — bounded so a fork-bomb workflow cannot starve Grant's interactive
|
||||
# session. Veron 1 is his workstation, not a dedicated build box.
|
||||
cpus: 12.0 # 12 of 24 cores
|
||||
mem_limit: 64g
|
||||
restart: unless-stopped
|
||||
|
||||
runner:
|
||||
image: docker.io/gitea/act_runner:0.2.11
|
||||
depends_on: [dind]
|
||||
environment:
|
||||
# The runner reaches its OWN daemon. Never the host's.
|
||||
DOCKER_HOST: tcp://dind:2375
|
||||
GITEA_INSTANCE_URL: http://gitea:3000
|
||||
GITEA_RUNNER_REGISTRATION_TOKEN: ${RUNNER_TOKEN:?set RUNNER_TOKEN}
|
||||
GITEA_RUNNER_NAME: veron-1
|
||||
CONFIG_FILE: /config.yaml
|
||||
volumes:
|
||||
- ./config.yaml:/config.yaml:ro
|
||||
- runner-data:/data
|
||||
networks: [jobs, forge]
|
||||
cpus: 2.0
|
||||
mem_limit: 4g
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
jobs:
|
||||
# Untrusted job containers live here. No route to the forge.
|
||||
internal: false # jobs legitimately need to fetch dependencies
|
||||
forge:
|
||||
# Pre-existing network owned by the forge compose project.
|
||||
external: true
|
||||
name: windy-git_default
|
||||
|
||||
volumes:
|
||||
dind-storage:
|
||||
runner-data:
|
||||
@@ -57,6 +57,17 @@ services:
|
||||
# G2.2 — OIDC only. No local password login, no self-registration.
|
||||
GITEA__service__DISABLE_REGISTRATION: "true"
|
||||
GITEA__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "true"
|
||||
# G3.1 — a Windy account IS the account. Signing in with Windy provisions
|
||||
# the Gitea user on first arrival; nobody is asked to invent a second
|
||||
# identity for the same person, and no local password ever exists.
|
||||
GITEA__oauth2_client__ENABLE_AUTO_REGISTRATION: "true"
|
||||
GITEA__oauth2_client__USERNAME: email
|
||||
GITEA__oauth2_client__ACCOUNT_LINKING: auto
|
||||
# The email is asserted by account-server, which is the authority on it.
|
||||
# Asking the user to re-verify an address their identity provider already
|
||||
# verified is friction that buys nothing.
|
||||
GITEA__oauth2_client__UPDATE_AVATAR: "false"
|
||||
GITEA__service__REGISTER_EMAIL_CONFIRM: "false"
|
||||
# G4.3 — heavy bytes to R2 at ZERO egress. Git object databases stay on
|
||||
# local NVMe (I-3); this covers LFS, attachments, packages, avatars and
|
||||
# Actions artifacts, which is where GitHub's painful bills actually come
|
||||
|
||||
@@ -17,6 +17,11 @@ dependencies = [
|
||||
"pydantic-settings>=2.6",
|
||||
"sqlalchemy[asyncio]>=2.0",
|
||||
"asyncpg>=0.30",
|
||||
# Alembic runs synchronously (env.py strips +asyncpg), so the image needs a
|
||||
# sync driver too. Without it migrations fail INSIDE the container while
|
||||
# passing on a developer machine that happens to have it — the kind of gap
|
||||
# that only shows up on a fresh deploy.
|
||||
"psycopg2-binary>=2.9",
|
||||
"alembic>=1.14",
|
||||
"httpx>=0.27",
|
||||
"boto3>=1.35",
|
||||
|
||||
Reference in New Issue
Block a user