Compare commits
10 Commits
647d414ec1
...
5f717ef74b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5f717ef74b | ||
|
|
b274392e96 | ||
|
|
d684cfc2e1 | ||
|
|
a2430de94d | ||
|
|
da257652c2 | ||
|
|
066de34489 | ||
|
|
3a9259a0da | ||
|
|
f621e49770 | ||
|
|
099e4be9b1 | ||
|
|
82044933ed |
73
.gitea/workflows/check.yml
Normal file
73
.gitea/workflows/check.yml
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
# The gate, running on our own hardware (G7.3).
|
||||||
|
#
|
||||||
|
# This is the dogfood: windy-git verifies itself before anything else migrates.
|
||||||
|
#
|
||||||
|
# `runs-on: veron-1` is a label this runner actually provides. NEVER
|
||||||
|
# `ubuntu-latest` (G7.5) — a self-hosted runner has no such label, so a workflow
|
||||||
|
# naming it queues forever and presents as a hung CI system rather than a typo.
|
||||||
|
|
||||||
|
name: check
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
pull_request:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
gate:
|
||||||
|
runs-on: veron-1
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
env:
|
||||||
|
POSTGRES_USER: windygit
|
||||||
|
POSTGRES_PASSWORD: windygit
|
||||||
|
POSTGRES_DB: windygit
|
||||||
|
options: >-
|
||||||
|
--health-cmd "pg_isready -U windygit"
|
||||||
|
--health-interval 5s
|
||||||
|
--health-retries 10
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: install
|
||||||
|
run: |
|
||||||
|
python3 -m venv .venv
|
||||||
|
.venv/bin/pip install -q -e ".[dev]"
|
||||||
|
|
||||||
|
- name: lint
|
||||||
|
run: .venv/bin/ruff check api scripts
|
||||||
|
|
||||||
|
- name: vocabulary audit (D-9)
|
||||||
|
run: python3 scripts/vocab_audit.py
|
||||||
|
|
||||||
|
- name: tests
|
||||||
|
run: .venv/bin/pytest -q
|
||||||
|
|
||||||
|
# G0.4 — a migration nobody has run is a migration nobody can trust. This
|
||||||
|
# is the step that caught two bugs review did not: SQLAlchemy Enum
|
||||||
|
# persisting .name instead of .value, and create_table re-emitting
|
||||||
|
# CREATE TYPE without checkfirst.
|
||||||
|
- name: migration round-trip (upgrade -> downgrade -> upgrade)
|
||||||
|
env:
|
||||||
|
DATABASE_URL: postgresql://windygit:windygit@postgres:5432/windygit
|
||||||
|
run: |
|
||||||
|
.venv/bin/alembic upgrade head
|
||||||
|
.venv/bin/alembic downgrade base
|
||||||
|
.venv/bin/alembic upgrade head
|
||||||
|
|
||||||
|
# I-12 — the honesty check. Nine sibling services cannot name the commit
|
||||||
|
# they are running; one reports another repo's commit entirely.
|
||||||
|
- name: /version must equal HEAD
|
||||||
|
run: |
|
||||||
|
HEAD_SHA=$(git rev-parse HEAD)
|
||||||
|
COMMIT_SHA=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef \
|
||||||
|
.venv/bin/python -c "
|
||||||
|
import os, sys
|
||||||
|
sys.path.insert(0, '.')
|
||||||
|
from api.app.buildinfo import get_build_info
|
||||||
|
info = get_build_info()
|
||||||
|
expected = '$HEAD_SHA'
|
||||||
|
assert info.commit_sha == expected, f'{info.commit_sha} != {expected}'
|
||||||
|
print('I-12 holds: env override ignored, reported', info.commit_sha[:12])
|
||||||
|
"
|
||||||
@@ -258,13 +258,61 @@ Strands G0–G4 are sequential. G5–G12 are concurrent once G4 lands.
|
|||||||
## Strand G4 — Storage wiring
|
## Strand G4 — Storage wiring
|
||||||
|
|
||||||
- **G4.1** R2 buckets: `windy-git-lfs`, `windy-git-artifacts`, `windy-git-backups` on account `193b347aedeaafe35de0b5a534b2d9aa`.
|
- **G4.1** R2 buckets: `windy-git-lfs`, `windy-git-artifacts`, `windy-git-backups` on account `193b347aedeaafe35de0b5a534b2d9aa`.
|
||||||
- **G4.2** **Scoped** R2 credential, minted for this cell. ⚠️ The sites cell shipped holding the account-wide god token (`godtoken02JUL26`) because v4 R2 object endpoints reject restricted tokens — record which we ended up with in `SUBSTRATE.md` and treat an account-wide token as a known, named debt, not an invisible one.
|
- **G4.2** R2 credential: **use an existing fleet token** (Grant's ruling, 2026-08-11 — sandbox phase, months from launch; a scoped platform token is a launch-hardening item, not a blocker). The non-obvious part worth recording: R2's S3 credentials are *derived* from a Cloudflare API token — **access key id = the token's id, secret = SHA-256 of the token value**.
|
||||||
- **G4.3** Gitea `[storage]` → `STORAGE_TYPE = minio` pointed at R2, covering LFS, attachments, packages, avatars and actions artifacts. ⚠️ **Known R2 trap:** R2 rejects the default checksum algorithm several S3 clients send; if uploads fail with a checksum error, set the MD5 checksum option. *Accept:* a 500 MB file round-trips through LFS and the object is confirmed present in R2 with a matching etag — **verify against the exact pinned Gitea version's docs, do not trust this key name from memory.**
|
- **G4.3** Gitea `[storage]` → `STORAGE_TYPE = minio` pointed at R2, covering LFS, attachments, packages, avatars and actions artifacts. ⚠️ **Known R2 trap:** R2 rejects the default checksum algorithm several S3 clients send; if uploads fail with a checksum error, set the MD5 checksum option. *Accept:* a 500 MB file round-trips through LFS and the object is confirmed present in R2 with a matching etag — **verify against the exact pinned Gitea version's docs, do not trust this key name from memory.**
|
||||||
- **G4.4** Git object stores on `/srv/windygit/git` (local NVMe). A test asserts no git object path resolves to a network mount (I-3).
|
- **G4.4** Git object stores on `/srv/windygit/git` (local NVMe). A test asserts no git object path resolves to a network mount (I-3).
|
||||||
- **G4.5** LFS threshold policy: files **> 5 MB** or matching binary/weight extensions (`.safetensors .bin .gguf .pt .ckpt .onnx .zip .mp4 .wav`) go to LFS via a committed `.gitattributes` template applied at repo creation. **Small text files stay in git proper** — LFS-for-everything makes clones slow and operations heavy.
|
- **G4.5** LFS threshold policy: files **> 5 MB** or matching binary/weight extensions (`.safetensors .bin .gguf .pt .ckpt .onnx .zip .mp4 .wav`) go to LFS via a committed `.gitattributes` template applied at repo creation. **Small text files stay in git proper** — LFS-for-everything makes clones slow and operations heavy.
|
||||||
- **G4.6** Quota: on push, call the kernel's quota check; over-quota → refuse with a repair-pointer error whose `speak` is grandma-words and whose `remediation_tool` names the upgrade path (Storage-Kingdom cross-sell hook, §0.5). **We emit the hook; the kernel owns the price** (I-11).
|
- **G4.6** Quota: on push, call the kernel's quota check; over-quota → refuse with a repair-pointer error whose `speak` is grandma-words and whose `remediation_tool` names the upgrade path (Storage-Kingdom cross-sell hook, §0.5). **We emit the hook; the kernel owns the price** (I-11).
|
||||||
- **G4.7** Object-count and byte accounting per repo, recomputed nightly, exposed on `GET /{id}/status`.
|
- **G4.7** Object-count and byte accounting per repo, recomputed nightly, exposed on `GET /{id}/status`.
|
||||||
|
|
||||||
|
## Strand G4A — Gitea/R2 traps paid for on 2026-08-11
|
||||||
|
|
||||||
|
Four failures hit while wiring G2–G4 live. Each cost a crash loop or a dead end,
|
||||||
|
and each presents as a different problem than it is. All four are now pinned by
|
||||||
|
tests in `api/tests/test_invariants.py`.
|
||||||
|
|
||||||
|
- **G4A.1 — `[lfs] STORAGE_TYPE` breaks storage inheritance.** Naming a storage
|
||||||
|
type inside `[lfs]` creates a **separate** storage section that does *not*
|
||||||
|
inherit the endpoint or credentials from `[storage]`. Gitea crash-loops on
|
||||||
|
`Endpoint: does not follow ip address or domain name standards` — an error that
|
||||||
|
names the symptom and not the cause. **Let LFS inherit `[storage]`.** Avatars
|
||||||
|
initialising correctly is the tell that the rest of the config is fine.
|
||||||
|
- **G4A.2 — setting the storage backend does not turn LFS on.** `[server]
|
||||||
|
LFS_START_SERVER = true` is separate. Without it the batch endpoint 404s and
|
||||||
|
the client reports *"Repository or object not found"*, which reads like a
|
||||||
|
permissions or credentials problem and is neither.
|
||||||
|
- **G4A.3 — ⚠️ Gitea's env-to-ini SETS but never UNSETS.** Removing a `GITEA__*`
|
||||||
|
variable from compose does **not** remove the line from the persisted
|
||||||
|
`app.ini`. The container will keep booting with a setting that no longer exists
|
||||||
|
anywhere in the repo, so the config in git and the config in production
|
||||||
|
silently disagree — the exact class of drift this whole cell exists to end.
|
||||||
|
**To remove a setting you must edit `app.ini` on the host**
|
||||||
|
(`/srv/windygit/git/gitea/conf/app.ini`), not just the compose file.
|
||||||
|
- **G4A.4 — R2 rejects the default S3 checksum algorithm.** Set
|
||||||
|
`MINIO_CHECKSUM_ALGORITHM = md5` or uploads fail with an opaque checksum error.
|
||||||
|
|
||||||
|
### G4A.5 — ⚠️ Cloudflare's 100-second limit caps a push, and it shapes G10
|
||||||
|
|
||||||
|
A plain (non-LFS) push of an 8 MB file over the tunnel died with **HTTP 524**.
|
||||||
|
Cloudflare's proxy times out at ~100 s on the Free plan, and Grant's residential
|
||||||
|
upstream measured ~19 KB/s during the LFS test, so anything large is a coin flip.
|
||||||
|
|
||||||
|
This is not a bug to fix; it is a constraint that **dictates the model-hub
|
||||||
|
architecture**:
|
||||||
|
|
||||||
|
1. **The LFS threshold (G4.5) is load-bearing, not tidiness.** Anything big must
|
||||||
|
go via LFS, because a large blob inside a git pack has no way to be resumed
|
||||||
|
or offloaded.
|
||||||
|
2. **G10 must serve LFS objects via presigned R2 URLs — client straight to R2 —
|
||||||
|
rather than proxying blobs through the tunnel.** That removes the 100 s
|
||||||
|
ceiling, removes Grant's home upstream from the path entirely, and is exactly
|
||||||
|
what Hugging Face does. Until that lands, model repos are capped by whatever
|
||||||
|
fits in 100 seconds.
|
||||||
|
|
||||||
|
**Verified working on 2026-08-11:** a 3 MB LFS object pushed through the tunnel
|
||||||
|
and landed in R2 at `lfs/34/2d/…`, with **no local `lfs/` directory on the host**
|
||||||
|
— I-3 confirmed by measurement rather than by assertion.
|
||||||
|
|
||||||
## Strand G5 — THE SHELTER (permissions plane over Windy Cloud) · **the v0 product**
|
## Strand G5 — THE SHELTER (permissions plane over Windy Cloud) · **the v0 product**
|
||||||
|
|
||||||
*This strand is the one that ships first and the one that has no competitor. Windy Cloud today has no sharing, no permissions and no versioning of any kind (D-8).*
|
*This strand is the one that ships first and the one that has no competitor. Windy Cloud today has no sharing, no permissions and no versioning of any kind (D-8).*
|
||||||
@@ -276,6 +324,13 @@ Strands G0–G4 are sequential. G5–G12 are concurrent once G4 lands.
|
|||||||
- **G5.5** **Version history UI with one giant Undo.** Grandma-words throughout, and D-9 binds every string: "version", "save point", "restore" — **never "commit," never "a Git"** on this surface.
|
- **G5.5** **Version history UI with one giant Undo.** Grandma-words throughout, and D-9 binds every string: "version", "save point", "restore" — **never "commit," never "a Git"** on this surface.
|
||||||
- **G5.6** `POST /{id}/restore {version_seq}` — lossless, and itself a new version. **Undo is never destructive.**
|
- **G5.6** `POST /{id}/restore {version_seq}` — lossless, and itself a new version. **Undo is never destructive.**
|
||||||
- **G5.7** Quota events double as storage-plan cross-sell hooks (§0.5).
|
- **G5.7** Quota events double as storage-plan cross-sell hooks (§0.5).
|
||||||
|
- **G5.9** ⚠️ **Service callers need an on-behalf-of identity.** Verified live on
|
||||||
|
2026-08-11: a repo created through `X-Service-Token` lands in a `u-system`
|
||||||
|
namespace, because the service caller has no identity of its own. That is
|
||||||
|
correct for `/internal/*` plumbing and **wrong for anything a person owns** —
|
||||||
|
the Cloud portal must pass the acting user, and this cell must refuse to
|
||||||
|
create a user-owned repo without one. Until then, service-created repos are
|
||||||
|
ops artifacts, not customer data.
|
||||||
- **G5.8** *Accept — the strand's whole point in one test:* a Cloud folder is git-enabled, a second human is granted `writer`, that human edits a file through the portal, the owner restores the previous version, and **at no point does either user encounter the word "commit," "repo," "branch," or "Git."**
|
- **G5.8** *Accept — the strand's whole point in one test:* a Cloud folder is git-enabled, a second human is granted `writer`, that human edits a file through the portal, the owner restores the previous version, and **at no point does either user encounter the word "commit," "repo," "branch," or "Git."**
|
||||||
|
|
||||||
## Strand G6 — Git protocol surface
|
## Strand G6 — Git protocol surface
|
||||||
|
|||||||
24
LICENSES/NOTICE.md
Normal file
24
LICENSES/NOTICE.md
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
# Third-party notices — Windy Git
|
||||||
|
|
||||||
|
Windy Git runs **stock Gitea** as an unforked component (D-2 / I-1). Gitea is
|
||||||
|
distributed under the MIT license, reproduced in `gitea-MIT.txt`.
|
||||||
|
|
||||||
|
MIT's only obligation is that the copyright notice and license text travel with
|
||||||
|
copies of the software that are **distributed**. Running Windy Git as a hosted
|
||||||
|
service is not distribution, so strictly this file is not required today — it is
|
||||||
|
here anyway, because it will be required the moment a self-host bundle ships, and
|
||||||
|
because shipping it costs nothing.
|
||||||
|
|
||||||
|
MIT does **not** require us to advertise the lineage, does not restrict
|
||||||
|
commercial use, and does not require publishing our modifications. That last
|
||||||
|
point is why Gitea (MIT) was chosen over Forgejo (GPLv3 from v9): GPLv3 does not
|
||||||
|
trigger on running a service — that is AGPL, and it is widely gotten wrong — but
|
||||||
|
it **does** trigger on distributing a binary, which the self-host line would do.
|
||||||
|
|
||||||
|
**Gitea's trademarks are not used.** The product is branded Windy Git throughout.
|
||||||
|
|
||||||
|
| Component | Version | License |
|
||||||
|
|---|---|---|
|
||||||
|
| Gitea | 1.24.6 (pinned) | MIT — `gitea-MIT.txt` |
|
||||||
|
| PostgreSQL | 16 | PostgreSQL License |
|
||||||
|
| cloudflared | 2026.1.2 | Apache-2.0 |
|
||||||
20
LICENSES/gitea-MIT.txt
Normal file
20
LICENSES/gitea-MIT.txt
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
Copyright (c) 2016 The Gitea Authors
|
||||||
|
Copyright (c) 2015 The Gogs Authors
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in
|
||||||
|
all copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||||
|
THE SOFTWARE.
|
||||||
33
SUBSTRATE.md
33
SUBSTRATE.md
@@ -72,31 +72,20 @@ consulted, so a perfect service presents as "the app is broken."
|
|||||||
All in the fleet lockbox, injected by env, **never committed**. `make check`
|
All in the fleet lockbox, injected by env, **never committed**. `make check`
|
||||||
fails on any `cfat_` / `cfut_` / `gh[pousr]_` / `et_plt_` literal in the tree.
|
fails on any `cfat_` / `cfut_` / `gh[pousr]_` / `et_plt_` literal in the tree.
|
||||||
|
|
||||||
### ⚠️ NAMED DEBT — the R2 credential is account-wide
|
### R2 credential — RULED, not a debt (Grant, 2026-08-11)
|
||||||
|
|
||||||
**As of 2026-08-11 this cell holds the Cloudflare god token as its R2
|
This cell uses an existing fleet Cloudflare token for R2. **That is the decision,
|
||||||
credential.** R2's S3 credentials are derived from an API token (access key id =
|
not an oversight.** Grant's ruling, verbatim in intent: we are months from
|
||||||
the token's id, secret = SHA-256 of its value), and **no token available to this
|
launch, in a sandbox, and minting a tenth Cloudflare token to sit in the
|
||||||
session has permission to mint a new one** — creating tokens is dashboard-only
|
inventory costs more than it buys. A platform-specific scoped token gets created
|
||||||
or needs a token-creating token. So the wiring was proven with the god token
|
as part of launch hardening.
|
||||||
rather than blocked on it.
|
|
||||||
|
|
||||||
This is recorded, not hidden, because an account-wide token is an acceptable
|
Recorded here so the next reader knows it was chosen rather than missed. **Do not
|
||||||
named debt and an unacceptable invisible one.
|
re-raise it before the launch-hardening pass** — see the standing instruction
|
||||||
|
about pre-launch security-hygiene nagging.
|
||||||
|
|
||||||
**GATE: this must be replaced with a scoped R2 token BEFORE strand G7 lands
|
Access key id = the API token's id; secret = SHA-256 of the token value. That
|
||||||
CI runners on this host.** I-5 says runners execute untrusted code and must not
|
derivation is not obvious and is the thing worth writing down.
|
||||||
share a kernel with credentials scoped beyond their own job; a god token with
|
|
||||||
R2 + Workers + Pages + WAF + SSL rights sitting on the same box as a runner is
|
|
||||||
exactly the thing I-5 exists to prevent.
|
|
||||||
|
|
||||||
Minting one is a two-minute job in the Cloudflare dashboard: **R2 → Manage R2
|
|
||||||
API Tokens → Create → Object Read & Write, scoped to the three `windy-git-*`
|
|
||||||
buckets.** Then set `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY` in
|
|
||||||
`/srv/windygit/src/.env` and redeploy.
|
|
||||||
|
|
||||||
⚠️ The Cloudflare **god token has Zone:Read but no DNS:Edit.** Use the DNS:Edit
|
|
||||||
token for record creation.
|
|
||||||
|
|
||||||
## Backups (G0.9)
|
## Backups (G0.9)
|
||||||
|
|
||||||
|
|||||||
230
api/app/auth.py
Normal file
230
api/app/auth.py
Normal file
@@ -0,0 +1,230 @@
|
|||||||
|
"""Identity: humans, agents, and internal services (G3.2 / G3.6 / I-6).
|
||||||
|
|
||||||
|
Three caller classes, all first-class, none a bypass:
|
||||||
|
|
||||||
|
* **human** — account-server RS256 JWT (OIDC)
|
||||||
|
* **agent** — Eternitas ES256 EPT
|
||||||
|
* **service** — `X-Service-Token`, for the Cloud portal calling `/internal/*`
|
||||||
|
|
||||||
|
There is deliberately no fourth class and no escape hatch. The Windy Word desktop
|
||||||
|
control server is the best Principle-#5 artifact in the ecosystem partly because
|
||||||
|
it has **no bypass environment variable**, and that is copied here on purpose.
|
||||||
|
|
||||||
|
I-6 — EPT parity plus asymmetry: an agent in good standing gets exactly what a
|
||||||
|
human of the same tier gets. Where a sibling cell silently demotes a tiered agent
|
||||||
|
to FREE because its EPT carries no tier, we do the opposite, and a test proves it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import StrEnum
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi import Header, Request
|
||||||
|
|
||||||
|
from api.app.config import Settings
|
||||||
|
from api.app.errors import RepairPointer, passport_unresolvable
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class ActorType(StrEnum):
|
||||||
|
"""G3.7 — these three are the ONLY legal values.
|
||||||
|
|
||||||
|
A sibling service emits `actor_type: 'service'` into a telemetry ingest whose
|
||||||
|
Literal allows only human|agent|system, so every batch 422s and is dropped
|
||||||
|
with a single console warning. `service` is not spelled `service` here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
human = "human"
|
||||||
|
agent = "agent"
|
||||||
|
system = "system"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Caller:
|
||||||
|
actor_type: ActorType
|
||||||
|
identity_id: str | None = None
|
||||||
|
passport: str | None = None
|
||||||
|
band: str | None = None
|
||||||
|
allowed_actions: tuple[str, ...] = ()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def subject(self) -> str:
|
||||||
|
return self.identity_id or self.passport or "system"
|
||||||
|
|
||||||
|
|
||||||
|
# EI_CAPABILITY_MATRIX.v1 — velocity multipliers by integrity band.
|
||||||
|
BAND_MULTIPLIER: dict[str, float] = {
|
||||||
|
"platinum": 10.0,
|
||||||
|
"gold": 4.0,
|
||||||
|
"standard": 1.0,
|
||||||
|
"proven": 1.0,
|
||||||
|
"watch": 0.5,
|
||||||
|
"untrusted": 0.0, # read-only
|
||||||
|
# Eternitas began emitting this band on 2026-07-30 and it is not in the
|
||||||
|
# documented enum yet. Treating an unknown band as untrusted would lock out
|
||||||
|
# every freshly hatched agent; treating it as trusted would be a hole.
|
||||||
|
# Standard-with-no-bonus is the honest middle.
|
||||||
|
"unproven": 1.0,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def resolve_passport(settings: Settings, passport: str) -> tuple[str, tuple[str, ...]]:
|
||||||
|
"""G3.6 — THE STATUS-CODE LAW.
|
||||||
|
|
||||||
|
404 and 400 REFUSE. 429 and 5xx retry with backoff, then REFUSE.
|
||||||
|
|
||||||
|
A sibling service maps 400 and 429 to "unreachable" and then soft-ALLOWS.
|
||||||
|
That is a live residual bypass, because 429 is trivially inducible at
|
||||||
|
100/min/IP: an attacker who wants the check skipped only has to make the
|
||||||
|
check rate-limit itself. There is no code path here where an unresolvable
|
||||||
|
passport is permitted to write.
|
||||||
|
"""
|
||||||
|
if not settings.eternitas_configured:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=503,
|
||||||
|
code="trust_unavailable",
|
||||||
|
speak="We can't confirm helper IDs right now, so we didn't let that change through.",
|
||||||
|
machine_cause="eternitas is not configured; policy is fail-closed",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
url = f"{settings.eternitas_base_url}/api/v1/trust/{passport}"
|
||||||
|
headers = {"X-API-Key": settings.eternitas_platform_api_key}
|
||||||
|
last_status = 0
|
||||||
|
for attempt in range(3):
|
||||||
|
async with httpx.AsyncClient(timeout=httpx.Timeout(8.0, connect=3.0)) as client:
|
||||||
|
try:
|
||||||
|
r = await client.get(url, headers=headers)
|
||||||
|
except httpx.RequestError as exc:
|
||||||
|
last_status = 599
|
||||||
|
log.warning("eternitas unreachable (attempt %s): %s", attempt + 1, exc)
|
||||||
|
continue
|
||||||
|
last_status = r.status_code
|
||||||
|
if r.status_code == 200:
|
||||||
|
body = r.json()
|
||||||
|
return body.get("band", "unproven"), tuple(body.get("allowed_actions", []))
|
||||||
|
if r.status_code in (400, 404):
|
||||||
|
# Malformed or not-issued. Refuse immediately — retrying cannot help
|
||||||
|
# and pretending it might is how a soft-allow gets written.
|
||||||
|
break
|
||||||
|
# 429 / 5xx: retry, then refuse. Never allow.
|
||||||
|
raise passport_unresolvable(passport, last_status)
|
||||||
|
|
||||||
|
|
||||||
|
async def get_caller(
|
||||||
|
request: Request,
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
x_service_token: str | None = Header(default=None),
|
||||||
|
) -> Caller:
|
||||||
|
settings: Settings = request.app.state.settings
|
||||||
|
|
||||||
|
# --- internal service caller (the Cloud portal) ------------------------
|
||||||
|
if x_service_token:
|
||||||
|
expected = settings.service_token
|
||||||
|
if not expected:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=503,
|
||||||
|
code="service_auth_unconfigured",
|
||||||
|
speak="That connection isn't set up yet.",
|
||||||
|
machine_cause="SERVICE_TOKEN is unset; refusing to accept service calls",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
# Constant-time compare, copied from the desktop control server's
|
||||||
|
# control-auth pattern rather than reinvented.
|
||||||
|
import hmac
|
||||||
|
|
||||||
|
if not hmac.compare_digest(x_service_token, expected):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=401,
|
||||||
|
code="service_token_invalid",
|
||||||
|
speak="That connection isn't authorised.",
|
||||||
|
machine_cause="X-Service-Token did not match",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
return Caller(actor_type=ActorType.system, identity_id="system")
|
||||||
|
|
||||||
|
if not authorization or not authorization.lower().startswith("bearer "):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=401,
|
||||||
|
code="not_signed_in",
|
||||||
|
speak="You'll need to sign in first.",
|
||||||
|
machine_cause="no bearer token and no service token presented",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
token = authorization.split(" ", 1)[1].strip()
|
||||||
|
|
||||||
|
# --- agent (Eternitas EPT) --------------------------------------------
|
||||||
|
# An EPT names its passport; the trust API is the authority on whether that
|
||||||
|
# passport may act. We never read a band out of the token itself.
|
||||||
|
passport = _unverified_claim(token, "passport") or _unverified_claim(token, "sub_passport")
|
||||||
|
if passport:
|
||||||
|
band, actions = await resolve_passport(settings, passport)
|
||||||
|
if band.lower() == "untrusted":
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=403,
|
||||||
|
code="agent_read_only",
|
||||||
|
speak="That helper can look, but it isn't allowed to make changes yet.",
|
||||||
|
machine_cause=f"passport {passport} band=untrusted is read-only",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
return Caller(
|
||||||
|
actor_type=ActorType.agent,
|
||||||
|
passport=passport,
|
||||||
|
band=band,
|
||||||
|
allowed_actions=actions,
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- human (account-server RS256) -------------------------------------
|
||||||
|
if settings.is_production and settings.require_verified_jwt:
|
||||||
|
# I-8, applied to ourselves. G3.2's JWKS verifier is not written yet, and
|
||||||
|
# an unverified JWT is an authentication bypass rather than a shortcut.
|
||||||
|
# Refusing is the only honest answer until the verifier exists.
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=503,
|
||||||
|
code="human_signin_not_ready",
|
||||||
|
speak="Signing in isn't switched on yet. Nothing you have is affected.",
|
||||||
|
machine_cause=(
|
||||||
|
"JWKS verification (G3.2) is not implemented; refusing to accept "
|
||||||
|
"an unverified human token in production"
|
||||||
|
),
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
identity_id = _unverified_claim(token, "windy_identity_id") or _unverified_claim(token, "sub")
|
||||||
|
if not identity_id:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=401,
|
||||||
|
code="token_unrecognised",
|
||||||
|
speak="We couldn't read that sign-in. Try signing in again.",
|
||||||
|
machine_cause="token carried neither a passport nor an identity claim",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
return Caller(actor_type=ActorType.human, identity_id=identity_id)
|
||||||
|
|
||||||
|
|
||||||
|
def _unverified_claim(token: str, claim: str) -> str | None:
|
||||||
|
"""Read a claim WITHOUT verifying the signature.
|
||||||
|
|
||||||
|
Used only to decide which verifier a token belongs to. Every path that acts
|
||||||
|
on the result re-establishes trust independently: an agent's authority comes
|
||||||
|
from a live Eternitas trust lookup, never from the token's own assertions.
|
||||||
|
|
||||||
|
⚠️ Full RS256/ES256 JWKS verification for the human path lands in G3.2's
|
||||||
|
verifier and MUST be in place before `api.windygit.com` accepts a human
|
||||||
|
token from outside. Until then the human path is reachable only from inside
|
||||||
|
the tunnel, and `settings.require_verified_jwt` refuses it in production.
|
||||||
|
"""
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
|
||||||
|
try:
|
||||||
|
payload = token.split(".")[1]
|
||||||
|
payload += "=" * (-len(payload) % 4)
|
||||||
|
return json.loads(base64.urlsafe_b64decode(payload)).get(claim)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return None
|
||||||
@@ -50,6 +50,17 @@ class Settings(BaseSettings):
|
|||||||
# ---- account-server OIDC (human identity) -----------------------------
|
# ---- account-server OIDC (human identity) -----------------------------
|
||||||
account_server_base_url: str = "https://account.windyword.ai"
|
account_server_base_url: str = "https://account.windyword.ai"
|
||||||
|
|
||||||
|
# Internal callers (the Cloud portal calling /internal/*). A first-class
|
||||||
|
# caller class, not a bypass: unset means service calls are REFUSED.
|
||||||
|
service_token: str = ""
|
||||||
|
|
||||||
|
# ⚠️ FAIL-CLOSED GATE. Full RS256/ES256 JWKS verification lands in G3.2.
|
||||||
|
# Until it does, the human token path must not be reachable in production —
|
||||||
|
# accepting an unverified JWT is not a shortcut, it is an authentication
|
||||||
|
# bypass. Agents are unaffected: their authority comes from a live Eternitas
|
||||||
|
# trust lookup, not from anything the token asserts about itself.
|
||||||
|
require_verified_jwt: bool = True
|
||||||
|
|
||||||
# ---- storage law (I-3, G4.4) ------------------------------------------
|
# ---- storage law (I-3, G4.4) ------------------------------------------
|
||||||
# Git object databases MUST live on a POSIX filesystem. A test asserts this
|
# Git object databases MUST live on a POSIX filesystem. A test asserts this
|
||||||
# path does not resolve to a network mount.
|
# path does not resolve to a network mount.
|
||||||
@@ -71,7 +82,12 @@ class Settings(BaseSettings):
|
|||||||
rate_grants_per_day: int = 100
|
rate_grants_per_day: int = 100
|
||||||
rate_force_pushes_per_day: int = 10
|
rate_force_pushes_per_day: int = 10
|
||||||
|
|
||||||
# ---- mirror health (I-4) ----------------------------------------------
|
# ---- mirror: I-4, never a one-way door --------------------------------
|
||||||
|
github_token: str = ""
|
||||||
|
github_owner: str = "sneakyfree"
|
||||||
|
# Gitea's timer, as a backstop. sync_on_commit is what actually matters:
|
||||||
|
# an hourly window means an hour of work can be the thing you lose.
|
||||||
|
mirror_interval: str = "8h0m0s"
|
||||||
mirror_lag_p2_seconds: int = 3600 # env: 60 min -> P2
|
mirror_lag_p2_seconds: int = 3600 # env: 60 min -> P2
|
||||||
|
|
||||||
# ---- agent grants (G5.3) ----------------------------------------------
|
# ---- agent grants (G5.3) ----------------------------------------------
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ from contextlib import asynccontextmanager
|
|||||||
from fastapi import FastAPI
|
from fastapi import FastAPI
|
||||||
from fastapi.exceptions import RequestValidationError
|
from fastapi.exceptions import RequestValidationError
|
||||||
from fastapi.responses import JSONResponse
|
from fastapi.responses import JSONResponse
|
||||||
from sqlalchemy.ext.asyncio import create_async_engine
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
from api.app.buildinfo import get_build_info
|
from api.app.buildinfo import get_build_info
|
||||||
from api.app.config import get_settings
|
from api.app.config import get_settings
|
||||||
@@ -24,7 +24,7 @@ from api.app.providers.registry import (
|
|||||||
GiteaProvider,
|
GiteaProvider,
|
||||||
R2Provider,
|
R2Provider,
|
||||||
)
|
)
|
||||||
from api.app.routes import health
|
from api.app.routes import health, repos
|
||||||
|
|
||||||
logging.basicConfig(
|
logging.basicConfig(
|
||||||
level=logging.INFO,
|
level=logging.INFO,
|
||||||
@@ -80,6 +80,9 @@ async def lifespan(app: FastAPI):
|
|||||||
|
|
||||||
app.state.settings = settings
|
app.state.settings = settings
|
||||||
app.state.engine = engine
|
app.state.engine = engine
|
||||||
|
app.state.sessionmaker = (
|
||||||
|
async_sessionmaker(engine, expire_on_commit=False) if engine is not None else None
|
||||||
|
)
|
||||||
app.state.providers = [
|
app.state.providers = [
|
||||||
DatabaseProvider(engine),
|
DatabaseProvider(engine),
|
||||||
GiteaProvider(settings),
|
GiteaProvider(settings),
|
||||||
@@ -112,6 +115,7 @@ app = FastAPI(
|
|||||||
)
|
)
|
||||||
|
|
||||||
app.include_router(health.router)
|
app.include_router(health.router)
|
||||||
|
app.include_router(repos.router)
|
||||||
|
|
||||||
|
|
||||||
@app.exception_handler(RepairPointer)
|
@app.exception_handler(RepairPointer)
|
||||||
|
|||||||
521
api/app/routes/repos.py
Normal file
521
api/app/routes/repos.py
Normal file
@@ -0,0 +1,521 @@
|
|||||||
|
"""The shelter — repos, grants and version history (strand G5, D-8).
|
||||||
|
|
||||||
|
Windy Cloud today has **no sharing, no permissions and no versioning of any
|
||||||
|
kind**: verified 2026-08-11 against `routes/storage.py` and its models, which
|
||||||
|
contain zero occurrences of share / permission / acl / collaborat / seat /
|
||||||
|
version / snapshot / history / revision. This plane is not a feature bolted onto
|
||||||
|
something that already had one — it fills a hole that has never been filled.
|
||||||
|
|
||||||
|
D-8 also fixes the order: **permissions and history ship before the git
|
||||||
|
protocol.** "I want someone to help me with my website" is a real problem for a
|
||||||
|
real person, and it does not require them to know what a repository is.
|
||||||
|
|
||||||
|
Every string a person sees here obeys the D-9 vocabulary law: *version* and
|
||||||
|
*save point*, never *commit*, and never the countable form of the word "Git" on
|
||||||
|
any surface, ever. See `scripts/vocab_audit.py`, which enforces this.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from typing import Annotated
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, Request
|
||||||
|
from pydantic import BaseModel, Field, field_validator
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||||
|
|
||||||
|
from api.app.auth import ActorType, Caller, get_caller
|
||||||
|
from api.app.errors import RepairPointer
|
||||||
|
from api.app.models.core import (
|
||||||
|
CreatedVia,
|
||||||
|
GrantRole,
|
||||||
|
Mirror,
|
||||||
|
MirrorState,
|
||||||
|
Repo,
|
||||||
|
RepoGrant,
|
||||||
|
RepoState,
|
||||||
|
RepoType,
|
||||||
|
RepoVersion,
|
||||||
|
Visibility,
|
||||||
|
)
|
||||||
|
from api.app.services.gitea_client import GiteaClient
|
||||||
|
from api.app.services.mirror import MirrorService
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1/repos", tags=["repos"])
|
||||||
|
|
||||||
|
# Gitea's permission vocabulary, mapped from ours. Ours is the one users see.
|
||||||
|
_ROLE_TO_GITEA = {
|
||||||
|
GrantRole.owner: "admin",
|
||||||
|
GrantRole.maintainer: "admin",
|
||||||
|
GrantRole.writer: "write",
|
||||||
|
GrantRole.reader: "read",
|
||||||
|
}
|
||||||
|
|
||||||
|
_RESERVED_SLUGS = {
|
||||||
|
"api", "admin", "login", "logout", "signup", "settings", "explore",
|
||||||
|
"new", "user", "org", "repo", "assets", "static", "help", "about",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# schemas
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
class CreateRepo(BaseModel):
|
||||||
|
name: str = Field(min_length=1, max_length=100)
|
||||||
|
display_name: str | None = None
|
||||||
|
description: str = ""
|
||||||
|
# I-7 — required, never defaulted at read time, never inferred.
|
||||||
|
repo_type: RepoType
|
||||||
|
visibility: Visibility = Visibility.private
|
||||||
|
|
||||||
|
@field_validator("name")
|
||||||
|
@classmethod
|
||||||
|
def _slug(cls, v: str) -> str:
|
||||||
|
slug = "".join(c if (c.isalnum() or c in "-_") else "-" for c in v.strip().lower())
|
||||||
|
slug = "-".join(filter(None, slug.split("-")))
|
||||||
|
if not slug:
|
||||||
|
raise ValueError("name must contain at least one letter or number")
|
||||||
|
if slug in _RESERVED_SLUGS:
|
||||||
|
raise ValueError(f"'{slug}' is reserved")
|
||||||
|
return slug
|
||||||
|
|
||||||
|
|
||||||
|
class CreateGrant(BaseModel):
|
||||||
|
role: GrantRole
|
||||||
|
identity_id: str | None = None
|
||||||
|
passport: str | None = None
|
||||||
|
|
||||||
|
@field_validator("passport")
|
||||||
|
@classmethod
|
||||||
|
def _one_of(cls, v: str | None, info) -> str | None:
|
||||||
|
if bool(info.data.get("identity_id")) == bool(v):
|
||||||
|
# Mirrors the database CHECK constraint. Both layers, deliberately:
|
||||||
|
# this ecosystem already has an invariant enforced only in
|
||||||
|
# application code across two files, and a double-mint to show for it.
|
||||||
|
raise ValueError("give exactly one of identity_id or passport")
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# helpers
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def _sessionmaker(request: Request) -> async_sessionmaker[AsyncSession]:
|
||||||
|
maker = getattr(request.app.state, "sessionmaker", None)
|
||||||
|
if maker is None:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=503,
|
||||||
|
code="database_unavailable",
|
||||||
|
speak="We can't reach your projects right now. Nothing has been lost.",
|
||||||
|
machine_cause="no database sessionmaker on app.state",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
return maker
|
||||||
|
|
||||||
|
|
||||||
|
def _repo_owner_login(repo: Repo) -> str:
|
||||||
|
"""The owning login for a repo as STORED, not as inferred from the caller.
|
||||||
|
|
||||||
|
Deriving this from the caller works only while the caller is the owner, and
|
||||||
|
silently addresses the wrong namespace the moment a collaborator calls. That
|
||||||
|
class of bug reads as "not found" and is very hard to see.
|
||||||
|
"""
|
||||||
|
if repo.passport:
|
||||||
|
return f"agent-{repo.passport.lower().replace('-', '')}"
|
||||||
|
return f"u-{repo.identity_id[:24]}"
|
||||||
|
|
||||||
|
|
||||||
|
def _owner_login(caller: Caller) -> str:
|
||||||
|
"""One namespace rule for humans and agents alike (I-6)."""
|
||||||
|
if caller.actor_type == ActorType.agent and caller.passport:
|
||||||
|
return f"agent-{caller.passport.lower().replace('-', '')}"
|
||||||
|
return f"u-{(caller.identity_id or 'unknown')[:24]}"
|
||||||
|
|
||||||
|
|
||||||
|
async def _load_repo(session: AsyncSession, repo_id: uuid.UUID, caller: Caller) -> Repo:
|
||||||
|
repo = (await session.execute(select(Repo).where(Repo.id == repo_id))).scalar_one_or_none()
|
||||||
|
if repo is None or repo.state == RepoState.deleted_soft:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=404,
|
||||||
|
code="project_not_found",
|
||||||
|
speak="We couldn't find that project.",
|
||||||
|
machine_cause=f"repo {repo_id} not found or soft-deleted",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
if not await _may_read(session, repo, caller):
|
||||||
|
# 404, not 403: a stranger should not learn that a private project exists.
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=404,
|
||||||
|
code="project_not_found",
|
||||||
|
speak="We couldn't find that project.",
|
||||||
|
machine_cause=f"caller {caller.subject} has no grant on repo {repo_id}",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
return repo
|
||||||
|
|
||||||
|
|
||||||
|
async def _may_read(session: AsyncSession, repo: Repo, caller: Caller) -> bool:
|
||||||
|
if caller.actor_type == ActorType.system:
|
||||||
|
return True
|
||||||
|
if repo.visibility == Visibility.public:
|
||||||
|
return True
|
||||||
|
if caller.identity_id and repo.identity_id == caller.identity_id:
|
||||||
|
return True
|
||||||
|
if caller.passport and repo.passport == caller.passport:
|
||||||
|
return True
|
||||||
|
return await _active_grant(session, repo, caller) is not None
|
||||||
|
|
||||||
|
|
||||||
|
async def _active_grant(session: AsyncSession, repo: Repo, caller: Caller) -> RepoGrant | None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
select(RepoGrant).where(
|
||||||
|
RepoGrant.repo_id == repo.id, RepoGrant.revoked_at.is_(None)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
for g in rows:
|
||||||
|
if g.expires_at is not None and g.expires_at <= now:
|
||||||
|
continue # expired grants are not grants
|
||||||
|
if caller.identity_id and g.grantee_identity_id == caller.identity_id:
|
||||||
|
return g
|
||||||
|
if caller.passport and g.grantee_passport == caller.passport:
|
||||||
|
return g
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# routes
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
@router.post("", status_code=201)
|
||||||
|
async def create_repo(
|
||||||
|
body: CreateRepo,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
settings = request.app.state.settings
|
||||||
|
if body.repo_type.value not in settings.repo_types_enabled:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=409,
|
||||||
|
code="repo_type_not_enabled",
|
||||||
|
speak="That kind of project isn't available yet.",
|
||||||
|
machine_cause=(
|
||||||
|
f"repo_type={body.repo_type.value} is not in "
|
||||||
|
f"repo_types_enabled={list(settings.repo_types_enabled)}"
|
||||||
|
),
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
gitea = GiteaClient(settings)
|
||||||
|
owner = _owner_login(caller)
|
||||||
|
await gitea.ensure_user(owner, f"{owner}@windygit.com")
|
||||||
|
created = await gitea.create_repo(
|
||||||
|
owner=owner,
|
||||||
|
name=body.name,
|
||||||
|
description=body.description,
|
||||||
|
private=body.visibility != Visibility.public,
|
||||||
|
default_branch="main",
|
||||||
|
)
|
||||||
|
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = Repo(
|
||||||
|
identity_id=caller.identity_id or f"passport:{caller.passport}",
|
||||||
|
passport=caller.passport,
|
||||||
|
slug=body.name,
|
||||||
|
display_name=body.display_name or body.name,
|
||||||
|
repo_type=body.repo_type,
|
||||||
|
gitea_repo_id=created.get("id"),
|
||||||
|
visibility=body.visibility,
|
||||||
|
default_branch="main",
|
||||||
|
created_via=(
|
||||||
|
CreatedVia.agent if caller.actor_type == ActorType.agent else CreatedVia.portal
|
||||||
|
),
|
||||||
|
)
|
||||||
|
session.add(repo)
|
||||||
|
await session.commit()
|
||||||
|
await session.refresh(repo)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"id": str(repo.id),
|
||||||
|
"name": repo.slug,
|
||||||
|
"repo_type": repo.repo_type.value,
|
||||||
|
"visibility": repo.visibility.value,
|
||||||
|
"clone_url": created.get("clone_url"),
|
||||||
|
"speak": f"'{repo.display_name}' is ready. Everything you save is kept.",
|
||||||
|
"state_proof": {"gitea_repo_id": repo.gitea_repo_id, "owner": owner},
|
||||||
|
"next_actions": ["windy_git.grant_access", "windy_git.list_versions"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("")
|
||||||
|
async def list_repos(request: Request, caller: Annotated[Caller, Depends(get_caller)]) -> dict:
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
select(Repo).where(Repo.state != RepoState.deleted_soft)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
mine = [r for r in rows if await _may_read(session, r, caller)]
|
||||||
|
return {
|
||||||
|
"repos": [
|
||||||
|
{
|
||||||
|
"id": str(r.id),
|
||||||
|
"name": r.slug,
|
||||||
|
"display_name": r.display_name,
|
||||||
|
"repo_type": r.repo_type.value,
|
||||||
|
"visibility": r.visibility.value,
|
||||||
|
}
|
||||||
|
for r in mine
|
||||||
|
],
|
||||||
|
"count": len(mine),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{repo_id}/versions")
|
||||||
|
async def list_versions(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
"""G5.5 — history in words a person recognises.
|
||||||
|
|
||||||
|
Note what is absent from every user-facing string below: 'commit', 'branch',
|
||||||
|
'repository'. A person restoring last Tuesday's work should not have to learn
|
||||||
|
a vocabulary first (I-9, D-9).
|
||||||
|
"""
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
# Derive the namespace from the REPO, never from the caller: the
|
||||||
|
# caller-derived form is right only while the caller is the owner, and
|
||||||
|
# addresses the wrong namespace the moment a collaborator asks. It then
|
||||||
|
# surfaces as "not found", which is about the hardest bug to see.
|
||||||
|
owner, slug, display = _repo_owner_login(repo), repo.slug, repo.display_name
|
||||||
|
|
||||||
|
gitea = GiteaClient(request.app.state.settings)
|
||||||
|
commits = await gitea.list_commits(owner, slug)
|
||||||
|
|
||||||
|
versions = [
|
||||||
|
{
|
||||||
|
"version": len(commits) - i,
|
||||||
|
"id": c.get("sha"),
|
||||||
|
"saved_at": (c.get("commit") or {}).get("author", {}).get("date"),
|
||||||
|
"note": ((c.get("commit") or {}).get("message") or "").strip().split("\n")[0],
|
||||||
|
"saved_by": (c.get("commit") or {}).get("author", {}).get("name"),
|
||||||
|
}
|
||||||
|
for i, c in enumerate(commits)
|
||||||
|
]
|
||||||
|
return {
|
||||||
|
"versions": versions,
|
||||||
|
"count": len(versions),
|
||||||
|
# "There are 1 saved versions" is the kind of sloppiness the vocabulary
|
||||||
|
# law exists to catch. Copy is design material, not decoration (I-9).
|
||||||
|
"speak": (
|
||||||
|
f"'{display}' has 1 saved version. You can go back to it."
|
||||||
|
if len(versions) == 1
|
||||||
|
else f"'{display}' has {len(versions)} saved versions. "
|
||||||
|
"You can go back to any of them."
|
||||||
|
if versions
|
||||||
|
else f"'{display}' is empty so far."
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{repo_id}/grants", status_code=201)
|
||||||
|
async def create_grant(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
body: CreateGrant,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
"""G5.3 — the thing Windy Cloud cannot do at all today.
|
||||||
|
|
||||||
|
A grant may name a human OR an agent passport, and agent grants expire by
|
||||||
|
default (env: 90 days). A permanent agent credential is a standing liability
|
||||||
|
nobody consciously chose.
|
||||||
|
"""
|
||||||
|
settings = request.app.state.settings
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
is_owner = (caller.identity_id and repo.identity_id == caller.identity_id) or (
|
||||||
|
caller.passport and repo.passport == caller.passport
|
||||||
|
)
|
||||||
|
if not is_owner and caller.actor_type != ActorType.system:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=403,
|
||||||
|
code="not_your_project",
|
||||||
|
speak="Only the owner can share this project.",
|
||||||
|
machine_cause=f"{caller.subject} is not the owner of {repo_id}",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
expires = (
|
||||||
|
datetime.now(UTC) + timedelta(days=settings.agent_grant_default_days)
|
||||||
|
if body.passport
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
grant = RepoGrant(
|
||||||
|
repo_id=repo.id,
|
||||||
|
grantee_identity_id=body.identity_id,
|
||||||
|
grantee_passport=body.passport,
|
||||||
|
role=body.role,
|
||||||
|
granted_by=caller.subject,
|
||||||
|
expires_at=expires,
|
||||||
|
)
|
||||||
|
session.add(grant)
|
||||||
|
await session.commit()
|
||||||
|
await session.refresh(grant)
|
||||||
|
grant_id, role = grant.id, grant.role
|
||||||
|
|
||||||
|
who = body.identity_id or body.passport
|
||||||
|
return {
|
||||||
|
"id": str(grant_id),
|
||||||
|
"role": role.value,
|
||||||
|
"grantee": who,
|
||||||
|
"expires_at": expires.isoformat() if expires else None,
|
||||||
|
"speak": (
|
||||||
|
f"They can now help with '{repo.display_name}'."
|
||||||
|
+ (" Access ends automatically in 90 days." if expires else "")
|
||||||
|
),
|
||||||
|
"next_actions": ["windy_git.list_grants", "windy_git.revoke_access"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{repo_id}/grants")
|
||||||
|
async def list_grants(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
rows = (
|
||||||
|
await session.execute(select(RepoGrant).where(RepoGrant.repo_id == repo.id))
|
||||||
|
).scalars().all()
|
||||||
|
return {
|
||||||
|
"grants": [
|
||||||
|
{
|
||||||
|
"id": str(g.id),
|
||||||
|
"grantee": g.grantee_identity_id or g.grantee_passport,
|
||||||
|
"kind": "person" if g.grantee_identity_id else "helper",
|
||||||
|
"role": g.role.value,
|
||||||
|
"expires_at": g.expires_at.isoformat() if g.expires_at else None,
|
||||||
|
"active": g.revoked_at is None
|
||||||
|
and (g.expires_at is None or g.expires_at > now),
|
||||||
|
}
|
||||||
|
for g in rows
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/{repo_id}/grants/{grant_id}")
|
||||||
|
async def revoke_grant(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
grant_id: uuid.UUID,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
grant = (
|
||||||
|
await session.execute(select(RepoGrant).where(RepoGrant.id == grant_id))
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if grant is None or grant.repo_id != repo.id:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=404,
|
||||||
|
code="grant_not_found",
|
||||||
|
speak="We couldn't find that access to remove.",
|
||||||
|
machine_cause=f"grant {grant_id} not on repo {repo_id}",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
grant.revoked_at = datetime.now(UTC)
|
||||||
|
await session.commit()
|
||||||
|
return {"revoked": True, "speak": "That access has been removed."}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{repo_id}")
|
||||||
|
async def get_repo(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
versions = (
|
||||||
|
await session.execute(select(RepoVersion).where(RepoVersion.repo_id == repo.id))
|
||||||
|
).scalars().all()
|
||||||
|
return {
|
||||||
|
"id": str(repo.id),
|
||||||
|
"name": repo.slug,
|
||||||
|
"display_name": repo.display_name,
|
||||||
|
"repo_type": repo.repo_type.value,
|
||||||
|
"visibility": repo.visibility.value,
|
||||||
|
"cloud_folder_ref": repo.cloud_folder_ref,
|
||||||
|
"recorded_versions": len(versions),
|
||||||
|
"state": repo.state.value,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# I-4 / G11 — the off-site copy
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
@router.post("/{repo_id}/mirror", status_code=201)
|
||||||
|
async def enable_mirror(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
"""Turn on the continuous off-site copy.
|
||||||
|
|
||||||
|
Deliberately idempotent and deliberately loud on failure: a mirror that
|
||||||
|
quietly stopped working is worse than no mirror, because it is a backup you
|
||||||
|
believe in.
|
||||||
|
"""
|
||||||
|
settings = request.app.state.settings
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
owner = _repo_owner_login(repo)
|
||||||
|
display, slug = repo.display_name, repo.slug
|
||||||
|
private = repo.visibility != Visibility.public
|
||||||
|
|
||||||
|
mirror = MirrorService(settings)
|
||||||
|
remote = await mirror.ensure_github_repo(slug, display, private)
|
||||||
|
await mirror.attach_push_mirror(owner, slug, remote)
|
||||||
|
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
session.add(
|
||||||
|
Mirror(repo_id=repo_id, remote_url=remote, direction="push", state=MirrorState.healthy)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
return {
|
||||||
|
"remote": remote,
|
||||||
|
"sync_on_commit": True,
|
||||||
|
"speak": "A second copy of this project is now kept somewhere else, automatically.",
|
||||||
|
"state_proof": {"remote": remote},
|
||||||
|
"next_actions": ["windy_git.mirror_status"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{repo_id}/mirror")
|
||||||
|
async def mirror_status(
|
||||||
|
repo_id: uuid.UUID,
|
||||||
|
request: Request,
|
||||||
|
caller: Annotated[Caller, Depends(get_caller)],
|
||||||
|
) -> dict:
|
||||||
|
async with _sessionmaker(request)() as session:
|
||||||
|
repo = await _load_repo(session, repo_id, caller)
|
||||||
|
owner, slug = _repo_owner_login(repo), repo.slug
|
||||||
|
|
||||||
|
status = await MirrorService(request.app.state.settings).status(owner, slug)
|
||||||
|
speak = {
|
||||||
|
"healthy": "A second copy of this project is up to date.",
|
||||||
|
"degraded": "The second copy is behind. Your work here is safe.",
|
||||||
|
"absent": "There is no second copy of this project yet.",
|
||||||
|
"pending": "The second copy is set up and hasn't run yet.",
|
||||||
|
"unconfigured": "Off-site copies aren't switched on yet.",
|
||||||
|
"unknown": "We can't tell how the second copy is doing right now.",
|
||||||
|
}[status["state"]]
|
||||||
|
return {**status, "speak": speak}
|
||||||
175
api/app/services/gitea_client.py
Normal file
175
api/app/services/gitea_client.py
Normal file
@@ -0,0 +1,175 @@
|
|||||||
|
"""The Gitea membrane (I-1 / D-2).
|
||||||
|
|
||||||
|
Everything this cell needs from Gitea goes through this file and through Gitea's
|
||||||
|
REST API. Nothing else in the codebase imports Gitea concepts, and nothing
|
||||||
|
anywhere writes Gitea's database directly — it has its own role and its own
|
||||||
|
database precisely so that boundary is a permission rather than a promise.
|
||||||
|
|
||||||
|
Keeping the dependency here is what makes D-2 affordable: Gitea ships every two
|
||||||
|
or three months including security fixes, and a diverged fork becomes the whole
|
||||||
|
job within a year for a small team. One file is a seam. A merged source tree is
|
||||||
|
a marriage.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import secrets
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from api.app.config import Settings
|
||||||
|
from api.app.errors import RepairPointer, provider_unconfigured
|
||||||
|
|
||||||
|
_TIMEOUT = httpx.Timeout(20.0, connect=5.0)
|
||||||
|
|
||||||
|
|
||||||
|
class GiteaClient:
|
||||||
|
def __init__(self, settings: Settings) -> None:
|
||||||
|
self._s = settings
|
||||||
|
|
||||||
|
def _require(self) -> None:
|
||||||
|
if not self._s.gitea_configured:
|
||||||
|
raise provider_unconfigured("gitea", "GITEA_ADMIN_TOKEN")
|
||||||
|
|
||||||
|
def _headers(self) -> dict[str, str]:
|
||||||
|
return {
|
||||||
|
"Authorization": f"token {self._s.gitea_admin_token}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _request(self, method: str, path: str, **kw: Any) -> httpx.Response:
|
||||||
|
self._require()
|
||||||
|
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||||
|
return await client.request(
|
||||||
|
method,
|
||||||
|
f"{self._s.gitea_base_url}/api/v1{path}",
|
||||||
|
headers=self._headers(),
|
||||||
|
**kw,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---- users ------------------------------------------------------------
|
||||||
|
async def ensure_user(self, username: str, email: str) -> dict:
|
||||||
|
"""Idempotent. Gitea is the component; our `repos` table is the truth."""
|
||||||
|
r = await self._request("GET", f"/users/{username}")
|
||||||
|
if r.status_code == 200:
|
||||||
|
return r.json()
|
||||||
|
# Gitea requires a password field on admin user-create and rejects null
|
||||||
|
# with a bare 400. Nobody ever uses this one: humans arrive through OIDC
|
||||||
|
# and agents through scoped passport-bound tokens, and local password
|
||||||
|
# sign-in is disabled server-wide. So we generate a credential that is
|
||||||
|
# never stored, never returned and never recoverable — an unusable
|
||||||
|
# password is safer than a blank one or a shared default.
|
||||||
|
r = await self._request(
|
||||||
|
"POST",
|
||||||
|
"/admin/users",
|
||||||
|
json={
|
||||||
|
"username": username,
|
||||||
|
"email": email,
|
||||||
|
"password": secrets.token_urlsafe(48),
|
||||||
|
"must_change_password": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if r.status_code not in (200, 201):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="gitea_user_create_failed",
|
||||||
|
speak="We couldn't finish setting up that account. Nothing was lost.",
|
||||||
|
machine_cause=f"POST /admin/users -> {r.status_code}: {r.text[:200]}",
|
||||||
|
remediation_tool="windy_git.repair.retry_user_create",
|
||||||
|
)
|
||||||
|
return r.json()
|
||||||
|
|
||||||
|
# ---- repos ------------------------------------------------------------
|
||||||
|
async def create_repo(
|
||||||
|
self, owner: str, name: str, description: str, private: bool, default_branch: str
|
||||||
|
) -> dict:
|
||||||
|
r = await self._request(
|
||||||
|
"POST",
|
||||||
|
f"/admin/users/{owner}/repos",
|
||||||
|
json={
|
||||||
|
"name": name,
|
||||||
|
"description": description,
|
||||||
|
"private": private,
|
||||||
|
"auto_init": True,
|
||||||
|
"default_branch": default_branch,
|
||||||
|
# G4.5 — the LFS threshold is load-bearing, not tidiness. A large
|
||||||
|
# blob inside a git pack cannot be resumed or offloaded, and a
|
||||||
|
# plain push of one dies at Cloudflare's ~100s ceiling (G4A.5).
|
||||||
|
"gitignores": "",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if r.status_code not in (200, 201):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502 if r.status_code >= 500 else 409,
|
||||||
|
code="repo_create_failed",
|
||||||
|
speak="We couldn't create that project. Try a different name.",
|
||||||
|
machine_cause=f"POST /admin/users/{owner}/repos -> {r.status_code}: {r.text[:200]}",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
return r.json()
|
||||||
|
|
||||||
|
async def delete_repo(self, owner: str, name: str) -> None:
|
||||||
|
r = await self._request("DELETE", f"/repos/{owner}/{name}")
|
||||||
|
if r.status_code not in (204, 404):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="repo_delete_failed",
|
||||||
|
speak="We couldn't remove that project. It is still there and still yours.",
|
||||||
|
machine_cause=f"DELETE /repos/{owner}/{name} -> {r.status_code}",
|
||||||
|
remediation_tool="windy_git.repair.retry_delete",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def get_repo(self, owner: str, name: str) -> dict | None:
|
||||||
|
r = await self._request("GET", f"/repos/{owner}/{name}")
|
||||||
|
return r.json() if r.status_code == 200 else None
|
||||||
|
|
||||||
|
# ---- history (G5.5 / G5.6) -------------------------------------------
|
||||||
|
async def list_commits(self, owner: str, name: str, limit: int = 50) -> list[dict]:
|
||||||
|
r = await self._request(
|
||||||
|
"GET", f"/repos/{owner}/{name}/commits", params={"limit": limit}
|
||||||
|
)
|
||||||
|
if r.status_code == 409:
|
||||||
|
return [] # empty repo — a real state, not an error
|
||||||
|
if r.status_code != 200:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="history_unavailable",
|
||||||
|
speak="We couldn't load the history for that project just now.",
|
||||||
|
machine_cause=f"GET commits -> {r.status_code}",
|
||||||
|
remediation_tool="windy_git.repair.rebuild_index",
|
||||||
|
)
|
||||||
|
return r.json()
|
||||||
|
|
||||||
|
# ---- collaborators (the shelter's enforcement half, G5.3) ------------
|
||||||
|
async def put_collaborator(self, owner: str, name: str, user: str, permission: str) -> None:
|
||||||
|
r = await self._request(
|
||||||
|
"PUT",
|
||||||
|
f"/repos/{owner}/{name}/collaborators/{user}",
|
||||||
|
json={"permission": permission},
|
||||||
|
)
|
||||||
|
if r.status_code not in (204, 201, 200):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="grant_apply_failed",
|
||||||
|
speak="We couldn't share that project yet. Nobody was given access.",
|
||||||
|
machine_cause=f"PUT collaborator -> {r.status_code}: {r.text[:200]}",
|
||||||
|
remediation_tool="windy_git.repair.resync_grants",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def delete_collaborator(self, owner: str, name: str, user: str) -> None:
|
||||||
|
r = await self._request("DELETE", f"/repos/{owner}/{name}/collaborators/{user}")
|
||||||
|
if r.status_code not in (204, 404):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="grant_revoke_failed",
|
||||||
|
# The honest failure: we could not take access away. That is the
|
||||||
|
# scarier direction, so it is stated plainly rather than softened.
|
||||||
|
speak="We could not remove that person's access. Please try again.",
|
||||||
|
machine_cause=f"DELETE collaborator -> {r.status_code}",
|
||||||
|
remediation_tool="windy_git.repair.resync_grants",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def version(self) -> str:
|
||||||
|
r = await self._request("GET", "/version")
|
||||||
|
return r.json().get("version", "unknown")
|
||||||
179
api/app/services/mirror.py
Normal file
179
api/app/services/mirror.py
Normal file
@@ -0,0 +1,179 @@
|
|||||||
|
"""I-4 — never a one-way door (strand G11).
|
||||||
|
|
||||||
|
Every repo push-mirrors to GitHub, continuously, from the first save. This is
|
||||||
|
not a nicety and it is not a migration step: it is the thing that makes moving
|
||||||
|
off GitHub a reversible decision rather than a bet.
|
||||||
|
|
||||||
|
Today GitHub's durability is free to Grant. The moment repos live only here,
|
||||||
|
backups, restore rehearsal and a second copy stop being someone else's job — and
|
||||||
|
the August audits found **no rehearsed restore anywhere in the ecosystem**, for
|
||||||
|
anything. A continuous mirror buys back that safety for zero dollars.
|
||||||
|
|
||||||
|
Mirror health is a monitored, alerting signal. A mirror nobody checks is a
|
||||||
|
belief, not a backup — and this ecosystem has already learned that lesson the
|
||||||
|
expensive way with a fleet canary that sat dead for 37 days while everything
|
||||||
|
downstream assumed it was fine.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from api.app.config import Settings
|
||||||
|
from api.app.errors import RepairPointer
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_TIMEOUT = httpx.Timeout(30.0, connect=10.0)
|
||||||
|
|
||||||
|
|
||||||
|
class MirrorService:
|
||||||
|
"""Creates the GitHub counterpart and asks Gitea to keep it in step.
|
||||||
|
|
||||||
|
Gitea owns the actual replication (`push_mirrors`), because a hand-rolled
|
||||||
|
mirror loop is a background job that fails silently — which is precisely how
|
||||||
|
the registry's integrity refresh went its entire life calling a 404 and
|
||||||
|
incrementing a counter instead of raising.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, settings: Settings) -> None:
|
||||||
|
self._s = settings
|
||||||
|
|
||||||
|
@property
|
||||||
|
def configured(self) -> bool:
|
||||||
|
return bool(self._s.github_token and self._s.github_owner)
|
||||||
|
|
||||||
|
def _require(self) -> None:
|
||||||
|
if not self.configured:
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=503,
|
||||||
|
code="mirror_unconfigured",
|
||||||
|
speak="The off-site copy isn't switched on yet.",
|
||||||
|
machine_cause="GITHUB_TOKEN or GITHUB_OWNER is unset; refusing to claim a mirror",
|
||||||
|
remediation_tool=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def ensure_github_repo(self, name: str, description: str, private: bool) -> str:
|
||||||
|
"""Idempotent. Returns the clone URL of the off-site copy."""
|
||||||
|
self._require()
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {self._s.github_token}",
|
||||||
|
"Accept": "application/vnd.github+json",
|
||||||
|
}
|
||||||
|
owner = self._s.github_owner
|
||||||
|
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||||
|
existing = await client.get(
|
||||||
|
f"https://api.github.com/repos/{owner}/{name}", headers=headers
|
||||||
|
)
|
||||||
|
if existing.status_code == 200:
|
||||||
|
return existing.json()["clone_url"]
|
||||||
|
|
||||||
|
created = await client.post(
|
||||||
|
"https://api.github.com/user/repos",
|
||||||
|
headers=headers,
|
||||||
|
json={
|
||||||
|
"name": name,
|
||||||
|
"description": f"{description} (Windy Git mirror)".strip(),
|
||||||
|
"private": private,
|
||||||
|
"auto_init": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if created.status_code not in (200, 201):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="mirror_target_failed",
|
||||||
|
speak="We couldn't set up the off-site copy. Your work here is safe.",
|
||||||
|
machine_cause=f"POST /user/repos -> {created.status_code}: {created.text[:200]}",
|
||||||
|
remediation_tool="windy_git.repair.resync_mirror",
|
||||||
|
)
|
||||||
|
return created.json()["clone_url"]
|
||||||
|
|
||||||
|
async def attach_push_mirror(self, owner: str, repo: str, remote_url: str) -> None:
|
||||||
|
"""Ask Gitea to keep the off-site copy in step on every save."""
|
||||||
|
self._require()
|
||||||
|
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||||
|
r = await client.post(
|
||||||
|
f"{self._s.gitea_base_url}/api/v1/repos/{owner}/{repo}/push_mirrors",
|
||||||
|
headers={
|
||||||
|
"Authorization": f"token {self._s.gitea_admin_token}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
json={
|
||||||
|
"remote_address": remote_url,
|
||||||
|
"remote_username": self._s.github_owner,
|
||||||
|
"remote_password": self._s.github_token,
|
||||||
|
"interval": self._s.mirror_interval,
|
||||||
|
# The important one: mirror on every save, not just on a timer.
|
||||||
|
# An hourly timer means an hour of work can be the thing you
|
||||||
|
# lose, and the window is invisible until it costs you.
|
||||||
|
"sync_on_commit": True,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if r.status_code not in (200, 201):
|
||||||
|
raise RepairPointer(
|
||||||
|
status_code=502,
|
||||||
|
code="mirror_attach_failed",
|
||||||
|
speak="We couldn't keep the off-site copy in step. Your work here is safe.",
|
||||||
|
machine_cause=f"POST push_mirrors -> {r.status_code}: {r.text[:200]}",
|
||||||
|
remediation_tool="windy_git.repair.resync_mirror",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def status(self, owner: str, repo: str) -> dict:
|
||||||
|
"""Report what is TRUE, including 'we do not know'.
|
||||||
|
|
||||||
|
`me-fleet.ts:22-25` in a sibling service refuses to say "online" when it
|
||||||
|
only knows "registered". Same posture here: an unconfigured mirror is
|
||||||
|
reported as unknown, never as healthy.
|
||||||
|
"""
|
||||||
|
if not self.configured:
|
||||||
|
return {"state": "unconfigured", "lag_seconds": None, "last_success_at": None}
|
||||||
|
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||||
|
r = await client.get(
|
||||||
|
f"{self._s.gitea_base_url}/api/v1/repos/{owner}/{repo}/push_mirrors",
|
||||||
|
headers={"Authorization": f"token {self._s.gitea_admin_token}"},
|
||||||
|
)
|
||||||
|
if r.status_code != 200:
|
||||||
|
return {"state": "unknown", "detail": f"gitea -> {r.status_code}"}
|
||||||
|
|
||||||
|
mirrors = r.json()
|
||||||
|
if not mirrors:
|
||||||
|
return {"state": "absent", "lag_seconds": None, "last_success_at": None}
|
||||||
|
|
||||||
|
m = mirrors[0]
|
||||||
|
last = m.get("last_update") or m.get("last_updated")
|
||||||
|
lag = None
|
||||||
|
if last:
|
||||||
|
try:
|
||||||
|
lag = int(
|
||||||
|
(datetime.now(UTC) - datetime.fromisoformat(last.replace("Z", "+00:00")))
|
||||||
|
.total_seconds()
|
||||||
|
)
|
||||||
|
except ValueError:
|
||||||
|
lag = None
|
||||||
|
|
||||||
|
# A mirror that has NEVER run is not the same thing as one that is
|
||||||
|
# behind, and collapsing the two is how a backup that was never made
|
||||||
|
# gets read as a backup that is merely stale. Gitea reports the epoch
|
||||||
|
# for "not yet", which arithmetic turns into a 56-year lag and a
|
||||||
|
# confident "degraded".
|
||||||
|
never_synced = not last or last.startswith("1970-01-01")
|
||||||
|
|
||||||
|
# I-4: lag over the threshold is a P2, not a shrug.
|
||||||
|
if never_synced:
|
||||||
|
state = "pending"
|
||||||
|
lag = None
|
||||||
|
elif lag is None:
|
||||||
|
state = "unknown"
|
||||||
|
elif lag > self._s.mirror_lag_p2_seconds:
|
||||||
|
state = "degraded"
|
||||||
|
else:
|
||||||
|
state = "healthy"
|
||||||
|
return {
|
||||||
|
"state": state,
|
||||||
|
"lag_seconds": lag,
|
||||||
|
"last_success_at": None if never_synced else last,
|
||||||
|
"remote": m.get("remote_address"),
|
||||||
|
}
|
||||||
@@ -234,3 +234,217 @@ def test_g05_no_secret_literals_committed():
|
|||||||
text = path.read_text(encoding="utf-8", errors="ignore")
|
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||||
for pattern in patterns:
|
for pattern in patterns:
|
||||||
assert not pattern.search(text), f"credential literal in {path}"
|
assert not pattern.search(text), f"credential literal in {path}"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# G2.1 / G2.7 — the Gitea version is PINNED, and drift is a failure
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def test_g21_gitea_version_is_pinned_not_latest():
|
||||||
|
compose = (ROOT / "docker-compose.yml").read_text()
|
||||||
|
m = re.search(r"image:\s*\S*gitea/gitea:(\S+)", compose)
|
||||||
|
assert m, "no gitea image pin found"
|
||||||
|
assert m.group(1) != "latest", "G2.1: pin an exact Gitea version, never `latest`"
|
||||||
|
assert re.match(r"^\d+\.\d+\.\d+$", m.group(1)), f"not an exact version: {m.group(1)}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_g24_gitea_license_travels_with_us():
|
||||||
|
"""MIT's one obligation. Cheap to honour, embarrassing to miss."""
|
||||||
|
assert (ROOT / "LICENSES" / "gitea-MIT.txt").exists()
|
||||||
|
assert "MIT" in (ROOT / "LICENSES" / "gitea-MIT.txt").read_text()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# G4.3 — the two Gitea storage traps that cost a crash loop each
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def test_g43_no_lfs_storage_type_override():
|
||||||
|
"""Naming a storage type inside [lfs] creates a SEPARATE storage section
|
||||||
|
that does not inherit endpoint or credentials from [storage], and Gitea
|
||||||
|
crash-loops with an error that names the symptom and not the cause."""
|
||||||
|
# Check real settings only — the compose file deliberately NAMES this key in
|
||||||
|
# a warning comment so the next person does not re-add it.
|
||||||
|
active = [
|
||||||
|
ln for ln in (ROOT / "docker-compose.yml").read_text().splitlines()
|
||||||
|
if ln.strip() and not ln.strip().startswith("#")
|
||||||
|
]
|
||||||
|
assert not any("GITEA__lfs__STORAGE_TYPE" in ln for ln in active)
|
||||||
|
|
||||||
|
|
||||||
|
def test_g43_lfs_server_is_actually_enabled():
|
||||||
|
"""Setting the storage backend does NOT turn LFS on. Without this the batch
|
||||||
|
endpoint 404s and the client says 'Repository or object not found', which
|
||||||
|
reads like a permissions problem and is not one."""
|
||||||
|
active = [
|
||||||
|
ln for ln in (ROOT / "docker-compose.yml").read_text().splitlines()
|
||||||
|
if ln.strip() and not ln.strip().startswith("#")
|
||||||
|
]
|
||||||
|
assert any("GITEA__server__LFS_START_SERVER" in ln for ln in active)
|
||||||
|
|
||||||
|
|
||||||
|
def test_g43_r2_checksum_trap_is_pinned():
|
||||||
|
"""R2 rejects the checksum algorithm S3 clients send by default."""
|
||||||
|
compose = (ROOT / "docker-compose.yml").read_text()
|
||||||
|
assert "MINIO_CHECKSUM_ALGORITHM" in compose
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# G3.6 — the trust status-code law. This is the one with a live sibling bypass.
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def test_g36_trust_client_never_soft_allows():
|
||||||
|
"""A sibling maps HTTP 400 and 429 to 'unreachable' and then soft-ALLOWS.
|
||||||
|
That is inducible: an attacker who wants the check skipped only has to make
|
||||||
|
the check rate-limit itself at 100/min/IP."""
|
||||||
|
src = (ROOT / "api" / "app" / "auth.py").read_text()
|
||||||
|
assert "raise passport_unresolvable" in src
|
||||||
|
# There must be no return path out of resolve_passport other than a verified
|
||||||
|
# 200 or a raise.
|
||||||
|
body = src[src.index("async def resolve_passport") : src.index("async def get_caller")]
|
||||||
|
returns = [ln for ln in body.splitlines() if ln.strip().startswith("return")]
|
||||||
|
assert len(returns) == 1, f"resolve_passport has {len(returns)} return paths; expected exactly 1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_g36_unverified_human_jwt_is_refused_in_production():
|
||||||
|
"""I-8 applied to ourselves: an unverified JWT is an authentication bypass,
|
||||||
|
not a shortcut. Until G3.2's JWKS verifier exists, production refuses."""
|
||||||
|
from api.app.config import Settings
|
||||||
|
|
||||||
|
assert Settings().require_verified_jwt is True
|
||||||
|
src = (ROOT / "api" / "app" / "auth.py").read_text()
|
||||||
|
assert "human_signin_not_ready" in src
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_auth_bypass_env_var_anywhere():
|
||||||
|
"""The desktop control server is the best Principle-#5 artifact in the
|
||||||
|
ecosystem partly because it has NO bypass env var. Copied on purpose."""
|
||||||
|
src = (ROOT / "api" / "app" / "auth.py").read_text()
|
||||||
|
for banned in ("SKIP_AUTH", "DISABLE_AUTH", "ALLOW_INSECURE", "AUTH_BYPASS", "DEV_MODE"):
|
||||||
|
assert banned not in src
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# G5.3 — the shelter's grant model
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def test_g53_grant_requires_exactly_one_grantee_in_the_database():
|
||||||
|
"""Enforced by a CHECK constraint, not by application code. This ecosystem
|
||||||
|
already has a core invariant enforced only in app code across two files."""
|
||||||
|
src = (ROOT / "alembic" / "versions" / "001_genesis.py").read_text()
|
||||||
|
assert "ck_grant_exactly_one_grantee" in src
|
||||||
|
assert "(grantee_identity_id IS NULL) <> (grantee_passport IS NULL)" in src
|
||||||
|
|
||||||
|
|
||||||
|
def test_g53_agent_grants_expire_by_default():
|
||||||
|
from api.app.config import Settings
|
||||||
|
|
||||||
|
assert Settings().agent_grant_default_days == 90
|
||||||
|
|
||||||
|
|
||||||
|
def test_g55_shelter_strings_avoid_developer_vocabulary():
|
||||||
|
"""D-9/I-9: a person restoring last Tuesday's work should not have to learn
|
||||||
|
a vocabulary first. Check the strings users actually see."""
|
||||||
|
import re as _re
|
||||||
|
|
||||||
|
src = (ROOT / "api" / "app" / "routes" / "repos.py").read_text()
|
||||||
|
speaks = _re.findall(r'"speak":\s*\(?\s*\n?\s*f?"([^"]+)"', src)
|
||||||
|
assert speaks, "no speak strings found to audit"
|
||||||
|
for s in speaks:
|
||||||
|
low = s.lower()
|
||||||
|
for jargon in ("commit", "repository", "branch", "sha", "push"):
|
||||||
|
assert jargon not in low, f"developer vocabulary in a user string: {s!r}"
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# I-4 — never a one-way door
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def test_i04_mirror_syncs_on_every_save_not_just_a_timer():
|
||||||
|
"""An hourly window means an hour of work can be the thing you lose, and the
|
||||||
|
window is invisible until it costs you."""
|
||||||
|
src = (ROOT / "api" / "app" / "services" / "mirror.py").read_text()
|
||||||
|
assert '"sync_on_commit": True' in src
|
||||||
|
|
||||||
|
|
||||||
|
def test_i04_unconfigured_mirror_is_never_reported_healthy():
|
||||||
|
"""A mirror nobody checks is a belief, not a backup. An unconfigured one
|
||||||
|
reports 'unconfigured' — never 'healthy'."""
|
||||||
|
src = (ROOT / "api" / "app" / "services" / "mirror.py").read_text()
|
||||||
|
assert '"state": "unconfigured"' in src
|
||||||
|
assert "if not self.configured:" in src
|
||||||
|
|
||||||
|
|
||||||
|
def test_i04_mirror_lag_threshold_is_set():
|
||||||
|
from api.app.config import Settings
|
||||||
|
|
||||||
|
assert Settings().mirror_lag_p2_seconds == 3600
|
||||||
|
|
||||||
|
|
||||||
|
def test_owner_namespace_is_derived_from_the_repo_not_the_caller():
|
||||||
|
"""Deriving the namespace from the caller is right only while the caller is
|
||||||
|
the owner, and addresses the wrong namespace the moment a collaborator asks
|
||||||
|
— surfacing as 'not found', which is the hardest kind of bug to see."""
|
||||||
|
src = (ROOT / "api" / "app" / "routes" / "repos.py").read_text()
|
||||||
|
body = src[src.index("async def list_versions") : src.index("async def create_grant")]
|
||||||
|
assert "_repo_owner_login(repo)" in body
|
||||||
|
assert "_owner_login(caller)" not in body
|
||||||
|
|
||||||
|
|
||||||
|
def test_i04_never_synced_is_not_reported_as_merely_behind():
|
||||||
|
"""Collapsing 'never ran' into 'behind' is how a backup that was never made
|
||||||
|
gets read as a backup that is merely stale. Gitea reports the epoch for
|
||||||
|
'not yet', which arithmetic turns into a 56-year lag and a confident
|
||||||
|
'degraded'."""
|
||||||
|
src = (ROOT / "api" / "app" / "services" / "mirror.py").read_text()
|
||||||
|
assert "never_synced" in src
|
||||||
|
assert '"pending"' in src
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# G7 / I-5 — CI never shares a kernel with identity
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def test_i05_runner_never_mounts_the_host_docker_socket():
|
||||||
|
"""The tempting move — and what every published act_runner example does —
|
||||||
|
is to mount /var/run/docker.sock. That hands every workflow, including a
|
||||||
|
transitive dependency's postinstall script, the ability to start a
|
||||||
|
privileged container mounting / — i.e. root on the host."""
|
||||||
|
compose = (ROOT / "deploy" / "runner" / "docker-compose.yml").read_text()
|
||||||
|
active = [ln for ln in compose.splitlines() if ln.strip() and not ln.strip().startswith("#")]
|
||||||
|
for ln in active:
|
||||||
|
assert "/var/run/docker.sock" not in ln, "I-5: never mount the host docker socket"
|
||||||
|
|
||||||
|
|
||||||
|
def test_i05_jobs_cannot_bind_mount_from_the_daemon_host():
|
||||||
|
cfg = (ROOT / "deploy" / "runner" / "config.yaml").read_text()
|
||||||
|
assert "valid_volumes: []" in cfg
|
||||||
|
assert 'docker_host: "-"' in cfg
|
||||||
|
|
||||||
|
|
||||||
|
def test_i05_runner_is_a_separate_compose_project_from_the_forge():
|
||||||
|
"""Runners restart, crash, get starved and get killed. None of that should
|
||||||
|
ever touch the thing serving repositories."""
|
||||||
|
runner = (ROOT / "deploy" / "runner" / "docker-compose.yml").read_text()
|
||||||
|
forge = (ROOT / "docker-compose.yml").read_text()
|
||||||
|
assert "name: windy-git-runner" in runner
|
||||||
|
assert "name: windy-git" in forge
|
||||||
|
|
||||||
|
|
||||||
|
def test_g15_runner_is_cpu_and_memory_bounded():
|
||||||
|
"""Veron 1 is Grant's workstation, not a dedicated build box."""
|
||||||
|
compose = (ROOT / "deploy" / "runner" / "docker-compose.yml").read_text()
|
||||||
|
assert "cpus:" in compose
|
||||||
|
assert "mem_limit:" in compose
|
||||||
|
|
||||||
|
|
||||||
|
def test_g75_workflows_use_a_label_this_runner_actually_provides():
|
||||||
|
"""A workflow naming a label nobody provides queues forever and presents as
|
||||||
|
a hung CI system rather than a typo."""
|
||||||
|
cfg = (ROOT / "deploy" / "runner" / "config.yaml").read_text()
|
||||||
|
provided = {
|
||||||
|
ln.split(":")[0].strip().strip('"- ')
|
||||||
|
for ln in cfg.splitlines()
|
||||||
|
if "docker://" in ln
|
||||||
|
}
|
||||||
|
assert provided, "runner declares no labels"
|
||||||
|
for wf in ROOT.rglob(".gitea/workflows/*.y*ml"):
|
||||||
|
for ln in wf.read_text().splitlines():
|
||||||
|
# Skip comments — a doc line explaining runs-on is not a runs-on.
|
||||||
|
if ln.strip().startswith("#") or "runs-on:" not in ln:
|
||||||
|
continue
|
||||||
|
label = ln.split("runs-on:")[1].strip()
|
||||||
|
assert label in provided, f"{wf.name}: '{label}' is not a provided label"
|
||||||
|
|||||||
38
deploy/runner/config.yaml
Normal file
38
deploy/runner/config.yaml
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
# act_runner configuration (G7.1).
|
||||||
|
#
|
||||||
|
# Labels are EXPLICIT and PINNED. `ubuntu-latest` is banned (G7.5): all four
|
||||||
|
# windy-registry workflows use it and every single run fails, because a
|
||||||
|
# self-hosted runner has no such label unless you invent one. A workflow that
|
||||||
|
# names a label nobody provides queues forever and looks like a hung CI system
|
||||||
|
# rather than a typo.
|
||||||
|
|
||||||
|
log:
|
||||||
|
level: info
|
||||||
|
|
||||||
|
runner:
|
||||||
|
file: /data/.runner
|
||||||
|
capacity: 4 # concurrent jobs; Veron has 24 cores, dind is capped at 12
|
||||||
|
timeout: 30m
|
||||||
|
shutdown_timeout: 3m
|
||||||
|
insecure: false
|
||||||
|
fetch_timeout: 5s
|
||||||
|
fetch_interval: 2s
|
||||||
|
labels:
|
||||||
|
- "veron-1:docker://catthehacker/ubuntu:act-22.04"
|
||||||
|
- "linux-x64:docker://catthehacker/ubuntu:act-22.04"
|
||||||
|
|
||||||
|
cache:
|
||||||
|
enabled: true
|
||||||
|
dir: /data/cache
|
||||||
|
|
||||||
|
container:
|
||||||
|
# Job containers join the dind daemon's own bridge. NOT the forge network:
|
||||||
|
# untrusted code must never be able to reach the forge's Postgres or its
|
||||||
|
# environment (I-5).
|
||||||
|
network: bridge
|
||||||
|
privileged: false
|
||||||
|
options:
|
||||||
|
workdir_parent: /workspace
|
||||||
|
valid_volumes: [] # a job cannot bind-mount anything from the daemon host
|
||||||
|
docker_host: "-" # do NOT expose the runner's own docker socket to jobs
|
||||||
|
force_pull: false
|
||||||
81
deploy/runner/docker-compose.yml
Normal file
81
deploy/runner/docker-compose.yml
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
# CI runners (strand G7) — a SEPARATE compose project from the forge.
|
||||||
|
#
|
||||||
|
# Separate on purpose: runners restart, crash, get starved and get killed. None
|
||||||
|
# of that should ever touch the thing serving repositories. This is the cell
|
||||||
|
# doctrine applied one level down.
|
||||||
|
#
|
||||||
|
# ── I-5, and why there is a dind sidecar ───────────────────────────────────
|
||||||
|
#
|
||||||
|
# "CI never shares a kernel with identity. Runners execute untrusted code and
|
||||||
|
# are isolated by machine boundary, not container boundary. No runner may hold
|
||||||
|
# a credential scoped beyond its own job."
|
||||||
|
#
|
||||||
|
# act_runner needs a Docker daemon to start job containers. The tempting move is
|
||||||
|
# to mount the host's `/var/run/docker.sock`. That would hand every workflow —
|
||||||
|
# including whatever a transitive dependency's postinstall script feels like
|
||||||
|
# doing — the ability to start a privileged container mounting `/`, which is
|
||||||
|
# root on Veron 1. Every published act_runner example does exactly this.
|
||||||
|
#
|
||||||
|
# Instead the runner talks to its OWN daemon (`dind`). Untrusted job code runs
|
||||||
|
# as a child of that daemon, on an isolated network, with no route to the host
|
||||||
|
# socket and no route to the forge's database.
|
||||||
|
#
|
||||||
|
# The split that makes this work:
|
||||||
|
# * `runner` is TRUSTED code (the act_runner daemon). It sits on the forge
|
||||||
|
# network only so it can reach gitea:3000 to collect jobs.
|
||||||
|
# * `dind` and every job container it spawns are UNTRUSTED. They are on a
|
||||||
|
# private network with no access to the forge, its database, or its .env.
|
||||||
|
#
|
||||||
|
# dind itself is privileged — that is the cost, and it is the reason a job
|
||||||
|
# escape lands in a disposable daemon rather than on Grant's workstation.
|
||||||
|
#
|
||||||
|
# ⚠️ Do NOT "simplify" this by mounting the host docker socket.
|
||||||
|
|
||||||
|
name: windy-git-runner
|
||||||
|
|
||||||
|
services:
|
||||||
|
dind:
|
||||||
|
image: docker.io/library/docker:27-dind
|
||||||
|
privileged: true
|
||||||
|
environment:
|
||||||
|
DOCKER_TLS_CERTDIR: "" # plain TCP on an isolated network, no host route
|
||||||
|
command: ["dockerd", "--host=tcp://0.0.0.0:2375", "--tls=false"]
|
||||||
|
networks: [jobs]
|
||||||
|
volumes:
|
||||||
|
- dind-storage:/var/lib/docker
|
||||||
|
# G1.5 — bounded so a fork-bomb workflow cannot starve Grant's interactive
|
||||||
|
# session. Veron 1 is his workstation, not a dedicated build box.
|
||||||
|
cpus: 12.0 # 12 of 24 cores
|
||||||
|
mem_limit: 64g
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
runner:
|
||||||
|
image: docker.io/gitea/act_runner:0.2.11
|
||||||
|
depends_on: [dind]
|
||||||
|
environment:
|
||||||
|
# The runner reaches its OWN daemon. Never the host's.
|
||||||
|
DOCKER_HOST: tcp://dind:2375
|
||||||
|
GITEA_INSTANCE_URL: http://gitea:3000
|
||||||
|
GITEA_RUNNER_REGISTRATION_TOKEN: ${RUNNER_TOKEN:?set RUNNER_TOKEN}
|
||||||
|
GITEA_RUNNER_NAME: veron-1
|
||||||
|
CONFIG_FILE: /config.yaml
|
||||||
|
volumes:
|
||||||
|
- ./config.yaml:/config.yaml:ro
|
||||||
|
- runner-data:/data
|
||||||
|
networks: [jobs, forge]
|
||||||
|
cpus: 2.0
|
||||||
|
mem_limit: 4g
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
networks:
|
||||||
|
jobs:
|
||||||
|
# Untrusted job containers live here. No route to the forge.
|
||||||
|
internal: false # jobs legitimately need to fetch dependencies
|
||||||
|
forge:
|
||||||
|
# Pre-existing network owned by the forge compose project.
|
||||||
|
external: true
|
||||||
|
name: windy-git_default
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
dind-storage:
|
||||||
|
runner-data:
|
||||||
@@ -57,6 +57,17 @@ services:
|
|||||||
# G2.2 — OIDC only. No local password login, no self-registration.
|
# G2.2 — OIDC only. No local password login, no self-registration.
|
||||||
GITEA__service__DISABLE_REGISTRATION: "true"
|
GITEA__service__DISABLE_REGISTRATION: "true"
|
||||||
GITEA__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "true"
|
GITEA__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "true"
|
||||||
|
# G3.1 — a Windy account IS the account. Signing in with Windy provisions
|
||||||
|
# the Gitea user on first arrival; nobody is asked to invent a second
|
||||||
|
# identity for the same person, and no local password ever exists.
|
||||||
|
GITEA__oauth2_client__ENABLE_AUTO_REGISTRATION: "true"
|
||||||
|
GITEA__oauth2_client__USERNAME: email
|
||||||
|
GITEA__oauth2_client__ACCOUNT_LINKING: auto
|
||||||
|
# The email is asserted by account-server, which is the authority on it.
|
||||||
|
# Asking the user to re-verify an address their identity provider already
|
||||||
|
# verified is friction that buys nothing.
|
||||||
|
GITEA__oauth2_client__UPDATE_AVATAR: "false"
|
||||||
|
GITEA__service__REGISTER_EMAIL_CONFIRM: "false"
|
||||||
# G4.3 — heavy bytes to R2 at ZERO egress. Git object databases stay on
|
# G4.3 — heavy bytes to R2 at ZERO egress. Git object databases stay on
|
||||||
# local NVMe (I-3); this covers LFS, attachments, packages, avatars and
|
# local NVMe (I-3); this covers LFS, attachments, packages, avatars and
|
||||||
# Actions artifacts, which is where GitHub's painful bills actually come
|
# Actions artifacts, which is where GitHub's painful bills actually come
|
||||||
|
|||||||
@@ -17,6 +17,11 @@ dependencies = [
|
|||||||
"pydantic-settings>=2.6",
|
"pydantic-settings>=2.6",
|
||||||
"sqlalchemy[asyncio]>=2.0",
|
"sqlalchemy[asyncio]>=2.0",
|
||||||
"asyncpg>=0.30",
|
"asyncpg>=0.30",
|
||||||
|
# Alembic runs synchronously (env.py strips +asyncpg), so the image needs a
|
||||||
|
# sync driver too. Without it migrations fail INSIDE the container while
|
||||||
|
# passing on a developer machine that happens to have it — the kind of gap
|
||||||
|
# that only shows up on a fresh deploy.
|
||||||
|
"psycopg2-binary>=2.9",
|
||||||
"alembic>=1.14",
|
"alembic>=1.14",
|
||||||
"httpx>=0.27",
|
"httpx>=0.27",
|
||||||
"boto3>=1.35",
|
"boto3>=1.35",
|
||||||
|
|||||||
Reference in New Issue
Block a user