API reference

The control plane's HTTP contract: what each endpoint does, who may call it, and what comes back. This is a reference — look things up here rather than reading it front to back.

Verified against code: 2026-08-26.

To get a backend running first, see Change the code. Every environment variable it reads is in Configuration.

---

Default Admin

Created automatically on first startup: - Email: admin@platform.com - Password: admin - ⚠️ Must change password on first login (must_change_password: true)

---

Authentication

Three credentials, strictly separated (spec §8.0/§17):

Credential	Header	Used by	Accepted on
JWT token	Authorization: Bearer eyJ...	Frontend (after login)	/v1/* dashboard endpoints
Personal API key	Authorization: Bearer gk-xxx	Programmatic user access	/v1/files, /v1/batches (via get_human_context)
Org worker API key	Authorization: Bearer gk-xxx	Worker daemons	/workers/* only (via get_worker_context)

There is no provider role and no per-user key column: every user gets a Personal Org (with owner membership) at signup; worker keys belong to organizations and are managed from the dashboard. The backend never issues keys during worker registration — it derives the owning org from the key.

---

API Endpoints

Auth — /v1/auth

Method	Endpoint	Auth	Description
POST	/v1/auth/signup	None	Register; creates the user + their Personal Org
POST	/v1/auth/login	None	Returns JWT (platform_role, must_change_password)
GET	/v1/auth/me	JWT	Current user profile + org memberships
POST	/v1/auth/change-password	JWT	Change password
POST	/v1/auth/api-keys/regenerate	JWT/personal key	Regenerate the caller's personal API key
POST	/v1/auth/forgot-password	None	Email a reset token
POST	/v1/auth/reset-password	None	Redeem a reset token
GET	/v1/admin/users	superadmin	List all users
GET	/v1/admin/workers	superadmin	List all workers across orgs
GET	/v1/admin/organizations	superadmin	List all organizations

POST /v1/auth/signup

// Request — no role field; platform_role defaults to "user"
{
  "email": "user@example.com",
  "password": "secret123",
  "full_name": "John Doe"
}

// Response (200) — no api_key; keys are created separately
{
  "id": "user-abc123",
  "email": "user@example.com",
  "full_name": "John Doe",
  "platform_role": "user",
  "is_active": true,
  "must_change_password": false,
  "created_at": 1780000000
}

Personal API keys — /v1/users/me/api-keys

Method	Endpoint	Description
POST	/v1/users/me/api-keys	Create a personal key (raw key returned once)
GET	/v1/users/me/api-keys	List own keys (prefix only)
PUT	/v1/users/me/api-keys/{key_id}	Rename / set expiry / revoke
DELETE	/v1/users/me/api-keys/{key_id}	Revoke

Organizations — /v1/orgs (JWT)

Method	Endpoint	Access	Description
GET	/v1/orgs	member	Orgs the caller belongs to
GET	/v1/orgs/{org_id}/api-keys	member	List org worker keys (prefix only)
POST	/v1/orgs/{org_id}/api-keys	owner/admin	Create a worker key; the raw gk-… is returned once and never again
DELETE	/v1/orgs/{org_id}/api-keys/{key_id}	owner/admin	Revoke a key — every daemon using it stops registering immediately
POST	/v1/orgs/{org_id}/api-keys/regenerate	owner/admin	Rotate the org worker key
GET	/v1/orgs/{org_id}/workers	member	Org's workers incl. status, activity, live VRAM, loaded models, policy
PATCH	/v1/orgs/{org_id}/workers/{worker_id}/policy	owner/admin	Set a worker's availability window and batch-size limit — see Provider policy

Files — /v1/files (OpenAI-compatible; JWT or personal key)

Method	Endpoint	Description
POST	/v1/files	Upload a .jsonl batch input (multipart: file, purpose="batch")
GET	/v1/files/{file_id}/content	Download raw file (also used by workers for job input)

Batches — /v1/batches (OpenAI-compatible; JWT or personal key)

Method	Endpoint	Description
POST	/v1/batches	Create a batch; async JSONL validation kicks off
GET	/v1/batches/{batch_id}/events	SSE stream of validation events
GET	/v1/batches/{batch_id}	Batch detail (owner or superadmin)
GET	/v1/batches	user sees own batches; superadmin sees all

Model handling: body.model is a model catalogue id (a platform slug from GET /v1/models), not a raw runtime string. It must be consistent across all JSONL lines, and validation rejects any body.model not in the catalogue (unsupported_model). See Model catalogue.

Models — /v1/models (OpenAI-compatible; JWT or personal key)

Method	Endpoint	Description
GET	/v1/models	Selectable catalogue entries (public + caller's org)

Pool — /v1/pool (auth optional)

Method	Endpoint	Description
GET	/v1/pool/capacity	Live pool capacity, aggregate only

The only endpoint that serves anonymous callers a view of the fleet. It exists so a batch user can answer "is anyone online, and can they run my model?" without being shown — or being able to infer — whose machine is whose. Per-worker rows stay on the superadmin GET /v1/admin/workers.

{
  "object": "pool.capacity",
  "workers_online": 6,
  "workers_idle": 3,
  "workers_busy": 3,
  "gpus_online": 14,
  "vram_total_gb": 312.0,
  "vram_largest_gpu_gb": 24.0,
  "ram_total_gb": 768.0,
  "models_servable": [
    {"id": "llama3.1-8b", "display_name": "Llama 3.1 8B", "parameter_size": "8B", "loaded": true}
  ],
  "as_of": 1756400000
}

• models_servable is what the scheduler would dispatch, not what the catalogue lists: an entry appears only when some online worker hosts the artifact, satisfies that runtime's fit rule, and is inside its provider's availability window, decided by scheduler.can_serve — the same predicate POST /workers/poll matches on. loaded says whether some online worker already has the artifact in memory (schedulable rows, digest enforced — the scheduler's own predicate, so a drift row does not count): a servable-but-not-loaded model costs a load on first use.
• vram_total_gb is the fleet sum; vram_largest_gpu_gb is what one job can actually have. A batch runs on a single worker, and on a single-device runtime on a single card, so the sum describes the size of the pool and nothing about what will fit. A pool totalling 312 GB whose biggest card is 24 GB cannot run a 40 GB model.
• Online means status = "online" and a heartbeat within HEARTBEAT_TIMEOUT_SECONDS. The sweeper only flips the column once a minute, so this endpoint applies the cutoff at read time as well.
• The hardware figures are null — vram_total_gb, vram_largest_gpu_gb, ram_total_gb and gpus_online — for anonymous callers, and for everyone while fewer than MIN_WORKERS_FOR_HARDWARE (3) workers are online: on a thin pool "141 GB" names a specific machine, which is the disclosure the aggregate exists to prevent. They travel together for that reason. GPU names are never returned at all.
• Org-private catalogue entries appear only for members of that org (superadmins see all), matching GET /v1/models.
• Never returned: hostnames, GPU names, worker ids, org ids.
• Responses are served from a 15-second in-process cache, so polling — and an unauthenticated caller hammering it — costs a constant number of queries. Poll no faster than the 30s heartbeat; anything quicker re-renders numbers that have not changed.

Workers — /workers (org worker key required)

Method	Endpoint	Description
POST	/workers/register	Register/re-register; backend assigns worker_id
POST	/workers/{worker_id}/heartbeat	Unified heartbeat: liveness + activity + dynamic capabilities
GET	/v1/models/quarantine	superadmin — worker-reported hashes the platform won't schedule (unregistered / drift), grouped by hash
POST	/v1/models/adopt	superadmin — promote a quarantined hash into a catalogue entry (status: unverified); its worker rows become schedulable at once
POST	/workers/poll	Claim the best-matching validated batch
POST	/workers/progress	Live prompt counts (time-throttled — see DAEMON_PROGRESS_INTERVAL_SECONDS)
POST	/workers/model-progress	Model download progress (logged; liveness)
POST	/workers/upload-results	Upload output JSONL + real completed/failed counts
POST	/workers/report-failure	Report failure → batch is requeued (up to 3 attempts)

All endpoints verify ownership: the worker must belong to the key's org, and result/progress/failure reports must come from the worker the batch is assigned to (403 otherwise).

POST /workers/{worker_id}/heartbeat

// activity: idle | busy | downloading_model (validated).
// Liveness (workers.status online/offline) is server-managed.
{
  "activity": "busy",
  "current_job_id": "batch-abc123",
  "progress": {"total_prompts": 100, "completed_prompts": 40, "failed_prompts": 1},
  "gpu_utilization": 87.5,
  "gpu_memory_used_gb": 14.2,
  "vram_total_gb": 24.0,
  "vram_available_gb": 9.8,
  "ram_available_gb": 41.2,
  "loaded_models": ["mistral-7b"],
  "inventory": [
    {"local_name": "mistral-7b", "sha256": "7485fe…", "size_bytes": 2497280256,
     "loaded": true, "runtime": "ollama",
     "details": {"quantization": "Q4_K_M", "parameter_size": "7.2B",
                 "context_length": 32768, "family": "llama"}}
  ],
  "runtimes": [
    {"type": "ollama", "status": "ready"},
    {"type": "vllm", "status": "unavailable"}
  ],
  "uptime_seconds": 3600
}

loaded_models and inventory[].loaded both mean resident in VRAM right now, not present on disk — Ollama reads them from /api/ps, vLLM from the models it serves. loaded_models carries no runtime tag, so on a mixed worker the backend can only apply it as a union; inventory[].loaded is tagged and therefore authoritative, and a daemon that sends no inventory keeps the union.

runtimes is each configured runtime's reachability, probed on the beat that carries it. Startup readiness cannot answer it: a provider who stops one server and starts another changes which models the worker can serve without restarting the daemon, and can_serve reads worker_runtimes.status to decide. A reported unavailable takes the runtime out of the pool on that beat; ready puts it back. draining is an operator decision rather than an observation, so a report never overwrites it, and an empty list means the daemon does not report liveness at all — the rows are then left alone, which is not the same as claiming every runtime is down.

A runtime whose liveness is reported but whose inventory is empty for two consecutive beats has its runtime_models rows marked missing, so a runtime that is up and holding nothing stops advertising what it used to hold. One beat is not enough: a single slow model listing would otherwise flap the pool. The count resets the moment inventory returns.

Two conditions gate it. The runtime must be named in runtimes — from a daemon that inventories one runtime and says nothing about the others, silence is not evidence. And some runtime must have reported artifacts on that beat: a payload carrying none at all reads the same as inventory reporting having broken, and de-listing every runtime on it costs a whole worker's capacity. One runtime reporting is enough to judge the rest.

inventory (additive; older daemons omit it) is the full on-disk artifact list with FILE hashes — Ollama manifest-layer digests, which equal the GGUF file's sha256 and join against model_catalog.digest (#116). It is resent whole every beat, so a model pulled manually on the box surfaces on the next heartbeat. Registration's runtimes[].inventory carries the same shape, so availability rows are born identity-carrying. details is what the runtime knows about the artifact — it pre-fills adopt and lets auto-adopt register a discovered model without a human. Ollama fills it from /api/tags + /api/show (cached per hash, one call per model, not per beat). vLLM fills it from the HF hub cache's config.json and adds source_ref / source_revision (repo + commit) — the cache is content-addressed, so sha256 is the first weight's blob hash — the repo's own weight_map order when it has one, else shard number — and files lists every weight ([{file, sha256, size_bytes}]); models served from a path outside the cache stay hash-less. The commit is only guessed when it is unambiguous: one snapshot dir in the cache is always served (vLLM can only run what is cached), several require a valid refs/main — otherwise the row reports no identity rather than pin the wrong commit.

Consequence: a worker advertises everything its runtime holds, not only DAEMON_MODELS — every on-disk Ollama model, every model vLLM serves — so a batch for a catalogued model may be routed to a box whose operator never listed it. Limiting what a box offers is the provider-control work in #104.

POST /workers/poll

// Request
{"worker_id": "worker-abc123"}

// Response — job found
{
  "job": {
    "job_id": "batch-abc123",
    "input_file_id": "file-abc123",
    "input_path": "/v1/files/file-abc123/content",
    "model": "mistral-7b"
  }
}

// Response — nothing compatible
{"job": null}

Matching: the scheduler offers a batch only to a worker that hosts the artifact and satisfies the fit rule of the runtime hosting it (scheduler.FIT_RULES), then prefers workers that already have the model loaded.

Fit is per-card, not per-machine: a single-device runtime needs one GPU large enough on its own, so a 2 × 12 GB worker is never offered a 20 GB model. Per-GPU sizes arrive at registration, so fit works before the first heartbeat; the machine aggregate is used only when a worker reports no per-GPU inventory. A worker that has stated no capacity at all — no GPU rows and no heartbeat — is not offered work, since every worker that can state one does so at registration (by probing, or via DAEMON_VRAM_GB).

POST /workers/upload-results

Content-Type: multipart/form-data
Fields:
  - job_id:    "batch-abc123"
  - worker_id: "worker-abc123"     (must hold the assignment)
  - completed: 98                  (optional — real success count)
  - failed:    2                   (optional — real failure count)
  - file:      (binary) output .jsonl

POST /workers/report-failure

// Request
{"job_id": "batch-abc123", "worker_id": "worker-abc123", "error": "OOM"}

// Response — requeued until attempts hit the max (3), then terminal
{"status": "validated", "batch_id": "batch-abc123", "attempts": 1, "error": "OOM"}

---

Fault Tolerance (spec §12)

• report-failure requeues the batch (status back to validated, attempts += 1, assignment voided); after 3 attempts it is marked failed terminally.
• A background sweeper (started at app startup, 60s interval) marks workers offline after 120s without a heartbeat and requeues their in-flight batches — a crashed daemon never strands a batch.

See sweeper.py for the thresholds.

---

Database Schema

Postgres with SQLAlchemy ORM. Tables (see models.py):

Table	Purpose / notable columns
users	platform_role (user/superadmin), must_change_password — no role/api_key columns
organizations	Ownership boundary; owner derived from memberships
organization_memberships	role: owner / admin / viewer
api_keys	Hashed keys; key_type: worker (org-scoped) or personal; prefix for UI
workers	Static specs; status (liveness, server-managed) vs activity (daemon-reported); aggregate vram_total_gb / vram_available_gb / ram_available_gb from heartbeats (null = unknown, not zero). ram_total_gb is a registration snapshot. Per-card sizes live on worker_gpus — that is what the fit rule reads, not the aggregate. Provider policy: window_start_min / window_end_min / window_tz, max_batch_prompts (null = no constraint)
worker_runtimes	Inference engines a worker exposes (spec §8.2): engine, base_url, status
runtime_models	Per-worker availability rows (spec §8.3): name, runtime_model_id, digest, status (on-disk) + loaded (in VRAM, heartbeat-updated). Lean by design — descriptive metadata lives on model_catalog.
worker_gpus	Physical GPUs per worker (spec §8.4): vendor, name, vram_gb, driver, cuda
model_catalog	Curated, pinned models users select for a batch (identity). See Model catalogue.
files	Uploaded inputs and generated outputs
batches	Lifecycle status, request counts, attempts (requeue counter)
batch_assignments	Which worker holds which batch (FK → workers.id, ON DELETE SET NULL — history survives worker removal)
password_reset_tokens	Forgot-password flow

Inventory is fully normalized per spec §8.2–8.4 — one row per runtime, model, and GPU rather than JSON blobs on workers.

---

Model catalogue — /v1/models

body.model is a catalogue id (a stable platform slug), not a raw runtime tag. Each model_catalog row is one pinned artifact — weights + quantization + runtime — so a batch never silently swaps precision or runtime (reproducibility). The raw runtime string (mistral:7b, an HF repo id) lives only in runtime_model_id; the backend hands the daemon that at poll time. See Model catalogue for the full design, scheduling, and curation runbook.

model_catalog columns: id (slug, = body.model), display_name, runtime, runtime_model_id, digest (reproducibility pin / match key), quantization, parameter_size, context_length, vram_gb (scheduling requirement), size_gb, task_type, source_*/homepage_url (provenance), org_id (NULL = public), status, enabled.

Scheduling (scheduler.py): poll resolves batch.model → entry, then matches a worker that fits vram_gb and hosts runtime_model_id, enforcing digest equality when both sides carry a digest (same tag + different digest ⇒ not matched); prefers a worker already serving it.

Curation: entries are seeded from backend/catalog/models.yaml at startup (upserted). Fill real digests + metadata from a live Ollama with python -m scripts.capture_catalog (see the runbook). Validation rejects a body.model not in the catalogue (unsupported_model).

---

Batch Status Lifecycle

validating → validated → in_progress → completed
     ↓            ↑            ↓
   failed         └────────────┘ requeue (failure/offline worker,
                                 max 3 attempts) → failed

The backend enforces this rather than merely describing it — VALID_TRANSITIONS in backend/routers/workers.py rejects anything else:

From	May become
validating	validated, failed
validated	in_progress
in_progress	completed, failed, validated (the last is a requeue)
completed	— terminal
failed	— terminal

There are exactly five. cancelling, cancelled, expired and finalizing exist in OpenAI's batch API and the dashboard styles badges for them, but this backend never produces them — there is no cancel endpoint.

Requeue (requeue_or_fail_batch in backend/sweeper.py) increments attempts, deletes the batch's assignment so another worker can claim it, and stores the error truncated to 2000 characters. On the third attempt (MAX_BATCH_ATTEMPTS = 3) the batch goes terminal: failed, completed_at stamped, and request_counts_failed set to the total.

Two things trigger it — a worker calling POST /workers/report-failure, and the sweeper finding a worker that has stopped heartbeating (HEARTBEAT_TIMEOUT_SECONDS = 120, checked every SWEEP_INTERVAL_SECONDS = 60), which marks the worker offline and releases whatever it was holding.

Metadata

POST /v1/batches accepts OpenAI's metadata: up to 16 string→string pairs (keys ≤ 64 chars, values ≤ 512). It is stored as given and returned on GET /v1/batches/{id} and the list. The scheduler ignores it; it exists so a client can find batches it submitted together — the dashboard's grid mode tags each arm with grid_id.

Runtime capabilities

Some body fields only some runtimes honour. Each daemon advertises its runtimes' capabilities at registration (logprobs, completions, prompt_scoring); the validator records which of them a file's rows ask for on the batch, and the scheduler offers that batch only to a runtime that advertises all of them.

Rows that…	Need	vLLM	llama.cpp	Ollama
set logprobs / top_logprobs > 0	logprobs	yes	yes	≥ 0.12.11
use POST /v1/completions	completions	yes	yes	no
set echo: true on completions (prompt scoring)	prompt_scoring	yes	no	no

If an online worker hosts the model but none of those hosts advertises the capability, validation fails with unsupported_capability, naming the field. If no host is online at all, the batch queues as usual and waits for a capable one. Daemons older than this contract advertise nothing and are never offered such batches.

Provider policy

An organisation owner or admin can limit what one of its workers is given:

// PATCH /v1/orgs/{org_id}/workers/{worker_id}/policy
{
  "window_start": "18:00",
  "window_end": "08:00",
  "window_tz": "Asia/Kolkata",
  "max_batch_prompts": 500
}

Only the fields sent change, and null clears one. The response, and each worker in GET /v1/orgs/{org_id}/workers, carries the result as policy.

• Availability window. Outside it the worker is not offered a batch, and it drops out of models_servable. A batch already running when the window closes finishes. Times are local to window_tz (an IANA name); a start later than the end wraps past midnight. The three fields are set together or cleared together, and start must differ from end — otherwise 400.
• max_batch_prompts. A batch with more requests than this is never offered to the worker. At least 1; null is no limit.

Both are checked in scheduler.can_serve, but they reach pool capacity differently. The window does: a worker outside it drops out of models_servable. The size cap does not, because capacity asks about a model without a batch size — a model servable for small batches stays listed even if every worker caps below yours. That is what the validation check below is for.

When every online worker that hosts a file's model caps below its size, validation fails with batch_too_large and names the largest limit; when none is online the batch queues. A worker outside its window still counts for that check — it is off for now, not unable.

Capacity is served from the 15-second cache above, so a window opening or closing, or a policy edit, reaches models_servable within 15 seconds. Dispatch reads the policy live and is never stale.

Validation errors

Validation is asynchronous: POST /v1/batches returns immediately with validating, and the batch reaches validated or failed on its own. Watch it with GET /v1/batches/{batch_id}/events (SSE) rather than polling.

Each error carries a code (missing_field, invalid_json, duplicate_custom_id, unknown_fields, unsupported_model, batch_too_large, …), the line it occurred on, and the field where applicable — null for whole-line JSON parse failures.

Stored errors are capped at MAX_STORED_ERRORS = 100 (backend/services/batch_validator.py), but the reported total is the true count. A batch can therefore report thousands of errors while listing a hundred; that is the cap, not the truth about the file.

---

Migration

Base.metadata.create_all() creates any missing tables on startup. No formal migration tool (Alembic) yet, and create_all() will not alter a table that already exists — a new column on an existing model reaches an existing database only if you add it by hand. For dev, drop and recreate the database and restart.

---

Project Structure

backend/
├── main.py                # App entry, CORS, sweeper + admin startup
├── database.py            # Postgres engine + session
├── models.py              # SQLAlchemy models
├── schemas.py             # Pydantic request/response models
├── auth.py                # JWT, hashing, key contexts (worker/personal/human)
├── scheduler.py           # Job-to-worker matching (VRAM + loaded models)
├── sweeper.py             # Requeue logic + stale-worker sweeper (spec §12)
├── rate_limit.py          # Key-creation rate limiting
├── requirements.txt
├── routers/
│   ├── auth.py            # /v1/auth/*, /v1/admin/*
│   ├── users.py           # /v1/users/me/api-keys (personal keys)
│   ├── organizations.py   # /v1/orgs/* (org keys, org workers)
│   ├── files.py           # /v1/files
│   ├── batches.py         # /v1/batches (+ SSE validation events)
│   └── workers.py         # /workers/* (daemon-facing)
└── services/
    ├── batch_validator.py # Async JSONL validation
    ├── sse_manager.py     # Validation event stream
    └── email_service.py   # Password-reset mail