Four questions, answered live: what we need, what it costs, what happens at scale, and how we optimize. Change any input — every answer recomputes, including the summary below.
1 · What do we need, given our projects?
click to open
The need isn't abstract — it's these planned projects. Check what's in scope and slide the roadmap; everything on this page recomputes from the plan itself. Every per-project number is editable (open a project's "workload numbers").
What the projects demand. Plus the people and coding agents using the machine:
What that maps to: the open-model menu. The AI model is the machine's engine — pick the quality tier, and the memory it needs picks the machine. All of these are openly available models we can run privately:
Evidence: "90% of the best models' quality" — where that claim comes from
Published evaluations of Qwen3-235B (vendor technical report plus independent leaderboards, mid-2025) against the closed frontier-tier models of the same period (GPT / Claude / Gemini class). Numbers rounded; releases move scores every few months — the claim is the tier, not the digit.
Skill (benchmark family)
Qwen3-235B
Best closed models
Why it matters here
Coding (LiveCodeBench-class)
~70
~75
the coding agents' daily job
Math & multi-step reasoning (AIME-class)
~85
~90
agents planning multi-step edits
Broad knowledge (MMLU-Pro-class)
~81
~85
general chat quality
Tool use (function-calling suites)
near parity
—
drives the document pipeline
What it means concretely: agent code edits and contract extraction land a shade below GPT/Claude quality — good enough for the daily work. It does not mean 90% of everything; obscure long-tail knowledge trails further. Where a mistake would be irreversible, route that one step to human review (the routing rule in "All the rules").
2 · How much will it cost?
click to open
Five machines, from a $22k gaming build to a $350k datacenter rack — each judged against our need, cheapest survivor recommended (green border). Cost per million tokens is at our usage: an idle machine makes expensive tokens, so that number falls as work grows.
Adjust the money assumptions — hours, electricity, power circuit, privacy, scheduling
3 · What happens when we scale to more projects?
click to open
The roadmap slider in §1 already grows the workload phase by phase. These switches test the harder what-ifs — a purchase that only works at today's numbers is already obsolete.
The upgrade path, cheapest step first
Trim reading sizes — free; smaller working sessions mean more workers fit.
Compress the working memory — one software setting, roughly doubles how many workers fit; free.
Add 2 cards to the empty slots — ~$17,000 plus an electrician visit (20-amp circuit); roughly doubles memory.
Second machine — ~$30,000+; also buys backup if one box goes down.
4 · How can we optimize the system?
click to open
Five levers, in the order we'd pull them. The first four are free or nearly free.
Keep it busy. The machine costs the same idle or working — so cost per token is set by how much it works. Running the document pipeline at night can cut cost per token severalfold. Costs nothing.
Never read the same thing twice. Caching remembers already-read text; for coding agents that erases ~90% of their reading work. One serving-software setting.
Compress the model. Half the memory and nearly double the writing speed, for a barely noticeable quality dip. Already our default. Free.
Compress the working memory. A second setting that halves what each worker's session occupies — roughly twice the workers on the same cards. Free.
Measure, then buy. Every speed number here is a conservative industry average. A ~$30 rental of the same cards replaces assumptions with facts before the order goes in. $30, once.
The one-page brief
click to open
Everything above, condensed for approval. Every claim traceable to a live number.
Standing caveat: every speed number rests on conservative industry averages (the dials in the engineer appendix) — not this machine's measured values. Before money moves: the ~$30 rental test. Design with averages; buy with measurements.
🔧 For engineers — how this page computes
Method: the 8-step server-design process from night school, compressed into the four questions above. One state object; every displayed number derives from the constants and formulas in this file's script.
Workload inputs (auto-filled from the §1 portfolio; edit to override)
Speed dials — conservative industry averages, replaced by rental-test measurements before ordering
The rules behind the math
📖 All the rules — everything the course and night school taught, one list
The physics
Everything is counted in tokens — about 4 characters each. course L1
Memory cost follows total weights; speed follows active weights. Mixture-of-experts (MoE) models — many specialists stored, only a few wake per word — are big to store, cheap to run. course L2
Writing is a memory-speed job (haul all active weights per token); reading is a crunching-speed job (one haul, thousands of tokens). night N1/N5
Writing speed ≈ memory GB/s × 0.5 ÷ active weight GB — two spec numbers predict any machine. night N1–N2
Silence before the first word = new context tokens ÷ reading speed (crunch × 0.4 ÷ 2× active weights). night N6
VRAM (the card's own fast memory) is the fast lane; spilling to system RAM doesn't crash — it runs ~20× slower. "Fits in card memory" is a hard yes/no. course L4
Sizing
Weights + every worker's context must fit in 90% of card memory — contexts run out before the model does. course L3, exam E1
Compression (INT4) halves memory and nearly doubles writing speed, for a sliver of quality. The standard move. course L5, night N4
Batching: one memory-read serves ~16 workers — aggregate speed scales until the cap, then workers share. course L6
Hardware sets the ceiling; serving software (vLLM-class, not single-user) decides how close you get. course L7
The prefix cache erases repeated reading (~90% for agents) — never new reading. Pipelines pay full price. course L8, night N7
Agents mostly wait (~25% busy) — they cost memory long before they cost speed. Memory is always the first wall. course L9–L10
Every tool call gets a timeout and retry budget — one hung agent must never starve the rest. course L11
Route by stage: the big model only where mistakes are irreversible; prep steps like turning documents into searchable number lists ("embeddings") and quick lookups use small or tiny models. course L12–L13
Money
Speed × hours = tokens; dollars ÷ tokens = price. That prices any card, rented or owned, in 30 seconds. night N3
An idle owned machine is the most expensive compute there is — utilization, not hardware price, decides cost per token. course L15
Buying is a ~3-year bet on sustained use; uncertain load favors renting. course L14
Justify a purchase with privacy + control + unlimited use at fixed cost, and verify the price roughly ties the APIs — if it only pencils at 100% utilization, rent. exam E3
Shopping
Deflate marketing numbers: ÷2 if "sparse," ÷2 per precision step up. Then compare cards. night N8
Pick the card family by where it lives: one card at home = gaming; several in an office = workstation (self-checking memory — "ECC", push-the-heat-out "blower" fans, 96GB cards); racks = datacenter. night N9
Count connection lanes (the data highways from processor to cards) first — 16 per GPU + 4 per fast drive. The lane count picks your CPU class before any benchmark does. night N10
System RAM ≥ card memory; a fast drive (NVMe) at least 2× your biggest model. Never let the $500 part bottleneck the $34k of cards. night N11
The wall socket is a spec: GPUs + ~400W must fit in 80% of the circuit's rating. Power picks your card variant. night N12
Two or more cards in one box need blower (or liquid) cooling — open-air cards feed on each other's heat. night N13
Check any parts list in dependency order: lanes → power → cooling → RAM. An early failure makes the rest fiction. exam E2
Integrator premium ≈ 15–20%, buying burn-in (pre-ship stress testing) + whole-box warranty + one phone number. Price the parts yourself first, then haggle. night N14
Running & growing
Scale up while there are empty slots (cards are cheaper than boxes, memory pools); scale out when slots run out or downtime becomes unacceptable. course L16
Upgrade ladder, cheapest first: trim contexts → compress the context cache → add cards → second box. scaling walk-through
Design with industry averages; buy with measurements — rent the target cards, measure, replace the dials with truth. step 5
Assumption audit — what the spec believes, and which beliefs expire first