The Handbook · entry 03
Open weights or frontier APIs: choosing your enterprise LLM
Enterprises keep asking the question as if it were ideological: open weights or paid frontier models? It is not ideological. It is an engineering decision with a small number of governing variables, and the honest answer is almost always a sentence nobody's procurement template has a box for: both, routed by evidence, task by task. Here is the framework we use, published because the reasoning matters more than the conclusion.
Where open weights win
Sovereignty and residency. When regulation, classification, or board policy says the data does not leave the building, an open-weights model (Qwen, Moonshot, Mistral, DeepSeek, Gemma) running in your VPC or fully air-gapped is not the cheaper option. It is the only option. Prior authorization inside a hospital network and underwriting files inside an insurer often live here.
Volume economics on routine cognition. Once a task is well understood, high-volume, and evaluated, per-token API pricing becomes the expensive way to do a cheap thing. Classification, extraction, routing, summarization at millions of units a month: a fine-tuned small open model on committed GPU capacity routinely beats API economics by an order of magnitude, provided utilization stays high. The proviso is doing real work in that sentence; a GPU cluster at 20 percent utilization is a subscription with worse ergonomics.
Deep specialization. When your advantage is proprietary data, fine-tuning open weights builds an asset you own outright: the adapted model, the training pipeline, the evals. Nothing about it renews annually or changes deprecation schedules without asking you.
Latency and the edge. Sub-second loops, on-premise factory floors, devices: local weights, no round trip.
Where frontier APIs win
The judgment steps. The hardest ten percent of any process, the ambiguous claim, the multi-document synthesis, the exception that touches four systems, is where frontier models (Claude, GPT, Gemini families) hold a real capability gap, and where that gap is worth paying for, because the judgment steps are where the number moves.
Agentic reliability. Long-horizon tool use, multi-step recovery from failure, sustained instruction-following across a workflow: the frontier is measurably ahead, and in agentic deployments reliability compounds; a model that is 5 percent better per step is not 5 percent better across a forty-step run.
Speed to first value. No infrastructure, no MLOps hiring, managed safety tooling, and the fastest path from workshop to working software. When the twelve-week clock is running, the API is how you launch. Optimization can move workloads later; that is what the optimize stage is for.
The upgrade treadmill, handled. The model you use today is the worst you will ever use. On an API, next quarter's better model is a string change. On self-hosted weights, it is a migration you own, and open-weights releases now arrive fast enough that refresh discipline is a standing engineering cost, not a one-time setup.
The mistake both camps make
Comparing token prices. The real comparison is total cost of a verified outcome: GPU commitment and utilization, fine-tuning and eval maintenance, MLOps headcount, upgrade cadence, and failure cost on the judgment steps, against API spend at the volumes the process actually runs. We have moved workloads in both directions after the reading. The spreadsheet, not the ideology, decides.
The question is never which model is best. It is which model earns each step of your process, on your data, at your volumes, with the proof read from your systems.
How we decide, in practice
Every futurify engagement runs the same protocol. The process is fixed first, so we know which steps are routine volume and which are judgment. Each step gets an eval built from your real cases, and candidate models, closed and open, run the same eval on the same data. The workload goes to whichever model earns it, and the architecture stays free to move when the frontier moves: frontier APIs on the judgment steps, fine-tuned open weights on the volume steps, in the same pipeline, behind the same observability. Deployed in your cloud, your VPC, or air-gapped where it must be. The result is not a model choice. It is a routing table with evidence attached, re-read every quarter, because the frontier will not sit still and neither should your architecture.