The bootstrapper's stack

The idea in one line: Training a small decision model now costs less than lunch. Whether your budget turns into a product depends on which labeller you buy from, which licence you build on, and how many runs you let fail.

Fine-tuning a 150M-parameter encoder or a roughly 1B-parameter decoder on 100,000 examples costs under $5 of rented GPU time. Labelling 1 million examples with a short rationale, on the other hand, costs anywhere from $48 (GPT-5-nano in batch) to $6,650 (GPT-6-astra in batch) depending on which model does the labelling: a spread of about 140x (derived from OpenAI, Anthropic and Gemini pricing, read 2026-10-28; the arithmetic is in distill a decision model). And a licence can wipe out a plan entirely: the newest TabPFN and TimesFM weights are non-commercial.

So the order-of-magnitude choices are model shape (a small encoder instead of an LLM), the labeller, and the licence. Which GPU brand or cloud you pick is 10% polish. This lesson gives you the real prices so you can stop worrying about compute and spend your attention where it pays.

Renting a GPU is like renting a car by the hour. Three choices change the price:

• On-demand means the machine is yours until you stop it.
• Interruptible (also called spot) means a cheaper rental that the host can take back at any moment, so your script must save checkpoints (snapshots of the training state) and resume from them.
• Community or marketplace clouds rent other people’s machines: cheaper, less reliable than “secure” datacentre tiers.

Sources: RunPod pricing (updated 2026-10-08), Vast.ai live offers (queried 2026-10-29 UTC; unvetted hosts, so these are market quotes, not list prices), Together pricing and Modal pricing (both read 2026-10-28).

Worked example (derived). Fully fine-tune ModernBERT-base (150M) on 100,000 examples of 128 tokens for 3 epochs on a 4090.

• The ModernBERT paper measured inference at 147,300 tokens/s on a 4090 (Warner et al., Table 2, 2024-12-18).
• A training step costs roughly 3x an inference pass (a rule of thumb: forward plus a backward pass twice as costly). So about 49,000 tokens/s, or about 380 examples/s.
• 300,000 example-passes ÷ 380 per second ≈ 790 s ≈ 13 minutes, so $0.07-0.19 at $0.34-0.89/h.

Even if the real throughput were 4x worse, a run costs under $2. The cost that matters is running many: 5 base models × 3 seeds × a few learning rates is 30-60 runs, still under $50.

For a 1B decoder, the anchor is Karpathy’s nanochat: a 561M model trained at about 135,000 tokens/s per H100 at about 48% MFU (model FLOPs utilisation, the share of the GPU’s peak arithmetic you actually use) (nanochat d20 walkthrough, 2025-10-13). Scaled by size, a 1B model sees about 75,000 tokens/s, so 100k examples of 226 tokens (22.6M tokens) take about 5 H100-minutes. Real fine-tuning runs are 3-10x less efficient, so expect $1-10 per run (derived).

Credit programmes sound big, but the six-figure tiers need investors. What a solo, incorporated, unfunded founder can realistically get:

That stacks to roughly $4,000-8,000 if you qualify for all of them. Hyperscaler credits are awkward for GPUs: GCP’s $300 trial “can’t add GPUs”, and hyperscaler H100s cost more than RunPod’s. Spend them on frontier-model labels for evaluation (through Bedrock or Vertex) and on storage.

MLX is Apple’s array framework for its own chips; mlx-lm is its library for running and fine-tuning language models. Apple Silicon shares one pool of memory between CPU and GPU, which is why a laptop can train models a gaming GPU of the same price can’t hold. On the 48 GB test machine, MLX recommended a GPU working set of 40.2 GB, about 78% of RAM (measured).

Measured with mlx-lm 0.32.2, batch size 8, examples of about 226 tokens (lower bounds, see the note above):

Worked example. 100,000 examples × 226 tokens = 22.6M tokens per epoch. At 1,380 tokens/s that is about 16,400 s, or 4.6 hours per epoch for LoRA on the 0.6B model. QLoRA on the 4B model takes about 28 hours per epoch. An encoder is faster still: ModernBERT-base fine-tuned in PyTorch on the Mac’s GPU (MPS) ran at 26.1 examples/s in bf16, so about 64 minutes per epoch for 100k examples (measured).

mlx-lm’s own docs give one figure, about 250 tokens/s for LoRA on Mistral-7B on a 32 GB M1 Max (mlx-lm LORA.md, read 2026-10-28). Third-party estimates for 8B on an M4 Max range from 380 to 800 tokens/s and conflict, and there are no official 2026 MLX training benchmarks. What fits, assuming the 78% ratio holds on other Macs (derived):

For typed decisions, a fully fine-tuned encoder or a 0.6-2B decoder is the right size anyway, so a 32 GB Mac covers the whole course.

A reminder from transformers in plain words: an encoder reads the whole input at once, in both directions, and outputs vectors, the natural shape for classification and scoring; a decoder writes text left to right but can also score known options in one pass (generating vs deciding).

Permissive picks (commercial use allowed; details from the Hugging Face model API, queried 2026-10-28):

Ettin’s paper shows why encoders are the default: with identical data and recipes, “a 400M encoder outperforms a 1B decoder on MNLI”, a sentence-pair reasoning benchmark (Ettin, 2025-07-15).

The licence traps. Several flagship models of this course can’t be used in a product:

Teacher terms matter as much as weights. Anthropic forbids using its services “to train competing AI models” (Commercial Terms, 2025-06-17), Google forbids models “that compete with the Services” (Gemini API terms, 2026-03-23), and OpenAI’s similar clause has an exception for internal classifiers whose exact wording could only be read from a search snippet. DeepSeek explicitly permits “training other models (such as model distillation)” (DeepSeek ToS, 2026-04-29). The data rights lesson covers the rest; distill a decision model turns this into a labelling plan.

Recall from three ways to teach a model that fine-tuning means continuing to train a pre-trained model on your data. You can change every weight or only a few.

• Full fine-tuning updates every weight. With the Adam optimizer it needs roughly 8-16 bytes per parameter plus activations: about 2.4 GB for a 150M encoder at 16 bytes, but 64 GB for a 4B decoder. For encoders it is cheap and the default.
• LoRA freezes the model and trains a small low-rank “patch” added to some weight matrices. Think of sticky notes on a textbook instead of a rewrite. The mechanism: a frozen matrix W becomes W + B·A, where A and B are thin. A 1,024 × 1,024 matrix holds 1,048,576 numbers; a rank-8 patch adds A (8 × 1,024) and B (1,024 × 8), 16,384 trainable numbers, 1.6% of that matrix. The original paper reports “10,000 times” fewer trainable parameters for GPT-3 175B and “no additional inference latency” once merged (Hu et al., 2021-06-17).
• QLoRA keeps the frozen base in a 4-bit format (“NF4”) and trains LoRA on top. Its paper fine-tuned a 65B model “on a single 48GB GPU” (Dettmers et al., 2023-05-23). On the Mac, QLoRA put a 4B model in 11.7 GB (measured).
• DoRA splits each weight into a size and a direction and applies LoRA to the direction; its authors claim it beats LoRA consistently (Liu et al., 2024-02-14).
• Gradient checkpointing stores fewer intermediate results and recomputes them during the backward pass. On the Mac it cut peak memory by 58% for about 15% of speed (measured).
• Packing fills each training row with several short examples so no compute goes to padding. Worked example: four examples of 40, 60, 120 and 230 tokens padded to 256 use 1,024 slots for 450 real tokens, 56% waste; packed into one 512-token row, 450 of 512 slots are real, 12% waste. TRL’s SFTConfig(packing=True, packing_strategy="bfd") does this, and its docs say packing needs FlashAttention, a fast attention kernel (TRL docs, read 2026-10-28). Encoders get the same effect from unpadding: on variable-length input BERT’s throughput halves (180.4k to 90.2k tokens/s) while ModernBERT holds 147.3k against 148.1k (ModernBERT).
• bf16 is a 16-bit number format that keeps fp32’s range; it is the standard training precision. fp8 halves the bits again and mainly helps large matrix multiplies on H100 and B200 GPUs. On the Mac, bf16 trained the encoder at 26.1 examples/s against 22.9 in fp32, about 14% faster (measured): honest 10% polish.

The state of the art. Three things moved in 2026. First, purpose-built decision trainers: Unsloth’s FastDecisionModel scores the options you provide and returns a choice with a probability, and it claims Qwen3.5-0.8B went from 36% to 73% on a typed-decisions benchmark using 4 GB and 42 minutes (Unsloth guide, vendor claim). Second, a frozen backbone plus a small adapter and head can lead: blockbrain’s Cygnet, a frozen Gemma-4-12B with a decision head, tied for first on JevBench 1.5, above Jev (BenchLM), and Cloudflare’s Clef trains a rank-256 LoRA and a routing head on a frozen backbone (remio). Third, pre-training itself got cheap: the GPT-2-small speedrun hit its target in 0.665 minutes on 8 H100s (record dated 2026-08-30, about $0.24-0.35 derived) (modded-nanogpt), and nanochat reaches GPT-2-level quality in 1.65 hours on one 8-GPU node (nanochat). Open problems: Unsloth’s page contradicts itself on LoRA rank (64 vs 16) and learning rate (2e-4 vs 2e-5) and doesn’t name its GPU; no official MLX training benchmarks exist; fp8 speedups for small models are undocumented.

C4 · L1 System context · Your decision-model stack

                   +-----------------------+
                   | [Person]              |
                   | You, the builder      |
                   | one Mac, a budget     |
                   +-----------------------+
                               |
                               | trains, tests, ships
                               v
                   +-----------------------+
                   | [Software system]     |
                   | Your training stack   |
                   | scripts, data, evals  |
                   +-----------------------+
                     |         |         |
          +----------+         |         +-----------+
          |                    |                     |
          | rents GPUs         | buys labels         | downloads
          v by the hour        v in batches          v base models
+-------------------+  +-------------------+  +-------------------+
| [External system] |  | [External system] |  | [External system] |
| GPU clouds and    |  | Teacher LLM APIs  |  | Model hubs        |
| free notebooks    |  | (check the terms) |  | (check licences)  |
+-------------------+  +-------------------+  +-------------------+

Your stack is mostly your own scripts and data on one Mac; it reaches out to three kinds of outside system, and two of them carry legal terms you must check. New to these boxes? See how to read the diagrams.

• Hu et al., LoRA — the original low-rank adaptation paper; short and readable.
• Dettmers et al., QLoRA — how 4-bit bases make big fine-tunes fit one GPU, with the authors’ own caveats about their benchmark.
• mlx-lm LORA.md — every flag you need to fine-tune on a Mac.
• Warner et al., ModernBERT — the encoder most of these plans start from, with throughput tables.
• Weller et al., Ettin — paired encoders and decoders at every size: the cleanest evidence for “start with an encoder”.
• Unsloth, Train your own decision model — the shortest path to a Jev-like model, read with its vendor-claim caveats.