Vertical foundation models

The idea in one line: vertical foundation models win by orders of magnitude where they replace an expensive physics or lab calculation and reality grades the answer for free; they get overstated where the baselines were weak and the grader was narrower than the claim.

A vertical foundation model is one pretrained once for a single industry or science domain, such as weather, proteins or crystals, then reused across that domain’s tasks (recall what a model is). The 10x never comes from model size. It comes from what the model replaces:

• ECMWF’s AI forecast uses about 1,000x less energy per forecast than the physics model beside it (ECMWF, 2025-02-25).
• One high-level quantum chemistry label in Meta’s OMol25 dataset cost about 60 CPU core-hours, while Meta’s UMA-S model scores a 1,000-atom system in about 1/16 of a second on one GPU.
• Chai Discovery says its Chai-2 model designs antibodies with a 16% hit rate, against under 0.1% for earlier computational methods (its own claim).

Money follows the expensive part: biology and materials labs, which run wet labs, raise $300M to $2B (Isomorphic $2.1B, Chai $400M), while time-series, weather and industrial teams raise $6M to $50M (the market map). Bigger models are mostly 10% polish: a 300M protein model matches one twice its size, its builder says.

Figures marked (secondhand) were read from search summaries, not the original page.

For any “foundation model for X” claim, ask:

1. What expensive thing does it replace? A physics solver, a quantum chemistry calculation, a round of lab experiments.
2. Who grades the answer, how fast and how cheaply? This is the verifiable reward: a signal checked automatically against ground truth (a resolved forecast, a sensor reading, an assay) rather than by human opinion. See verifiable rewards and evals.
3. What does the strongest simple baseline score? Ridge regression, an average, last season’s value, a full-data specialist.

A worked example on question 1. At a hit rate under 0.1%, you test more than 1,000 designs per binder. Chai-2’s technical report gives 68 binders from 496 designs, 14% (secondhand): about 7 designs per binder. That is more than 100x fewer experiments, if the claim holds, and experiments are what pharma pays for; inference cost is a rounding error next to it.

Numerical weather prediction (NWP) forecasts by solving the physics equations of the atmosphere on a supercomputer, as ECMWF’s IFS system does (its high-resolution run is HRES; its ensemble is ENS). An ensemble is many slightly different forecasts whose spread gives probabilities. AI models learn from ERA5 reanalysis: a free, decades-long, gap-free reconstruction of past weather.

Sources: GraphCast, GenCast, AIFS, AIFS ENS, WeatherNext 2.

This is the only vertical running daily at national forecasting centres. In the 2025 Atlantic season, Google DeepMind’s cyclone ensemble edged the US National Hurricane Center’s official track forecasts through 72 hours (secondhand, NHC’s 2025 report), though NHC documents don’t independently confirm Google’s claimed one-day lead-time gain.

Every headline ratio compares running a trained network with running a physics model. None counts producing ERA5 or training, and physics stays upstream: AIFS starts from the same roughly 60 million quality-controlled observations the IFS system assimilates.

Only the last is an operational measurement, so it is the most decision-relevant (NeuralGCM; FourCastNet). Training is cheap by LLM standards: GraphCast took about 4 weeks on 32 TPU v4 chips (secondhand), about 21,500 TPU-hours.

Zhang, Fischer, Zscheischler and Engelke tested record-breaking heat, cold and wind. GraphCast, Pangu-Weather and FuXi had errors “consistently larger” than HRES at nearly all lead times. They under-predicted hot records and over-predicted cold ones, yet the same models beat HRES on average temperature and wind (arXiv 2508.15724, 2025-08-21). Extrapolating beyond the training data is the failure mode. NeuralGCM’s own paper says it “does not extrapolate to substantially different future climates” (secondhand), and ECMWF treats AI and physics as complementary.

The best power-system test is Za’ter and Hodge’s study on Texas (ERCOT) data. nMAE is mean absolute error as a percentage of capacity or the mean; lower is better.

(arXiv 2604.22077, 2026-04-23). The authors call the zero-shot errors “far too high for practical grid operations”. Fine-tuned, the foundation model wins, and it generalises better: on unseen sites, TimesFM’s solar error rose from 3.71% to 4.73%, the LSTM’s from 6.71% to 12.35%. The course’s research found one of the paper’s claims unsupported: at 20% of the data, the foundation models did not beat LSTMs at 100% on solar (7.8-8.4% versus 6.71%).

So the product is few-shot; zero-shot is for cold start. Batteries lag further. The BatteryLife benchmark merged 16 datasets covering 59 chemistries and found that “models popular in other time series fields can be unsuitable” for battery life (arXiv 2502.18807). The research found no peer-reviewed zero-shot battery foundation model.

A protein language model (PLM) is trained to predict masked or next amino acids; its internal representations end up encoding structure and function. Meta’s ESM-2 scaled to 15B parameters. EvolutionaryScale’s ESM3 98B used about 1.07×10²⁴ FLOPs and designed esmGFP, a fluorescent protein with 58% identity to its nearest natural relative, from two rounds of 96 candidates (both secondhand; ESM3).

Then scale deflated. EvolutionaryScale says ESM C 300M performs like ESM-2 650M and ESM C 600M rivals ESM-2 3B (ESM C, 2024-12-04). Independently, Vieira, Handojo and Wilke found that with limited data, medium models perform “comparably to, and in some cases outperform” large ones, and ESM-2 650M and ESM C 600M fell “only slightly behind” the 15B and 6B models (secondhand, bioRxiv). ESM C, including 6B, is now MIT-licensed (repo). The default for most people: a frozen 300M-650M model plus a linear head.

Arc Institute’s Evo 2 (preprint 2025-02-19) comes in 7B and 40B. The 40B model saw 9.3T tokens from more than 128,000 genomes and trained on more than 2,000 H100s for several months. Weights, code and data are all open (Arc). It is state of the art at zero-shot scoring of ClinVar non-coding single-letter variants. On coding single-letter variants it ranks only 4th and 5th, behind AlphaMissense, ESM-1b and GPN-MSA (bioRxiv). A small classifier on its embeddings reached an AUROC (the chance a model ranks a random positive above a random negative; 0.5 is chance) of 0.94 on BRCA1 coding variants, the “foundation model plus tiny head” pattern again. On regulatory tasks, Tang and Koo found DNA language models “do not offer substantial advantages over conventional machine learning approaches” (bioRxiv, 2024-03-04).

Co-folding predicts a protein’s 3D structure together with its drug molecules or partners in one pass. AlphaFold 3 (2024-05-08) leads, but its weights are academic-only and its terms bar using outputs to train similar models (secondhand): a licence turned into a moat. MIT’s Boltz-1 (2024-11-16) reached comparable accuracy, by its authors’ account, about six months later under an MIT licence, training data included. Boltz-2 (2025-06-06) is billed as “the first deep learning model to approach the accuracy of physics-based FEP methods, while running 1000x faster” (Boltz-2, its builders’ claim). FEP, free-energy perturbation, is the slow physics simulation that is the industry’s gold standard for binding strength. On its builders’ benchmark, Boltz-2’s correlation with measured binding was 0.62, against about 0.72 for full FEP (secondhand).

The critiques: on the Runs N’ Poses benchmark, accuracy “significantly declines” on complexes unlike the training set (secondhand). In a prospective test on 557 newer complexes, AlphaFold 3, Boltz-2 and Chai-1 each placed more than half the drug poses within 2 Å, but success tracked training-set similarity, and AlphaFold 3’s confidence scores separated true ligands from decoys worse than old docking scores (secondhand, eLife). The lesson for decision models: confidence isn’t calibration.

A single-cell foundation model is pretrained on millions of single-cell gene-expression profiles to transfer to tasks like cell typing or perturbation prediction (forecasting how a cell’s genes respond when one is knocked down). This is the field’s clearest case of overstatement:

• Zero-shot scGPT and Geneformer lost to simple pipelines (highly-variable-gene selection, scVI, Harmony) at clustering and batch integration (Kedzierska et al., 2025, secondhand).
• Ahlmann-Eltze, Huber and Anders tested five foundation models plus GEARS and CPA on CRISPR screens. For double perturbations, every model’s error was “substantially higher than the additive baseline”. None beat a “no change” baseline at finding genetic interactions, and no deep model consistently beat a mean or linear baseline on unseen single perturbations (Nature Methods, 2025-08-04).
• One ICML 2026 paper pushes back: deep models beat baselines on “resistant” perturbations, but only for which genes respond and in which direction, not by how much (single source).

Arc said its State model, trained on about 170M observational and more than 100M perturbed cells, is the first to “consistently beat simple linear baselines” (Arc, 2025-06-23, its own claim; non-commercial licence). Arc’s 2025 Virtual Cell Challenge then tested the field in public: more than 5,000 registrants, more than 1,200 submitting teams, two $100k grand prizes (GEN, 2025-12-08). Arc’s Hani Goodarzi said mean absolute error stopped influencing optimisation mid-competition. One participant’s post-mortem (a single source) reported that:

• the winning entry’s mean absolute error was more than 37x the “average” baseline’s;
• clipping negative scores to zero made that error irrelevant, and without clipping all top-50 entries would have scored below zero;
• his one-person pseudobulk kernel ridge regression (averaging all cells per condition into one clean profile) ranked 38th overall, came first on correlation, and beat State (LaPierre, 2025-12-11).

Lesson: when the metric can be gamed, the leaderboard measures the gaming.

Visual anomaly detection flags defective parts in images. AUROC is defined above; AU-PRO scores defect segmentation, crediting each defect region equally whatever its size.

Zero-shot sits about 7 points short of the specialist; one normal example closes 2-3. MMAD’s authors say GPT-4o “falls far short of industrial requirements” (arXiv 2410.09453). On the harder MVTec AD 2 (transparent objects, unusual lighting, tiny defects), the state of the art scores below 60% AU-PRO (arXiv 2503.21622). Industrial models (Siemens’, Archetype AI’s Newton) publish no weights or benchmarks, so their claims can’t be checked.

Why manufacturing data is scarce and siloed:

1. Defects are rare by design, so the field trains on normal images only.
2. Data is the plant’s IP. Siemens says the data that matters is “not available on the open internet” (Siemens, 2025-11-26).
3. Public benchmarks are non-commercial. MVTec AD is CC BY-NC-SA 4.0; MVTec AD 2’s ground truth is server-only (MVTec).
4. Pooling is still a pilot. A cross-company federated training hackathon found the data volume “was not sufficient for training purposes” (secondhand).

Density functional theory (DFT) is the standard quantum-mechanical calculation of a material’s energy: accurate but expensive. A universal interatomic potential (MLIP) is a neural network trained on DFT results that predicts energies and forces for almost any atoms, thousands to millions of times faster. Energy above the convex hull says how far a crystal sits above the most stable mix of competing phases; at or below zero means “predicted stable”.

Meta’s UMA trained on about 459M structures with about 10²² FLOPs, roughly 100x less than ESM3 98B; the small model took about 5 days on 128 H200 GPUs (UMA). The expensive part was the labels:

OMol25 labels:  6 billion core-hours / 100 million calculations
              = about 60 core-hours per label
UMA-S:          16 evaluations per second, 1,000 atoms, one H100
              = about 1/16 second per structure

That asymmetry makes search and reinforcement learning in materials affordable (Rowan, 2025-05-23). On the Matbench Discovery leaderboard, the stability F1 (balancing missed stable crystals against false “stable” calls) has saturated: as of October 2026 the top ten score 0.906-0.931, most have 28.7M-82.9M parameters, and a 730M model scores 0.924, no better (Matbench Discovery). The leader depends on how you sort: UMA’s paper reports 0.929, while EPFL claimed the top spot in January 2026 (secondhand). Permissive options exist: MACE-MP-0 (MIT), Orb-v3 (Apache 2.0), and UMA, which is “commercially permissive” but gated.

Google DeepMind’s GNoME (2023-11-29) reported 2.2M new crystals, about 380k of them stable, which would raise known stable materials from about 48k to 421k (DeepMind). Two critiques followed (both secondhand). Cheetham and Seshadri found “scant evidence” of novelty, credibility and utility: many entries were already known, or ordered versions of compounds “unlikely to be ordered in the real world”. Leeman and colleagues re-examined the 43 materials the companion A-Lab paper said it made and concluded “no new materials have been discovered in that work”. DeepMind stands by its claims. The lesson: “stable according to DFT” is a verifiable reward, but it isn’t “new, makeable and useful”. The verifier was narrower than the claim.

The workable pattern is cheap-then-expensive: generate, score with an MLIP, check the top few with DFT, synthesise a handful (see how to read the diagrams).

C4 · L1 System context · Materials screening loop

+-------------------+  sets a target  +------------------------+
| [Person]          |  property       | [Software system]      |
| Materials         |---------------->| Screening loop         |
| researcher        |                 | ranks candidates       |
+-------------------+                 +------------------------+
                                        |                  |
                      checks the top    |                  | sends the
                      few with DFT      v                  v top handful
                    +---------------------+  +---------------------+
                    | [External system]   |  | [External system]   |
                    | CPU cluster (DFT)   |  | Synthesis lab       |
                    +---------------------+  +---------------------+

The researcher sets a target; the loop ranks, and calls out only for the expensive checks.

C4 · L2 Container · Materials screening loop

............... Materials screening loop [system] ................
:                                                                :
:  +---------------------+  candidates  +---------------------+  :
:  | [Container: Python] |  (CIF files) | [Container: GPU]    |  :
:  | Generator           |------------->| MLIP scorer         |  :
:  | proposes crystals   |              | UMA-S, Orb or MACE  |  :
:  +---------------------+              +---------------------+  :
:             ^                                    |             :
:             | reward: energy above               | scores      :
:             | the hull (SQL)                     v (SQL)       :
:             |                         +---------------------+  :
:             +-------------------------| [Database: SQLite]  |  :
:                                       | Ranked candidates   |  :
:                                       +---------------------+  :
:                                                  |             :
:..................................................|.............:
                                                   | top few
                                                   v (CIF files)
                                        +---------------------+
                                        | [External system]   |
                                        | CPU cluster (DFT)   |
                                        +---------------------+

Inside, the generator proposes crystals, the MLIP scorer rewards low energy above the hull, and only the best few leave for DFT. CrystalFormer-RL uses this reward because DFT inside the loop is “infeasible” on cost (secondhand, arXiv 2504.02367).

Chemistry shows what a checkable answer buys: FutureHouse’s ether0 (24B, Apache-2.0) was trained with reinforcement learning rewarded “only on the correctness of the final answer”, on about 50k examples per task, and beats frontier models on some molecule-design tasks, by its builders’ account (FutureHouse, 2025-06-05).

The drug figures are a BCG analysis (HT World, 2024-05-10): Phase I mostly tests safety, which AI chemistry optimises, while Phase II tests whether the biology was right. Behind the right column: weak baselines, a verifier narrower than the claim, leaked test data, or evaluation on average conditions only.

1. Strong-baseline audits. Test a vertical claim (perturbation, cell typing, DNA regulatory, battery life) against ridge, pseudobulk, gradient boosting and seasonal naive. Almost no compute, and such audits have repeatedly reset the field.
2. Calibrated typed decisions. A wrapper that returns “binder, stable or anomalous: yes or no, plus a calibrated probability” on top of open models. Co-folding confidence fails here today (from probability to automation).
3. Few-shot anomaly detection on MVTec AD 2, where the best is below 60% AU-PRO, plus commercially licensed datasets.
4. Zero-shot remaining-useful-life benchmarks on public vibration data (IMS, CWRU): the research found none.
5. Battery-native pretraining on BatteryLife (MIT-licensed), since generic forecasters are unsuitable.
6. Out-of-distribution weather: correction heads for record extremes and heavy rain.
7. Honest evaluation: similarity-binned co-folding tests and perturbation metrics that can’t be gamed.
8. Surrogate-reward loops: MLIP rewards checked by sparse DFT, and ether0-style task suites for spectroscopy or reaction conditions.

Robot foundation models follow the same playbook, with weaker verification. Physical Intelligence open-sourced π0 on 2025-02-04 and says “between 1 and 20 hours of data was sufficient” to fine-tune it for a variety of tasks (openpi). NVIDIA released GR00T N1 on 2025-03-18 and says it generated 780,000 synthetic trajectories in 11 hours, equal to about 6,500 hours of human demonstrations, with a 40% gain from mixing synthetic and real data (NVIDIA, its own claim). Simulation plays the verifier, as DFT does in materials, and the sim-to-real gap plays the part of GNoME’s novelty gap.

• ECMWF, AI forecasts become operational (2025-02-25): the one operational energy comparison.
• Zhang et al., record-breaking extremes (arXiv 2508.15724): where AI weather fails.
• Ahlmann-Eltze, Huber and Anders (Nature Methods, 2025): the baseline audit that deflated perturbation prediction.
• Za’ter and Hodge, foundation models for power systems (arXiv 2604.22077): zero-shot versus fine-tuned on real grid data.
• Matbench Discovery: the live materials leaderboard.
• LaPierre, the real value of the Virtual Cell Challenge (2025-12-11): a participant’s account of how the metric was gamed.