Go to Groundby Jaidev Karthickeyan
← All notes Infrastructure · November 2026

QuarryIf the same model runs eight times faster on one cloud than another at the same price, what is a neocloud actually selling?

The neocloud GTM reset: sell the GPU-hour and you give away the engineering

Six providers charge exactly the same to run OpenAI's gpt-oss-120b. The fastest is eight times quicker than the slowest. Yet 98% of CoreWeave's revenue is still rented capacity, and capacity is the one thing getting dearer.

What happened

Six companies will sell you OpenAI's open-weight gpt-oss-120b for exactly the same price: 15 cents per million tokens in, 60 cents out. In Artificial Analysis's measurements to 4 November, the fastest of them, Groq, produced 394 tokens a second. The slowest, DeepInfra's Turbo tier, produced 46. Microsoft Azure managed 323, Nebius 204 and Amazon Bedrock 120. Three of the six also fell short of the model's reference accuracy.

Same model, same price, an eightfold difference in what arrives. That shouldn't happen to a commodity. It happens because the speed and cost of a model depend less on the chip than on everything wrapped around it: how requests are batched, how memory is managed, how the work is split across machines.

That is the case for a reset in how neoclouds sell. The first generation sold a scarce physical thing, the GPU-hour. The next would sell what the hour produces: tokens at a promised speed, at a known cost per finished task. My claim is narrower than the slogan. The engineering is now the product, but under the contracts neoclouds actually sign, the customer keeps all of it.

The filings show how far the business sits from the slogan. In the second quarter of 2026, 98% of CoreWeave's revenue came from committed contracts, "which are take-or-pay", and its three largest customers took 36%, 26% and 10% of it. Nebius prices new deals in dollars per megawatt. Lambda shut its inference API in September 2025 and went back to renting GPUs.

The arithmetic

One megawatt of GB200 capacity, per year (model base case)
GPUs per MW of IT load600
Capital cost, all-in$32.5m
Revenue, rented on take-or-pay terms (Nebius Q2 deals; IREN)$20.0m
Payback on that rent (IREN says about two years)2.1 years
Revenue selling tokens at DeepSeek's list price, 70% utilisation$41.0m
Token price that only matches the rent at 70% utilisation$0.37 per M

The capital side of the model reproduces the paybacks the companies disclose, which is some comfort. The token side rests on one independent benchmark and an assumed utilisation, which is less. Both are in the model, with every input sourced or marked as an assumption.

Going to ground

First order

The serving stack is the product. Inference has two halves with opposite appetites. Reading the prompt is "compute-bound", in the words of the Sarathi-Serve paper; writing the answer is "memory-bound", one token per request at a time. Memory is the bill. Llama 3.1 70B keeps 320 KiB of attention state for every token in context, so one 128,000-token conversation holds 42.9 GB, more than half of an H100's 80 GB.

Everything clever in serving is a way to waste less of that memory and time. PagedAttention, the idea behind vLLM, delivered two to four times the throughput at the same latency, because older systems used only 20.4% to 38.2% of the memory they reserved. Splitting prompt-reading from answer-writing (DistServe) served 7.4 times as many requests within latency targets. On identical Nvidia GB300 racks, the throughput reported in MLPerf for DeepSeek-R1 rose from 2,907 to 8,064 tokens a second per GPU between the 2025 and 2026 rounds, gains Nvidia credits to software.

These gains don't stack, and they depend on the workload. Speculative decoding gives 3.0 to 6.5 times at a batch of one and 1.38 times at a batch of 64. In a simulated agent run, re-reading a growing context accounts for 88% of the serving cost, so prefix caching is worth more to an agent than a faster decoder. Knowing which trick fits which workload is the job.

Second order

The pricing unit decides who keeps the gain. Rent a GPU by the hour and any throughput the provider engineers into it belongs to the tenant. In the model, a 20% gain on one megawatt is worth about $8.2m a year of extra tokens at list price. Under take-or-pay, the provider's share is nothing.

Sell tokens instead and the provider keeps between $1.5m and $8.2m of it, depending on whether competition forces the saving into prices and whether demand grows to fill the freed capacity. That explains an awkward fact. The fastest-growing inference businesses rent their chips. In July, IREN named Fireworks AI and Together AI among its cloud customers, and Fireworks has reported annualised revenue above $1bn. The neoclouds are their landlords.

Their own inference lines are small. CoreWeave's managed-inference bookings reportedly passed $100m of annual revenue, against $2.6bn of revenue in the second quarter alone. Crusoe's contracted figure passed $100m, about $65m of it one dedicated cluster for Thinking Machines Lab. Nebius says inference workloads on its Token Factory "more than tripled" in the quarter, and gives no revenue figure.

Third order

Selling workloads moves the sales team, and the risk with it. A capacity sale qualifies megawatts, credit and a delivery date. A workload sale qualifies token shapes, latency targets and cache hit rates, then proves the economics with a benchmark on the customer's own traffic. The hiring has started: Nebius is recruiting an "Account Executive, Token Factory" for adoption and consumption, and a principal solutions architect for it at $208,000 to $261,000.

From GPU rental to workload partner: evidence score, 0 to 6
CoreWeave, Nebius, Crusoe, AWS, Azure4.5
Google Cloud (top rung not assessed)4
Lambda (inference API closed), IREN2
Applied Digital (landlord)1

Nobody reaches the sixth rung, pricing on outcomes. And the risk travels with the meter. Take-or-pay leaves idle hours with the tenant; selling tokens hands them to the provider. At $20m a megawatt for capacity, tokens at DeepSeek's list price beat the rent above about 34% utilisation. At $40m, it takes 68%.

Lenders notice. Committed contracts are what finance the build: 70% of Nebius's second-quarter deals came with prepayments covering 50% to 60% of the capital cost. A token business has to win that credit back with consumption history. That takes years, not quarters.

Ears up: what to watch

  • CoreWeave's committed share of revenue, 98% for two years running, and managed inference against its reported $250m target for the end of 2026.
  • Token Factory revenue. Nebius discloses inference volume; the first dollar figure will say more.
  • Contract value per megawatt: whether Nebius's $40m short-term deals hold or slide back towards $12m to $20m.
  • Throughput-priced products from a neocloud. Of the providers studied, only the hyperscalers sell reserved throughput units.
  • Same-model gaps on Artificial Analysis: if speed and accuracy converge at one price, serving is commoditising too.

Bone of contention

The strongest case against my argument is that capacity isn't commoditising at all. It is getting dearer. Nebius's contract value per megawatt rose from $12m in its 2026 base to over $20m on second-quarter deals and over $40m on short-term deals in the third, and one-year H100 contracts reportedly rose from $1.70 to $2.35 an hour between October 2025 and March 2026. Nebius itself names "the two constraints in this industry: capital and capacity". While that holds, renting the rack is the better business, and the frontier chart says so.

The biggest buyers also want the rack, not the tokens. Labs and hyperscalers run their own serving stacks, and they are most of the revenue. The evidence has limits too. Speeds come from one Google Cloud region over five days, and the accuracy runs are months old. The model's throughput comes from a single benchmark, which may count input and output tokens together, and its utilisation is an assumption. No neocloud discloses how its revenue splits between training and inference.

So I hold the thesis as a direction, not a description. The engineering is real, and it will matter more as capacity loosens. For now, the money is in the megawatt.

J. & the researcher

Sources
  1. Artificial Analysis, gpt-oss-120b providers: list prices, output speed, time to first token and endpoint accuracy, observed 4 November 2026; methodology v3.0.0 (30 October 2026)confirmed
  2. CoreWeave, Form 10-Q for Q2 2026: committed contracts 98% of revenue, take-or-pay, top three customers 36%, 26% and 10%, remaining performance obligations $103.7bn; Q2 2026 results: revenue $2,575mconfirmed
  3. Nebius, Q2 2026 shareholder letter (Form 6-K): contract value per MW, prepayments, Token Factory inference workloads, "capital and capacity"confirmed
  4. IREN, FY26 results (Form 8-K): contracts above $20m revenue per MW of IT, about two-year paybackconfirmed
  5. IREN, 20 July 2026: Fireworks AI and Together AI as customers; Fireworks run-ratereported
  6. CoreWeave Q2 2026 call: managed-inference bookings and the $250m target; Crusoe and Thinking Machines Labreported
  7. Lambda, Inference API documentation: shut down 25 September 2025reported
  8. Nebius job listings, Account Executive, Token Factory and Principal ML Solutions Architect, seen 4 November 2026reported
  9. Papers: Sarathi-Serve (prefill and decode), vLLM / PagedAttention, DistServe, EAGLE-3; Llama 3.1 70B configuration from the Llama 3 paperconfirmed
  10. MLPerf Inference DeepSeek-R1 results on GB300 NVL72, v5.1 and v6.0, per GPU derived from system resultsreported
  11. SemiAnalysis InferenceX: DeepSeek R1 throughput on GB200 NVL72, 22 May 2026 (model input)reported
  12. One-year H100 contract prices, citing SemiAnalysisreported
  13. Go to Ground model: GB200 NVL72 at 600 GPUs per MW of IT load, capital cost from a SemiAnalysis estimate, EIA industrial power price, CBRE colocation rent, 70% utilisation and other assumptions as listedscenario

Model outputs: neocloud-gtm-model.csv; every input, its range and its source: neocloud-gtm-assumptions.json. Provider scores use one rule: a rung counts only with a dated, generally available product or disclosed commercial evidence, and an announcement or preview counts as half. Artificial Analysis figures are quoted with attribution and are not offered as a download.