Capital & Compute

Is Local AI a Waste of DRAM? Altman's Claim Checked

Partly. Sam Altman is right that idle RAM makes a local token cost more, but data-center HBM, not home PCs, is what is eating the world supply of DRAM.

· ai· local-llms· hardware· economics· By Capital & Compute
A Crucial DDR4 laptop RAM module installed in a laptop motherboard memory slot
Photo: Crucial CT4G4SFS8213 on laptop motherboard 20171218 by Diego Torres Silvestre, CC BY 2.0, via Wikimedia Commons. Cropped.

Local AI is not a meaningful waste of DRAM. Sam Altman is right that a home machine sits idle most of the day, which makes a local token cost up to 13 times more than a busy one. But the shortage is driven by data-center HBM, which burns about three times the wafer per gigabyte that laptop RAM does.

That is the short answer to the claim the OpenAI CEO made on the Cheeky Pint podcast on October 29, 2026, which r/LocalLLaMA promptly turned into a thread summed up as “Guy selling product”. The reaction misses that half of his argument holds up. The other half does not, and the numbers show where it breaks.

What did Sam Altman say about local inference?

Altman made the remark about 25 minutes into Why Sam Altman is against a single superintelligence, a 53-minute episode of Stripe co-founder John Collison’s Cheeky Pint show published on October 29, 2026. Collison had asked why memory makers were not flooding the market when prices are this high. Altman answered that DRAM makers remember their boom-and-bust history and keep capital spending conservative, and that “the invisible hand is failing right now.” Then he turned to local inference. From the official episode transcript on Cheeky Pint’s Substack:

A lot of people want local inference. And somehow, they assume that the problem is solved now. Because, we are going to have open-source models. We're going to be able to run them on our computers. And so we're not going to need all the stuff in the data center. But from a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer. And, being used 5% of the time. You want maximum utilization in the cloud if you're trying to deal with this very supply constrained environment.
Sam Altman, CEO, OpenAI, on Cheeky Pint with John Collison, October 29, 2026

The headline phrase that went viral, “a terrible allocation of scarce resources”, is a paraphrase. He said “quite terrible” from “a global allocation perspective.” The meaning is close, but the argument is narrower than the headline: it is about DRAM in a shortage, not about local AI in general.

Two claims sit inside that quote. One is economic: idle hardware makes expensive tokens. The other is about supply: personal computers are hoarding a scarce resource. They need testing separately.

Is he right that local hardware sits idle?

Yes, and this is the part the backlash skips. A personal machine runs a model only while its owner is typing or an agent is working. The rest of the time its memory holds weights that produce nothing. A cloud cluster batches thousands of users onto the same GPUs, so the same memory serves requests almost continuously.

The site’s own self-hosted LLM cost model puts numbers on it. A used triple RTX 3090 rig running flat out, 24 hours a day, produces output tokens near $0.74 per million. Push a heavy personal workload of 10 million tokens a month through the same box and the fixed cost spreads over far fewer tokens: about $10 per million, roughly 13.5 times more. Hosted open-model APIs sit between $0.10 and $0.90 per million.

Cost per million output tokens: busy hardware vs personal useA log-scale ladder of cost per million output tokens. The cheapest hosted open-model API is about $0.10, a triple RTX 3090 rig running flat out 24/7 is about $0.74, the priciest hosted open-model API is about $0.90, and the same rig at a personal 10 million tokens a month is about $10, roughly 13.5 times the flat-out cost.$0.10$1.00$10.00cost per million output tokens, USD (log scale)Cheapest hosted open-model API$0.10Local rig, running 24/7$0.74Priciest hosted open-model API$0.90Local rig, 10M tokens a month$10.00
Cost per million output tokens: busy hardware vs personal use
ToolCost per taskMultiple of baseline
Cheapest hosted open-model API$0.10-
Local rig, running 24/7$0.74-
Priciest hosted open-model API$0.90-
Local rig, 10M tokens a month$10.00-
The same local hardware is cheap per token only when it is busy. Idle time, not the hardware, is what makes a personal token expensive.Source: Capital & Compute self-hosted cost model (3x RTX 3090), hosted open-model API list prices, checked October 31, 2026

So on cost alone, Altman’s intuition matches the data. For most individuals, the hosted API is cheaper than owning the hardware, and the reason is exactly the one he gave: utilization.

How much of the DRAM shortage do personal computers cause?

Very little, and here the argument turns around. The 2026 memory squeeze is not a story of laptops absorbing RAM. It is a story of memory makers moving wafers to high-bandwidth memory (HBM), the stacked DRAM bolted next to every AI accelerator.

According to a June 2026 TrendForce estimate covered in the site’s 2027 shortage forecast, HBM takes about 22% of the top three makers’ DRAM wafer input in 2026 but delivers only about 9% of the bits, rising to 30% of wafers for 13% of bits in 2027. A widely used industry rule of thumb, reported by Tom’s Hardware, is that a gigabyte of HBM consumes about three times the wafer capacity of a gigabyte of DDR5, with HBM4 possibly closer to five times.

That reverses the allocation logic. Measured in wafers, the scarce input, a gigabyte of memory in a data-center GPU costs the world roughly three gigabytes’ worth of PC memory. The cloud runs it at higher utilization, which claws some of that back, but it starts from a large deficit. Cloud inference is not a free-lunch use of DRAM; it is the most wafer-hungry form of it.

Altman is open about his position. In the same conversation he described OpenAI’s own inference chip and a plan to “make a lot of really cheap compute”, and later he said, on a separate point about token spending, “I’m really talking about my book here.” The supply argument is coming from the largest buyer in the market.

Does running a local LLM make RAM prices worse?

At the margin, a little. A person who buys a 64GB or 128GB machine to run a 70B model is adding real demand, and enthusiasts who upgraded in 2026 paid for it: conventional DRAM contract prices are still rising, by 10 to 15% quarter on quarter in 4Q26 per TrendForce, after much larger jumps earlier in the year. The memory price tracker shows where retail kits stand now.

But most local inference does not add DRAM at all. It runs on memory that was already in the laptop, the Mac or the gaming PC for other reasons. Altman’s “5% of the time” figure is true of that memory whether or not it ever holds a model; it was idle before local AI arrived. Running a 7B or 14B model on a 16GB or 32GB machine you already own takes nothing new from the supply chain. The RAM sizing guide shows what fits at each size.

The demand that moves prices is measured in hundreds of thousands of wafers a month and comes from hyperscalers signing long-term agreements, which is why the site’s explainer on why RAM is so expensive traces the shortage to AI servers rather than to consumers.

When does local AI still make sense?

Utilization is one reason to pick a deployment, not the only one. Here is how the two options compare on the things a buyer actually weighs.

What you care about Local inference Cloud API
Price per token at personal volume Higher (about $10 per million on a 3x 3090) Lower ($0.10 to $0.90 for open models)
Price if hardware is already owned Electricity only Same as above
Availability Works offline and during provider outages Depends on the provider and rate limits
Privacy Data never leaves the machine Governed by the provider’s policy
Refusals and policy changes You choose the model and its behavior Provider can restrict requests at any time
Model quality ceiling Open models only Frontier models included

The refusal row is the one r/LocalLLaMA kept returning to. Developer Péter Szilágyi put it on X by pairing Altman’s quote with a ChatGPT refusal screen for a cybersecurity request. That is a control argument, not a cost argument, and a utilization chart does not answer it. For the strongest open models to run this way, see the best local LLMs that rival paid AI.

The verdict on Altman’s claim

Half right. He is correct that idle memory makes local tokens expensive, and most people would pay less per token in the cloud. He is wrong that personal computers are what is squeezing the world’s DRAM: the squeeze comes from HBM for data centers, which uses about three times the wafer per gigabyte, and his own company holds letters of intent for up to 900,000 DRAM wafer starts a month.

The practical rule follows from both halves. Run local models on the memory you already have, and whenever privacy, offline access or control matters. Do not buy new RAM at shortage prices purely to beat API token rates, because at personal volume the cloud wins that comparison.

Frequently asked questions

What did Sam Altman say about local inference?
On the Cheeky Pint podcast published October 29, 2026, Sam Altman said that from a global allocation perspective it is "quite terrible" for everyone to keep part of the world's limited DRAM supply in their own computers, used about 5% of the time, and that the cloud offers maximum utilization in a supply-constrained market.
Is local AI a waste of DRAM?
Not meaningfully. Local AI usually runs on RAM people already own, so it adds little new demand. The 2026 DRAM shortage is driven by data-center HBM, which takes about 22% of DRAM wafers for about 9% of bits and uses roughly three times the wafer per gigabyte of DDR5.
Is running a local LLM cheaper than using an API?
Usually not at personal volume. A triple RTX 3090 rig costs about $0.74 per million output tokens when run 24/7, but about $10 per million at 10 million tokens a month, while hosted open-model APIs charge $0.10 to $0.90 per million.
Does running local LLMs make RAM prices go up?
Only slightly, and only when people buy extra memory for it. Hyperscaler demand, including OpenAI's Stargate letters of intent for up to 900,000 DRAM wafer starts a month, dwarfs anything hobbyists buy.

Sources

Get each breakdown before it makes the rounds

You get one email when a new source-backed analysis goes live: what AI agents actually cost, which models are worth running, and what the benchmarks really mean. No hype.

No spam. Unsubscribe anytime.

← Back to Local AI & hardware