What is HBM Memory, and Why Does It Matter for AI Pricing

The basic problem HBM solves. Training a large language model is, at its core, a data-movement problem. The GPU's processing cores are fast enough — the bottleneck is feeding them data quickly enough to keep them busy. Conventional DRAM, mounted on a PCB some distance from the chip, cannot do that. HBM solves it by stacking multiple DRAM dies vertically, connecting them through thousands of through-silicon vias (TSVs), and mounting the entire stack directly on the same silicon interposer as the logic die. The result is a memory interface orders of magnitude wider and faster than anything a standard slot can achieve.

The numbers are stark. A DDR5 stick delivers roughly 67 GB/s. HBM3E — the standard in 2026's flagship accelerators — delivers 1,180 GB/s per stack: nearly 18× more. HBM4, now in volume production, pushes past 1,500 GB/s. An NVIDIA B200 carries eight HBM3E stacks; aggregate bandwidth available to its compute cores exceeds 8 TB/s. No other memory architecture comes close.

Why the cost matters so much right now. HBM is not cheap to make. Each gigabyte consumes roughly three times the wafer capacity of equivalent DDR5, reflecting yield loss from stacking, the TSV process, and wafer thinning requirements. The result is a 5–6× price premium per gigabyte over standard DRAM. HBM3E costs approximately $8–10 per GB against DDR5's $1.50–2. That arithmetic reaches into every AI accelerator on the market: HBM now accounts for 30–40% of the total manufacturing cost of a flagship AI chip. For the B200, memory alone costs roughly $2,400 — exceeding the logic die itself.

The supply side compounds this. Three companies — SK hynix, Samsung and Micron — control essentially all HBM production. SK hynix holds 50–55% of the market and is sold out through 2026. Samsung began commercial HBM4 shipments in February 2026. Micron is expanding aggressively but from a small base. With hyperscalers signing multi-year lockup contracts covering an estimated 35–40% of global DRAM wafer capacity through 2029, the spot market for HBM does not meaningfully exist.

The ripple effect on AI pricing. When memory is both scarce and expensive, the cost does not stay inside the data centre. Cloud providers renting GPU instances to AI developers pay through higher hardware acquisition costs. Developers building on those instances pay through instance pricing that does not fall as fast as per-token model costs. And device makers — from laptop OEMs to smartphone brands — pay because every HBM wafer start is a consumer DRAM wafer start that did not happen. Gartner estimates DRAM contract prices rose 125% across 2026, with meaningful relief not expected before late 2027.

The generation transition makes this more acute, not less. HBM4 introduces a custom logic base die — the base layer can now be specified by the chip buyer. What that means in practice: already-scarce advanced packaging capacity gets further fragmented by customer-specific variants. The bespoke nature of 2027's HBM market means supply cannot simply be reallocated when demand shifts.

Three things to watch. The HBM market is not a normal commodity cycle, and the signals that matter are not the usual ones.

First: wafer allocation. Any revision to the multi-year DRAM wafer lockups held by hyperscalers will move prices faster than new capacity additions. Second: HBM4E customisation traction. The degree to which large AI labs adopt custom base dies determines whether the supply pool splinters — raising costs — or consolidates into standard parts. Third: China's domestic HBM program. CXMT is working toward domestic HBM production, but TSV and packaging steps remain well behind Korean leaders. If China closes the gap faster than expected, it adds capacity to a constrained market; if it stays constrained, Huawei's Ascend accelerator program hits a ceiling in memory, not logic.

The HBM market will reach an estimated $58 billion in 2026 — up from $16 billion two years prior — on track for $90 billion in 2027, with Micron's CEO guiding a total addressable market above $100 billion by 2030.

What customisation means for pricing power. The introduction of custom base dies in HBM4E changes competitive dynamics in ways headline market-share numbers do not capture. When a hyperscaler embeds its own logic into the base die, it becomes technically dependent on a specific vendor's process — creating switching costs that did not exist when HBM was a standard part. That favours memory makers in the short term; it locks in relationships and justifies higher pricing for bespoke configurations.

The Micron–TSMC partnership for HBM4E is the clearest expression of this logic: Micron brings DRAM expertise, TSMC brings the advanced logic process for the base die, and together they can offer a fully integrated solution that neither could supply alone. Samsung's advantage is that it can do both in-house — but it first needs to prove yield consistency on 12-high HBM3E before those integrated ambitions translate into market share.

The forecast, and the honest uncertainty around it. HBM demand grows 77% in 2026 and 68% in 2027 per TrendForce. Those numbers rest on one assumption: that the hyperscaler capex programmes underpinning them do not slow materially. Cloud giants have guided aggregate 2026 data-centre investment above $725 billion; if that number revises down in late 2026 earnings calls, HBM demand projections follow within one quarter.

The structural case is sound regardless of timing. Every new GPU generation demands more HBM per unit — NVIDIA's Rubin Ultra targets 512 GB per GPU, versus 80 GB for the H100 two years ago. That is a 6× increase in memory per chip in a single product generation. The industry can debate when AI capex plateaus. It cannot debate the direction of per-chip memory requirements. HBM's share of accelerator cost goes up before it comes down, and any organisation that buys, builds or prices around AI infrastructure needs to model that assumption explicitly — not treat it as a footnote.