Samsung outlines a three-phase HBM roadmap: moving towards true 3D ZHBM, directly stacking DRAM onto the computing chip.

Samsung's three-stage HBM evolution roadmap aims to achieve the "zHBM" architecture, which vertically stacks DRAM on top of computing chips. Samsung claims that zHBM offers a 70% reduction in power consumption and a 230% increase in bandwidth compared to HBM4E, while also freeing up an additional 100W of thermal headroom for the GPU.

Recently, at the Hot Chips technology conference, Sangwook Han from Samsung's DRAM design team gave a presentation, officially revealing Samsung's three-stage roadmap for the evolution of HBM. The endpoint of this roadmap is a brand-new architecture called zHBM—which directly and vertically stacks DRAM on computing chips such as GPUs and TPUs, completely eliminating the 2.5D interposer.

According to a report by wccftech on August 23, Samsung stated that compared to the standard HBM4E, the zHBM solution claims to achieve a 70% reduction in power consumption, a 230% increase in DRAM bandwidth, and a 100W power saving per DRAM module, while freeing up 8.3% of additional power headroom for the GPU.

What is HBM, and where are the bottlenecks?

HBM, or high-bandwidth memory, is one of the most critical storage components in current AI training and inference systems. It consists of two types of chips:

C-die (core chip) : Contains DRAM memory cells and can be vertically stacked up to 16 layers.

B-die (Base Die) : Located at the bottom of the entire stack structure, it is responsible for handling various DRAM control functions and communicating with computing chips such as GPUs through the PHY (Physical Interface Layer).

The two are connected via a TSV (Through-Silicon Via) – a vertical conductive channel that penetrates the chip.

According to Wccftech, the bandwidth of each HBM4 stack currently exceeds 3TB/s, HBM4E further advances to the 4TB/s range, and HBM5 doubles the bandwidth of HBM4, with a capacity exceeding 60GB.

However, the continued expansion of bandwidth faces two major constraints: the physical limits of the number and spacing of TSVs , and the upper limit of the number and speed of I/O of the PHY interface within the Base Die . At the same time, the process generation gap between the B-die and the computing chip (xPU SoC) is narrowing generation by generation, which is both a challenge and an entry point for Samsung's roadmap.

Phase 1: Making room for computing chips

The core objective of the first phase of Samsung's roadmap is to "reclaim" silicon area from xPUs (computing chips) .

Samsung has applied D1c and 4nm logic processes to the HBM4 Base Die (B-die), primarily to reduce power consumption and shrink the effective area. This marks the beginning of the true integration of DRAM and advanced logic processes.

Specific measures include:

Replacing the traditional HBM PHY with a D2D interface shortens the channel length, directly improving energy efficiency, while freeing up valuable silicon space for the XPU.

Offloading the memory controller from the XPU to the cHBM B-die is expected to free up 5% to 10% of the XPU area, corresponding to a 10% to 20% performance improvement.

Introducing a fine-grained SRAM-based repair scheme (Near-MC SRAM-Based Cell Repair), utilizing idle space on the B-die to deploy SRAM repair resources.

One side effect of the reduced area is the hotspot issue. To address this, Samsung has introduced Heat Path Block (HPB) technology, built on top of the cHBM4 solution, which can reduce peak temperatures by more than 35% and cover 50% of the PHY area.

Phase Two: The B-die begins to "grow" computing power

The focus of the second phase shifted from "creating space" to "adding features," which Samsung defines as the feature expansion phase .

Expanding memory capacity. As the context window of large AI models expands dramatically, the demand for key-value caches (the memory area used to store intermediate states during model inference) is growing exponentially. Samsung plans to integrate a memory expansion controller and PHY on the unused silicon area of the base die, expanding the system's available memory capacity through external LPDDR or HBM solutions.

Integrated Processing Units (PEs). Samsung also proposed integrating some computing processing elements (Processing Elements) on the Base Die, offloading some of the computations that would otherwise need to be performed on the xPU to the memory side, thereby reducing D2D bandwidth requirements, power consumption, and thermal burden. This form factor is called AHBM (Advanced HBM).

Enhance reliability and testing capabilities. The second phase also includes integrating advanced RAS (Reliability, Availability, Serviceability) sensors and real-time telemetry functions, as well as on-chip self-test (ATIP) capabilities, into the Base Die to improve yield and test coverage.

Phase 3: zHBM – Eliminating the interposer layer, DRAM is directly applied to the computing chip.

The third phase is the ultimate form of the entire roadmap: zHBM .

Current mainstream AI systems use a 2.5D packaging architecture, where the GPU and HBM are placed side-by-side on the same interposer, transmitting data through horizontal interconnects. The zHBM approach is to "vertically" integrate this structure—by stacking DRAM directly onto the xPU chip, forming true 3D vertical integration.

Samsung describes the key features of zHBM as including:

Distributed I/O : Minimizing data transfer distance within the HBM stack

3D structure : Eliminates traditional 2D interfaces, significantly improving system efficiency.

The target I/O power consumption is approximately 0.5 pJ/bit : achieved by removing redundant modules such as SerDes.

Bandwidth increased by more than 2.3 times, system thermal headroom reaches 100W.

Samsung's demonstration solution involved stacking a quad-stack zHBM layer on top of an XPU.

To achieve these goals, Samsung is developing two key packaging technologies: WoW (Wafer on Wafer) and HCB (Hybrid Cube Bonding) to achieve ultra-high I/O density and ultimately build a unified SoC-DRAM co-design system.

The core logic behind the roadmap

The core narrative of Samsung's presentation was to reposition HBM's Base Die from a "passive data transfer station" to an "intelligent partner with proactive computing capabilities."

The evolution logic of the three stages is clear: first, the area is reduced and xPU space is freed up through process upgrades (first stage); then, more functions are integrated and capacity and computing power are expanded by utilizing the freed space (second stage); finally, the system architecture is completely reconstructed through 3D vertical integration, achieving breakthroughs in power consumption, bandwidth and thermal management simultaneously (third stage).

In its concluding remarks, Samsung stated, "By mastering advanced packaging and unified SoC-DRAM co-design, we will overcome the power consumption, area, and capacity bottlenecks that constrain AI systems, paving the way for higher efficiency, higher performance, and greater scalability in the coming years."