A.I.
Hardware · Reference
HBM memory
HBM, or high-bandwidth memory, is a stack of memory chips placed right beside an AI chip in the same package. It holds the model's weights (the numbers the model learned) and its working data. Because the chip has to read those numbers for every word it writes, memory speed often limits AI speed more than raw math does. The memory is DRAM, the same kind of working memory a laptop uses, stacked vertically and wired through the stack.
Generations
SK hynix, one of the main HBM makers, lists six generations: HBM, HBM2, HBM2E, HBM3, HBM3E, and HBM4 (SK hynix, 12 Sep 2025). SK hynix says HBM4 doubles the data lanes to 2,048, from 1,024, runs above 10 Gb/s per lane against the 8 Gb/s JEDEC standard, and is over 40% more power efficient than HBM3E. JEDEC is the industry body that writes memory standards. These are SK hynix's own figures.
What current chips carry
| Chip | Memory | Bandwidth | Source page |
|---|---|---|---|
| Nvidia Rubin GPU | 288 GB HBM4 | 19.2 TB/s | Rubin |
| AMD MI455X | 432 GB HBM4, 12 stacks | 23.3 TB/s | MI455X |
| Microsoft Maia 300 | 288 GB HBM4, 8 stacks | 18 TB/s | Maia |
| OpenAI Jalapeño | 216 GiB HBM4 | 15.4 TB/s | Jalapeño |
| Google TPU 8i | 288 GB HBM | not stated | TPU 8 |
| AWS Trainium3 | 144 GiB HBM3e | 4.9 TB/s | Trainium3 |
Each figure is the chip maker's own and comes from the page linked in its row. GB is gigabytes; a GiB is about 7% more. TB/s is terabytes per second.
The other path: SRAM
Some inference chips lean on SRAM, a smaller, much faster memory built into the chip itself. Groq 3 LPX uses 500 MB of SRAM per chip and no HBM, and Cerebras puts 44 GB of SRAM on one wafer-sized chip.