Chip Expert: AI Computing Power Bottleneck Is Shifting from Chips to Packaging, Interconnects, and Storage

Beating AI News Flash: In an interview on the Frictionless Podcast, senior chip designer and hardware analyst "Mr. Bubble" stated that the core bottleneck of AI compute infrastructure is shifting from simply improving chip compute power to advanced packaging, system interconnect latency, and memory hierarchy management.

Mr. Bubble believes that the rapid rise in advanced process costs, combined with limited gains in transistor density, is posing an economic challenge to the traditional Moore's Law. Rather than continuing to shrink process nodes, the industry is increasingly connecting multiple chips into a single computing system through advanced packaging. He pointed out that the basic unit of future AI compute may no longer be a single chip or server, but a complete rack or even multiple racks, with PCB, packaging, and interconnect capabilities becoming key limiting factors.

On the memory front, he believes that as AI inference context windows expand, massive historical context should not all occupy expensive HBM. The industry may increasingly adopt flash offloading, keeping high-frequency data in HBM while migrating low-frequency context to larger-capacity, lower-cost storage tiers. He further predicts that 3D DRAM may become an important direction for next-generation memory technology.

Regarding AI infrastructure investment, Mr. Bubble believes that as prefill and decode gradually adopt disaggregated architectures, the importance of system-level hardware design will further increase. He also stated that compared to directly betting on large model companies, enterprises with hardware, packaging, storage, and other key infrastructure capabilities may play a longer-term role in the AI industry chain.