TSMC announces plans for giant AI processors to meet the surging demand for compute

Big picture: As TSMC prepares for a physical expansion of its chip packaging, the semiconductor industry is nearing a significant milestone. At its recent North American Technology Symposium, the company detailed plans for a new generation of CoWoS (Chip-on-Wafer-on-Substrate) technology, enabling the assembly of multi-chiplet processors much larger than those currently in production.

High-end processors today, including those that power data centers and AI workloads rely on multichiplet designs in order to meet the soaring demands of performance and memory bandwidth. TSMC’s CoWoS solutions are able to accommodate interposers as large as 2,831 mm2, which is more than three times larger than a standard photomask reticle. EUV lithography limits the size to 830 – 858 mm2.

The technology is already used in products such as AMD’s Instinct MI300X GPUs and Nvidia B200 GPUs which combine large compute chipsets with stacks high-bandwidth memories.

However, as AI and high-performance computing applications continue to grow in complexity, the appetite for even more silicon is only increasing. To address this, TSMC is The development of is a new CoWoS packaging technology that will be launched as early as next summer. It supports interposers up to 4,719 mm2, which is about 5.5 times larger than the reticle limitation. Substrates measuring 100x100mm are required. This will allow for up to 12 stacks high-bandwidth memories, a significant improvement over current capabilities.

TSMC estimates that chips built using this technology will deliver over three-and-a-half times the compute performance compared to today’s leading designs. This could meet the needs of upcoming CPUs such as Nvidia Rubin GPUs.

TSMC is planning to push the envelope even further with a larger package. A 7,885 mm2 Interposer mounted on a substrate of 120×150 mm, a footprint that is slightly larger than a CD case. This is a 9.5-fold improvement over the previous 8x-reticle package.

Such a massive assembly could host four 3D-stacked systems-on-integrated chips, twelve HBM4 memory stacks, and multiple input/output dies, setting a new benchmark for performance and integration.

TSMC offers its System-on-Wafer technology (SoW-X), which allows the integration of entire chips into a single wafer, to customers with the highest performance requirements. TSMC expects that wafer-level integration will be used more widely as the demand for supersized chips increases. Cerebras and Tesla are currently the only companies using this technology for specialized AI processing.

These behemoth CPUs present a number of engineering challenges. Innovative solutions are needed to deliver power to large multi-chiplet assemblies, which can draw kilowatts, far more than traditional server designs can handle.

TSMC integrates advanced power management circuits into the chip package to combat this. The company embeds monolithic ICs for power management and on-wafer-inductors in the CoWoS L substrate using its N16 FinFET Technology, allowing the power to be routed through the package efficiently.

Using this approach, electrical resistance is reduced and power integrity is improved. This allows for dynamic voltage scaling and rapid responses to changing workloads. Embedded deep-trench capacitors stabilize electrical performance by filtering out voltage fluctuations, ensuring reliable operation even under heavy computational load.

The advances in this area reflect a shift towards system-level optimization, where power delivery and packaging are treated as interconnected components rather than separate concerns.

The move to larger chip packages is not without complications. The new substrates are larger than existing module standards, such as OAM 2.0. This may require new approaches to system and board design.

Thermal Management is another critical challenge. As processors become larger and consume more power, they generate a lot of heat. Hardware manufacturers are exploring advanced techniques of cooling, such as direct liquid cooling and immersion to keep these chips operating efficiently.

TSMC and partners have already developed immersion cooling solutions for data centres. These solutions can reduce energy consumption and stabilize chip temperature, even when under heavy workloads. The future will be a challenge in integrating these cooling techniques directly into chip packages.