From copper to CPO: The next shift in AI interconnects

As optical transceivers continue to advance in bandwidth, power efficiency, and integration level, their influence is extending beyond the data-centre network and into the way AI systems themselves are architected. Increasingly, the transition toward optical interconnects is not driven by raw bandwidth demand alone, but by the growing tension between electrical SerDes scaling, system power budgets, and physical architecture constraints.

To understand how this shift is unfolding, it is useful to first distinguish between scale-up fabrics, connectivity in AI systems.

Scale-up vs. Scale-out, and Serdes challenges
Scale-up refers to maximising performance within a tightly coupled system, such as a single server or accelerator domain. The goal is to aggregate more compute, memory, and bandwidth while maintaining extremely low latency and tight synchronisation.

Physically, scale-up fabrics operate over short reach, typically within a server or across a single rack, often well under ten metres. In this domain, high-speed copper interconnects remain dominant, supported by mature electrical SerDes and protocols such as NVLink and emerging open alternatives.

Scale-out, by contrast, distributes workloads across many servers to increase total system throughput. Once communication extends beyond a rack or row, optical interconnects become essential. Ethernet and InfiniBand therefore form the backbone of today's large-scale AI clusters, enabling high-bandwidth, power-efficient communication over tens to hundreds of metres.

A simplified AI accelerator architecture illustrates how these two domains coexist. At the compute layer, accelerators connect upward into L1 compute switches using very high-bandwidth copper links. These are classic scale-up connections: short, dense, and optimised for moving massive volumes of data with minimal latency. L1 switches then interconnect with each other, again over copper, forming a tightly coupled fabric that allows multiple accelerators to behave almost like a single large device from a software perspective.

As traffic moves higher up the hierarchy, it aggregates into L2 network switches that interface with the broader data-centre network. At this level, optical pluggables dominate, as the system must support longer reach, higher port counts, and scalable bandwidth growth.

The growing challenge across both domains is that electrical SerDes, while still advancing, is increasingly constrained at the system level. On silicon, SerDes continues to scale from 112G to 224G PAM4 and beyond. However, as data rates rise, the electrical channel, including packages, substrates, PCB traces, connectors, and cables, becomes the limiting factor. Maintaining signal integrity over distance requires increasingly aggressive equalisation and DSP, driving up power per bit and adding thermal load.

For large AI switches and accelerator fabrics with thousands of SerDes lanes, even modest increases in energy per bit translate into hundreds of watts at the rack level. As a result, SerDes is no longer just a circuit-level concern; it has become a first-order architectural constraint.

Co-packaged optics (CPO) -- what role does it play?
This is where optics, and in particular co-packaged optics (CPO), begins to reshape system design.

According to the IDTechEx report "Co-Packaged Optics (CPO) 2026-2036: Technologies, Market, and Forecasts", the short- to mid-term evolution of AI system architectures will be incremental rather than disruptive. Copper remains highly effective in the scale-up domain as long as links stay short, controlled, and power-efficient.

Leading platforms, particularly those led by NVIDIA, continue to push copper aggressively for scale-up fabrics, citing its low latency, cost advantages, and, critically, its reliability at scale. From this perspective, optics is not yet a compelling replacement for copper within tightly coupled GPU fabrics.

The most immediate pressure point instead lies at the network switch layer.

Today's scale-out connectivity relies on pluggable optics mounted at the switch front panel. However, as switch ASIC bandwidth increases from tens to hundreds of terabits per second, this pluggable model faces growing constraints in power consumption, signal integrity, and front-panel density. Simply scaling electrical SerDes to ever higher data rates becomes increasingly inefficient once links must traverse long electrical paths to the front panel.

CPO addresses this by placing optical engines much closer to the switch ASIC, often within the same package. By dramatically shortening the electrical path, CPO reduces I/O power, improves signal integrity, and enables higher aggregate bandwidth scaling without relying on increasingly complex electrical channels. In effect, optics absorbs the reach problem, while electrical SerDes is confined to its most efficient operating regime.

This selective deployment strategy aligns with how different industry players view optical adoption. While NVIDIA remains firmly copper-first for scale-up, it has been clear that optics, including CPO, plays a critical role in scaling network fabrics. Other players, such as Marvell and Broadcom appear more open to introducing optical links in scale-up fabrics.

In the longer term, the boundary between scale-up and scale-out may itself become less rigid. As accelerator counts per logical node increase and systems span larger physical footprints, copper-based scale-up fabrics will face mounting pressure from power density, airflow, and cabling complexity, even if they remain electrically viable. 

In such scenarios, optical I/O may begin to play a role in scale-up as well, particularly for inference-optimised architectures where throughput per watt outweighs ultra-low latency.

Key takeaway
The key takeaway is that optics will not replace copper everywhere at once. Instead, AI system architectures are evolving through a pragmatic division of labour: copper dominates where latency and reliability matter most and reach is short, while optics expands wherever electrical SerDes scaling collides with power, reach, and density limits. Co-packaged optics represents a critical inflection point in this evolution, not by eliminating SerDes, but by containing it within its optimal physical and economic envelope.

-- Dr Yu-Han Chang, IDTechEx Principal Technology Analyst, USA.