Skip to main content
Data floating in grid on a blue background

WHITE PAPER

Performance Analysis of Various Modulation Techniques Over 448Gbps Channels in Data Centers

Artificial intelligence (AI) and machine learning (ML) are the fastest-growing areas of technology in recent memory, and their growth has driven unprecedented demand for newer, faster, more efficient data center infrastructure. As we advance toward the next generation of high-speed serial communication interfaces, several critical implementation challenges must be addressed to enable 3.2 terabit/port Ethernet technologies. Achieving this bandwidth will require either doubling the number of 224G lanes or increasing the speed of each lane to operate at 448G.

To handle the growing level of AI data processing,   the data center requires scaling AI clusters as well as the ability to transmit more data within each cluster.   As clusters expand to hundreds or thousands of nodes, the interconnect fabric becomes the major bandwidth-limiting element in these communication channels. Copper interconnects remain a viable choice for scaling out AI clusters, but the use of copper at 448G per lane requires a deeper understanding of signal integrity (SI) as it relates to connector design and construction.  

This paper presents results of a study by Molex and a leading global semiconductor vendor, examining the right ways to design the data center and supporting components for optimal performance given the intense demand and high data rates. Specifically, the paper explores one of the most impactful design considerations: the feasibility of using copper interconnects with three modulation options: PAM-4, PAM-6 and PAM-8. Industry standards groups are currently debating these modulation options, and the signal integrity challenges involved in connector construction and interconnect design will depend on which modulation option is ultimately implemented for 448G.

System Architecture at 448G


Rising data rates in serial communication protocols are reshaping system architectures across data centers. In decades past, data rates and channel bandwidth requirements were small enough that losses recognized in the package as well as the PCB could effectively be ignored. This meant connectors required for building server-to-server interconnects could be placed anywhere with respect to an application-specific integrated circuit (ASIC) or processor.

As data rates increased, the cabled connectors used to build scale-up/scale-out architectures moved progressively closer to the ASIC or processor to reduce PCB losses. Systems using today’s fastest serial interfaces bypass the PCB entirely by implementing co-packaged optics or co-packaged copper, a technology in which copper cables (and, in many cases, optical fibers) are integrated directly into the chip package to provide ultra-high-speed connectivity within a device or between chips. With co-packaged copper, those scale-up/scale-out connectors sit on the ASIC package itself, and data is routed from them to the external I/O or backplane connectors through twinax cable, as shown in Figure 1.

Illustration of typical scale-up/scale-out topologies.

Figure 1: Typical scale-up/scale-out topologies

Connectors for co-packaged copper interconnects are surface-mount components, making them vulnerable to signal integrity problems found in other high-density, high-pin-count surface-mount device (SMD) connectors. Any of the following factors can contribute to bandwidth limiting in connectors for co-packaged copper:

  • Stubs in the connector mating interfaces and SMD mounting 
  • Isolation between I/O pins to suppress crosstalk 
  • Small holes that enable electrical connection between layers (vias) in the package routing into an SMD pin on the connector

These challenges would normally be exacerbated when connectors are placed on the PCB, particularly due to vias in the PCB and the insertion loss in long routes where standard PCB materials are used. The extent to which these design factors inhibit the use of specific modulation options in 448G is an open question. Therefore, there is a distinct need for a study to determine the feasibility of the various modulation options with co-packaged copper for 448G channels.

Study on Connector Designs Supporting 448G


The study conducted by Molex and a leading global semiconductor vendor focuses on design factors influencing signal integrity in surface mount technology (SMT) connectors as well as co-packaged copper connectors for 448G channels. It specifically examines how connector design influences insertion loss bandwidth, as well as how noise contributes to signal-to-noise ratios at high frequencies. In essence, the study evaluates the feasibility of using SMT connectors or co-packaged copper to support 448G channels for three modulation options: PAM-4, PAM-6 and PAM-8.

The analysis methodology involved using idealized channel models for each element in the design, such as ball grid array (BGA) and connector attach, to determine which modulation formats can be best supported by existing technology.

Study Findings and Conclusions


Insertion Loss and Channel Reach

To first examine the feasibility of PAM-4, PAM-6 or PAM-8 in 448G channels, it is worth comparing the case of PCB routing against co-packaged copper in terms of insertion loss and channel bandwidth. As expected, initial findings showed that losses in the PCB channel are much higher than those in co-packaged copper. This can be seen quite clearly in Figure 2, which compares routing to an I/O connector on a PCB versus routing between co-packaged copper and an I/O connector over twinax cable.

Illustration and graphs representing comparison insertion loss in 448G channels with PCB routing (left) and co-packaged copper (right).

Figure 2: Comparison of insertion loss in 448G channels with PCB routing (left) and co-packaged copper (right)

While both PCB routing and co-packaged copper with twinax appear to provide sufficient bandwidth beyond the 56 GHz Nyquist frequency for 224Gbps-PAM-4, routing over long distances on the PCB incurs significant loss as shown in the left graph of Figure 2. Co-packaged copper channels allow for longer reach compared to PCB channels for the same insertion loss or, alternatively, co-packaged copper channels have lower insertion loss for the same reach in PCB channels.

Ultimately, nonlinear insertion loss arises near 80 GHz, where steep roll-off effectively terminates the channel bandwidth. Using co-packaged copper with twinax cable between the processor and I/O connector significantly increases the channel reach beyond the 56 GHz Nyquist frequency, which enables 224Gbps-PAM-4 channels. However, neither option provides enough bandwidth to enable the use of PAM-4 in 448G channels.

Significant variation in the channel bandwidth limits can be seen due to channel impairments, as shown in Figure 2. Figures 3 and 4 show which connector design factors can create the variations seen in Figure 2. The results in Figures 3 and 4 show how the insertion loss spectra are affected by changes in the connector mating interface stub length (Figure 3) and the J-lead PCB attachment-related stub length (Figure 4).

Graph representing the variation in insertion loss.

Figure 3: Variation in insertion loss

Figure 3 shows that a baseline design, qualified for use in a 224Gbps-PAM-4 channel, could fail to support 448G at PAM-6 or PAM-8, as these require minimum channel bandwidths of 90 or 75 GHz, respectively. Strong insertion losses arise due to the mating contact tip length and the size of the mating pad stub. Reducing the sizes of both elements in a standard connector design can extend the bandwidth to the point where a copper channel can support 448Gbps-PAM-6 or 448Gbps-PAM-8.

Insertion loss plots showing reduction in SMD pin stub length in baseline design.

Figure 4: Insertion loss plots showing how reduction in SMD pin stub length in a baseline design extends channel bandwidth.

Figure 4 shows the effect of modifying the stub size on the connector’s SMD pad. A reduction of the SMD pad’s stub length also pushes the channel bandwidth limit up to higher frequencies. The increase in channel bandwidth up to higher frequencies is sufficient to enable the use of 448Gbps-PAM-6 and 448Gbps-PAM-8. In all the results presented so far, PAM-4 continues to remain useful for 224G in PCBs and co-packaged copper channels, but not for 448G.

Crosstalk and Noise Injection

High-density connectors with high pin counts cannot always isolate neighboring differential pairs, making it vital to understand the impact of differential crosstalk between neighboring channels.

To better understand the effects of crosstalk on total noise budgets in these channels, Figures 5 and 6 show the power-sum crosstalk (PSNEXT and PSFEXT) penalty in channels with different levels of insertion loss and different insertion loss roll-off frequencies. The baud rate associated with various modulations is also compared.

Three graphs comparing the total noise budget consumed in copper channels with different levels of PSNEXT.

Figure 5: Fraction of total noise budget consumed in copper channels with different levels of PSNEXT.

Figure 6: Fraction of total noise budget consumed in copper channels with different levels of PSFEXT.

These graphs illustrate the noise penalty due to crosstalk as a function of signal-to-noise ratio and signal bandwidth. They demonstrate that, for a given signal level, the noise penalty diminishes alongside a reduction in crosstalk levels. Alternatively, for a given crosstalk level, the noise penalty and signal loss increase together. Co-packaged copper channels have lower signal loss and perform better than PCB-based channels, and PAM-6 modulation has a higher noise penalty than PAM-8 modulation. This is due to the fact that crosstalk levels are significantly higher at 90 GHz for PAM-6 compared to 75 GHz for PAM-8.

Graph showing noise budget consumed in various channels at each modulation based on a combination of PSNEXT and PSFEXT values.

Figure 7: Noise budget consumed in various channels at each modulation based on a combination of PSNEXT and PSFEXT values.

Finally, we see how signal losses and crosstalk contribute to the total noise for each channel: PAM-6 outperforms PAM-8 modulation when channel bandwidth is sufficient for PAM-6 operation. To improve channel reach with either modulation, crosstalk reduction requires a new shielding methodology and new mating interface technologies that can scale co-packaged copper to support 448G.

The Path Forward to 448G


Co-packaged copper eliminates one of the major bandwidth-limiting elements: the interface between semiconductor device packages and the BGA footprint in the PCB. Signals crossing this interface must pass through PCB vias and lossy PCB materials (dielectrics that attenuate high-frequency signals because of their inherent dielectric losses), both of which limit signal propagation. Co-packaged copper bypasses this interface and keeps routing inside the package; connector construction then becomes the main bandwidth-limiting factor. 

By selecting co-packaged copper to support 448G data rates, new challenges emerge related to the design and construction of connectors:

  • The separable mating interface in connectors may contain stubs which create strong insertion losses. 
  • If J-lead attachments on connectors are used, via and SMT pad stub minimization will be required. 
  • When twinax cable is used, the connector transition to the twinax must have broad bandwidth.

This in-depth study found that copper channels can provide sufficient bandwidth to support PAM-6 and PAM-8 modulation in 448G channels, with PAM-8 being better suited for channels with the lowest bandwidth. Outstanding signal integrity questions remain surrounding crosstalk and equalization:

  • Can connector construction improve insertion loss resonance and crosstalk in 448G channels? 
  • Will new equalization schemes be able to reduce the impact of nonlinear insertion loss and higher crosstalk in 448G channels?

These challenges still need to be addressed to ensure high-fidelity data transmission at 448G and beyond.


VIDEO SERIES PART 1

448G and AI Clusters

Explore why 448G technology is critical for the future of AI clusters and hyperscale data centers. Learn about the limitations of existing copper technology and how Molex is helping to advance high-speed data connectivity through innovation and 448G performance analysis.

VIDEO SERIES PART 2

A Study on 448G and Data Transfer

Unpack critical findings from Molex’s study on high-speed interconnect design, informing broader 448G performance analysis and development of next-gen solutions. Explore how: idealized channel models reveal key performance tradeoffs, modulation schemes impact complexity and bandwidth, and equalization techniques support reliable signal integrity.

VIDEO SERIES PART 3

Challenges with 448G and AI Clusters

Learn how to tackle the toughest design challenges revealed by the Molex 448G performance analysis. Learn why copper interconnects may still be feasible for 448G and explore the critical need for innovation in materials, manufacturing, interface designs and methodologies in the pursuit of next-generation scalability. 

Shaping the Next Phase of Connectivity

Molex is laying the foundation for 448G interconnect technology through extensive research and deep engineering expertise, building on our proven leadership at 112G and 224G speeds. By advancing connector architecture and signal integrity, Molex empowers data centers to move information faster and with greater signal clarity, meeting the performance demands of emerging AI-powered data environments.

Data center with blue overlay