Tomorrow’s systems that employ high speed serial connections offer the ability to design a truly distributed computing system. Serial connections with low latency and high bandwidth means multiple processors and memory can be connected across a backplane
in a very effective manner. This has many system benefits; scalability thru card upgrades, power management thru distribution
on the heat producers, and system optimization thru the use of task specific processors. Choosing the best technology for this high-speed data plane fabric requires a detailed understanding of both features and useable bandwidth.

Serial interconnect is specified in both VITA41and VITA 46, but multiple choices have slowed adoption rates as companies
wait for a clear winner to emerge. These serial fabric choices – ranging from Ethernet, InfiniBand, PCI Express, HyperTransport, and RapidIO – narrow significantly
when key criteria are applied. Low latency, high bandwidth, and error management
are required by VME customers to build distributed computing platforms with the reliability and scalability expected. Additionally, companies should look for ecosystem
momentum, open standards, and an active trade association to ensure the serial interconnect thrives and evolves.

The Benefits of Distributed Computing Architecture

A fundamental challenge in electronics is to put more functions in the same space. More functionality always results in greater heat generation despite shrinking silicon geometries. As any thermal engineer will attest, it is not the power but power density
that creates the cooling challenge. So despite much hype about System-on-Chip (SoC) integration, cost-effective products distribute the power hungry compute nodes throughout the system chassis.

In addition to lowering system cooling costs, distributed computing enables the use of multiple task-specific computing elements. Network Processing Units (NPU), ASICs, DSPs, exist because they are more cost-efficient
at certain tasks than the CPU or SoC. Being able to hook these elements together provides modularity and scalability through replacement of individual computing elements
as needed. The challenge is that distributed computing requires a very fast data pipe between computing elements or performance suffers. Serial interconnect is providing that data pipe in many embedded applications. This connection can be on a card or across a backplane.

Distributed Computing Requires a High-Performance Switch

Serial switches like Tundra’s RapidIO Tsi578TM or Tsi568ATM are used to create the point-to-point connections required in serial communication. There is no multi-drop bus at these Gigahertz speeds. Low latency and high bandwidth are key switch features. Simply put, switch latency is measured
as the time data takes to traverse a switch. So a low latency value indicates a single fast data pipe. Latency varies widely between vendors’ silicon and depends on both the design and the type of forwarding mode used. But to prevent traffic congestion
and truly claim high performance, the switch must also be non-blocking. A switch can claim low latency on one path, but if it isn’t non-blocking, other data paths will be waiting while that one transaction completes.

A company should also look for the lowest power so that the power budget can be spent on processing. Having ports that are independently
configurable in both speeds and widths means that power can be minimized by not using ports or bandwidth you don’t need. To conclude, one needs to shop carefully
for switches because they are not all the same and there are many variables.

Interconnect Standard’s Impact on Performance

Okay but what serial standard is the right choice? It needs to be designed to ensure hi bandwidth and reliable data transmission. Bandwidth is a product of both efficiency and the physical transmission rate. Efficiency can be measured by a ratio of the bytes available for data, to the bytes required for overhead (addressing and error checking). The effective bandwidths presented in Table 1 are calculated using efficiency and the data rates available today. How packets are terminated is also important. Termination is essentially taking the header apart in order to read addresses and perform error checking. Gigabit Ethernet (GigE) using TCP/IP can take a significant amount of computing power as only up to level 2 of the protocol stack is handled in silicon; the rest is dealt with in software. Schemes like UDP are employed in GbE to help reduce the overhead. In PCIe and RapidIO, termination is done in the silicon up to level 3.

Protocol Data Unit (PDU) = header, data, CRC, and so forth. Effective bandwidth includes the related CRC encoding scheme. GigabitEthernet figures assume a UDP implementation (TCP/IP would be worse and Layer 2 slightly better).

Reliability in the context of an interconnect standard means that it should support multipoint
connection topology – a fabric capable of topologies like dual star or mesh – and offer advanced error handling features. PCIe is really a host-centric architecture. A PCIe application has a single CPU mastering the traffic – called the root complex. This leaves the system vulnerable to root complex failure. It also means that mapping distributed
memory so it can be globally shared by multiple processors is very difficult in PCIe. RapidIO offers all of the features required to build a VME VXS or VPX system with the greatest bandwidth and reliability.

In addition to performance and features for reliability, what else does a company need to consider when choosing an interconnect
standard? A wise choice also has a developed ecosystem of suppliers, is found as a native port on the processing elements thereby reducing the need for additional conversion or “bridge” chips, has an active road map, an active trade association and is an open standard. Right now only RapidIO, PCIe, and GigE can claim to be native ports on processors and to have multiple vendors selling silicon.

RapidIO – The Interconnect of Choice for Distributed Computing Systems

Considering all of the challenges above, the serial interconnect specification that best addresses these is RapidIO. RapidIO is found as a native port on many processing elements, has multiple suppliers with silicon available today, has features to support error management with small overhead, and has the highest bandwidth to enable distributed processing. With 25 years of products behind it, VME has succeeded not just because it is an open standard, but also because it promotes backwards compatibility, has a large supply base, and has continued to evolve over time. Any serial interconnect should offer the same potential. Tundra’s RapidIO products do. So now you can DESIGN.CONNECT.GO.™