COM Express Type 7 Adds 10 GigE for New Server Paradigm
PICMG’s COM Express 3.0 Type 7 pinout definition was released early to market with big plans for new server applications.
The glacial pace of evolving standards is like watching paint dry: slow moving and not terribly exciting. But this time around, PICMG lit a fire under the new Type 7 COM Express spec. Officially part of COM Express 3.0 (“COM.0, rev 3.0”), the Type 7 pinout was just recently shared with PICMG members—although not officially “released”—amid the clamor of market demand. You see, Type 7 COMe adds four 10 Gigabit Ethernet ports and opens the door to installing Intel’s latest Xeon® processor D on Type 7 (Figure 1).
“Big, deal,” you say. But it is a big deal to server companies like Dell, HP and ADLINK. With up to 16 cores and 32 threads in the Xeon processor D (non-embedded versions), never before has so much processing power and 10 Gig I/O been available in such a small package. Xeon D is a system-on-chip (SoC) processor with CPU and PCH on the same die. In principle, 10 or more 95 x 125mm Type 7 COM modules will fit on a typical 1U full-depth server tray. That’s 160 cores or virtual machines, and each Xeon processor D only consumes between 25W-45W. 160 cores at under 500W? Incredible! This is 25% to 50% the power of competing 1U shelf servers.
Type 6 Gives It Up for Type 7
According to ADLINK CTO Jeff Munch, COM Express was last revised about five years ago, and, although it is a popular spec, it’s never offered enough bandwidth for the data center. Sure, there was PCI Express Gen 3, HDMI and DVI—but the data center wants network ports and lots of ‘em. Previous versions of COMe offered 1 GigE port, but data center clients want 10 GigE KR. (To be fair, COM Express—although developed by the enterprise-oriented PICMG standards group—was always intended for embedded applications, not data centers.)
Adding 10 GigE to COMe was not an original idea, says Munch. A number of vendors are already doing it by modifying the COM Express Type 6 pinout while removing one or more of the 3x DVI ports. “What’s the point of having a standard,” Munch told me, “if every vendor’s pinout is proprietary?”
Type 7 solves this problem, precisely by eliminating graphics and reducing the number of SATA ports in Type 6 while adding four 10 GigE KR ports to the pinout. Although officially a part of the COM Express specification, work on Rev 3 started in late 2015, and months were spent defining, simulating (three different simulations, says ADLINK) and studying the Type 7 pinout in an effort to achieve a robust interconnect free from signal integrity (SI) problems. No single vendor’s pinout was used, giving no one vendor an early-mover advantage. Extra pins were added to support copper and SFP+ (optical) carrier boards and backplanes.
By no coincidence, Type 7 is “backward compatible enough,” says ADLINK’s Munch, such that a Type 7 board will plug onto a Type 6 carrier and work with the legacy I/O present on both 7 and 6. That’s an impressive standardization payoff.
Lucky Type 7
To make room for the 10 GigE ports, the COM Express committee—chaired by ADLINK’s Munch—reasoned that SATA drives are moving to PCIe. Both Intel and Samsung, for example, are making big inroads converting data center drives to the faster NVMe PCIe Gen 3 interface at the expense of SATA III. At IDF16 this month (August 2016), motherboard manufacturer Supermicro was showcasing improved server speeds using NVMe drives from Intel. Marrying NVMe with COMe Type 7 is a no-brainer.
Type 7 removed two of the four SATA ports found on Type 6, but increased to 32 the number of PCIe Gen 3 lanes. And although the current generation Intel Xeon processor D contains “only” two 10 GigE ports (and 24 PCIe Gen 3), future versions are expected to have more. Table 1 lists the relevant signals/ports found in the Type 7 pinout.
While two 10 GigE ports on Xeon D are compelling, even more so is the variety of processor cores versus power consumption. A quick look at Intel’s Xeon processor D page reveals 16 cores at 45W (server variant), down to 4 cores at 20W (embedded). Putting Xeon D on a tiny Basic COMe module (95 x 125mm) allows companies like ADLINK to deploy many-core server processing boards into all kinds of embedded form factors. As we’ve said, Xeon processor D on COMe Type 7 might give Intel’s high end Xeon E5 some competition. But companies like ADLINK also have other plans.
“D” for EmbeDDed
At IDF15 a year ago, several server vendors showed off new electrical and mechanical architectures that move away from PICMG’s ATCA server and data center paradigm. Using pluggable modules instead of a fixed motherboard, IT and data center managers can easily swap processor, I/O and storage modules. At IDF16, companies like ADLINK showcased more of these new architectures. ADLINK, for example, announced its membership in the Open Compute Project and showed off a modular “sled” based server that one could easily imagine using a dozen or more Type 7 COMe Xeon D modules.
The company announced the Express-BD7 COMe board with 4, 8, 12 and 16-core Xeon D versions (Figure 2), and showcased a notional carrier board that supports 4x 10 GigE ports that can be KR-to-KR, KR-to-optical fiber, or KR-to-copper. A white paper entitled “COM Express Type 7: Bringing 16-core Server Grade Support and 10G to COM Express” also recently became available.
But ADLINK, a company moving from mere modules to more vertical applications based upon embedded technology, could easily migrate a Type 7 COMe Xeon D module into many of its other product lines besides servers. Its mobile edge computer SETO-1000, designed for outside installations (Figure 3), its HPERC “Extreme Rugged” coldplate system, and its IoT intelligent gateway MXE-202i could all benefit from multiple virtual machines supplied by a Xeon D-based Type 7 COMe board.
And what about that COM Express 3.0 revision? It’s expected to be ratified by as early as the end of this year. It’s a certainty that the Type 7 pinout will be a big selling feature with server and embedded companies alike. That’s good deal for ADLINK, which falls comfortably into both camps.
This article was sponsored by ADLINK Technology.