Technology, Philosophy, and Kitty Litter: An Interview with VITA’s Ray Alderman

By: Chris A. Ciufo, Editor, Embedded Systems Engineering

Chairman of the Board, Ray Alderman, presents a unique view of how embedded companies compete, thrive and die in the COTS market.

One never knows what Ray Alderman is going to say, only that it’s going to be interesting.  As Chairman of the Board of VITA (and former Executive Director), Ray is a colorful character. We caught up with him to discuss a recent white paper he wrote entitled: “RAW – How This Embedded Board and Systems Business Works.” We posed a series of questions to Ray about his musings; edited excerpts follow.

Chris “C2” Ciufo: Ray, you reference the Boston Consulting Group matrix that places companies in four quadrants, arguing that most of the companies in our embedded COTS industry are Low Volume (LV)/High Margin (HM) “Niche” players. The place not to be is the LV/LM “Graveyard”—right where technologies like ISA, S-100, Multibus and PCI Gen 2 are. But…PCI Express?

RayAldermanRay Alderman: I was careful to say “PCI Express Gen 2.” That’s because Gen 3 is on our doorstep, and then there will be Gen 4, and so on. Gen 2 will be EOL [end of life] before too long. The niche players in our market—all embedded boards, not just VME/VPX—rarely take leadership in mainstream technology. That position is reserved for the four companies that control 75% of the commercial embedded market segment, or $1.5 billion. They are ADLINK, Advantech, congatec, and Kontron: these guys get the inside track with technology innovators like Intel and Nvidia; they’ll have PCIe Gen 4 product ready to ship before the niche players even have the advanced specs. Everyone else has to find other ways to compete.

C2: You said that “in the history of this industry, no company has ever reached $1 billion in sales” because as the volumes go up, customers shift to contract manufacturers to lower their prices. Only three companies ever came close to the HV/LM quadrant. Who were they?

Ray: Advantech, Kontron and Motorola Computer Group (MCG). MCG, you’ll recall, was amalgamated with Force when sold by Solectron, and then morphed into Emerson Computer Group. MCG damn near ruled the VME business back then, but as my model points out—it was unsustainable. Advantech and Kontron are still around, although Kontron is going through some—ahem!—realignment right now. My model and predictions still hold true.

C2: What’s causing this growth-to-bust cycle in the embedded market? Not all markets experience this kind of bell curve: many keep rising beyond our event horizon.

Ray: Since about 1989, the companies that had to sell out or went out of business made one of two basic mistakes: (1) they entered into a commodity market and could not drive their costs down fast enough, or (2) they entered a niche market with a commodity strategy and the volumes never materialized.

I’ve been saying this for a while—it’s practically “Alderman’s Law”—but our military embedded board and system merchant market (all form factors) is about $1.2 billion. The cat litter market in the U.S. is about $1.8 billion, and their product is infinitely less complicated.

C2: Wait—are you really comparing kitty litter to embedded technology?

Ray: By contrast. Cat litter margins are low, volumes are high and they use a complex distribution system to get the litter to cats. Our margins are high, our volumes are low, and we deal direct with the users. The top three companies in the military segment—Abaco [formerly GE Intelligent Platforms], Curtiss-Wright Defense Solutions and Mercury—total up to about $750 million. They’re around $200 million each. They add intellectual value and enjoy high GPM [gross profit margin].

On the other hand, the commercial embedded board market for telecom, industrial, commercial and transportation totals to about $2.0 billion. Using kitty logic, the dry cat food market in the U.S. is about $3.8 billion. Their margins are low, volumes are high, and they use a complex distribution system. The players in the commercial board market have low margins, low volumes (compared to other segments), and sell directly to end users. It’s a terrible place to be. Kitty litter or cat food?

C2: What’s your advice?

Ray: I’m advocating for the military market, where margins are higher. About 61% of the military embedded board/system market is controlled by the three vendors, $750 million. The remaining $450 million (39%) is shared by many small niche vendors: nice, profitable niches. Several smaller companies do $30-50 million in this segment.  In contrast, only four companies control 75% of the commercial embedded boards market, or roughly $1.5 billion. That leaves a mere $500 million (25%) for all of the other smaller companies. Thus there are not many fairly large or profitable niches for these smaller guys—and not many of them do more than $10-15 million. Kitty litter, anyone?

C2: Can you offer some specific advice for board vendors?

Ray: There are only three values you can add to make money in these markets: manufacturing value, service value, and intellectual value. Adding intellectual value is where you add high-level technical skills that other companies do not have. Examples: high speed A-to-D boards where companies like Mercury and Pentek live. You can also add DSPs with unique IP inside. Again, Mercury and Pentek come to mind. In fact, Mercury (then Mercury Computer Systems) proved this model nicely when they invented the RACEway inter-board link and created ASICs to implement it. If you want to raise your GPM, this is how you do it.

In fact, Mercury is still doing it. They bought Echotek some years ago for their I/O boards and just recently bought three divisions of Microsemi. With this latest acquisition, they gain secure storage memories, crypto IP, and a bunch of RF capabilities to add to their existing RF portfolio. Today, RF technology is “magical” and Mercury will be able to charge accordingly for it to maximize their GPM.  Most of the embedded board military suppliers add their value to the market through intellectual value. It makes the most sense.

C2: Is the recipe for success merely targeting niche markets and adding intellectual value?

Ray: I’ll let you in on a little secret. The margin on boards is much higher than the margin on systems. It’s ironic, because every board guy seems to want to get into the systems business, and there have been lots of M&A [mergers and acquisitions] over the past several years. If you’re going to do systems, you’ve got to raise the price, especially if you’re selling air-cooled [convection] systems. Conduction-cooled systems command a higher price, but they’re harder to design.

You also need to choose the niche carefully, but that goes without saying. If you can add intellectual value to your niche—such as high performance GPGPU processing—you can command higher prices, whether at the board- or systems level.

There are only three ways to be successful in the embedded boards and systems business. Be first, be smarter, or cheat. Let me explain.

Being first is usually relegated to the big guys, like Abaco, Curtiss-Wright, or Mercury. They get access to the latest semiconductor technology, which is a fundamental driver in all of our markets. Examples here would be in-advance knowledge of Intel’s Kaby Lake follow-on to the Skylake Core i7 processor, or Nvidia’s plans for their next GPU. The smaller board vendors won’t get access, so they usually can’t be first.

One other thing, the big guys can also adapt a market to them. That is, they have enough influence that they can actually move an entire market. The smaller guys just have to find other ways.

But they can be smarter. Force Computer couldn’t (at the time) beat Motorola’s Computer Group because Motorola was inventing the 68xxx processors back then. So Force switched to the SPARC processor and built a successful business around it.  In effect, Force adapted to a market that was hungry for processing power—it didn’t have to be 68020 or 68040 processing power. [Editor’s note: in fact, the 68040 wasn’t successful because Motorola themselves introduced their PowerPC processor to the market, which was co-developed with IBM. The market moved away from the 68xxx CISC processor to the PPC60x RISC processor; the rest is “history.”]

C2: And lastly, how should companies “cheat” to win?

Ray: It’s hard to cheat in the open market, against big entrenched players. The best way to cheat is to fragment an existing market. Sun Tzu called this the “Divisional” strategy. Companies can create a niche such as by creating an open standard for your version of a board or system architecture. Creating a niche is like being smarter, but is marketing-based instead of being engineering-based.

At VITA/VSO, the policies and procedures allow any company, along with two other sponsors, to write a new standard without interference. There are countless examples of this within VITA, and many of these “fragmented niches” have become successful standards that we use today, including FMC, PMC, and XMC [mezzanine cards]. Older standards like Greenspring [mezzanine modules] were successful but now mostly obsolete. There are other new standards such as the three for rugged small form factors [VITA 73, 74, 75]. And the various OpenVPX profiles are other examples, such as new “Space VPX” and “Space VPX Lite”.

C2: Any last thoughts?

Ray: As Albert Einstein once said, “We cannot solve problems by using the same kind of thinking we used when we created them.” My point: look to new architectures beyond von Neumann’s architecture that the semiconductor guys keep forcing on us. Consider fiber interconnects as a way to get off the copper-trace technology curve. Create a niche—“cheat” if you have to. Just don’t end up following a kitty litter business strategy, else you’ll be taken out with the trash.

How Does One “Zeroize” Flash Devices?

By Chris A. Ciufo, Editor Embedded Systems Engineering

Editor’s Note: This is Part 1 of a two-part article on the topic of securely erasing data in flash devices such as memories and SSDs. In Part 2, I examine the built-in flash secure erase feature intended to eradicate sensitive data and see if it meets DoD and NIST specifications.

I was recently asked the question of how to go about “zeroizing” flash memory and SSDs. I had incorrectly assumed there was a single government specification that clearly spelled out the procedure(s). Here’s what several hours of research revealed:

DoD has no current spec that I could find besides DoD 5220.22-M “National Industrial Security Program[1]. This 2006 document prefaced by the Under Secretary of Defense cancels a previous 1995 recommendation and discusses some pretty specific procedures for handling classified information. After all, the only reason to sanitize or zeroize flash memory is to eradicate classified information like data, crypto keys, or operating programs (software). The document makes reference to media—including removable media (presumably discs, CDs and USB drives at that time)—and the need to sanitize classified data. However, I was unable to identify a procedure for sanitizing the media.

There is, however, a reference to NIST document 800-88Guidelines for Media Sanitization” published in DRAFT form in 2012. A long document that goes into extensive detail on types of media and the human chain of command on handling classified data, Appendix A provides lengthy tables on how to sanitize different media. Table A-8 deals with flash memory and lists the following steps (Figure 1):

-       Clear: 1. Overwrite the data “using organizationally approved and validated overwriting technologies/methods/tools” and at least one pass through by writing zeros into all locations. 2. Leverage the “non-enhanced” ATA Secure Erase feature built into the device, if supported.

-       Purge: 1. Use the ATA sanitize command via a) block erase and b) Cryptographic Erase (aka “sanitize crypto scramble”). One can optionally apply the block erase command after the sanitize command. 2. Apply ATA Secure Erase command, but the built-in (if available) sanitize command is preferred. 3. Use the “Cryptographic Erase through TCG Opal SSC or Enterprise SSC”—which relies on media (drives, including SSDs) that use the FIPS 140-2 self-encrypting feature.

-       Shred, Disintegrate, Pulverize, or Incinerate the device. This literally means mechanically destroy the media such that if any 1’s and 0’s remain on the floating transistor gates, it’s not possible to reconstruct these bits into useful data.

Figure 1: Recommended ways to sanitize flash media per NIST 800-88 DRAFT Rev 1 (2012).

Figure 1: Recommended ways to sanitize flash media per NIST 800-88 DRAFT Rev 1 (2012).

Of note in the NIST document is a footnote that states that Clear and Purge must each be verified. Crypto Erase only needs verification if performed prior to a Clear or Purge. In all of these cases, all procedures except for mechanical eradication rely on mechanisms built into the drive/media by the manufacturer. There is some question if this is as secure as intended and the NSA—America’s gold standard for all things crypto—has only one recommended procedure.

The NSA only allows strong encryption or mechanical shredding, as specified in “NSA/CSS Storage Device Sanitization Manual.” This 2009 document is now a bit difficult to find, perhaps because the NSA is constantly revising its Information Assurance (IA) recommendations to the changing cyberspace threats due to information warfare. Visiting the NSA website on IA requires a DoD PKI certificate per TLS 1.2 and a “current DoD Root and Intermediate Certificate Authorities (CA) loaded” into a browser. Clearly the NSA follows its own recommendations.

The manual is interesting reading in that one has only the choice to cryptographically protect the data (and the keys) and hence not worry about sanitization. Or, one can render the media (drive) completely unrecognizable with zero probability of any data remaining. By “unrecognizable,” think of an industrial shredder or an iron ore blast furnace. When it’s done, there’s nothing remaining.

Recent discussions with government users on this topic reminded me of the Hainan Island Incident in 2001 where a Chinese fighter jet attempting an intercept collided with a US Navy EP-3 SIGINT aircraft. The EP-3 was forced to make an emergency landing on China-controlled Hainan, giving unauthorized access to classified US equipment, data, algorithms and crypto keys (Figure 2). It was a harrowing experience, sadly causing the death of the Chinese pilot and the near-fatalities of the 24 Navy crew.

The crew had 26 minutes to destroy sensitive equipment and data while in the air using a fire axe, hot coffee and other methods, plus another 15 minutes on the ground, but it was widely reported to be only partially successful. While this sounds far-fetched, the topic of sanitizing data is so critical—yet so unresolved, as described above—that allegedly some current-generation equipment includes a visible “Red X” indicating exactly where an operator is to aim a bullet as a last ditch effort to mechanically sanitize equipment.

Figure 2: US Navy EP-3 SIGINT plane damaged in 2001 by collision with Chinese fighter jet. The crew did only a partial sanitization of data. (Image courtesy of Wikipedia.org and provided by Lockheed Martin Aeronautics.)

Figure 2: US Navy EP-3 SIGINT plane damaged in 2001 by collision with Chinese fighter jet. The crew did only a partial sanitization of data. (Image courtesy of Wikipedia.org and provided by Lockheed Martin Aeronautics.)

From Pulverize to Zeroize

There’s a lot of room between the DoD’s wish to have classified data and programs zeroized and the NSA’s recommendation to pulverize. The middle ground is the NIST spec listed above that relies heavily on flash memory manufacturer’s built-in secure erase options. While there are COTS recommendations for secure erase, they are driven not from a military standpoint but from the need to protect laptop information, Sarbanes-Oxley (corporate) legislation, health records per HIPAA, and financial data.

In Part 2 of this article, I’ll examine some of the COTS specifications built into ATA standards (such as Secure Erase), recommendations presented at Flash Memory Summit meetings, and raise the question of just how much trust one can place in these specifications that are essentially self-certified by the flash memory manufacturers.


[1] Previously, DoD relied on NISPOM 8-306; NSA had NSA 130-2 and NSA 9-12; Air Force had AFSSI-5020; Army had AR 380-19; and Navy had NAVSO P-5239-26. These all appear to be out of date and possibly superseded by the latest 5220.22-M. As a civilian, it’s unclear to me—perhaps a reader can shed some light?