The Soft(ware) Core of Qualcomm’s Internet of Everything Vision

Qualcomm supplements silicon with multiple software initiatives.

Qualcomm Snapdragon
Update 1: Added attribution to figures.
The numbers are huge: 50B connected devices; 7B smartphones to be sold by 2017; 1000x growth in data traffic within a few years. Underlying all of these devices in the Internet of Things…wait, the Internet of Everything…is Qualcomm. Shipping 700 million chipsets per year on top of a wildly successful IP creation business in cellular modem algorithms, plus being arguably #1 in 3G/4G/LTE with Snapdragon SoCs in smartphones, the company is now setting its sights on M2M connectivity. Qualcomm has perhaps more initiatives in IoT/IoE than any other vendor. Increasingly, those initiatives rely on the software necessary for the global M2M-driven IoT/IoE trend to take root.

Telit Wireless Devcon
Speaking at the Telit Wireless Devcon in San Jose on 15 October, Qualcomm VP Nakul Duggal of the Mobile Computing Division painted a picture showing the many pieces of the company’s strategy for the IoT/E. Besides the aforementioned arsenal of SnapDragon SoC and Gobi modem components, the company is bringing to bear Wi-Fi, Bluetooth, local radio (like NFC), GPS, communications stacks, and a vision for heterogeneous M2M device communication they call “dynamic proximal networking”. Qualcomm supplies myriad chipsets to Telit Wireless, and Telit rolls them into higher order modules upon which Telit’s customers add end-system value.

Over 8 Telit Wireless modules are based upon Qualcomm modems.

Over eight Telit Wireless modules are based upon Qualcomm modems, as presented at the Telit Wireless Devcon 2013.

But it all needs software in order to work. Here are a few of Qualcomm’s software initiatives.

Modem’s ARM and API Open to All
Many M2M nodes–think of a vending machine, or the much maligned connected coffee maker–don’t need a lot of intelligence to function. They collect data, perform limited functions, and send analytics and diagnostics to their remote M2M masters. Qualcomm’s Duggal says that the ARM processors in Qualcomm modems are powerful enough to perform that computational load. There’s no need for an additional CPU so the company is making available Java (including Java ME), Linux and ThreadX to run their 3rd generation of Gobi LTE modems.

Qualcomm is already on its 3rd generation of Gobi LTE modems.

Qualcomm is already on its 3rd generation of Gobi LTE modems.

Qualcomm has also opened up the modem APIs and made available their IoT Connection Manager software to make it easier to write closer-to-the-metal code for modem. Duggal revealed that Qualcomm has partnered with Digi International in this effort as it applies to telematics market segments.

Leverage Smartphone Graphics
And some of those M2M devices on the IoE may have displays–simple UIs at first (like a vending machine)—but increasingly more complex as the device interacts with the consumer. A restaurant’s digital menu sign, for example, need not run a full blown PC and Windows Embedded operating system when a version of a Snapdragon SoC will do. After all, the 1080p HDMI graphics needs of an HTC One with S600 far outweigh those of a digital sign. Qualcomm’s graphics accelerators and signal processing algorithms can easily apply to display-enabled M2M devices. This applies doubly as more intelligence is pushed to the M2M node, alleviating the need to send reams of data up to the cloud for processing.

Digital 6th Sense: Context
Another area Duggal described as the “Digital 6th Sense” might be thought of as contextual computing. Smartphones or wearable fitness devices like Nike’s new FuelBand SE might react differently when they’re outside, at work, or in the home. More than just counting steps and communicating with an App, if the device knows where it is…including precisely where it is inside of a building…it can perform different functions. Qualcomm now includes the Atheros full RF spectrum of products including Bluetooth, Bluetooth LE, NFC, Wi-Fi and more. Software stacks for all of these enable connectivity, but code that meshes (no pun) Wi-Fi with GPS data provides outside and inside position information. Here, Qualcomm’s software melds myriad infrastructure technologies to provide inside positioning. A partnership with Cisco will bring the technology to consumer locations like shopping malls to coexist with Cisco’s Mobility Services Engine for location-based Apps.

Smart Start at Home
Finally, the smart home is another area ripe for innovation. Connected devices in the home range from the existing set-top box for entertainment, to that connected coffee pot, smart meter, Wi-Fi enabled Next thermostat and smoke/CO detector, home health and more. These disparate ecosystems, says Duggal, are similar only in their “heterogeneousness” in the home. That is: they were never designed to be interconnected. Qualcomm is taking their relationships with every smart meter manufacturer, their home gateway/backhaul designs, and their smartphone expertise, and rolling it into the new AllJoyn software effort.

The open source AllJoyn initiative, spearheaded by Qualcomm, seeks to connect heterogeneous M2M nodes. Think: STB talks to thermostat, or refrigerator talks to garage door opener.

The open source AllJoyn initiative, spearheaded by Qualcomm, seeks to connect heterogeneous M2M nodes. Think: STB talks to thermostat, or refrigerator talks to garage door opener. Courtesy: Qualcomm and AllJoyn.org .

AllJoyn is an open source project that seeks to set a “common language for the Internet of Everything”. According to AllJoyn.org, the “dynamic proximal network” is created using a universal software framework that’s extremely lightweight. Qualcomm’s Duggal described the ability for a device to enumerate that it has a sensor, audio, display, or other I/O. Most importantly, Alljoyn is “bearer agnostic” across all leading OSes or connectivity mechanism.

AllJoyn connectivity diagram.

AllJoyn connectivity diagram. Courtesy: www.alljoyn.org .

If Qualcomm is to realize their vision of selling more modems and Snapdragon-like SoCs, making them play well together and exchange information is critical. AllJoyn is pretty new; a new Standard Client (3.4.0) was released on 9 October. It’s unclear to me right now how AllJoyn compares with Wind River’s MQTT-based M2M Intelligent Device Platform or Digi’s iDigi Cloud or Eurotech’s EveryWhere Device Framework.

Qualcomm’s on a Roll
With their leadership in RF modems and smartphone processors, Qualcomm is laser focused on the next big opportunity: the IoT/E. Making all of those M2M nodes actually do something useful will require software throughout the connected network. With so many software initiatives underway, Qualcomm is betting on their next big thing: the Internet of Everything. Software will be the company’s next major “killer app”.

“Mirror, Mira” on the Car’s IVI Screen: Two Different Standards?

You might be hearing about a new technology called MirrorLink that mimics your smartphone’s screen on the larger nav screen in your “connected car”. Or, you might be following the news on Miracast, a more open standard now baked into Android that offers Apple AirPlay-like features to stream smartphone content to devices like connected TVs.

You’d be forgiven if you think the two similarly-named standards are trying to accomplish the same thing. I didn’t understand it either, so I did some digging. Here’s what I found out.

The Smart, Connected Car
When I attended the Paris Auto Show last Fall specifically to investigate in-vehicle infotainment (IVI) trends for the Barr Group under contract to Intel, I got spun up “right quick” on all manner of IVI. From BMW’s iDrive to Chevrolet’s MyLink, the connected car is here. In fact, it’s one of the biggest trends spotted at last week’s 2013 CES in Las Vegas. MirrorLink is being designed into lots of new cars.

BMW's iDrive IVI uses a native system and doesn't rely on smartphone mirroring.

BMW’s iDrive IVI uses a native system and doesn’t rely on smartphone mirroring. (Courtesy of BMW.)

The biggest question faced by every auto manufacturer is this: in-car native system, or rely on the apps in one’s smartphone? Ford’s industry breakthrough MyFord Touch with SYNC by Microsoft is native and based upon Microsoft Auto Platform (now called Windows Embedded Automotive 7). Elsewhere, premium brands like BMW, Lexus and Cadillac have designed self-contained systems from the ground up. Some, like BMW, include in-car cellular modems. Others rely on the smartphone only for music and Internet access, but that’s it.

2013 Chevrolet MyLink IVI uses MirrorLink with smartphone apps

2013 Chevrolet MyLink IVI uses MirrorLink with smartphone apps. (Courtesy of Chevrolet.)

Still others, like Toyota and Chevrolet use a technology called MirrorLink to “mirror” the smartphone’s screen onto the car’s larger IVI. For all apps that make sense to be viewed on the IVI, the system will display them — usually identically to what the user sees on the smartphone (subject to safety and distraction caveats).

MirrorLink is now a trademarked standard owned by the Car Connectivity Consortium that’s designed specifically for cars and smartphones. That means the standard worries about driver distractions, apps that make sense for drivers (such as Google Maps) and those that don’t (such as a panoramic camera stitching application). Apps have to be qualified for use with MirrorLink.

As well, MirrorLink replaces the phone’s touch I/O with in-car I/O such as steering wheel controls, console joysticks, or the IVI head unit’s touchscreen or bezel buttons. Equally as important, audio input from microphones is routed from the car to the phone, while output uses the car’s speakers. The car’s antennae for radio and GPS will be given preference over the phone’s, improving the signal reception.  The protocols between smartphone and car also take input from the vehicle’s CANbus, including speed. This means that you can check your email when parked, but not while driving. A great resource for how it works and what the future holds is here.

MirrorLink started as a Nokia idea that was intended for smartphone-to-car connectivity. Now at version 1.1, it’s a client-server architecture where the IVI head unit is the USB host.  It uses industry-standard protocols such as Internet Protocol (IP), USB, Wi-Fi, Bluetooth (BT HFP for telephony, BT A2DP for media), RTP, and UPnP. Recent additions use The Trusted Computing Group concepts of device attestation protocols with SKSD/PKSD keys via authentication. The actual screen sharing uses the VNC protocol.

MirrorLink and Trusted Computing Group authentication process for trusted content.

MirrorLink and Trusted Computing Group authentication process for trusted content. (Courtesy of Car Connectivity Consortium.)

What MirrorLink doesn’t yet support is video streaming, since drivers watching video is a no-no is cars (tell that to the Japanese who I’ve seen with TVs mounted in their cars!).

Android and Miracast
Miracast, on the other hand, is all about streaming. It’s a Wi-Fi Alliance spec recently demoed at CES 2013 that’s designed to stream video and photos from smartphones, tablets, and future embedded devices. Like Apple’s AirPlay, it moves stuff from a small screen onto a big TV screen. It’s based upon Wi-Fi’s not-new-but-rarely-used Wi-Fi Direct standard (WiDi 3.5) that avoids routers to establish peer-to-peer connectivity.

The Wi-Fi Alliance Miracast standard streams video from small to large screens, as shown in this excerpt from a YouTube video. (Courtesy of YouTube and Wi-Fi Alliance.)

The Wi-Fi Alliance Miracast standard streams video from small to large screens, as shown in this excerpt from a YouTube video. (Courtesy of YouTube and Wi-Fi Alliance.)

Miracast supports 1080p HD video, 5.1 surround, and CPUs from nVidia, TI, Qualcomm, Marvell and others have announced plans to support it. Built into the spec is the ability to stream DRM and HDCP protected content using already established HDMI and DisplayPort style copy protection schemes. I guess they figure if you’ve got the rights to play it on your phone, might as well play it on your TV too.

Last Fall, Google updated Android Jelly Bean to 4.2 and included Miracast as part of the update, and I’m thrilled that my Nexus 7 tablet can now, in theory, stream content to my Samsung Smart TV. As Android proliferates throughout the embedded market, I can envision commercial applications where a user might do more than stream a video to another embedded device. Sharing the entire smartphone’s screen can be useful for PowerPoint presentations or demoing just about any Android app in existence. If it’s on the phone’s screen, it can get mirrored via Wi-Fi to another screen.

Will MirrorLink and Miracast Converge?
I doubt the two standards will merge. MirrorLink is exclusively aimed at IVI systems in cars, and the closely curated standard is intended to vet applications to assure safe operation in a vehicle. Miracast is similar in that it mirrors a smartphone’s screen, but there are no limitations on moving between screens, so Miracast is clearly the superset standard to a broader market.

Ironically, as the Car Connectivity Consortium looks to release MirrorLink Version 2.0, they’re examining Miracast as a way to provide an “alternative video link” for streaming H.264 1080p@30 FPS into the car cabin.

Why? For passenger entertainment. Think about minivans (shudder) and Suburbans loaded with kids.