Gaming Devices and Wearables Sense that Biometrics are a Healthy Way Forward
Collecting and making sense of information to better our health and add new dimensions to gaming, VR and AR experiences.
At the recent MEMS and Sensors Executive Congress, Ryan Kraudel, VP Marketing at Valencell, a biometric sensor technology company, spoke with EECatalog. Edited excerpts of our conversation follow.
EECatalog: Please introduce our readers to Valencell.
Ryan Kraudel, Valencell: We make the biometric sensor systems that go into wearable devices of all kinds including smartwatches, fitness bands, and ear buds. Most of our customer base is in the consumer electronics and wearables realm, and includes Bose, LG, Samsung, Jabra, and similar companies. For example, we’ve created a sensor module that resides in the bottom of an earbud (Figure 1). The earbud is worn like any other and measures your biometrics while you are listening to music, working out, or running.
EECatalog: Where are you seeing traction?
Kraudel, Valencell: Healthcare and gaming are two areas where we are seeing a tremendous amount of traction. There is a great deal of activity around the concept of medical device companies making consumer wearable devices. They want these devices to have medical-grade accuracy and to collect data from outside a medical facility—something people wear which continues to collect data.
The devices will be arriving in all kinds of different form factors, and the use cases underneath that are mostly in disease management and/or determining whether specific treatment protocols would be appropriate, particularly in cardiovascular treatments and care.
And [with the] gaming sector we are seeing a lot of interest from game developers and from the hardware companies in gaming who are looking at building biometrics and real-time biometric input into the game play. The player’s biometrics impact game play in real time.
For our example, one of our founders, a Star Wars fan, imagines a Star Wars game where in order to use your Jedi powers, you need to lower your heart rate to open up different capabilities and actions for your character.
At the MEMS Executive Congress in November, we put together a simple game that measures (in the background) medically relevant biometric data while you are playing the game. The game we demonstrated has real-time biometric feedback and input as well, making it possible to collect medically relevant data while someone is doing something they do as part of their everyday life.
This is an area where you are going to see significantly more activity and projects around companies looking to collect biometric data in the background, not necessarily in devices designed for data collection—for example, jewelry. We are seeing a lot of implementations of this technology in jewelry to collect biometric data in the background while the user is wearing something they want to wear anyway, rather than wearing a device that would not otherwise be on their wrist.
EECatalog: Are any chicken-and-egg type challenges occurring as the biometric gaming and medical wearable markets take off?
Kraudel, Valencell: In the gaming market in particular we are in a bit of a chicken-and-egg scenario where the game makers, the software developers, are hesitant to build in real-time biometrics when there are no (at least none at scale) hardware platforms to support it. And the hardware platforms aren’t going to build it in unless there are games available to utilize the hardware. We are in a period between new platforms coming out—the next PlayStation or the next Xbox. There is still some time before those next platforms arrive.
EECatalog: Do Augmented Reality and Virtual Reality platforms figure into Valencell’s plans?
Kraudel, Valencell: Virtual Reality and Augmented Reality are new emerging hardware platforms that are coming much more to the forefront, particularly at a consumer level.
Biometric sensors can also go into AR and VR platforms to add another dimension to the experience of those devices. VR is a fairly immersive experience. So, being able to collect biometric data at the same time to understand how someone is reacting to what is going on in the VR platform is important, not just to the user experience itself, but to being able to adjust things that are going on in that virtual reality experience based on biometrics.
In the augmented reality experience, you could have—and are already seeing—early implementations of this in sports and fitness realms, where heads up displays for competitive road biking, for example, use biometric sensors to measure heart rate and other biometrics. That data can then be projected to display on the lenses of the glasses worn by the rider.
Someone who is riding a bike very fast doesn’t need to look down at a watch or bike computer to see where they stand in terms of their current heartrate zone and where they stand with overall cardiovascular output.
EECatalog: Drivers also want to know where they stand—are they alert enough to continue driving?
Kraudel, Valencell: That’s right, and we are working with a company now which is building a device for the long haul trucking drivers to provide alerts and alarms. As soon as someone starts nodding off at the wheel, there is an alarm that goes off. They are triangulating that information partially with biometric data that these biometric sensors are putting forth and also using other MEMS sensors like accelerometers and gyroscopes to see body position and head position in particular.
EECatalog: How is Valencell supporting ease of programming and development?
Kraudel, Valencell: Our primary customers are the designers and engineers at consumer electronics, wearables, and medical device companies who are designing these products. There are two primary ways that we provide our technology to these companies.
One way is that we make pre-packaged modules called BenchmarkTM (Figures 2, 3) that have all of the hardware, all of the opto-mechanical design, all of the firmware and the algorithms that process the data coming off the sensor—all of that is packaged up in a module that you can plug and play with communication over UART and I2C.
And then data is fed into the host platform and presented in a user experience, whether that is a watch, a mobile app, or whatever the user experience is for that device. This is the most common way of working with our technology. A second way is a licensing model, applicable more to custom applications, custom form factors, that can’t necessarily use the pre-packaged module.
With the first way, our customers design around the module. In the licensing module, we essentially teach our customers how to integrate this module into their design. Typically, they come to us with an industrial design and a form factor that they want to build. Next, our engineering teams work with their engineering teams to build a design that integrates our technology in a way that maintains the level of accuracy that we expect and that our customers expect as well.
The licensing model is more for custom form factors or custom use cases that aren’t supported by our standard Benchmark module. The biometric ear buds we did with Jabra, called the Jabra Elite Sport (Figure 4), offer a good example of the licensing model. Jabra was very specific about the true wireless form factor and the user experience it wanted with the earbuds, and so we worked with Jabra to integrate this technology into the device.To give another example, a very well-known sports and fitness watch company wanted its devices to be waterproof. Some individuals use the devices as dive watches, and the watches need to be waterproof down to 100 meters. Putting these optical sensors into a pressure tank was not something we had done before with these pre-packaged modules, so we worked with the watch company on a licensing module for a custom solution.
Our development kits include the pre-packaged modules described earlier. They include reference designs either for a wrist-based device or an ear bud that are the most common starting points for our customers to begin testing out the technology. That comes with all of the API documentation and communications documentation that goes along with that. Someone can take that development kit and take that dev board to a bench and start, first, seeing how it works, and second, how that would get integrated into the design of the device.
EECatalog: Have your development kits evolved in response to customer feedback?
Kraudel, Valencell: We did not initially have development kits, but our business model has transitioned a bit. As of last year, we were exclusively a licensing company, and did not yet offer the pre-packaged modules.
The change to offering the pre-packaged modules came directly in response to customer feedback, particularly from small- and mid-sized companies. Licensing works well with the large organizations that have the resources and the time and the expertise to work with us to integrate our technology in their design. The small and mid-sized companies don’t necessarily have all those things, so they asked about buying the pre-packaged module.
EECatalog: Yet the smaller companies still face the same time to market pressures the big guys do.
Kraudel, Valencell: Right, and Benchmark significantly accelerates time to market for our customers to be able to plug this in, design around it, and go.
So that was direct response to customer feedback, which was great on a lot of levels because it allowed us to address a larger part of the market at scale that we were not able to address before with the licensing model.
EECatalog: Anything we have not talked about yet which you would like to share with our readers?
Kraudel, Valencell: There are several other initiatives we are working on from a development standpoint, such as educational sessions around what to consider in building a biometric wearable. We have written a few blog posts on this to present the expertise and experience we have in building out these biometric wearables.
Many consumer electronics and wearables companies have tremendous expertise in building a hardware device or a wearable device that is comfortable and that looks good and that people want to wear. But as soon as you start adding biometrics to it, particularly optical heart rate biometrics, it is a completely different animal because that technology now has to interact with the body.
Let’s take a smartwatch as an example. A smartwatch without biometric sensing doesn’t need to know about the human physiology of the wrist. We are all different, and so this technology interacts with different people in different ways. It must account for things like different fitness levels, different skin tones, different use cases and how people use these devices, how they wear them on their wrist or whatever location they happen to be wearing the device. There are a variety of factors along those lines that impact the user experience and the accuracy of these devices. And that is not something that most wearable companies, particularly consumer electronic companies, have expertise in. We have looked at a variety of different educational mechanisms to teach people about the things they need to consider when they are looking to build a biometric wearable and how to address some of the challenges that will inevitably come up in those projects.