Testing IoT at Scale Using Realistic Data, Part I
IoT can involve vast numbers of sensors, devices, and gateways that connect to a cloud. How do you comprehensively test your whole IoT system and not just the sum of its parts? KnowThings.io introduces a novel solution to perform quality testing on a massive scale using machine learning and dramatically reduces the engineering workload… (read more)
Editor’s Note: Embedded Systems Engineering’s Editor-in-Chief, Lynnette Reese, sat down with KnowThings.io, an aspiring, smart startup accelerator project within CA Technologies that created a tool for IoT developers to realistically test IoT applications, from small to very large scale by leveraging machine learning. They currently call it the Self-learning IoT Virtualizer. KnowThings.io’s CEO, Anand Kameswaran, talked with Reese about its mission to make realistic IoT simulation and testing effective and easy, including cloud interaction, internet foibles, massive numbers of IoT sensors and connections, and IoT chaos in general.
Lynnette Reese (LR) Embedded Systems Engineering: You’re the CEO of a very busy start-up; I am glad you made the time for this interview.
Anand Kameswaran, (AK) KnowThings: You’re welcome.
LR: I understand that KnowThings.io has a tool that makes testing the whole integrated IoT/Cloud /network arrangement much easier for developers. You call the KnowThings tool an IoT Virtualizer. Can you briefly explain how the tool works?
AK: It’s an IoT simulator that creates a virtual device model, so we like to call it an IoT Virtualizer. In short, it can mimic real device system interactions accurately within minutes. It’s a specific type of simulation that interacts at the network layer, using patented self-learning or machine learning algorithms to simulate up to hundreds of thousands of individual data sensors, device inputs, and their interaction with the cloud. Think of it as a “no surprises” test harness for an entire IoT network, even if that IoT system is very, very large.
It allows you to test sections as you develop, try out new tweaks to see how it affects everything else, and test your final revisions so it’s as good as it can be before you send it out into the real world.
LR: So, it allows you to experience the larger picture of thousands of IoT devices on the internet, interacting with each other and the cloud? But without having to have the actual IoT devices online yet?
AK: Yes. There’s individual bugs that occur on a device level or with the application that runs analytics, processing etc., but there are also issues that can develop simply based on the number of devices you’re dealing with. When you have thousands of devices collecting data that coalesce into valuable trends and knowledge crucial to good decision-making, there’s a level of collaboration that has to be worked out as the sum of thousands of IoT parts creates a whole different beast.
In one case, we were working with a company doing shipping container monitoring. And their use case is to have 50,000 containers on a ship, with all containers communicating with a single, central node. When we were working with the prototype virtualizer, we were already engaged with customers who were trying to accomplish goals at that kind of scale. We are still working with customers today that are testing at large scale across several industries, whether it’s supply chain and transportation, smart agriculture in monitoring fairly large fields, and so on. That’s one of the reasons why we are interested in those who want to get into our early adopter program, our beta test program, so we can expose the IoT Virtualizer to as many unique industry problems as we can.
LR: Can you tell us more about the prototype tool and how it’s coming along?
AK: IoT is a really big space, covering many industries. We are having a new community edition preview on our website and it’s free to download today from our site, knowthings.io.
LR: Why are you giving away this incredible tool for free?
AK: We value getting direct feedback from customers and shaping the direction from that. There is still a lot to be learned. We are bringing together people from the embedded space, cloud networking, and other areas; they’re all coming together, so the best practices going forward form the best development tool we can provide.
LR: Who would the tool benefit most, and why?
AK: Any IoT developer or solution architect or tester working with IoT applications who wants to test their solutions to a large scale quickly, cheaply, and thoroughly. It can improve time to market, reduce labor costs, and reduce the confusion and frustration that can occur as an engineer straddles both embedded hardware and network cloud integration to implement a viable system in the real world with as few surprises as possible.
LR: The Virtualizer is a new idea, then? No one else offers this?
AK: Though device virtualization exists today through a few vendors, usage of machine learning to generate realistic data scenarios is something unique about our solution. The community edition pre-release is the preview of our free edition that will always remain free for customers to try before they purchase our commercial product. It runs on a desktop or even the Raspberry Pi Zero, but we are planning to release a cloud-based one soon. Those that are interested in working closely with us can apply to participate in our early adopter program.
LR: How did you get the idea for this tool?
AK: It started a few years ago, as an idea to borrow techniques from work CA Technologies was doing on service virtualization based on genomic sequencing. We’ve been working on the underlying machine learning with the University of Swinburne, Australia, as part of a partnership that predates KnowThings. We found that we could take these algorithms that we use for doing genomic sequencing and apply them to learn computer messages. Sequencing a bunch of genes is actually not a whole lot different than sequencing bytes in working towards understanding what that information is telling us.
There is a type of machine learning associated with genome sequencing, which is a learning algorithm and data mining method that analyzes recorded samples. After this was successfully applied in a service virtualization solution, we felt a similar approach can be taken in the IoT space for the KnowThings solution. We flipped that machine learning to automatically create a simulation of IoT devices with individual, asynchronous behaviors represented at different nodes, or locations on the network. It came out of a collaboration with Computer Associates’ strategic research in continuing to advance the underlying algorithm for the IoT Virtualizer. IoT presents a different type of data space versus genome sequencing, so we can make assumptions that let us take some shortcuts outside of the original genome sequencing algorithm and end up with a very efficient algorithm for IoT simulation.
After several years in research, KnowThings is now on the third generation of the technology. Indeed, some of the previous versions were run in service virtualization solutions with real customers on high-performance computers, dealing with hundreds of thousands of transactions per second. So, we have a history of real-life testing already, and we know that the code and the simulation can successfully accommodate a huge scale simulation.
KnowThings has some close early adopters and is working with customers that need an environment at that sort of scale.
LR: What kind of applications would typically require this sort of scale?
AK: The verticals that KnowThings is currently working with for existing customers include smart agriculture, smart transportation, and facets of retail that include IoT. Smart ag would include hydroponics farms. The Virtualizer for smart transportation would help with the operations and logistics side. And an IoT retail channel might include smartphone tracking via Bluetooth beacons to establish behaviors for very targeted marketing through customer smartphones that act as IoT devices through, say, a coupon app that is also tracking customer behavior to some degree. It’s one thing to test a digital coupon-to-smartphone interaction with one or two participating smartphones, but what happens on Black Friday? KnowThings.io ’s product assists developers to honestly answer the question, “What could go wrong?”
The KnowThings IoT Virtualizer would work well for any IoT application that needs to test on a scale that’s too large to simulate by one’s self. It can save time, for one thing.
LR: How can developers get their hands on this tool?
AK: We are in the early adopter stage right now and offering a role in beta testing. There’s an opportunity for us to partner with those customers that are part of the early adoption program. Not only will partners shape what everything should look like, but they will also help in developing best practices in a very challenging development environment. We want to know about the real challenges IoT is facing and concentrate on solving the problems that IoT developers care about.
Anyone interested in trying it out and contributing suggestions to improving the tool can download the community edition pre-release or sign up for the early adopter program at https://knowthings.io/. Commercial product launch is in mid-summer.
For more information, go to the KnowThings.io Self-Learning IoT Virtualizer FAQ online.
Part II of this story can be found at: http://eecatalog.com/IoT/2018/05/15/testing-iot-at-scale-using-realistic-data-part-ii/
Lynnette Reese is Editor-in-Chief, Embedded Intel Solutions and Embedded Systems Engineering, and has been working in various roles as an electrical engineer for over two decades