Reconfigurable Image Sensors Make Imagination the only Limit for Innovative IoT Use



The improvements in image sensors in the key areas of resolution, power and pixel low light performance evidenced in today’s camera phone products can be leveraged for an array of IoT devices.

With an estimated 13 billion electronic devices already connected to the Internet, interest in the potential explosive growth of the Internet of Things (IoT) market has greatly increased and attracted a large amount of technology investment. While initial product applications have incorporated a number of sensor technologies, the limitations of unintelligent sensor technology have hamstrung efforts to provide real-time information such as images and video.

Figure_1

Figure 1. An Internet-connected smoke detector device.

Image sensor technology allows IoT devices to become smarter and provide real-time data. Improvements in image sensors in the key areas of resolution, power and pixel low light performance evidenced in today’s camera phone products can be leveraged for an array of IoT devices. In the case of IoT devices, however, there is an additional need for image processing at the point of image capture (“edge”) in order to ensure quick and accurate “decision making” by these smart devices. Image processing at the sensor also enables improved video performance (faster frame rates, HDR video, etc.) Finally, the ability to field reconfigure these smart devices by adding more functionality and correcting system errors without the need for product recalls is invaluable to both the customer and manufacturer of connected devices.

While the technology building blocks necessary to integrate imaging capabilities into smart devices have been available, a low-cost, integrated way to meet the size and power constraints of most IoT devices has not.

Beyond Smoke Detection

By leveraging the technology advances made at Forza Silicon at the sensor level, through more than 10 years of custom design experience, with similar advances made in programmable logic design at Lattice Semiconductor, a successful “proof of concept” platform has been developed. Leveraging parallel technology developments in the area of image sensor packaging using 3D stacking, Forza Silicon has demonstrated an Internet-connected smoke detector device (Figure 1) with the enhanced ability to monitor room condition, transmit an alert to the homeowner’s smartphone and stream real-time video of the actual event. This application was chosen to both validate the ability to incorporate these types of functions in a small-profile, low-cost, integrated device as well as the ability to field update the device remotely for additional capabilities or firmware changes without the need for replacement of the physical product. In addition, the reconfigurable capability of this same device means it can be programmed for other product applications and feature sets.

The possibilities for novel uses of imaging in IoT devices is expanded drastically by enhancing the image sensor’s timing and control logic with a programmable logic device. With Forza Silicon’s reconfigurable architecture, customer algorithms can control pixel integration and readout in non-standard, unique ways. For example, instead of being forced into a frame-based readout of sequential rows, as is typically the case in standard image sensors, the reconfigurable system can read rows in any arbitrary sequence, and react immediately to the latest pixel data by adjusting the readout sequence (Figure 2). Pixel integration time can also be adjusted on the fly, as determined by the customer algorithm. The FPGA’s configurable I/Os also allow flexibility to choose the best embedded processor, microcontroller or application processors for the IoT application. These examples only scratch the surface of how the full flexibility of this reconfigurable image sensor will be harnessed for future IoT devices, as the FPGA community can only guess at the diverse applications opened up by programmable logic itself.

Figure_2

Figure 2. Consumers and businesses could see the imaging possibilities for IoT devices greatly expand thanks to strides made in the fields of sensors and programmable logic.

Lattice’s MachXO/2/3 architecture complements the Forza Silicon image sensor. The MachXO families have driven down the size, power and cost of small FPGAs to the point where high-volume IoT applications can now incorporate FPGAs. The goal of this architecture is to balance capability and value for image sensor based applications. By meeting the technology challenges of the IoT market, the reconfigurable image sensor provides an optimal solution for additional markets such as automotive, surveillance and industrial applications where the implementation of image sensors and the need for enhanced image processing at the point of capture is critical.


Shawn Maloney Forza

Shawn Maloney is Executive Vice President of Business & Product Strategy at Forza Silicon. He has over 30 years of strategic sales and marketing experience in the imaging semiconductor industry and played a leading role in the introduction of two breakthrough technologies (CMOS image sensors and MEMS) into the mobile imaging market.

Kambiz Khalilian_Lattice Semicondutor_Headshot

Kambiz Khalilian is marketing director at Lattice Semiconductor, responsible for driving strategy for the company’s cost-effective and flexible industrial and automotive solutions, which address the market’s need to interface with new image sensors and display technologies and process ever increasing image resolutions and frame-rates. For over the past two decades, he has worked in the technology field developing his expertise in video solutions, serving in various product marketing and engineering roles.

Share and Enjoy:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • TwitThis

Tags: