Minimizing Latency in Mission-Critical Video Processing Applications

Why FPGAs occupy a crucial role for video processing applications as they pare down latency

Three video processing applications—degraded visual environments, autonomous vehicles, and active protection systems—benefit from three core processing architectures. These architectures are FPGAs, GPPs, and GPUs. Common to these applications is the need for a custom stream processor that minimizes latency—the goal of any mission-critical video processing application. The FPGA is a critical piece of processing technology for these three applications because it addresses both what’s common and what varies in the applications’ environments.

A Jeep Wrangler Rubicon earlier this year at the Woomera Test Range in South Australia. U.S. Army Tank Automotive Research, Development and Engineering Center (TARDEC) engineers across the globe in Warren, Michigan operated the vehicle. (Photo by Isiah Davenport) (Photo Credit: U.S. Army)

The Modularity Gap
While this ability to ingest myriad I/O types is important—especially when upgrading legacy systems or interfacing with older video interfaces—it can be problematic for systems that were not designed to be modular. For example, many legacy cameras simply produce a data stream using Gigabit Ethernet. Others use lower latency interfaces such as Camera Link. If your application requires performing a technology insertion and upgrading just the processors in the system, it can be challenging to do so if the components are not modular.

The Lightning platform from Abaco Systems is one example of an approach to solving this problem of I/O diversity and upgradability with flexible and modular product technologies. Our FPGA Mezzanine Cards (FMCs) deliver high performance I/O and our patented Micro-Mezzanine System (MMS) is intended for a broader mix of slower speed I/O. For video interfaces, our FMC430 gives users a direct Ethernet connection to a FPGA system cards such as the VP880, which is built on Xilinx Zynq Ultrascale+ and Virtex Ultrascale device families.


Figure 1: One example of a solution which aims to reduce NRE costs and enable users to modernize their systems by including the latest FPGAs is the Abaco Systems FMC422 FPGA Mezzanine Card, a low-cost Camera Link FMC.

To reduce NRE and the need for one-off custom system designs, we’ve recently introduced our low-cost Camera Link FMC, the FMC422 (Figure 1), which allows you to upgrade your existing systems to the latest FPGA devices. At the same time, it makes it possible to implement a modular approach to ease future upgrades.

VITA 57.1 FMC compliant, the FMC422 is designed for demanding, mission-critical video processing applications that require high-performance capture or output together with FPGA processing. The FMC422 suits high bandwidth deployments such as the three applications noted earlier: degraded visual environments, active protection systems, and autonomous vehicles.

Reduced Integration Risk
When the goal is to substantially minimize the time, cost, and risk of developing mission-ready systems for low-latency video applications, a complete single source solution significantly reduces integration risk. A comprehensive support package is another factor which has the potential to lessen development effort, decreasing cost and time-to-deployment.

Modular solutions that leverage proven interfaces can enable you to refresh your legacy systems rather than having to completely re-architect them.

Camera Link, currently in version 2.0, has been on the market a long time, and its low latency and high bandwidth characteristics make it a dependable option for many video processing applications. However, monolithic video processing boards with Camera Link inputs can make it challenging to upgrade to the latest technology. Transitioning to the FMC422 and an FPGA board can enable tech insertion today, as well as simplify the path to future FMC module upgrades. This modular approach can also significantly mitigate the impact of obsolescence.

Table 1: A case of ‘use the right tool for the job.’

Faster Data Rate Potential
A direct LVDS connection to the FPGA from the FMC should be considered when comparing solutions because older generation transceiver technology cannot operate with modern FPGAs, and a direct connection offers the potential to run at a faster data rate than the standard. Another consideration is whether integration complexity can be alleviated with the choice of one partner for multiple aspects of your system development.

Camera links up to and beyond the industry standard are provided by the FMC422, while Power over Camera Link (PoCL) reduces the requirement to add power cables to the camera. Pairing the FMC422 with an FPGA carrier enables extensive support for legacy cameras as well as future products.

Adopting Standards to Keep Options Open
In cases where technology provides regular and frequent opportunities to do more, different and better, our experience is that embedded designers and developers prefer to keep their options open. Adopting industry standards is one way of doing that, as is basing developments on open architectures. Implementing functionality in software, rather than in hardware, is also becoming more popular, given the greater ease of modification and upgrade. Increasingly, however, architects of high-performance solutions are also looking to see how a modular approach may be beneficial, enabling simpler, more cost-effective upgrades as new opportunities present themselves. Combining the inherent flexibility of FPGA technology with a modular hardware architecture which complies with industry standards makes sense.

Haydn Nelson is Director of Product Management for RF and DSP, based at Abaco Systems’s DSP Innovation Center in Austin, Texas. Having been an engineer most of his career, Nelson is passionate about technology—especially FPGAs and RF. Having worked in a number of industries from mil/aero research to RF semiconductor test, his broad experience and knowledge of EW and communications systems gives him a unique view of multi-disciplinary technology. Starting as a research engineer then becoming a field applications engineer, he joined Abaco as part of the 4DSP acquisition. 


White Paper: Addressing the challenges of low latency video system requirements for embedded applications[1]


Share and Enjoy:
  • Digg
  • Sphinn
  • Facebook
  • Mixx
  • Google
  • TwitThis


Extension Media websites place cookies on your device to give you the best user experience. By using our websites, you agree to placement of these cookies and to our Privacy Policy. Please click here to accept.