SoC Sees No Shortage of Applications
Why the Intel® Apollo Lake SoC brings the right combination of features to numerically intensive and image capture tasks like those increasingly demanded by virtual reality, autonomous driving, 3D surveillance, and more.
The Intel® Atom® Processor E3900 Series (formerly known as Apollo Lake) SoC couples its multicore CPU with the sophisticated graphics signal and image processing of the Gen9 graphics engine and a robust display controller, yielding a solution for a wide variety of modern applications. What follows is a closer look at the display controller, power savings, and graphics engine architecture, as well as the rich software support that implementing open standards achieves.
The Intel Atom Processor E3900 Series SoC integrates the next generation of the Intel processor Core, Graphics, Memory Controller, and I/O interfaces. The graphics and image processing components are a 9th generation Intel graphics processor and display controller. The Gen9 LP graphics engine contains three groups (subslices) of eight execution units (EUs) each. The EUs are connected to the CPU cores and the System Agent using a ring topology that supports a 32-bit bi-directional data bus, with separate lines for request, snoop, and acknowledge. The ring attached System Agent provides access to the SoC’s DRAM memory management unit, the display controller, and other off-chip IO controllers such as PCIe.
The Apollo Lake’s display controller includes three display pipes and supports simultaneous displays using either the MIPI DSI, HDMI, or eDP interfaces. Capable of resolution up to and including 3840×2160 @ 60Hz, with a 24bpp color palette, it supports multiple scaling options using any of the display interfaces. The display controller also supports:
- Three display pipes, simultaneous multi-streaming on all three display pipes (1x Internal and 2x External Displays)
- Two MIPI-DSI 1.1 ports
- Three DDI ports to enable eDP 1.3, DP 1.2, or HDMI 1.4b
- 1x Internal Display (eDP 1.3). DDI2 port is dedicated to eDP
- Supports 2x External Display (DP 1.2, HDMI 1.4b). DDI0 and DDI1 can be used for external displays
- HD audio on DP and HDMI
- Multi Plane Overlay (MPO)
- Intel Display Power Saving Technology (DPST) 6.3, Panel Self Refresh (PSR) and Display Refresh Rate Switching Technology (DRRS)
The display controller supports advanced power saving technologies, including Graphics Render C-state (RC6), dynamic display frequency scaling, dynamic backlight brightness, panel self-refresh, and the ability to dynamically adjust the core display clock and the display’s frame rate.
The SoC employs dynamic frequency and power sharing to efficiently conserve energy by varying burst frequency depending on the type of instructions being executed. This approach makes it possible to boost the power level to achieve performance gains for high intensity “dynamic” workloads using intelligent power averaging algorithms to manage power and thermal headroom.
Graphics and Media Engine
The Apollo Lake Gen9 LP has built-in hardware decoders for many video compression standards including H.264, MPEG2, VC-1, WMV9, HEVC, VP8, VP9 and JPEG/MJPEG. Built-in hardware encoders are included for H.264, HEVC, VP8, and JPEG/MJPEG video compression standards.
Graphics Engine Architecture
The foundational building block of the Gen9 LP compute architecture is the execution unit (Figure 1). Its architecture combines simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT). The EUs are compute processors that drive multiple issue, single instruction, multiple data arithmetic logic units (SIMD ALUs) pipelined across multiple threads for high throughput floating-point and integer compute.
The fine-grained nature of the EUs insures continuous streams of ready-to-execute instructions while also enabling latency hiding of longer operations.
The grouping of EUs into subslices, with each subslice containing a locale thread dispatch unit and its own local instruction cache, allows each subslice to support 56 simultaneous threads.Imaging
The Apollo Lake’s MIPI CSI2 controller and Image Signal Processor can capture concurrent streams from four (4) operating sensors (cameras). Able to save streams to memory for offline processing, the imaging block can also pipeline them through the Intel 9th generation (Gen 9) LP graphics and media encode/decode engine. The engine includes three slices of six EUs each (3×6), with each slice supporting six threads for a total of 108 available threads.
Engine features also encompass:
- Support for 3-D rendering, media compositing, and video encoding
- Graphics Burst enabled through energy counters
- 4x anti-aliasing
- Support for content protection using PAVP 2.0 and HDCP 1.4/2.0.
The Intel® Atom® Processor E3900 Series graphics engine supports the industry standard DirectX 12 and OpenGL 4.2 graphics APIs for 2D and 3D rendering of vector graphics.
Using the OpenCL framework to support parallel computing using task and data-based parallelism, the SoC harnesses the power of multicore and vector processing capabilities.
Intel has ported the OpenCV (computer vision) API to the SoC’s Gen9 LP graphics engine. Offering support for computer/machine vision applications, the OpenCV API also makes the SoC suitable for medical imaging, digital surveillance, biometric identification, autonomous driving, and optical correction designs.
OpenVX is another computer vision API that can be used to accelerate Apollo Lake computer vision applications. OpenVX is targeted at real-time mobile and embedded platforms that are constrained by low power requirements. Unlike OpenCV, with its wide scope of functions and camera APIs/interfaces, OpenVX focuses on core hardware accelerated functions for mobile vision. OpenVX has lower precision requirements than OpenCL, which permits its use in low-cost and low-power system designs.
Support for Intel® RealSense™ technology allows the implementation of perceptual computing algorithms, useful in such fields as medical imaging, computer vision, digital surveillance, and factory machine vision. The RealSense SDKs from Intel contain libraries and example programs that, when used with a supported RealSense camera, demonstrate facial expression recognition, hand gesture recognition, depth perception, and 3D rendering of objects.
Intel has also created the Integrated Performance Primitives (IPP) package, a function library and collection of tools providing low-level building blocks for image- signal- and data processing (compression/decompression/cryptography). IPP functions are highly optimized for performance using Intel’s Streaming SIMD Extensions (Intel SSE) and Intel’s Advanced Vector Extensions (Intel AVX/AVX2). The IPP enhances OpenCV by providing functionality optimized to Intel’s wide range of x86 architectures.
The Apollo Lake’s graphics and image processing capabilities are useful in many industrial application areas.
The OpenCL framework allows numerically intensive applications the ability to offload complex mathematic operations onto the graphics engine and its EUs. The parallelism available in OpenCL improves the efficiency of multi-threaded applications by allowing the software designer the ability to distribute tasks across a collection of heterogeneous compute units. The collection of compute units and how they are used within the application is left up to the software developer, which permits the application to customize computing resources to the application-specific tasks.
Image Capture/Processing Applications
The Apollo Lake’s on-die integrated processor graphics architecture offers outstanding real time 3D rendering and media performance. These image processing capabilities allow the system designer to implement complex image capture and image processing applications.
The OpenCL API allows the use of the Gen9 LP to efficiently analyze images to maximize clarity and minimize visual noise and halos. Figure 3 shows an example: Using OpenCL before and after the Smart Sharpen feature in Adobe Photoshop.
Photoshop’s “Intelligent upsampling” feature can be also be accelerated using OpenCL and Intel’s processor graphics to accelerate the upsampling operations while preserving details and sharpness without the introduction of visual noise.
Perceptual Computing Applications
Intel’s perceptual computing mission statement is “Add human like sensing and intelligence to devices and machines.” To this end, Intel has developed a suite of components that may be used to provide a device or machine the ability to better understand and interact with its environment. RealSense cameras provide 3D depth sensing capabilities, and RealSense tracking modules allows devices to comprehend position and orientation, providing the ability to navigate the world. RealSense middleware enables devices to locate, sense, identify and interact in both real and virtual worlds.
With the Intel Simultaneous Localization and Mapping (SLAM) middleware library devices gain the ability to understand position and orientation. The SLAM library has functions for six-degrees-of-freedom tracking with real world scale, fast initialization, and no prior knowledge of the environment. Re-localization gives machines the ability to learn an area, save/load a tracking state, and recover localization with saved data. 3D dense reconstruction uses depth data to create live 3D models and maps for use in virtual/mixed reality landscapes, and 2D dense reconstruction uses depth data to allow creation of 3D occupancy maps as inputs to robots.
The Person middleware library permits sensing, recognition and understanding of body posture, body tracking, and body gestures. Additional functionality supports person detection (locating persons within a scene) and tracking body parts movements (with understanding of pre-defined body movements such as pointing and waving). Tracking of face movements and facial expression detection (emotion detection: smile=happy, frown=sad) are additional Person library features. Body direction (front/back/side) and body posture (sitting/lying/standing) comprehension is also supported.
An Object library allows software applications to identify and locate objects within image streams. This includes a library of components that can be utilized in market-specific applications. Deep learning cloud services allow for accuracy improvement of existing objects and for collecting data to increase the number of supported objects.
Using room and object localization, machines will understand what objects and rooms are and where they are within image streams. With a moving camera, object tracking allows a machine to not lose the understanding of objects of interest.
The Hands library can detect hand movement and interact with objects in virtual environments. Possible applications include user interface control and navigation. Interaction with objects in virtual reality is enabled through the 3D mesh of hands and collider-based interaction with virtual objects.
All of these perceptual computing abilities enable system designs for use in the virtual reality, robotics, and autonomous vehicle markets. Additional markets include shipping/logistics, PC peripherals, home/retail surveillance, and mobile 3D scanning.[ii]
Mr. Philp received a BS degree in Mathematical Sciences from the University of Texas at Dallas in 1989 and has almost 30 years of experience in the design and implementation of embedded software for use in real-time and embedded systems. He is currently employed as a Senior Software Engineer at WinSystems, Inc.
[i] Graphic does not accurately reflect the actual Apollo Lake’s EU topology.
[ii] No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
WinSystems disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your WinSystems representative to obtain the latest forecast, schedule, specifications and roadmaps.
The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request.