NET Open Camera Concept (OCC) for iam

Owned by NET GmbH, created
Last updated: Sept 13, 2024 by Andreas Kramer | NET GmbH
5 min read

iam provides 64-bit processor scalability while combining real-time control with soft and hard engines for graphics and video. With the NET SDK and GigE Vision toolboxes shown below customers can start comfortably to build their unique vision system with iam. The open system architecture of iam enables customers to use both CPU and FPGA processing resources for their application.

 

Introducing the NET Open Camera Concept for iam

In section you will find a various software example application where we briefly described how to optimize the application code by using hardware acceleration by synthesizing a FPGA co-processor into the programmable logic of the camera.

Video: The NET Open Camera Concept (OCC)

See all Video Tutorial Session .

Example Repository

The iam_apps example repository is structured into the steps below.

  1. Set up a Virtual Host System for cross-compiling including FPGA synthesis.

  2. Check out example repository net-gmbh/iam_apps
    git clone https://bitbucket.org/net-gmbh/iam_apps.git

  3. Build the iam software platform project.
    Download latest platform source achive file (download section).

  4. Check out and chose one of our example projects including iAMGigEServeriAMGigEServer or build one of hundreds Xilinx Vitis vision library examples from the repositories below.

    1. net-gmbh/iam_apps

    2. Xilinx/Vitis_Libraries

  5. Deploy your software to camera using app2cam.sh script.

The figure below illustrates the described steps above and the following table summarizes the example projects.

 

More details and step-by-step guidlining can be found on Bitbuket net-gmbh/iam_apps

ccode_synview

cross

The example code_synview uses HLS hw acceleration written in C-code.

The main processing method can be selected among following choices:

  • no processing

  • simple C-code software processing

  • a optimized C-code version with ARM NEON instructions

  • processing with OpenCV functions

  • Hardware accelerated processing based on a C-code source

After processing the image is sent out via GigE Server.

remap_synview

cross

The example remap_synview uses the hardware acceleration function xf::cv::remap from the Vitis vision library.

After processing, the image is sent out via GigE Server.

remap_boxfilter_synview

cross

The example remap_boxfilter_synview uses the hw acceleration function xf::cv::remap from the Vitis vision library and the cv::remap and cv::boxfilter function from OpenCV.
It demonstrates the execution of multiple sw- and hw-functions in different processing threads. The connection between the threads is established with two image-buffers. One is written, the other is read and vice versa.

After processing, the image is sent out via GigE Server.

rtl_threshold_synview

cross

This example uses RTL hardware acceleration written in verilog.

After processing, the image is sent out via GigE Server.

rtl_threshold_synview_dfx

cross

This example uses RTL hardware acceleration written in verilog. It consists of two different threshold accelerators, but only one of them can be loaded into the programmable logic. The module can be changed dynamically during running acquisition. This function is called DFX (Dynamic Function eXchange) and can help to use different accelerators that wouldn't fit together in the fpga part.

After processing, the image is sent out via GigE Server.

strm_boxfilter_synview

cross

This example uses HLS hardware acceleration with the xfOpenCV function xf::cv::boxfilter in the streaming path.
The module is located after the sensor interface and before the dma. The AXI-Stream data width convertion will be done by either axiStrm_wInc/axiStrm_wDec (HLS-implementation) or axiStrm_wInc_rtl/axiStrm_wDec_rtl (RTL-implementation).

After processing, the image is sent out via GigE Server.

strm_rtl_threshold_synview

cross

This example uses RTL hardware acceleration in the streaming path.
The module is located between sensor interface and dma.
It consists of a 32-bit master and 32-bit slave axi-stream interface for sensor pixel data and a 32-bit axi-lite control interface for register access.

After processing, the image is sent out via GigE Server.

sw_scaler_synview

cross

This example uses no hardware acceleration.
But it uses the same infrastructure like other hardware accelerated examples.
The main processing consists of a simple C-code scaler for horizontal and vertical rescaling.

After processing the image is sent out via GigE Server.

sw_frm_buf_config_synview

cross

This example uses no hardware acceleration But it uses the same infrastructure like other hw accelerated examples. The application does no processing. The purpose of the app is to demonstrate how to configure (buffer-number, number of max. buffer delay) the input frame buffer manager.

After processing the image is sent out via GigE Server.

View this page on Bitbucket:
/net-gmbh/iam_apps/

 

 

 

Background and Strategy

Hardware Acceleration: Optimizing Effects

The processing power of iam can be improved by FPGA optimization effects such as pipelining, parallelization, co-processing and quantization.

The flow graph illustrates different optimization effects such as pipelining (FPGA implementation), parallelization (FPGA implementation), co-processing (CPU || FPGA) compared to the frame-based CPU processing in the first line of the figure.

 

Hardware Acceleration: Kernels Types

The figure below shows different methodes of writing acceleration kernels for iam. Most of the different ways are covered by a software example in the net-gmbh/iam_apps repository.

Introduction to Cross Compiling

Cross-compilation is the act of compiling code for one computer system (often known as the target) on a different system, called the host.

 

 

Xilinx Vitis IDE TM

Introducing Vitis IDE TM

The Xilinx Vitis unified software platform enables the development of embedded software and accelerated applications on heterogeneous Xilinx platforms including FPGAs, SoCs, and Versal ACAPs. It provides a unified programming model for accelerating your iam applications.

 

  • Develop and deploy hardware accelerated application

  • Work on an application level

  • Comfortable eclipse editor

  • Several desing examples from NET net-gmbh/iam_apps

  • Even more from Xilinx Vitis Vision Library

  • Profiling and live debugging on iam

More information can be found at Vitis Unified Software Platform

Include Examples Repository to Vitis IDE

View this page on Bitbucket:
/iam_apps/apps_gui/

 

 

 

 

© Copyright 2020 NET GmbH. Privacy Statement