PSNPE Introduction

Purpose

Executing multiple SNPE instances on heterogeneous computation resources (e.g. HVX, HMX) in parallel was named as parallel SNPE (PSNPE) that can significantly speed up the bulk-input processing task.

Structure

One PSNPE instance contains one or more SNPE instances that has three optional execution modes including synchronous, output asynchronous and input/output asynchronous. The creation of multiple SNPE instances helps to create multiple worker threads. The worker threads allow multiple execution calls to be queued, ready to be dispatched when the hardware is ready, improving throughput.

• In synchronous mode, the processing steps are in sequence. Firstly is loading bulk input data, then executing multiple SNPE instances parallely, and lastly getting output data.

• In output asynchronous mode, loading input data is synchronous, however executing SNPE instances and getting output is asynchronous. The callback fuction is used to implement asynchrony.

• In input/output asynchronous mode, loading input data, executing SNPE instance and getting output data are all asynchronous. Currently user buffer memory mapped is not supported in this mode.

Features

1. Synchronous mode

2. Output asynchronous mode

3. Input/output asynchronous mode

The following image shows the three different modes.