UDO DSP tutorial on Windows for Quantized DLC

Overview

This tutorial describes the steps needed to create a UDO package for DSP runtime and execute the Inception-V3 model using the package. The Softmax operation has been chosen in this tutorial to demonstrate the implementation of a UDO with Qualcomm® Neural Processing SDK. Also describe the offline cache generation for DSP V68.

The Qualcomm® Neural Processing SDK for Linux provides the resources for this example under

  • $SNPE_ROOT/examples/SNPE/NativeCpp/UdoExample/Softmax

Information on UDO in general is available at Overview of UDO.

Information on running the Inception-V3 network without UDO is available at Inception-V3 Tutorial on Windows

Prerequisites

The following tutorial assumes that general Qualcomm (R) Neural Processing SDK setup has been followed to support SDK environment on Linux, TensorFlow environment, and desired platform dependencies. The steps listed in this tutorial use the Tensorflow model in the form of inception_v3_2016_08_28_frozen.pb. For details on acquiring the Inception-V3 model visit Tutorials Setup

Introduction

Here are the steps to develop and run a UDO

1.) Generation on Linux

2.) Framework Model Conversion to a DLC on Linux

3.) Package Implementation on Linux

4.) Package Compilation on Linux

5.) Package Compilation on Windows

6.) Model Execution on Windows

UDO on Windows is largely similar to UDO on Linux, but only with few more steps required to be performed on Windows. Here summarizes the workflow:

  • Step 1 is executed on Linux, and with an additional flag to generate CMakeLists.txt.

  • Step 2-4 are executed on Linux, and basically the same as in UDO DSP tutorial for Quantized DLC.

  • Step 5 is run on the x86_64 Windows host to compile the package.

  • Step 6 provides information on execution using the Qualcomm® Neural Processing SDK command-line executable snpe-net-run.

Step 1: Package Generation on Linux

Similar to the Linux example, we use the config of v68 devices (Softmax_Htp.json) as the UDO plugin in this example. Please reference Package Generation for more information about the json and the detailed steps.

Generate the SoftmaxUdo Package using the following:

export SNPE_UDO_ROOT=$SNPE_ROOT/share/SNPE/SnpeUdo
export QNN_SDK_ROOT=<path to Qualcomm® AI Direct SDK>
snpe-udo-package-generator -p $SNPE_ROOT/examples/SNPE/NativeCpp/UdoExample/Softmax/config/Softmax_Htp.json -o $SNPE_ROOT/examples/Models/InceptionV3/ --gen_cmakelists

Please note that \-\-gen_cmakelists is required for package compilation on Windows in Step 5.

Step 2-4: Prepare DLC and UDO package on Linux

These steps are basically the same as in the Linux example. Please reference

  1. Step 2: Framework model Conversion to a DLC

  2. Step 3: Package Implementations

  3. Step 4: Package Compilation

Step 5: Package Compilation on Windows

Information for compiling package is available at Compiling a UDO package for Windows

Here are the instructions for this example:

  1. Copy UDO package from Linux to Windows

  2. Open Developer Powershell for Visual Studio

  3. Execute the environment setup script, set the environment variable $env:QNN_SDK_ROOT and run CMake commands

The expected artifact is

  • The UDO package registration library: SoftmaxUdoPackage/libs/arm64_windows/UdoSoftmaxUdoPackageReg.dll

Step 6: Model Execution on Windows Devices

Execution using snpe-net-run

Executing Inception-V3 for UDO is largely the same as use of snpe-net-run without UDO.

For UDO, snpe-net-run consumes the registration library through the \-\-udo_package_path option, and the following artifacts should also be placed on the target device.

  1. Quantized dlc with UDO (for more information, please reference quantization for DSP V68 at Hexagon DSP Execution in UDO DSP tutorial for Quantized DLC)

  2. libUdoSoftmaxUdoPackageImplDsp.so (generated from Step 4)

  3. UdoSoftmaxUdoPackageReg.dll (generated from Step 5)