Overview

The Qualcomm® Neural Processing SDK allows for the preparation and execution of AI models on Qualcomm supported hardware. It comes with tools to prepare your model for a target device, runtimes for executing your model on that device, and an API to control how your model interacts with your device’s processors.

In order to use your model on the target device, there are two major stages:

  1. Building and Executing your Model on your target device.

  2. Creating an app that runs on your target device to use your model..

The tools / binaries / executables that are needed for each stage will vary based on your host machine, target device, model, and which optimizations you enable.

Model Workflow

../images/snpe.png

Model training is performed on a popular deep learning framework (PyTorch, TFLite, ONNX, and TensorFlow models are supported by Qualcomm® Neural Processing SDK). After training is complete the trained model is converted into a DLC file that can be loaded into the Qualcomm® Neural Processing SDK runtime. This DLC file can then be used to perform forward inference passes using one of the Snapdragon accelerated compute cores.

The basic Qualcomm® Neural Processing SDK workflow consists of only a few steps:

  1. Convert the network model to a DLC file that can be loaded by Qualcomm® Neural Processing SDK.

  2. Optionally quantize the DLC file for running on the Hexagon DSP.

  3. Prepare input data for the model.

  4. Load and execute the model using Qualcomm® Neural Processing SDK runtime.

Understanding the docs

These docs are organized based on the workflow above.

  1. “Setup” explains how to install SNPE and all necessary dependencies to work with your model and target device.

  2. “Tutorials” gives step by step guides for how to use SNPE in a variety of situations.

  3. “Network Models” explains which network operations (ex. SoftMax) which your model may use are supported for which processors.

  4. “Input Data and Preprocessing” explains how to format input data in a way your model can interpret.

  5. “Benchmarking and Accuracy” explains how to test and analyze your model’s performance to optimize for your target device.

  6. “Tools” provides a reference for all CLI tools that are used with SNPE.

  7. “Debug Tools” provides a reference for additional debugging tools which may help resolve issues.

  8. “API” gives a detailed reference for how to use the SNPE API when developing your app / executable.

  9. “Limitations” contains known issues and workarounds.

  10. “Revision History” notes changes for each new version of the SNPE SDK.

  11. “References” contains a glossary of terms.

The best place to start for new readers is the Building and Executing a Model Tutorial as that will walk you through all the steps needed to work with your model (including the Setup steps).

Detailed Capabilities

The Qualcomm® Neural Processing SDK is a Qualcomm Snapdragon software accelerated runtime for the execution of deep neural networks. With the Qualcomm® Neural Processing SDK, users can:

  • Execute an arbitrarily deep neural network.

  • Execute the network on the Snapdragon CPU, the Adreno GPU or the Hexagon DSP.

  • Debug the network execution on x86 Ubuntu Linux.

  • Convert PyTorch, TFLite, ONNX, and TensorFlow models to a Qualcomm® Neural Processing SDK Deep Learning Container (DLC) file.

  • Quantize DLC files to 8 or 16 bit fixed point for running on the Hexagon DSP.

  • Debug and analyze the performance of the network with Qualcomm® Neural Processing SDK tools.

  • Integrate a network into applications and other code via C++ or Java.

Supported Snapdragon Devices

Linux/Android

Snapdragon Device/Chip

Supported Tool Chains

CPU

GPU

DSP Hexagon Arch

AIP

SD 8 Elite Gen 5 (SM8850)

aarch64-android

Yes

Yes

V81

No

SD 8 Gen 4 (SM8750)

aarch64-android

Yes

Yes

V79

No

SD 8 Gen 3 (SM8650)

aarch64-android

Yes

Yes

V75

No

SD 8 Gen 2 (SM8550)

aarch64-android

Yes

Yes

V73

No

SD 8s Gen 3 (SM8635)

aarch64-android

Yes

Yes

V73

No

SD 8+ Gen 1 (SM8475)

aarch64-android

Yes

Yes

V69

No

SD 8 Gen 1 (SM8450)

aarch64-android

Yes

Yes

V69

No

888+ (SM8350P)

aarch64-android

Yes

Yes

V68

No

888 (SM8350)

aarch64-android

Yes

Yes

V68

No

7+ Gen 3 (SM7675)

aarch64-android

Yes

Yes

V73

No

7 Gen 1 (SM7450)

aarch64-android

Yes

Yes

V69

No

778G (SM7325)

aarch64-android

Yes

Yes

V68

No

QCM6490

aarch64-android, aarch64-ubuntu-gcc9.4, aarch64-oe-linux-gcc11.2

Yes

Yes

V68

No

865 (SM8250)

aarch64-android

Yes

Yes

V66

Yes

765 (SM7250)

aarch64-android

Yes

Yes

V66

Yes

750G (SM7225)

aarch64-android

Yes

Yes

V66

Yes

690 (SM6350)

aarch64-android

Yes

Yes

V66

Yes

QRB5165U

aarch64-ubuntu-gcc9.4

Yes

Yes

V66

Yes

QRB5165LE

aarch64-oe-linux-gcc9.3, aarch64-oe-linux-gcc11.2

Yes

Yes

V66

Yes

QCS7230LA

aarch64-android

Yes

Yes

V66

Yes

QCS7230LE

aarch64-oe-linux-gcc9.3, aarch64-oe-linux-gcc11.2

Yes

Yes

V66

Yes

695 (SM6375)

aarch64-android

Yes

Yes

V66

No

680 (SM6225)

aarch64-android

Yes

Yes

V66

No

480 (SM4350/6325)

aarch64-android

Yes

Yes

V66

No

460 (SM4250)

aarch64-android

Yes

Yes

V66

No

662 (SM6115)

aarch64-android

Yes

Yes

V66

No

QCS610LA

aarch64-android

Yes

Yes

V66

No

QCS610LE

aarch64-oe-linux-gcc9.3

Yes

Yes

V66

No

QCS410LA

aarch64-android

Yes

Yes

V66

No

QCS410LE

aarch64-oe-linux-gcc9.3

Yes

Yes

V66

No

QCM4290

aarch64-android

Yes

Yes

V66

No

QCM6125

aarch64-android

Yes

Yes

V66

No

QRB4210LE

aarch64-oe-linux-gcc9.3

Yes

Yes

V66

No

QCM4490

aarch64-android

Yes

Yes

N/A

No

XR2 Gen 2 (SXR2230P)

aarch64-android

Yes

Yes

V69

No

XR2 Gen 2 (SXR2250P)

aarch64-android

Yes

Yes

V69

No

QCS/QCM8550LA

aarch64-android

Yes

Yes

V73

No

QCS/QCM8550LE

aarch64-oe-linux-gcc11.2

Yes

Yes

V73

No

QCS9100

aarch64-oe-linux-gcc11.2

Yes

Yes

V73

No

QCS/QCM6690

aarch64-android

Yes

Yes

V73

No

QCS/QCM2290

aarch64-android

Yes

Yes

N/A

No

QCS8625

aarch64-oe-linux-gcc11.2, aarch64-android

Yes

No

V75

No

Windows

Snapdragon Device/Chip

Supported Tool Chains

CPU

GPU

DSP Hexagon Arch

AIP

Snapdragon X2 Elite Extreme (SC8480XP)

aarch64-windows-msvc, arm64x-windows-msvc

Yes

No

V81

No

Snapdragon 8cx Gen 4 (SC8380XP)

aarch64-windows-msvc, arm64x-windows-msvc

Yes

No

V73

No

Snapdragon 8cx Gen 3 (SC8280X)

aarch64-windows-msvc

Yes

No

V68

No

Snapdragon 8cx Gen 2 (SC8180X)

aarch64-windows-msvc

Yes

No

V66

No

Snapdragon 7c gen 2 (SC7280X)

aarch64-windows-msvc

Yes

No

V68

No

Note

See Windows ARM64X Support for more information about arm64x-windows-msvc toolchain.

What’s included in the SDK

Linux/Android

SDK Asset

Type

Compiler

C++ STL

Description

bin/x86_64-linux-clang

binary

clang

libc++ 14

x86 Linux binaries

bin/aarch64-android

binary

clang

libc++

Aarch64 Android binaries

bin/aarch64-ubuntu-gcc9.4

binary

gcc9.4

gnustl

Aarch64 Ubuntu binaries

bin/aarch64-oe-linux-gcc8.2

binary

gcc8.2

gnustl

Aarch64 Linux binaries

bin/aarch64-oe-linux-gcc9.3

binary

gcc9.3

gnustl

Aarch64 Linux binaries

bin/aarch64-oe-linux-gcc11.2

binary

gcc11.2

gnustl

Aarch64 Linux binaries

lib/android

lib

-

-

Android aar file used to include SNPE into your application

lib/x86_64-linux-clang

lib

clang

libc++ 14

x86 Linux libraries

lib/aarch64-android

lib

clang

libc++

Aarch64 Android libraries

lib/hexagon-<dsp_version>

lib

-

-

Hexagon DSP runtime libraries

lib/aarch64-ubuntu-gcc9.4

lib

gcc9.4

gnustl

Aarch64 Ubuntu libraries

lib/aarch64-oe-linux-gcc8.2

lib

gcc8.2

gnustl

Aarch64 Linux libraries

lib/aarch64-oe-linux-gcc9.3

lib

gcc9.3

gnustl

Aarch64 Linux libraries

lib/aarch64-oe-linux-gcc11.2

lib

gcc11.2

gnustl

Aarch64 Linux libraries

lib/python

lib

-

-

Qualcomm (R) Neural Processing SDK Python model tools modules

include/SNPE

include dir

-

-

Qualcomm (R) Neural Processing SDK API header files

examples

examples dir

-

-

Source code sample applications in Native C, C++ and Android Java

docs

documents

-

-

Reference Guide

benchmarks

scripts

-

-

Benchmark framework to gather runtime performance data on devices

Windows

SDK Asset

Type

Compiler

C++ STL

Description

bin/x86_64-windows-msvc

binary

MSVC

MSVC

x86_64 Windows binaries

bin/aarch64-windows-msvc

binary

MSVC

MSVC

ARM64 Windows binaries

bin/arm64x-windows-msvc

binary

MSVC

MSVC

ARM64X Windows binaries

lib/x86_64-windows-msvc

lib

MSVC

MSVC

x86_64 Windows libraries

lib/aarch64-windows-msvc

lib

MSVC

MSVC

ARM64 Windows libraries

lib/arm64x-windows-msvc

lib

MSVC

MSVC

ARM64X Windows libraries

lib/hexagon-v66

lib

-

-

Hexagon DSP v66 runtime libraries

lib/hexagon-v68

lib

-

-

Hexagon DSP v68 runtime libraries

lib/hexagon-v73

lib

-

-

Hexagon DSP v73 runtime libraries

lib/python

lib

-

-

Qualcomm (R) Neural Processing SDK Python model tools modules

include/SNPE

include dir

-

-

Qualcomm (R) Neural Processing SDK API header files

examples

examples dir

-

-

Source code sample applications in Native C++

docs

documents

-

-

Reference Guide

share

resources

-

-

Qualcomm (R) Neural Processing SDK Udo resources

Important Files and Locations

$SNPE_ROOT refers to the base directory where the SDK is installed.

Linux/Android

File

Type

Details

Location

envsetup.sh

script

Sets up environment variables needed to run SDK tools and binaries

  • $SNPE_ROOT/bin

snpe-onnx-to-dlc

script

Converts an ONNX model to a DLC file

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-tensorflow-to-dlc

script

Converts a TensorFlow model to a DLC file

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-tflite-to-dlc

script

Converts a TfLite model to a DLC file

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-pytorch-to-dlc

script

Converts a Pytorch model to a DLC file

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-dlc-quant

executable

Quantize a DLC file using 8/16 bit quantization

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-dlc-graph-prepare

executable

Prepares (offline) a graph for HTP on the host

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-dlc-quantize

script

Invokes snpe-dlc-quant and snpe-dlc-graph-prepare (for backward compatibility)

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-diagview

executable

Displays Neural Processing SDK timing output

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-dlc-info

script

Prints DLC file information

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-dlc-viewer

script

Displays a DLC file as an HTML file

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-dlc-diff

script

Compares two different DLC files

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe-udo-package-generator

executable

Generates a UDO package

  • $SNPE_ROOT/bin/x86_64-linux-clang

snpe_bench.py

script

Executes DLC model on device and collect benchmark information

  • $SNPE_ROOT/benchmarks

snpe-net-run

executable

Executes neural networks using SDK APIs

  • $SNPE_ROOT/bin/x86_64-linux-clang

  • $SNPE_ROOT/bin/aarch64-android

  • $SNPE_ROOT/bin/aarch64-oe-linux-gcc8.2

  • $SNPE_ROOT/bin/aarch64-oe-linux-gcc9.3

libSNPE.so

library

Qualcomm® Neural Processing SDK runtime for host and device development

  • $SNPE_ROOT/lib/x86_64-linux-clang

  • $SNPE_ROOT/lib/aarch64-android

  • $SNPE_ROOT/lib/aarch64-oe-linux-gcc8.2

  • $SNPE_ROOT/lib/aarch64-oe-linux-gcc9.3

libHtpPrepare.so

library

Library for offline graph preparation for HTP

  • $SNPE_ROOT/lib/x86_64-linux-clang

libSnpeDspV66Skel.so

library

Hexagon DSP runtime library for v66 targets

  • $SNPE_ROOT/lib/hexagon-v66/unsigned

libSnpeHtpVxxSkel.so

library

Hexagon DSP runtime library for v68/69/73/75/79 targets

  • $SNPE_ROOT/lib/hexagon-v68/unsigned

  • $SNPE_ROOT/lib/hexagon-v69/unsigned

  • $SNPE_ROOT/lib/hexagon-v73/unsigned

  • $SNPE_ROOT/lib/hexagon-v75/unsigned

  • $SNPE_ROOT/lib/hexagon-v79/unsigned

Windows

File

Type

Details

Location

snpe-net-run.exe

executable

Executes neural networks using SDK APIs

  • $SNPE_ROOT/bin/x86_64-windows-msvc

  • $SNPE_ROOT/bin/aarch64-windows-msvc

  • $SNPE_ROOT/bin/arm64x-windows-msvc

snpe-onnx-to-dlc

script

Converts an ONNX model to a DLC file

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-tensorflow-to-dlc

script

Converts a TensorFlow model to a DLC file

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-dlc-quant

executable

Quantizes a DLC file using 8/16 bit quantization

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-dlc-graph-prepare

executable

Prepares (offline) a graph for HTP on the host

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-diagview

executable

Displays Neural Processing SDK timing output

  • $SNPE_ROOT/bin/x86_64-windows-msvc

  • $SNPE_ROOT/bin/aarch64-windows-msvc

snpe-dlc-info

script

Prints DLC file information

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-dlc-viewer

script

Displays a DLC file as an HTML file

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-dlc-diff

script

Compares two different DLC files

  • $SNPE_ROOT/bin/x86_64-windows-msvc

snpe-udo-package-generator

executable

Generates UDO package

  • $SNPE_ROOT/bin/x86_64-windows-msvc

envsetup.ps1

script

Sets up environment variables needed to run SDK tools and binaries

  • $SNPE_ROOT/bin

SNPE.dll

library

Qualcomm® Neural Processing SDK runtime for host and device development

  • $SNPE_ROOT/lib/aarch64-windows-msvc

  • $SNPE_ROOT/lib/arm64x-windows-msvc

  • $SNPE_ROOT/lib/x86_64-windows-msvc

SnpeHtpPrepare.dll

library

Library for offline graph preparation for HTP

  • $SNPE_ROOT/lib/aarch64-windows-msvc

  • $SNPE_ROOT/lib/arm64x-windows-msvc

libSnpeDspV66Skel.so

library

Hexagon DSP runtime library for v66 targets

  • $SNPE_ROOT/lib/hexagon-v66/unsigned

libSnpeHtpV68Skel.so

library

Hexagon DSP runtime library for v68 targets

  • $SNPE_ROOT/lib/hexagon-v68/unsigned

libSnpeHtpV73Skel.so

library

Hexagon DSP runtime library for v73 targets

  • $SNPE_ROOT/lib/hexagon-v73/unsigned