QNN LPAI Execution¶

Transfer model and input files to the target device

Set up a dedicated test directory on the target device or x86 host (for simulation). Copy the following into this directory:

The compiled model binary

Input data files

A predefined input_list file (for QNN-NET-RUN)

Ensure that all required QNN and LPAI runtime components are included based on the target platform Execute model using qnn-net-run to learn more about necessary components for every configuration.

Execute the model using qnn-net-run on test platform (simulator or target device)

Execute the model on supported platforms (x86, ARM, DSP). For execution instructions, refer to:

x86 Lpai backend. The same backend used during offline model generation. It can also simulate how the model would run in real-time, making it useful for testing and validation purposes.

ARM Lpai backend. This backend is less efficient due to latency introduced by FastRPC communication, which impacts overall performance.

Native DSP Lpai backend. This backend type is designed to execute directly on the DSP, to run models on this backend, qnn-net-run should be used in direct mode.

LPAI Simulation behavior illustrates the LPAI simulation on x86 Linux/Windows OS.

QNN LPAI Backend Emulation¶

The LPAI backend compiled for x86 platform supports both offline model generation and direct execution using a simulator. This capability allows clients to debug and deploy their models more quickly on an x86 machine without needing to interact directly with the target device.

Refer the offline model generation page to prepare configuration files ahead Offline LPAI Model Generation.

QNN LPAI Emulation on Linux x86¶

LPAI x86 Linux Simulation

Note

If full paths are not given to qnn-net-run, all libraries must be added to LD_LIBRARY_PATH and be discoverable by the system library loader.

From Quantized model:

$ cd ${QNN_SDK_ROOT}/examples/QNN/converter/models
$ ${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-net-run \
              --backend ${QNN_SDK_ROOT}/lib/x86_64-linux-clang/libQnnLpai.so \
              --model ${QNN_SDK_ROOT}/examples/QNN/example_libs/x86_64-linux-clang/libQnnModel.so  \
              --input_list ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt \
              --config_file /path/to/config.json

From Serialized buffer:

$ cd ${QNN_SDK_ROOT}/examples/QNN/converter/models
$ ${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-net-run \
              --backend ${QNN_SDK_ROOT}/lib/x86_64-linux-clang/libQnnLpai.so \
              --retrieve_context ${QNN_SDK_ROOT}/examples/QNN/converter/models/qnn_model_8bit_quantized.serialized.bin \
              --input_list ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt \
              --config_file /path/to/config.json

Tip

Add the necessary libraries to your LD_LIBRARY_PATH:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${QNN_SDK_ROOT}/lib/x86_64-linux-clang

QNN LPAI Emulation on Windows x86¶

LPAI x86 Windows Simulation

Follow these steps to run the LPAI Emulation Backend on a Windows x86 system:

Note

If full paths are not given to qnn-net-run.exe, all libraries must be added to PATH and be discoverable by the system library loader.

From Quantized model:

$ cd ${QNN_SDK_ROOT}/examples/QNN/converter/models
$ ${QNN_SDK_ROOT}/bin/x86_64-windows-msvc/qnn-net-run.exe \
              --backend ${QNN_SDK_ROOT}/lib/x86_64-windows-msvc/QnnLpai.dll \
              --model ${QNN_SDK_ROOT}/examples/QNN/example_libs/x86_64-windows-msvc/QnnModel.dll \
              --input_list ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt \
              --config_file /path/to/config.json

From Serialized buffer:

$ cd ${QNN_SDK_ROOT}/examples/QNN/converter/models
$ ${QNN_SDK_ROOT}/bin/x86_64-windows-msvc/qnn-net-run.exe \
              --backend ${QNN_SDK_ROOT}/lib/x86_64-windows-msvc/QnnLpai.dll \
              --retrieve_context ${QNN_SDK_ROOT}/examples/QNN/converter/models/qnn_model_8bit_quantized.serialized.bin \
              --input_list ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt \
              --config_file /path/to/config.json

Important

Ensure that the QNN_SDK_ROOT environment variable is set correctly:

set QNN_SDK_ROOT=C:\path\to\qnn_sdk

Tip

Add the necessary libraries to your PATH:

set PATH=%PATH%;%QNN_SDK_ROOT%\lib\x86_64-windows-msvc

Outputs from the run will be located at the default ./output directory.

LPAI ARM Backend Type illustrates the LPAI ARM backend type execution.

QNN LPAI ARM Backend Type¶

LPAI ARM Backend Type Execution

Running the LPAI Backend on an Android device via an ARM target is supported exclusively for offline-prepared graphs. This tutorial outlines the process of preparing the graph on an x86 host and subsequently transferring the serialized context binary to the device’s LPAI Backend for execution.

To ensure compatibility with a specific target platform, it is essential to use libraries compiled for that particular target. Examples are provided below. The QNN_TARGET_ARCH variable can be utilized to specify the appropriate library for the target.

Setting Environment Variables on x86 Linux¶

# Example for Android targets (Not all targets are supported for Android)
$ export QNN_TARGET_ARCH=aarch64-android

# Example for LE Linux targets (If applicable)
$ export QNN_TARGET_ARCH=aarch64-oe-linux-gcc<your version>

# Example for QNX targets (If applicable)
$ export QNN_TARGET_ARCH=aarch64-qnx800

# For LPAI v6 HW version
$ export HW_VER=v6

Prepare config.json file¶

{
   "backend_extensions": {
   "shared_library_path": "/data/local/tmp/LPAI/libQnnLpaiNetRunExtensions.so",
   "config_file_path": "./lpaiParams.conf"
   }
}

Note

To run the LPAI backend on an Android device, the following requirements must be fulfilled:

${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpaiSkel.so has to be signed by client
qnn-net-run to be executed with root permissions

Create test directory on the device¶

$ adb shell mkdir -p /data/local/tmp/LPAI/adsp

Push the quantized model to the device¶

$ adb push ./output/qnn_model_8bit_quantized.serialized.bin /data/local/tmp/LPAI

Push the input data and input lists to the device¶

$ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_data_float /data/local/tmp/LPAI
$ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt /data/local/tmp/LPAI

Push the qnn-net-run tool¶

$ adb push ${QNN_SDK_ROOT}/bin/aarch64-android/qnn-net-run /data/local/tmp/LPAI

Set up the environment on the device¶

$ adb shell
$ cd /data/local/tmp/LPAI
$ export LD_LIBRARY_PATH=/data/local/tmp/LPAI;/data/local/tmp/LPAI/adsp
$ export ADSP_LIBRARY_PATH="/data/local/tmp/LPAI/adsp"

Execute the LPAI model using qnn-net-run¶

$ ./qnn-net-run --backend ./libQnnLpai.so --device_options device_id:0 --retrieve_context ./qnn_model_8bit_quantized.serialized.bin --input_list ./input_list_float.txt

LPAI Native DSP Backend type illustrates the LPAI Native DSP Backend type execution.

QNN LPAI Native aDSP Backend Type¶

LPAI Native DSP Backend Type Execution

Overview¶

The QNN LPAI Native aDSP Backend Type is designed to enable efficient execution of the LPAI backend by providing direct access to the DSP (Digital Signal Processor) hardware. This approach eliminates the overhead associated with data and control transfer via the IPC (Inter-Process Communication) mechanism, resulting in significantly reduced latency and improved runtime performance. Applications that can access input sources such as audio, camera, or sensors directly on the aDSP can run independently of the main operating system (Linux/Android). To use the native aDSP path, these applications must be integrated into either the audio or sensor power domain (PD).

Execution on a physical device using the native aDSP target is supported exclusively for offline-prepared graphs. This mode does not support dynamic graph compilation or runtime graph generation.

To run on a specific target platform, you must use binaries compiled for that platform. The appropriate library can be selected using the QNN_TARGET_ARCH environment variable (see more details below).

Target Platform Configuration¶

To deploy the LPAI backend on a specific target, configure the environment using the correct architecture-specific binaries. Set the QNN_TARGET_ARCH variable as shown below:

export QNN_TARGET_ARCH=<target_arch>

Supported target architectures include:

aarch64-android for Android-based ARM64 platforms
aarch64-oe-linux-gcc<your version> for LE Linux-based ARM64 platforms
aarch64-qnx800 for QNX-based ARM64 platforms
hexagon-v<version> for Qualcomm Hexagon DSP platforms

Important

Not all target architectures are supported for Android. Some platforms are lack HLOS (High-Level Operating System) support entirely. In such cases, HLOS deployment instructions do not apply. Ensure that your target platform supports the necessary runtime environment for LPAI execution. Refer to the Available Backend Libraries table for platform-specific compatibility and deployment guidance.

Setting Environment Variables on HLOS Android/Linux¶

To configure your development or deployment environment on an x86 Linux host, set the following environment variables:

# Example for Android targets (Not all targets are supported for Android)
$ export QNN_TARGET_ARCH=aarch64-android

# Example for LE Linux targets (If applicable)
$ export QNN_TARGET_ARCH=aarch64-oe-linux-gcc<your version>

# Example for QNX targets (If applicable)
$ export QNN_TARGET_ARCH=aarch64-qnx800

# For Hexagon versrion
$ export HEX_VER=81
$ export HEX_ARCH=hexagon-v${HEX_VER}

# For LPAI v6 HW version
$ export HW_VER=v6

Note

To execute the LPAI backend on an Android device, the following conditions must be met:

The following Lpai artifacts in ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned must be signed by the client:
- libQnnLpai.so
- libQnnLpaiNetRunExtensions.so
The following qnn-net-run artifacts in ${QNN_SDK_ROOT}/lib/${HEX_ARCH}/unsigned must be signed by the client:
- libQnnHexagonSkel_dspApp.so
- libQnnNetRunDirectV${HEX_VER}Skel.so
qnn-net-run must be executed with root permissions.

Prepare config.json file for direct-mode, where is_persistent_binary is required for direct-mode:

{
   "backend_extensions": {
      "shared_library_path": "/data/local/tmp/LPAI/adsp/libQnnLpaiNetRunExtensions.so",
      "config_file_path": "./lpaiParams.conf"
   },
   "context_configs": {
      // This parameter should be set for native aDSP LPAI backend
      "is_persistent_binary": true
   }
}

Create test directory on the device¶

$ adb shell mkdir -p /data/local/tmp/LPAI/adsp

Push the offline LPAI generated model to the device¶

$ adb push ./output/qnn_model_8bit_quantized.serialized.bin /data/local/tmp/LPAI

Push the Lpai libraries to the device¶

$ adb push ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpai.so /data/local/tmp/LPAI/adsp
$ adb push ${QNN_SDK_ROOT}/lib/lpai-${HW_VER}/unsigned/libQnnLpaiNetRunExtensions.so /data/local/tmp/LPAI/adsp

Push the qnn-net-run libraries to the device¶

$ adb push ${QNN_SDK_ROOT}/lib/${HEX_ARCH}/unsigned/libQnnHexagonSkel_dspApp.so /data/local/tmp/LPAI/adsp
$ adb push ${QNN_SDK_ROOT}/lib/${HEX_ARCH}/unsigned/libQnnNetRunDirectV${HEX_VER}Skel.so /data/local/tmp/LPAI/adsp

Push the input data and input lists to the device¶

$ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_data_float /data/local/tmp/LPAI
$ adb push ${QNN_SDK_ROOT}/examples/QNN/converter/models/input_list_float.txt /data/local/tmp/LPAI

Push the qnn-net-run tool and its dependent libraries¶

$ adb push ${QNN_SDK_ROOT}/bin/${QNN_TARGET_ARCH}/qnn-net-run /data/local/tmp/LPAI
$ adb push ${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnNetRunDirectV${HEX_VER}Stub.so /data/local/tmp/LPAI

Set up the environment on the device¶

$ adb shell
$ cd /data/local/tmp/LPAI
$ export LD_LIBRARY_PATH=/data/local/tmp/LPAI;/data/local/tmp/LPAI/adsp
$ export ADSP_LIBRARY_PATH="/data/local/tmp/LPAI/adsp"
$ export HW_VER=v6

Execute the LPAI model using qnn-net-run¶

$ ./qnn-net-run --backend asdp/libQnnLpai.so --direct_mode --retrieve_context ./qnn_model_8bit_quantized.serialized.bin --input_list ./input_list_float.txt --config_file <config.json>

QNN LPAI Execution¶

QNN LPAI Backend Emulation¶

QNN LPAI Emulation on Linux x86¶

QNN LPAI Emulation on Windows x86¶

QNN LPAI ARM Backend Type¶

Setting Environment Variables on x86 Linux¶

Prepare config.json file¶

Create test directory on the device¶

Push the quantized model to the device¶

Push the LPAI related libraries to the device¶

Push the input data and input lists to the device¶

Push the qnn-net-run tool¶

Set up the environment on the device¶

Execute the LPAI model using qnn-net-run¶

QNN LPAI Native aDSP Backend Type¶

Overview¶

Target Platform Configuration¶

Setting Environment Variables on HLOS Android/Linux¶

Create test directory on the device¶

Push the offline LPAI generated model to the device¶

Push the Lpai libraries to the device¶

Push the qnn-net-run libraries to the device¶

Push the input data and input lists to the device¶

Push the qnn-net-run tool and its dependent libraries¶

Set up the environment on the device¶

Execute the LPAI model using qnn-net-run¶