Linux Setup

Instructions for Linux Host (Ubuntu, WSL, and other distros)

Follow these instructions to install the Qualcomm AI Engine Direct SDK (commonly referred to as the QNN SDK) and all necessary dependencies.

Note

This guide is for a host machine running Linux. Ubuntu 22.04 and Ubuntu 20.04 are the supported OS versions. If you are using a Windows machine, please follow the instructions for a Windows setup, here

The QNN SDK allows you to convert an AI model (e.g., a .pt model from PyTorch) into instructions that can be run on a target device’s various processing units (CPU, GPU, HTP, cDSP, or LPAI).

Note

To learn about acronyms which are new to you, see the glossary.

You will need the following to use the QNN SDK:

Step 1: The QNN SDK binary and its immediate dependencies.

Step 2: Install QNN SDK Dependencies

Step 3: Install model frameworks to interpret your AI model files. (e.g., PyTorch)

Step 4: Install Target Device OS-Specific Tool Chain Code

Step 5: Install Dependencies for Target Hardware Processors

Step 6 (Optional step): Additional Packages for Evaluating Model Accuracy.

Note

This guide contains many recommendations about specific versions of code to use that have been verified to work. Other versions of those dependencies may or may not work, so use them at your own risk.

Step 1: Install Qualcomm AI Engine Direct (aka the “QNN SDK”)

1. Go to the QNN SDK product page: https://www.qualcomm.com/developer/software/qualcomm-ai-engine-direct-sdk

2. Click “Get Software” to download the QNN SDK

   1. Note: The QNN SDK is ~2 GB when unzipped.

3. Unzip the downloaded SDK.

   1. The zipped file should have a name like v2.22.6.240515.

   2. When unzipped, the folder will contain a folder named qairt with a sub-folder that has a very similar name to the top-level folder like 2.22.6.240515.

Set up your environment

1. Open a terminal.

2. Navigate to qairt/<SDK_VERSION> inside the unzipped QNN SDK.

   1. Replace <SDK_VERSION> with the name of the file directly under qairt - it should look something like 2.22.6.240515.

3. Setup the environment using the below commands:

   Run cd bin
   Run source ./envsetup.sh
   

   This will automatically set the environment variable QAIRT_SDK_ROOT which points to this SDK.

   Warning

   The envsetup.sh script only updates environment variables for the current terminal session. If you need to set QAIRT_SDK_ROOT again, then just re-run this script. Alternatively, consider persisting the environment variables set by the script to your shell environment.

   Warning

   The QNN_SDK_ROOT and SNPE_ROOT environment variables are currently set to QAIRT_SDK_ROOT for backwards compatibility. They will be deprecated in favor of QAIRT_SDK_ROOT in a future release. Please update any relevant workflows to use QAIRT_SDK_ROOT instead of QNN_SDK_ROOT or SNPE_ROOT.

4. Run sudo ${QAIRT_SDK_ROOT}/bin/check-linux-dependency.sh.

   This will install the Linux build tools.

   When you have installed all the necessary dependencies, the script will say “Done!!”.

   Note

   As the script is running, you will have to confirm additional downloads by pressing “Enter”. The script may take several minutes to complete.

5. Run ${QAIRT_SDK_ROOT}/bin/envcheck -c.

   This will verify that you have installed the required toolchain successfully.

Step 2: Install QNN SDK dependencies

1. Install python 3.10 by running the following commands:

   sudo apt-get update && sudo apt-get install python3.10 python3-distutils libpython3.10
   

2. Verify the installation worked by running:

   python3 --version
   

   Warning

   Ensure you have Python 3.10. The QNN SDK was verified with Python version 3.10.4.

3. Run cd ${QAIRT_SDK_ROOT}.

4. Install python3.10-venv if you don’t have it installed already by running:

   sudo apt install python3.10-venv
   

5. Run the following command to create and activate a new virtual environment (you may rename MY_ENV_NAME to any name you prefer):

   python3 -m venv MY_ENV_NAME --without-pip
   source MY_ENV_NAME/bin/activate
   python3 -m ensurepip --upgrade
   

   Warning

   We have to use the flag --without-pip on Debian/Ubuntu systems to avoid a crash since venv will call ensurepip at the system level (which is disabled on Debian/Ubuntu). Once we activate our venv, we can then safely call ensurepip to create a local pip for installing packages.

6. Run which pip3 to verify that the virtual environment has a local version of pip3.

   You should see a path that is inside your virtual environment (Ex. /MY_ENV_NAME/bin/pip3)

7. Update all dependencies by running the following command:

   python "${QAIRT_SDK_ROOT}/bin/check-python-dependency"
   

   Warning

   If you run into an error with a specific version of a package, run pip install PACKAGE_NAME to get a more up-to-date version of the package then re-run the above script.

   Note

   The above installs all required. To install optional packages pass -o|--with-optional

Step 3: Install Model Frameworks

QNN supports the following model frameworks.

Install the ones that are relevant for the AI model files you want to use.

Warning

You do not need to install all packages here.

Note

You can install a package by running pip3 install package==version. e.g., pip3 install torch==1.13.1

Package	Version	Description
torch (SNPE/QNN)	1.13.1	PyTorch is used for building and training deep learning models with a focus on flexibility and speed. Used with .pt files. Install by downloading the proper binary from PyTorch previous versions if pip install does not work.
torch (QAIRT)	2.4.0	PyTorch is used for building and training deep learning models with a focus on flexibility and speed. Used with .pt files. Install by downloading the proper binary from PyTorch previous versions if pip install does not work.
torchvision	0.14.1 (SNPE/QNN) / 0.19.0 (QAIRT)	Torchvision is used for computer vision tasks with PyTorch, providing datasets, model architectures, and image transforms.
tensorflow	2.10.1	TensorFlow is used for building and training machine learning models, particularly deep learning models. Used with .pb files. .. note:: The envcheck script will incorrectly say this file is not installed on Ubuntu.
tflite	2.18.0	TFLite is used for running TensorFlow models on mobile and edge devices with optimized performance. Used with .tflite files.
onnx	1.17.0	ONNX stands for Open Neural Network Exchange. It is used for defining and exchanging deep learning models between different frameworks. Used with .onnx files.
onnxruntime	for ubuntu22.04 - 1.22.0, for ubuntu20.04 - 1.19.2	ONNX stands for Open Neural Network Exchange. It is used for running ONNX models with high performance across various hardware platforms. Used with .onnx files.
onnxsim	0.4.36	Onnxsim is used for simplifying ONNX models to reduce complexity and improve inference efficiency. Used with .onnx files.

Note

You can verify your installation by calling ${QAIRT_SDK_ROOT}/bin/envcheck -a which will check to see whether these dependencies are installed. These are optional, so just verify that the dependencies you intended to install are actually installed.

Step 4: Install Target Device OS-Specific Toolchain Code

Depending on the target device’s operating system, there may be additional installation requirements.

Working with an Android Target Device

For working with Android devices, you will need to install a corresponding Android NDK (Native Developer Kit). You can install the recommended version (Android NDK version r26c) by following these steps:

1. Download the Android NDK: Android NDK r26c

2. Unzip the file.

   Warning

   If your environment is in WSL, the Android NDK must be unzipped under $HOME with the WSL unzip command.

3. Open a terminal.

4. Replace <path-to-your-android-ndk-root-folder> with the path to the unzipped android-ndk-r26c folder then run:

   echo 'export ANDROID_NDK_ROOT="<path-to-your-android-ndk-root-folder>"' >> ~/.bashrc
   

5. Add the location of the unzipped file to your PATH by running:

   echo 'export PATH="${ANDROID_NDK_ROOT}:${PATH}"' >> ~/.bashrc
   source ~/.bashrc
   

6. Verify your environment variables by running:

   ${QAIRT_SDK_ROOT}/bin/envcheck -n
   

Working with a Linux Target Device

For Linux target devices, you will likely need to use clang++14 to build models for them using the QNN SDK. Later versions may work but have not been verified.

You can verify if you have clang++14 by running:

${QAIRT_SDK_ROOT}/bin/envcheck -c

If not, please install clang++14 from LLVM: Clang++14

Step 5: Install Dependencies for Target Hardware Processors

Some of the target processors (CPU, GPU, HTP, cDSP, or HTA) require additional dependencies to build models.

CPU (Central Processing Unit)

The x86_64 targets are built using clang-14. If working with this kind of target, please install clang++14 from LLVM: Clang++14

The ARM CPU targets are built using the Android NDK (see working with an Android target).

GPU (Graphical Processing Unit)

The GPU backend kernels are written based on OpenCL. The GPU operations must be implemented based on OpenCL headers with a minimum version of OpenCL 1.2.

HTP and DSP

Compiling for both HTP and DSP hardware requires the use of the Qualcomm Hexagon SDK and Hexagon SDK Tools which you can install by following these steps.

1. If you do not already have Qualcomm Package Manager 3 (QPM3) installed, install it by following the steps below:

   1. Sign into Qualcomm Package Manager 3

      Warning

      You may have to reclick the link after logging in to have it load properly.

   2. Download the Linux version of Qualcomm Package Manager 3 (QPM3).

   3. Open the installer and click “Install”.

   4. Once installed, open “Qualcomm Package Manager”.

      1. For Ubuntu, click “Activities” and search for “Qualcomm” then click the blue application.

2. Start QPM3 by running the QPM3 executable.

3. Create a Qualcomm account or log in to your existing one when prompted by QPM3.

4. Once logged in to QPM3, click on “Tools.”

5. Search for “Hexagon SDK” and download the proper version for your target device.

   Note

   You can find the “Hexagon Architecture” of your chip by looking up your chip type in the supported Snapdragon devices” table.

   Backend	Hexagon Architecture	Hexagon SDK Version	Additional Notes
   HTP	V75	5.4.0
   HTP	V73	5.4.0
   HTP	V69	4.3.0
   HTP	V68	4.2.0 (5.4.0 for Automotive platform)	Not pre-packaged with Hexagon 4.2.0 SDK Tools.
   DSP	V66	4.1.0	Not pre-packaged with Hexagon SDK Tools.
   DSP	V65	3.5.2	Must be downloaded manually Hexagon DSP SDK Tools instead of from QPM3. Not pre-packaged with Hexagon SDK Tools.

   Note

   Hexagon SDK Tools version 8.4.09/8.4.06/8.3.07 is not currently pre-packaged into Hexagon SDK version 4.2.0/4.1.0/3.5.2 respectively. It needs to be downloaded separately and placed at the location $HEXAGON_SDK_ROOT/tools/HEXAGON_Tools/.

6. Search for “Hexagon Tools” and download the proper version for your target device.

   Based on your device’s architecture version, use the following table to determine which versions to download.

   Note

   You can find the “Hexagon Architecture” of your chip by looking it up in the supported Snapdragon devices” table.

   Backend	Hexagon Architecture	Hexagon Tools Version	Additional Notes
   HTP	V75	8.7.03
   HTP	V73	8.6.02
   HTP	V69	8.5.03
   HTP	V68	8.4.09	Not pre-packaged with Hexagon 4.2.0 SDK Tools.
   DSP	V66	8.4.06	Not pre-packaged.
   DSP	V65	8.3.07	Not pre-packaged.

   Note

   Hexagon SDK Tools version 8.4.09/8.4.06/8.3.07 is not currently pre-packaged into Hexagon SDK version 4.2.0/4.1.0/3.5.2 respectively. It needs to be downloaded separately and placed at the location $HEXAGON_SDK_ROOT/tools/HEXAGON_Tools/.

7. Install clang++ so you can compile code for HTP/DSP hardware.

8. Follow the instructions at $HEXAGON_SDK_ROOT/docs/readme.html (where $HEXAGON_SDK_ROOT is the location of the Hexagon SDK installation).

LPAI (Low Power AI)

The LPAI backend is designed for offline model preparation only. In order to use QNN with LPAI, you must download the LPAI code using Qualcomm Package Manager.

1. If you do not already have Qualcomm Package Manager 3 (QPM3) installed, install it by following the steps below:

   1. Sign into Qualcomm Package Manager 3

      Warning

      You may have to reclick the link after logging in to have it load properly.

   2. Download the Linux version of Qualcomm Package Manager 3 (QPM3).

2. Start QPM3 by running the QPM3 executable.

3. Create a Qualcomm account or log in to your existing one when prompted.

4. Once logged in to the Qualcomm Package Manager, open the “Tools” category.

5. Search for “LPAI” and click the “Extract” button.

   1. This will attempt to install the LPAI code, but will likely fail because it is missing the Hexagon SDK dependency.

   2. Use the error message to determine which version of the Hexagon SDK you need to install first.

      1. e.g., For v2.4.0 of the LPAI code, it requires version 5.x of the Hexagon SDK.

   3. Install the necessary version of the Hexagon SDK.

      Warning

      If an installation fails because QPM3 cannot access a folder, move the installation path to one which does not require sudo access.

   4. Then re-install the LPAI code.

Step 6 (Optional): Additional Packages

For Generative AI and Model Accuracy Evaluation use-cases, you may also need to install some of the following dependencies.

You can install each relevant package by running:

pip3 install dependency==version
# e.g., pip3 install pycocotools==2.0.7

Package	Version	Description
pycocotools	2.0.7	Use for working with COCO datasets for object detection, segmentation, and keypoint tasks.
transformers	4.44.0	Use for leveraging pre-trained models for various NLP tasks like text classification, translation, and more.
tokenizers	0.19.1	Use for efficient tokenization of text data, particularly when dealing with large datasets.
sacrebleu	2.3.1	Use for evaluating the quality of machine translation models.
scikit-learn	1.3.0	Use for implementing machine learning algorithms and models for various predictive tasks.
OpenNMT-py	2.3.0	Use for building and training neural machine translation models.
sentencepiece	0.2.0	Use for unsupervised text tokenization and preprocessing for language models.
onnxruntime-genai	0.8.2	Use for running LLMs with ONNX Runtime.
gguf	0.9.1	Use for reading and writing binary files in the GGUF (GGML Universal File) format.
onnx-graphsurgeon	0.5.8	Used to modify and create ONNX Models
onnxscript	0.3.2	Enables developers to author ONNX functions and models in Python.
islpy	2025.2.5	Used for manipulating sets and relations of integer points bounded by linear constraints.

Next Steps

Now that you’ve finished setting up QNN SDK and its dependencies, you can use the QNN SDK. Follow the CNN to QNN tutorial to learn how to transform your AI models, build them for your target device, and use them to generate inferences on the information processing cores you choose. Use the QNN SDK to allow your AI models to execute on your specific target device’s cores.