Windows Setup¶

Instructions for Windows Host (10, 11, and Snapdragon)¶

Follow these instructions to install the Qualcomm AI Engine Direct SDK (commonly referred to as the QNN SDK) and all necessary dependencies.

Note

This guide is for a host machine running Windows 10, 11, or Windows on Snapdragon. If you are using a Linux machine or are using a WSL environment, follow the instructions here.

The QNN SDK allows you to convert an AI model (ex. a .pt model from PyTorch) into instructions that can be run on a target device’s various processing units (CPU, GPU, HTP, cDSP, or LPAI).

Note

To learn about acronyms which are new to you, see the glossary.

You will need the following to use the QNN SDK:

Step 1: The QNN SDK binary and its immediate dependencies.

Step 2: Install QNN SDK Dependencies

Step 3: Install model frameworks to interpret your AI model files. (e.g., PyTorch)

Step 4: Install Target Device OS-Specific Tool Chain Code

Step 5: Install Dependencies for Target Hardware Processors

Step 6 (Optional step): Additional Packages for Evaluating Model Accuracy.

Note

This guide contains many recommendations about specific versions of code to use that have been verified to work. Other versions of those dependencies may or may not work, so use them at your own risk.

Step 1: Install Qualcomm AI Engine Direct (aka the “QNN SDK”)¶

Go to the QNN SDK product page: https://www.qualcomm.com/developer/software/qualcomm-ai-engine-direct-sdk
Click “Get Software” to download the QNN SDK
1. Note: The QNN SDK is ~2 Gb when unzipped.
Unzip the downloaded SDK.
1. The zipped file will have the version name, for example: v2.22.6.240515.
2. When unzipped, the folder should be named qairt with a sub-folder that looks like 2.22.6.240515.

Set up your environment¶

Open an admin Powershell terminal by:
1. Pressing the “Windows” button to search for applications
2. Searching for “Powershell”
3. Right clicking “Windows Powershell” and selecting “Run as Administrator”
4. Saying “Yes” when prompted for permission to run this application as an administrator.
Navigate to qairt/<SDK_VERSION> inside the unzipped QNN SDK.
1. Replace <SDK_VERSION> with the name of the file directly under qairt - it should look something like 2.22.6.240515.

Setup the environment using the below commands:

cd bin
Unblock-File ./envsetup.ps1
./envsetup.ps1

This will automatically set the environment variable QAIRT_SDK_ROOT.

Warning

The envsetup.ps1 script only updates environment variables for this Powershell instance. If you need to set QAIRT_SDK_ROOT again, then just re-run this script. Alternatively, consider persisting the environment variables set by the script to your shell environment.
Run Unblock-File "${QAIRT_SDK_ROOT}/bin/check-windows-dependency.ps1".
Run & "${QAIRT_SDK_ROOT}/bin/check-windows-dependency.ps1"

This will install the core Windows build tools.
1. Each time you run the script it will install more dependencies. You will likely have to re-run the command after it helps you install Visual Studio and Visual Studio Build Tools.
2. When you have installed all the necessary dependencies, the script will say “All Done”.
Run Unblock-File "${QAIRT_SDK_ROOT}/bin/envcheck.ps1".

Run & "${QAIRT_SDK_ROOT}/bin/envcheck.ps1" -m.

This will verify that you have installed the Microsoft Visual Studio C++ (MSVC) toolchain successfully.

Note

It’s normal to see that only one of “Visual C++(x86)” and “Visual C++(arm64)” is installed. The installed version should match the hardware of your host machine.

You should see an output like this:

Checking MSVC Toolchain
--------------------------------------------------------------
WARNING: The version of VS BuildTools 14.40.33807 found has not been validated. Recommended to use known stable VS
BuildTools version 14.34
WARNING: The version of Visual C++(x64) 19.40.33812 found has not been validated. Recommended to use known stable
Visual C++(x64) version 19.34
WARNING: The version of CMake 3.28.3 found has not been validated. Recommended to use known stable CMake version 3.21
WARNING: The version of clang-cl 17.0.3 found has not been validated. Recommended to use known stable clang-cl version
15.0.1
Name              Version
----              -------
Visual Studio     17.10.35027.167
VS Build Tools    14.40.33807
Visual C++(x86)   19.40.33812
Visual C++(arm64) Not Installed
Windows SDK       10.0.22621
CMake             3.28.3
clang-cl          17.0.3
--------------------------------------------------------------

Step 2: Install QNN SDK dependencies¶

Install Python 3.10.4 - python.org/downloads/release/python-3104/

Warning

Ensure you are downloading the proper version. New releases beyond Python 3.10.4 may not work as expected with the QNN SDK.
Verify the installation worked by running:
```
py --list
```
You should see -V:3.10 as one of the options.
Open the folder at QAIRT_SDK_ROOT in Visual Studio.
Open a new Powershell terminal in Visual Studio by going to the top-menu “Terminal” and clicking “New Terminal”.

Warning

Ensure the new terminal is a Powershell instance.
From the QAIRT_SDK_ROOT folder, run the following command to create and activate a new virtual environment:
```
py -3.10 -m venv "venv"
& "venv\Scripts\Activate.ps1"
```
Update all dependencies by running the following commands in Powershell:
```
python -m pip install --upgrade pip
python "$env:QAIRT_SDK_ROOT\bin\check-python-dependency"
```
Note

The above installs all required packages. To install optional packages pass o|--with-optional

Step 3: Install Model Frameworks¶

QNN supports the following model frameworks.

Install the ones that are relevant for the AI model files you want to use.

Warning

You do not need to install all packages here.

Note

You can install a package by running pip install package==version. e.g., pip install torch==1.13.1

Package	Version	Description
torch (SNPE/QNN)	1.13.1	PyTorch is used for building and training deep learning models with a focus on flexibility and speed. Used with .pt files. Install by downloading the proper binary from PyTorch previous versions if pip install does not work.
torch (QAIRT)	2.4.0	PyTorch is used for building and training deep learning models with a focus on flexibility and speed. Used with .pt files. Install by downloading the proper binary from PyTorch previous versions if pip install does not work.
torchvision	0.14.1 (SNPE/QNN) / 0.19.0 (QAIRT)	Torchvision is used for computer vision tasks with PyTorch, providing datasets, model architectures, and image transforms.
tensorflow	2.10.1	Use for building and training machine learning models, particularly deep learning models. Used with .pb files.
tflite	2.18.0	Use for running TensorFlow models on mobile and edge devices with optimized performance. Used with .tflite files.
onnx	1.17.0	ONNX stands for Open Neural Network Exchange. It is used for defining and exchanging deep learning models between different frameworks. Used with .onnx files.
onnxruntime	for ubuntu22.04 - 1.22.0, for ubuntu20.04 - 1.19.2	ONNX Runtime is used for running ONNX models with high performance across various hardware platforms. Used with .onnx files.
onnxsim	0.4.36	It is used for simplifying ONNX models to reduce complexity and improve inference efficiency. Used with .onnx files.

Note

You can verify your installation by calling & "${QAIRT_SDK_ROOT}/bin/envcheck.ps1" -a which will check to see whether these dependencies are installed. These are optional, so just verify that the dependencies you intended to install are actually installed.

Step 4: Install Target Device OS-Specific Toolchain Code¶

Depending on the target device’s operating system there may be additional installation requirements.

Working with an Android Target Device¶

For working with Android devices, you will need to install a corresponding Android NDK (Native Developer Kit). You can install the recommended version (Android NDK version r26c) by following these steps:

Download the Android NDK: https://dl.google.com/android/repository/android-ndk-r26c-linux.zip
Unzip the file.

Warning

If your environment is in WSL, the Android NDK must be unzipped under $HOME with the WSL unzip command.
Copy the path to your unzipped Android NDK root (the folder should be named android-ndk-r26c).
Open a Powershell terminal.

Add the location of the unzipped file to your PATH by running:

Warning

Ensure you update the <path-to-your-android-ndk-root-folder> below to be the path to the unzipped android-ndk-r26c folder before running the below commands! (Ex. /home/usr/Documents/android-ndk-r26c)

$env:ANDROID_NDK_ROOT = "<path-to-your-android-ndk-root-folder>"
[System.Environment]::SetEnvironmentVariable("ANDROID_NDK_ROOT", "<path-to-your-android-ndk-root-folder>", "User")

$env:PATH = "$env:ANDROID_NDK_ROOT;$env:PATH"
[System.Environment]::SetEnvironmentVariable("PATH", "$env:ANDROID_NDK_ROOT;$env:PATH", "User")

Run Unblock-File "${QAIRT_SDK_ROOT}/bin/envcheck.ps1".
Run & "${QAIRT_SDK_ROOT}/bin/envcheck.ps1" -n.

This will verify your environment variables are properly configured.

Working with a Linux Target Device¶

For Linux target devices you will likely need to use clang++14 to build models for them using the QNN SDK. Later versions may work but have not been verified.

You can update your Visual Studio to use a specific version of clang++ by following these instructions: https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170.

Step 5: Install Dependencies for Target Hardware Processors¶

Some of the target processors (CPU, GPU, HTP, cDSP, or HTA) require additional dependencies to build models.

CPU (Central Processing Unit)¶

The x86_64 targets are built using clang-14. You can update your Visual Studio to use a specific version of clang++ by following these instructions: https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170.

The ARM CPU targets are built using the Android NDK (see Working With an Android Target Device).

GPU (Graphical Processing Unit)¶

The GPU backend kernels are written based on OpenCL. The GPU operations must be implemented based on OpenCL headers with minimum version OpenCL 1.2.

HTP and DSP¶

Compiling for both HTP and DSP hardware requires the use of the Qualcomm Hexagon SDK and Hexagon SDK Tools which you can install by following these steps.

If you do not already have Qualcomm Package Manager 3 (QPM3) installed, install it by following the steps below:
1. Sign into https://qpm.qualcomm.com/#/main/tools/details/QPM3.
  
  Warning
  
  You may have to reclick the link after logging in to have the QPM3 product page load properly.
2. Download the Windows version of Qualcomm Package Manager 3 (QPM3).
Start QPM3 by running the QPM3 executable. It should have a similar name to QPM.3.0.92.3.Windows-AnyCPU.exe.
Create a Qualcomm account or log in to your existing one when prompted.
Look up which “Hexagon Architecture” your chip supports in the supported Snapdragon devices table. (It should look like “V00”)

Use the below table to determine the proper Hexagon SDK version for your Hexagon architecture:

Note

You can find the “Hexagon Architecture” of your chip by looking it up in the supported Snapdragon devices table - it should look something like “V00”.

Backend	Hexagon Architecture	Hexagon SDK Version	Additional Notes
HTP	V75	5.4.0	Pre-packaged.
HTP	V73	5.4.0	Pre-packaged.
HTP	V69	4.3.0	Pre-packaged.
HTP	V68	4.2.0	Not pre-packaged with Hexagon SDK Tools.
DSP	V66	4.1.0	Not pre-packaged with Hexagon SDK Tools.
DSP	V65	3.5.2	Must be downloaded manually here instead of from QPM3. Not pre-packaged with Hexagon SDK Tools.

Warning

Hexagon SDK Tools version 8.4.09/8.4.06/8.3.07 is not currently pre-packaged into Hexagon SDK version 4.2.0/4.1.0/3.5.2 respectively. It needs to be downloaded separately and placed at the location $HEXAGON_SDK_ROOT/tools/HEXAGON_Tools/.

In the QPM3, search for “Hexagon SDK”.
Download the specified version in the above table.

If your version of Use the table below to determine which version of the Hexagon Tools software corresponds to your Hexagon Architecture from before. (You can look it up again in the supported Snapdragon devices table)

Backend	Hexagon Architecture	Hexagon Tools Version	Additional Notes
HTP	V75	8.7.03	Pre-packaged.
HTP	V73	8.6.02	Pre-packaged.
HTP	V69	8.5.03	Pre-packaged.
HTP	V68	8.4.09	Not pre-packaged.
DSP	V66	8.4.06	Not pre-packaged.
DSP	V65	8.3.07	Not pre-packaged.

Warning

Hexagon SDK Tools version 8.4.09/8.4.06/8.3.07 is not currently pre-packaged into Hexagon SDK version 4.2.0/4.1.0/3.5.2 respectively. It needs to be downloaded separately and placed at the location $HEXAGON_SDK_ROOT/tools/HEXAGON_Tools/.

In QPM3, search for “Hexagon Tools”.
Install the proper version from the above table for your Hexagon Architecture.
Install clang++ so you can compile code for HTP / DSP hardware.
Follow the instructions at $HEXAGON_SDK_ROOT/docs/readme.html (where $HEXAGON_SDK_ROOT is the location of the Hexagon SDK installation).

LPAI (Low Power AI)¶

The LPAI backend is designed for offline model preparation only. In order to use QNN with LPAI, you must download the LPAI code using Qualcomm Package Manager.

If you do not already have Qualcomm Package Manager 3 (QPM3) installed, install it by following the steps below:
1. Sign into https://qpm.qualcomm.com/#/main/tools/details/QPM3.
  
  Warning
  
  You may have to reclick the link after logging in to have it load properly.
2. Download the Windows version of Qualcomm Package Manager 3 (QPM3).
Start QPM3 by running the QPM3 executable. It should have a similar name to QPM.3.0.92.3.Windows-AnyCPU.exe.
Create a Qualcomm account or log in to your existing one when prompted.
Once logged in to the Qualcomm Package Manager, search for “LPAI” and click the “Extract” button.
1. This will attempt to install the LPAI code, but will likely fail because it is missing the Hexagon SDK dependency.
2. Use the error message to determine which version of the Hexagon SDK you need to install first.
  1. Ex. For v2.4.0 of the LPAI code, it requires version 5.x of the Hexagon SDK.
3. Install the necessary version of the Hexagon SDK.
4. Then re-install the LPAI code.

Step 5 (Optional): Additional Packages for Evaluating Model Accuracy¶

If you want to evaluate model accuracy in real time, you may also need to install some of the following dependencies.

You can install each relevant package by running:

pip3 install dependency==version

Ex. pip3 install pycocotools==2.0.7

Package	Version	Description
pycocotools	2.0.7	Use for working with COCO datasets for object detection, segmentation, and keypoint tasks.
transformers	4.31.0	Use for leveraging pre-trained models for various NLP tasks like text classification, translation, and more.
tokenizers	0.19.1	Use for efficient tokenization of text data, particularly when dealing with large datasets.
sacrebleu	2.3.1	Use for evaluating the quality of machine translation models.
scikit-learn	1.3.0	Use for implementing machine learning algorithms and models for various predictive tasks.
OpenNMT-py	2.3.0	Use for building and training neural machine translation models.
sentencepiece	0.1.98	Use for unsupervised text tokenization and preprocessing for language models.

Next Steps¶

Now that you’ve finished with setting up QNN SDK and its dependencies, you can use the QNN SDK. Follow the CNN to QNN tutorial to learn how to transform your AI models, build them for your target device, and use them to generate inferences on the information processing cores you choose. Use the QNN SDK to allow your AI models to execute on your specific target device’s cores.