CNN to QNN for Windows Host¶

This guide will teach you how to convert your CNN model into an executable that can be run on a target device’s processors using Qualcomm AI Engine Direct (aka the QNN SDK).

In order to do that, you will learn how to:

Convert your Convolutional Neural Net (CNN) model to a Qualcomm Neural Net (QNN) Model.
Build that model for a specific target device operating system. (Ex. Android)
Transfer and use the model to make inferences on the desired processing unit. (Ex. GPU)

Step 1: Tutorial Setup¶

Install the QNN SDK¶

Follow the instructions in Setup to install the QNN SDK.
1. Make sure to install the optional Tensorflow dependency as this tutorial will use a Tensorflow model. (See Step 3 in Setup for more instructions).

Note

Using the same Powershell terminal for the Setup and these steps will speed up the process as some necessary setup steps only affect the terminal scope.

Check that QNN_SDK_ROOT is set to the folder just inside qairt by running echo $env:QNN_SDK_ROOT.
1. You should see the path to the folder name inside qairt (Ex. .../qairt/2.22.6.240515)
2. If QNN_SDK_ROOT is not set:
  
  Navigate to qairt/<QNN_SDK_ROOT_LOCATION>/bin
  
  Run ./envsetup.ps1 to set the environment variable. Note: These changes will only apply to the current Powershell instance.
  
  Check again by running echo $env:QNN_SDK_ROOT and run ./envsetup.ps1 to set it.
Ensure you are in the proper virtual environment for Python.
1. If you are not in a venv, see Step 2 of Setup to install / activate your environment.

Set Up The Example Tensorflow Model¶

Run python -m pip show tensorflow to verify that tensorflow is installed properly.
1. If tensorflow is not found, follow the steps in Step 3 of the Setup instructions to install Tensorflow.
Run the following 3 commands below to set the TENSORFLOW_HOME environment variable.

$tensorflowLocation = (python -m pip show tensorflow | Select-String -Pattern '^Location: ' | ForEach-Object { $_.ToString().Split(' ')[1].Trim() });
[System.Environment]::SetEnvironmentVariable('TENSORFLOW_HOME', "$tensorflowLocation\tensorflow", 'User');
$env:TENSORFLOW_HOME = [System.Environment]::GetEnvironmentVariable('TENSORFLOW_HOME', 'User');

Note

TENSORFLOW_HOME is needed because the setup_inceptionv3.py script uses TensorFlow utilities like optimize_for_inference.py, which are present in the TensorFlow installation directory.

Run $env:TENSORFLOW_HOME to verify that the environment variable was set properly.
Run python ${QNN_SDK_ROOT}/examples/Models/InceptionV3/scripts/setup_inceptionv3.py -a ~/tmpdir -d to download the example data for this tutorial.
1. A model file at: ${QNN_SDK_ROOT}/examples/Models/InceptionV3/tensorflow/inception_v3_2016_08_28_frozen.pb 2. Raw images at: ${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped

2. Converting the CNN model into a QNN model¶

Converting models into QNN format allows them to be built for specific target device operating systems and processors.

This tutorial is using a tensorflow model, so we can convert by running the qnn-tensorflow-converter. If you are using another type of model, you can look at the Tools page for a table of potential scripts to help convert them into QNN format. They will have a similar qnn-model-type-converter naming convention.

You can use QNN to convert either full precision models or quantized models.

Warning

HTP and DSP target devices MUST use quantized models.

Full Precision Model Conversion¶

Run the following script:

python ${QNN_SDK_ROOT}\bin\x86_64-windows-msvc\qnn-tensorflow-converter `
  --input_network "${QNN_SDK_ROOT}/examples/Models/InceptionV3/tensorflow/inception_v3_2016_08_28_frozen.pb" `
  --input_dim input 1,299,299,3 `
  --out_node InceptionV3/Predictions/Reshape_1 `
  --output_path "${QNN_SDK_ROOT}/examples/Models/InceptionV3/model/Inception_v3.cpp"

The flags are used to specify:

--input_network - The path to the source framework model.
--input_dim - The input name followed by the dimensions of that input.
--out_node - The name of the graph’s output Tensor Names. Multiple output names should be provided separately.
--output_path - This indicates where to put the output files.

Note

qnn-tensorflow-converter supports a wide variety of input types. Run the script with the --help flag to see all options available to you.

The above qnn-tensorflow-converter command will produce the following artifacts:

Inception_v3.cpp - This contains the sequence of API calls.
Inception_v3.bin - This has the static data associated with the model.
Inception_v3_net.json - This describes the model data in a standardized manner. (This is not needed to build the QNN model later on)

You can find the artifacts in the $env:QNN_SDK_ROOT/examples/Models/InceptionV3/model directory.

Quantized Model Conversion¶

To create a quantized model instead of a full precision model, you will need to pass in the --input_list flag to specify the input.

In a real situation, you would need to prepare your input data before running this command. For this tutorial, that data has already been prepared in ${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/raw_list.txt. (No action required)
Run the below command:

python ${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-tensorflow-converter `
  --input_network "${QNN_SDK_ROOT}/examples/Models/InceptionV3/tensorflow/inception_v3_2016_08_28_frozen.pb" `
  --input_dim input 1,299,299,3 `
  --input_list "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/raw_list.txt"
  --out_node InceptionV3/Predictions/Reshape_1 `
  --output_path "${QNN_SDK_ROOT}/examples/Models/InceptionV3/model/Inception_v3.cpp" `

The flags are used to specify:

--input_network - The path to the source framework model.
--input_dim - The input name followed by the dimensions of that input.
--input_list - The absolute path to input data. (Required for quantization)
--out_node - The name of the graph’s output Tensor Names. Multiple output names should be provided separately.
--output_path - This indicates where to put the output files.

The above qnn-tensorflow-converter command will produce the following artifacts:

Inception_v3.cpp - This contains the sequence of API calls.
Inception_v3.bin - This has the static data associated with the model.
Inception_v3_net.json - This describes the model data in a standardized manner. (This is not needed to build the QNN model later on)

You can find the artifacts in the $env:QNN_SDK_ROOT/examples/Models/InceptionV3/model directory.

3. Build the Model for a Target Device¶

Based on the target architecture for your target device, choose one of the following guides to follow: