SNPE Tutorial for Linux Target Device from Linux Host¶

Note

This is the second section of this tutorial. Please complete it if you haven’t already done so.

Note

Please use the same terminal on your host device as you did in the previous section, as it contains environment variables we use throughout these steps.

Part 6: Transfer Files to Your Target Device¶

Now that you have a .dlc version of your model, the next step is to transfer the built model and all necessary files to the target processor, then to run inferences on it.

Install all necessary dependencies from Setup.
Follow the below SSH setup instructions.
Follow the instructions for each specific processor you want to run your model on.

Warning

For cases where the “host machine” and “target device” are the same (ex. you want to build and run model inferences on your Snapdragon for Windows device), you can skip the SSH instructions and instead adapt the steps to handle the files locally.

Sub-Step 1: If you haven’t already, ensure that you follow the processor-specific Setup instructions.¶

Sub-Step 2: Set up SSH on the target device.¶

Ensure that both the host device and the target device are on the same network for this setup.
1. Otherwise, OpenSSH requires port-forwarding to connect.
On your target device, install OpenSSH.
1. Open a terminal.
2. Run the following command to install OpenSSH Server:
```
sudo apt update
sudo apt install openssh-server
```
3. Type y when prompted to confirm.
Once installed, start the ssh server on your target device by running:
```
sudo systemctl enable ssh
sudo systemctl start  ssh
```
You can verify that the ssh server is live by running:
```
sudo systemctl status ssh
```
Note

You can turn off the OpenSSH Server service by running sudo systemctl stop ssh on your target device.
On your target device, get its IP address by running:
```
hostname -I
```
On your host machine, set a console variable for your target device’s ipv4 address from above (replacing 127.0.0.1 below):
```
export TARGET_IP="127.0.0.1"
```
Set the username you want to use to sign into your target device (you can find it by looking at the path to a user folder like Documents):
```
export TARGET_USER="yourusername"
```

Sub-Step 3: Transferring Relevant Files¶

There are several files we need to transfer over:

Our model file (ex. inception_v3_model.dlc) - The built .dlc file containing our model.
Input data (ex. notice_sign.raw) - Each file here will be used with snpe-net-run to do inferences using our model. The paths to these files will be specified by the input_list.txt.
input_list.txt - A list of paths to input data above, one path per line.
libSNPE.so - This contains the primary backend logic to interpret the .dlc file on your target device.
snpe-net-run - This example application pulls together your .dlc file, input data, and the SNPE backend to run inferences using your model.
- For practical applications, you will need to implement your own application using the SNPE API, as snpe-net-run is just for testing purposes (it is relatively slow, and not tailored to your use case). See this tutorial for more details on how to build an application that uses your model on the target device.
Additional runtimes based on your use case. (Ex. libSnpeHtpV68Stub.so):
- (HTP Only) The cached model file (ex. inception_v3_model_cached.dlc) - This is generated at the end of Part 5 (from the previous page) via snpe-dlc-graph-prepare in order to speed up inferences on HTP devices.

These files work together to allow your model to run on the target device (producing output data for each input file).

Steps to Transfer Files¶

Warning

Throughout these steps, we will be switching between the host machine (where you have SNPE installed) and the target device (where your model will be run). Pay attention to the bolded directions indicating which terminal to do commands in.

Decide what folder you want to use for your destination folder:
- For Android or OE Linux, we recommend you use /data/local/tmp/snpeexample/.
- For other forms of Linux, we recommend you use /tmp/snpeexample/.
Note

/data/local/tmp can be written to without any permissions, but is temporary. Being temporary is fine for our example, but may not be fine outside of testing.
On the target device, open a terminal or connect over ssh.
Set a variable for your DESTINATION on the target device by running:
```
export DESTINATION="/tmp/snpeexample/"
```
Make the destination folder(s) on the target device for transferred files by running the following:
```
mkdir -p $DESTINATION
```
On the host machine, set an environment variable for the destination folder you chose earlier:
```
export DESTINATION="/tmp/snpeexample/"
```
Warning

Ensure that the directory you created on your target device matches the DESTINATION you set on your host machine!

Run the following on your target device to see which architecture, OS, and gcc version you have:

uname -m
cat /etc/os-release
gcc --version

You should see an output like:

x86_64
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

Based on your target device’s architecture, OS, and gcc version, choose the proper folder:

Operating System	Architecture	GCC Version	Folder Name
Linux (64-bit)	x86_64		x86_64-linux-clang
Android (64-bit devices)	arm64		aarch64-android
QNX (Qualcomm)	arm64		aarch64-qnx
OpenEmbedded Linux	arm64	11.2	aarch64-oe-linux-gcc11.2
OpenEmbedded Linux	arm64	9.3	aarch64-oe-linux-gcc9.3
OpenEmbedded Linux	arm64	8.2	aarch64-oe-linux-gcc8.2
Ubuntu Linux	arm64	9.4	aarch64-ubuntu-gcc9.4

On your host machine, run the following command with the corresponding folder from above to set your TARGET_DEVICE_ARCH, for example:
```
export TARGET_DEVICE_ARCH="x86_64-linux-clang"
```
Note

This is similar to what we did earlier when setting HOST_MACHINE_ARCH for our host machine’s details. TARGET_DEVICE_ARCH helps ensure we’re moving executables and libraries that are built to work with the target device’s architecture / OS / tool stack.
Use scp to transfer the example built model from your host machine to your target device. The rest of these scp commands will be run on your host machine:
```
scp "${SNPE_ROOT}/examples/Models/InceptionV3/data/inception_v3_model.dlc"  "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
```
Note

You will need to sign in when using SSH for each scp request.

Transfer the input data and script from the SNPE examples folder into ~\snpe_test_package on the target device using scp in a similar way:

scp -r "${SNPE_ROOT}/examples/Models/InceptionV3/data/cropped"  "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
scp "${SNPE_ROOT}/examples/Models/InceptionV3/data/imagenet_slim_labels.txt"  "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
scp "${SNPE_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py"  "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"

Transfer the input list with file paths for all files that should be inferenced for our test:
```
scp "${SNPE_ROOT}/examples/Models/InceptionV3/data/target_raw_list.txt"  "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
```
Note

The target_raw_list.txt is generated for our example model via the initialization script $SNPE_ROOT/examples/Models/InceptionV3/.

Transfer the primary runtime libSNPE.so:

scp "$SNPE_ROOT/lib/${TARGET_DEVICE_ARCH}/libSNPE.so" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"

Transfer snpe-net-run to the target device:

scp "$SNPE_ROOT/bin/$TARGET_DEVICE_ARCH/snpe-net-run" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"

Transfer the example interpreter script which is used to turn the direct outputs of snpe-net-run into an easy-to-read terminal output. For your applications, you may want to write your own interpreter:
```
scp "${SNPE_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
```
Transfer any additional dependencies based on your use case. It’s worth looking at the .so files within these folders to see if they are relevant for your application. They are not needed for the Inception_v3 model:
- See all libs which are for SNPE or shared between QNN and SNPE (excludes QNN specific files):
```
ls ${SNPE_ROOT}/lib/${TARGET_DEVICE_ARCH}/ | grep -v Qnn
```
- If they’re relevant, write an scp command to transfer the value over to "${TARGET_USER}@${TARGET_IP}:${DESTINATION}":
```
scp "$SNPE_ROOT/lib/${TARGET_DEVICE_ARCH}/<RELEVANT_FILE>.so" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
```

Additional files to transfer for DSP / HTP / AIP¶

Warning

This is only required if you plan to use a DSP, HTP, or AIP processor. Otherwise skip to Part 7 below.

Determine your target device’s SnapDragon architecture by looking your chipset up in the chipset table and finding the “DSP Hexagon Arch”; for example, “SD 8 Gen 3 (SM8650)” | V66.
Update the “X” values below and run the commands to set HEXAGON_ARCH to match the version number found in the table above.
- Only the 2 digits at the end should update, and they should have the same version. Ex. For “V68”, the proper value would be hexagon-v68:
```
export HEXAGON_VERSION="XX"
export HEXAGON_ARCH="hexagon-v${HEXAGON_VERSION}"
```

(DSP Only) Use scp to transfer DSP specific runtimes as well as other necessary executables from your host machine to ~\snpe_test_package on the target Windows device:

scp "$SNPE_ROOT/lib/${HEXAGON_ARCH}/unsigned/libSnpeDspV${HEXAGON_VERSION}Skel.so" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"
scp "$SNPE_ROOT/lib/${TARGET_DEVICE_ARCH}/libSnpeDspV${HEXAGON_VERSION}Stub.so" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"

(HTP Only) If you are planning on using an HTP backend, copy over the HtpPrepare library:

scp "$SNPE_ROOT/lib/${TARGET_DEVICE_ARCH}/libSnpeHtpPrepare.so" "${TARGET_USER}@${TARGET_IP}:${DESTINATION}"

See if any of the other files within the ${HEXAGON_ARCH} folder are relevant (this command ignores files which are QNN specific):
```
ls ${SNPE_ROOT}/lib/${HEXAGON_ARCH}/unsigned/ | grep -v Qnn
```
If there are any relevant files, write an scp command similar to above to transfer the data over to "${TARGET_USER}@${TARGET_IP}:${DESTINATION}".

Part 7: Executing Your Model With `snpe-net-run`¶

At this point, we have moved over all the necessary files to use our model to execute inferences on the target device and verify the outcome.

Setting Environment Variables¶

Open a terminal in your target device:

Note

You can alternatively connect to your target device using ssh by running: "${TARGET_USER}@${TARGET_IP}"
Update the PATH to point to where the exectuable (bin) files are located:
```
export PATH=$PATH:${DESTINATION}
```
Update the LD_LIBRARY_PATH to point to where the libraries (lib) are located:
```
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${DESTINATION}
```
(DSP Only) Set the ADSP_LIBRARY_PATH to point to where libraries (lib) needed by the DSP are located by running:
```
export ADSP_LIBRARY_PATH="${DESTINATION};/system/lib/rfsa/adsp;/system/vendor/lib/rfsa/adsp;/dsp"
```
- ADSP_LIBRARY_PATH indicates which folders should be loaded into the DSP on the target device, and operates similar to LD_LIBRARY_PATH (although it is delimited by semicolons instead of colons).
- Each path we add here covers a place where our .so files may live:
  - The first path is our ${DESTINATION} which is where we uploaded our files as an example.
  - /system/lib/rfsa/adsp is a default location on some Androids for preinstalled DSP binaries.
  - /system/vendor/lib/rfsa/adsp is where vendor partners sometimes pre-install DSP binaries.
  - /dsp on some platforms DSP files appear directly under /dsp.

Executing¶

From the target device shell, navigate to the directory containing the test files:
```
cd ${DESTINATION}
```

Run the following to confirm that the files have been transferred and that you are in the proper folder:

ls

You should see the files you transferred, like so:

cropped                   libSnpeHtpPrepare.so
imagenet_slim_labels.txt  libSNPE.so
inception_v3_model.dlc    show_inceptionv3_classifications.py
libSnpeDspV66Skel.so      snpe-net-run
libSnpeDspV66Stub.so      target_raw_list.txt

Run the following command on the target device to execute an inference:
```
./snpe-net-run \
  --container "./inception_v3_model.dlc" \
  --input_list "./target_raw_list.txt" \
  --output_dir "./output"
```
Note

When calling your application (like snpe-net-run) you can decide which processors to use dynamically. In this case, if you wanted to specify a backend you could pass in --use_cpu, --use_gpu, --use_dsp, or --use_aip as long as you passed the appropriate backends to this device as part of Part 6: Transferring Files. See the reference docs for more details.

Verify that you see an output like this:

-------------------------------------------------------------------------------
Model String: N/A
SNPE v2.33.0.250327124043_117917
-------------------------------------------------------------------------------

Processing graph : inception_v3_model
Processing DNN input(s):
cropped/notice_sign.raw
Processing DNN input(s):
cropped/trash_bin.raw
Processing DNN input(s):
cropped/plastic_cup.raw
Processing DNN input(s):
cropped/chairs.raw
Successfully executed graph inception_v3_model

(Optional) If you don’t want to install python on your target device, you can scp the output folder back to your host machine by running:
1. This command to extract the output folder:
```
scp -r "${TARGET_USER}@${TARGET_IP}:${DESTINATION}/output" .
```
2. Run the commands below on your host machine instead of your target device.
  - Replace python3 with python if you are in a virtual environment (venv).

Interpret the results using show_inceptionv3_classifactions.py by running:

Warning

If the show_inceptionv3_classifications.py does not work because the script expects a different folder structure, you can also try running the alternate script below (which also depends on python), but handles the proper output structure.

pip install numpy
python3 show_inceptionv3_classifications.py -i target_raw_list.txt -o output -l imagenet_slim_labels.txt

Alternate script:

#!/bin/bash
echo
echo "Classification results"
idx=0
while IFS= read -r input_file || [ -n "$input_file" ]; do
    rawfile="output/Result_${idx}/InceptionV3/Predictions/Reshape_1:0.raw"
    if [ ! -f "$rawfile" ]; then
        printf "%-22s %.6f %3d %s\n" "$input_file" 0.0 0 "missing_file"
    else
        python_output=$(python3 -c "import numpy as np; a=np.fromfile('$rawfile', dtype=np.float32); print(f'{np.max(a)} {np.argmax(a)}')" 2>/dev/null)
        if [ -z "$python_output" ]; then
            printf "%-22s %.6f %3d %s\n" "$input_file" 0.0 0 "parse_error"
        else
            read maxval maxidx <<< "$python_output"
            label=$(sed "$((maxidx+1))q;d" imagenet_slim_labels.txt)
            printf "%-22s %.6f %3d %s\n" "$input_file" "$maxval" "$maxidx" "$label"
        fi
    fi
    idx=$((idx+1))
done < target_raw_list.txt

You should see an output that looks like this:

Classification results
cropped/notice_sign.raw 0.130224 459 brass
cropped/trash_bin.raw  0.719755 413 ashcan
cropped/plastic_cup.raw 0.989595 648 measuring cup
cropped/chairs.raw     0.380808 832 studio couch

Note

If you are using a different model, you will likely want to create your own interpretation script similar to the above to turn the raw output tensors into a human-readable output.

With that, you’ve successfully gone through each part to build and execute your AI model on your target device!

In Summary¶

You have installed SNPE and its dependencies, built your model into a .dlc, transferred it onto the target device, and used snpe-net-run to execute inferences using the processors you chose!

When applying this to your own model, be sure to consider the key variables which may change how you use the SNPE tools along the way:

What model do you want to use?
1. How will you download it?
2. How will you get the input data?
3. How will you format the input data to feed it into your model?
Which framework is the model using (ex. ONNX, PyTorch, Tensorflow, etc.)?
What is the OS and architecture of your host machine?
What is the OS and architecture of your target device?
Which processor(s) do you want to use for your AI models?

With those answers, you can adapt this tutorial to work with a model of your choice on your host machine, with any supported target device.

From here, the most common next steps are to:

Use this guide to build and execute your own model instead of the example model.
Create an application which uses the model on the target device (replacing snpe-net-run). See this tutorial for more details (The SNPE API supports C++, C, or Java).
Optimize your model using tools like snpe-bench (link).

If you have any questions, consider reaching out on Qualcomm’s Developer Discord here.

SNPE Tutorial for Linux Target Device from Linux Host¶

Part 6: Transfer Files to Your Target Device¶

Sub-Step 1: If you haven’t already, ensure that you follow the processor-specific Setup instructions.¶

Sub-Step 2: Set up SSH on the target device.¶

Sub-Step 3: Transferring Relevant Files¶

Steps to Transfer Files¶

Additional files to transfer for DSP / HTP / AIP¶

Part 7: Executing Your Model With snpe-net-run¶

Setting Environment Variables¶

Executing¶

In Summary¶

Part 7: Executing Your Model With `snpe-net-run`¶