Running a model with UDO

Executing neural networks with UDO

This section describes how to provide the set of libraries representing a UDO to Qualcomm® Neural Processing SDK, to execute network models containing the UDO. Execution requires the model containing the UDO be converted into DLC format using Qualcomm® Neural Processing SDK converter tools as explained in Preparing a model with UDO. Execution additionally requires the creation of registration and implementation libraries compatible with UDO APIs as described in Overview of UDO.

UDOs can be registered with a process and be utilized by any instance of Qualcomm® Neural Processing SDK that employs them throughout the lifetime of the process. They are registered with Qualcomm® Neural Processing SDK using the Snpe_Util_AddOpPackage() API:

SNPE_API
int Snpe_Util_AddOpPackage(const char* regLibraryPath );

Note that the absolute path to the registration library is explicitly provided to Qualcomm® Neural Processing SDK using the above API, whereas the implementation libraries can exist anywhere on the system that should be discovered by the library loader (using LD_LIBRARY_PATH on Unix systems).

In addition to the native C API, the Qualcomm® Neural Processing SDK provides a Java API for use in Android applications. The API is a part of the SNPE class:

public static boolean
addOpPackage(final Application application, String regLibraryPath)

Running snpe-net-run command-line tool with UDO-based DLC

This section outlines the use of snpe-net-run command line tool with a UDO-based DLC. The use of snpe-net-run is largely unchanged from its typical usage. The snpe-net-run tool registers a UDO registration library through the command line option –udo_package_path which accepts the absolute path to the registration library.

Example usage is as follows:

snpe-net-run --container <path_to_dlc> --input_list <path_to_input_list> --udo_package_path <path_to_registration_lib>

Running a UDO-based DLC on a signed process domain

To run on an signed process domain, the same principles as the above section on utilizing the snpe-net-run command line tool apply with a few extensions. Firstly, signed process domains only apply to the DSP target. Second, for a signed process domain to be created, an argument must be passed to the command line to request it. This comes in the form of the platform_options key:value argument pairing. To enable model execution on a signed process domain, simply add the following to the command line:

--use_dsp --platform_options unsignedPD:OFF

Example usage is as follows:

snpe-net-run --container <path_to_dlc> --input_list <path_to_input_list> --udo_package_path <path_to_registration_lib> --use_dsp --platform_options unsignedPD:OFF

Note: snpe-net-run requires implementation libraries to be discoverable by the system library loader.