Preparing a model with UDO

This section talks about the steps required to convert a framework model with user defined operations (UDO).

Converting a network model with UDO into DLC

snpe-<framework>-to-dlc tools support UDO functionality by accepting configuration file(s) with the option –udo_config_paths. For input UDO config file specifications, see Defining a UDO. Currently UDO functionality is supported on TensorFlow and ONNX models.

Note: Any modifications in the UDO configuration file should be followed up with re-generation of DLCs to reflect the changes.

Converting Tensorflow model with UDO to DLC

The following syntax showcases the way TensorFlow models can be converted using UDO:

snpe-tensorflow-to-dlc -i <input-tensorflow-model>
                       -d <input-name> <input-dim>
                       --out_node <output-node-name>
                       --udo_config_paths <input-model.json>
                       -o <output-model.dlc>

where the option –udo_config_paths allows users to specify the UDO configuration file to be used in the conversion.

See snpe-tensorflow-to-dlc and TensorFlow Model Conversion for further details.

Converting ONNX model with UDO to DLC

The following syntax showcases the way ONNX models can be converted using UDO:

snpe-onnx-to-dlc -i <input-onnx-model>
                 --udo_config_paths <input-model.json>
                 -o <output-model.dlc>

where the option –udo_config_paths allows users to specify the UDO configuration file to be used in the conversion.

See snpe-onnx-to-dlc and ONNX Model Conversion for further details.

Quantizing a DLC with UDO

Additionally, users may want to quantize converted models having UDOs to run on fixed-point runtimes. Qualcomm® Neural Processing SDK provides the tool snpe-dlc-quantize for this purpose. This is an offline tool that can be run on the host x86 platform. Since it runs inferences with a representative data-set in order to determine quantization ranges for all layers in the network including UDOs, users will need to provide a UDO package containing CPU reference implementation to the tool. Refer to Creating a UDO Package and Compiling a UDO package for further instructions on creating such a package for the x86 platform.

The following syntax showcases the way DLCs with UDOs can be quantized with snpe-dlc-quantize:

snpe-dlc-quantize --input_dlc <model.dlc>
                  --input_list <input-list.txt>
                  --udo_package_path <udo-registration-lib>
                  --output_dlc <quantized-model.dlc>

where the option –udo_package_path allows users to specify the absolute path to the UDO registration library. LD_LIBRARY_PATH must also be updated to include the runtime-specific artifacts generated from package compilation during x86 Host Compilation .

Note: If your UDO contains static inputs or parameters, you need to handle both floating-point and fixed-point data types in CPU reference implementation. Refer to provided example $SNPE_ROOT/examples/SNPE/NativeCpp/UdoExample/Conv2D/src/CPU/Conv.cpp for details.