Create a QNN converter Op Package shared library¶

qnn-op-package-generator generates skeleton code and makefiles used to create a QNN converter Op Package shared library. qnn-op-package-generator takes an XML configuration as an input that describes the package attributes and the --converter_op_package or -cop option that specifies whether to generate the converter Op Package and produce a QNN converter Op Package directory structure.

Note

Executing qnn-op-package-generator with the --converter_op_pacakge or -cop option creates a Converter Op Package in addition to the QNN Op Package. The name of the Converter Op Package generated will be QNN Op Package appended with the string Converter_Op_Package. The option is not supported on Windows platform yet.

Linux Platform¶

Create a Converter Op Package skeleton - Using ReluOpPackage as an example.

Note

These instructions assume that the general setup instructions 1 have been followed and that qnn-op-package-generator is accessible on the command line. For Linux users, the converter Op Package can be generated and compiled on a linux-86_64 platform.
1. Define an XML OpDef configuration. This operation definition XML describes package information like name, version, and domain. See XML OpDef Schema Breakdown for more information about XML schema.
  
  Sample configurations can also be found at Example XML Op Def Configs and in the SDK at:
```
${QNN_SDK_ROOT}/examples/QNN/OpPackageGenerator
```
  Note
  
  The XML configuration used to generate the QNN Converter Op Package is same as the XML configuration used in QNN Op Package generation.
2. Run the qnn-op-package-generator tool and pass the XML configuration using the --config_path or -p option along with the --converter_op_pacakge or -cop option.
```
qnn-op-package-generator -p <QNN_SDK_ROOT>/examples/QNN/OpPackageGenerator/ReluOpPackageCpu.xml -cop -o <output_dir>
```
  This command creates an Op Package named ReluOpPackage and a converter Op Package named ReluOpPackage_Converter_Op_Package. 2
  
  Note
  
  The -p command line option can be specified multiple times to generate different packages, provided each package name is distinct. If the package name is not distinct, the tool will merge all ops defined in each configuration into a single package directory.
- Directory structure
  
  The package directory tree is shown and explained below:
```
|-- Makefile
|-- ConverterOpPackage
    |-- ConverterOpPackage.cpp
    `-- Makefile.linux-x86_64
```
  - Makefile – Contains make targets and rules to compile the package source files for x86 targets.
  - ConverterOpPackage – Contains generated source files for all operations defined in the configuration and target specific makefile.

Note

The converter Op Package must be generated and compiled on a linux-86_64 platform.

Implement the skeleton code
- Op source file – ConverterOpPackage.cpp
  
  The source file contains empty functions bodies that should be completed by users. The code used in this section references the generated package in Directory Structure. The following example highlights the output for the Relu op defined in the example configuration.
  
  The following autogenerated code contains a structure that takes opConfig of type Qnn_OpConfig_t as its parameter. OpConfig is the operations configuration for storage and retrieval of inputs, outputs, and parameters.
```
 1EXPORT_API Qnn_ErrorHandle_t ReluShapeInference(Qnn_OpConfig_t *opConfig) {
 2
 3/**
 4* Add code here
 5**/
 6
 7return QNN_SUCCESS;
 8}
 9
10Qnn_ErrorHandle_t (*ReluOutputInfoInferencePtr)(Qnn_OpConfig_t *)  = &ReluShapeInference;
```
  Complete the function by implementing the logic to populate output shapes using the opConfig and add any necessary validation. The function is called by a function pointer associated by the converter.
```
 1EXPORT_API Qnn_ErrorHandle_t ReluDataTypeInference(Qnn_OpConfig_t *opConfig) {
 2
 3/**
 4* Add code here
 5**/
 6
 7return QNN_SUCCESS;
 8}
 9
10Qnn_ErrorHandle_t (*ReluDataTypeInferencePtr)(Qnn_OpConfig_t *)  = &ReluDataTypeInference;
```
  Complete the function by implementing the logic to populate output datatypes using the opConfig and add any necessary validation. The function is called by a function pointer associated by the converter
  
  Note
  
  Users can populate both the output shapes and output datatypes within a single function named operation name followed by ShapeInference (Eg: ReluShapeInference for Relu operation). This ensures backward compatibility. Users should note that populating both the output shapes and output datatypes within the same function will be deprecated and the logic to populate the output shapes must be written in OpNameShapeInference function and and output datatypes must be written in OpNameDataTypeInference function respectively.
  
  Note
  
  The operation name must be a valid C++ identifier, meaning that it can only contain alphanumeric characters and underscores.
  
  Note
  
  The inputs, outputs, and parameters order in the opConfig must follow the order they are present in the model. See <QNN_SDK_ROOT>/examples/QNN/OpPackageGenerator/generated/CONVERTER for sample Relu implementation.
Compile the package for x86 targets.
1. Set up the QNN environment.
```
$ source <QNN_SDK_ROOT>/bin/envsetup.sh
```
2. Required for x86: Ensure clang compiler is in your path, or set CXX to point to a valid compiler path.[3]_
Note

Steps a-b can also be set manually in the makefiles or as a command line option to make without the use of scripts.
1. Compile the package for x86 targets :
  make cpu or make cpu_x86
  This creates a shared library at <current_dir>/<package_name>_Converter_Op_Package/ConverterOpPackage/lib<ConverterOpPackage>.so

1: For instructions on QNN tool setup, see Setup.
2: If the directory already exists, the tool will only generate new files and attempt to append to existing files. To force a new package to be generated, use the --force-generation option.
3: The script, <QNN_SDK_ROOT>/bin/check-linux-dependency.sh, can also be used to download the appropriate Clang version if it is not present.