Op Package Migration Guide¶

Since QNN HTP does not currently guarantee backward compatibility for Op Packages, they may need to be rebuilt from source or modified at the source code level to comply with later versions of the QNN HTP API. This page is a guide for resolving any such compatibility breakages and is designed to help external developers update their Op Packages for compatibility with the latest QNN SDK.

Migration Guide for QNN HTP API version 5.15.0 / QNN SDK version 2.15.0¶

QNN HTP API version 5.15.0 introduces a pair of new requirements on Op Packages:

Hexagon targets of Op Packages must be rebuilt from source with the -DPREPARE_DISABLED compiler option
- Only the Hexagon target(s) must add -DPREPARE_DISABLED; x86_64-linux and aarch64-android targets should be left alone.
- Note that the qnn-op-package-generator tool, provided by QNN SDK 2.15.0, will generate a Makefile that is compliant with this new requirement.
Any external shape, constraint, and replacement functions, and whatever helper code they depend on, must now be wrapped in an #ifndef PREPARE_DISABLED / #endif guard
- These are the functions referenced by invocations of the EXTERNAL_SHAPEFN(), EXTERNAL_CONSTRAINT(), and EXTERNAL_REPLACE() elements of the package optimization grammar.
- Note that only Op Packages that define these kinds of functions need to make this change.

Migration Guide for QNN HTP API version 5.10.0 / QNN SDK version 2.10.0¶

Op Packages are not backward compatible after QNN HTP API Version 5.10.0. This document outlines the process for updating the Op Packages for QNN HTP to use the new registration system.

An Op Package contains two parts, both of which require updates:

Part 1: The Op Package interface file, which references the Op source file(s)
Part 2: Each Op implementation has an individual source file (or files)

Part 1¶

To adapt the new registration system, the interface.cpp file must contain the following points:

Include new header files:

#include "HTP/core/unique_types.h"
#include "HTP/core/simple_reg.h"

Add macros and order of declarations list

DEFINE_UNIQ_TY()
BEGIN_PKG_OPS_OPTS_LIST()

// Append the Op file name in between the begin and end call
DECLARE_PKG_OPS_OPTS_LIST(PKG_SOFTMAX)
DECLARE_PKG_OPS_OPTS_LIST(PKG_RELU)
DECLARE_PKG_OPS_OPTS_LIST(PKG_RELUFP16)
DECLARE_PKG_OPS_OPTS_LIST(PKG_MAXPOOL)

END_PKG_OPS_OPTS_LIST()

Note: The Op file name depends on the name you give to your op.cpp file. The order matters; always append the new one at the bottom

Part 2¶

For each Op source file, make sure to add the following changes:

Add new include header file:
#include "HTP/core/simple_reg.h

Add BEGIN and END macros:

BEGIN_PKG_OP_DEFINITION(<name>)
...
...
END_PKG_OP_DEFINITION(<name>)

Before invoking any variant of REGISTER_OP or DEF_OPT, call BEGIN_OP_DEFINITION(<name>), where <name> is a C++ identifier that uniquely IDs the source file. At the bottom of the file, call END_OP_DEFINITION(<name>), using the same <name> as BEGIN.

Declare your source file name inside Interface file:
Make sure to append the line DECLARE_PKG_OPS_OPTS_LIST(<name>) to the interface file, as mentioned in the second point of Part 1.

NEW MACRO for registering ops

We now have three ways to register ops¶

//Default Cost Function and Default Flags
DEF_PACKAGE_OP
//Built-in cost value and Flags
DEF_PACKAGE_OP_AND_COST_AND_FLAGS
//NEW MACRO, user-specified cost function and Flags
DEF_PACKAGE_OP_AND_COST_F_AND_FLAGS

Op Registration Restrictions¶

COST VALUE: DO NOT use STRING format of COST VALUE

OPNAME: DO NOT use any STRING VARIABLE for NAME ENTRY, use STRING LITERAL directly

examples/OpPackage/HTP/ExampleOpPackageRelu.cpp in QNN SDK¶

    //Do NOT use string variable for name entry
    //static constexpr auto sg_opNameReluMinMax = "ReluMinMax";

    DEF_PACKAGE_OP_AND_COST_AND_FLAGS((reluMinMaxImpl<QUint8CroutonTensor, Tensor>),
     //Use string literal directly
     "ReluMinMax",
     //DO NOT use string for cost value
     // "SNAIL",
     SNAIL,
     Flags::RESOURCE_HVX)