Op Packages¶
Operation Packages in Qualcomm® AI Engine Direct, also called ‘Op Packages’ refer to a collection of operations that are made available to a backend in order to be utilized in creating and executing Qualcomm® AI Engine Direct graphs representing network models.
Qualcomm® AI Engine Direct software architecture is designed to allow the memory footprint of applications that use QNN to be highly customized, depending on the required backends and operations to run their models with. Operation packages are separated from the backends by virtue of being compiled into shared libraries distinct from the core backend libraries. This allows users to compile lean applications by packaging any number of operations they need for their use-cases into Op packages. This is especially desirable in deeply embedded IOT-type use-cases which operate within strict memory limits and target just a handful of network models tailored for their needs.
A Qualcomm® AI Engine Direct backend registers Op packages supplied by users with the API QnnBackend_registerOpPackage().
Multiple Op packages can be registered with a backend, and are shared between contexts and graphs created within the backend as individual, distinct instances. Their lifetime and scope matches the scope of the backend they are registered in, and hence they can outlast any graphs that need them. Crucially, this also means that external parties can easily create and integrate their custom Op packages with Qualcomm® AI Engine Direct using the same mechanism used by Qualcomm® AI Engine Direct Op packages. All Op packages are required to implement the interfaces defined in QnnOpPackage.h.
See Tutorials for a simple example Op package that demonstrates how external parties can create and use custom ops within the Qualcomm® AI Engine Direct framework.
See the following section for info on generating Op Package skeleton code, which can then be implemented as a full-fledged Qualcomm® AI Engine Direct Op Package.