XML OpDef Schema Breakdown¶
This page provides an overview of the schema used to define Operation Definitions (Op Defs) in QNN. Op Defs specify the inputs, outputs, parameters, and descriptive metadata that constitute an operation.
The schema is formalized using Extensible Markup Language (XML) and XML Schema Definition (XSD).
The XSD file that defines Op Defs, as well as the constructs used to define multiple operations
for use with the qnn-op-package-generator, is located at:
${QNN_SDK_ROOT}/lib/python/qti/aisw/op_package_generator/translator/OpDef.xsd
The remainder of this page will identify and detail the elements defined in the XSD that are most relevant
for use with the qnn-op-package-generator. Example usage of the schema is available
at Example.
Operation Definition Schema¶
The following elements will be explored in detail:
A simple UML diagram describing the relationship between these entities is given here:
QNN OpDef Schema UML
Note
members prefixed by @ are XML attributes, and those with no prefix are XML elements.
OpDef¶
An OpDef is an XML element that describes an operation at the highest level. Namely, it contains the following. Elements with content are required, and elements that are empty are optional.
<OpDef>
<Name>OpName</Name>
<Description>
<Content></Content>
<Code></Code>
</Description>
<Reference Source="" Url=""></Reference>
<!--Requires at least one input-->
<Input>
<Name>in[0]</Name>
<Mandatory>true</Mandatory>
<Constraint id="" Type=""></Constraint>
<Datatype>QNN_DATATYPE_FLOAT_32</Datatype>
<Shape>
<Rank>1D</Rank>
<Layout></Layout>
<Text></Text>
</Shape>
<Default></Default>
<Repeated></Repeated>
<IsStaticTensor></IsStaticTensor>
</Input>
<!--Requires at least one output-->
<Output>
<Name>out[0]</Name>
<Mandatory>true</Mandatory>
<Constraint id="" Type=""></Constraint>
<Datatype>QNN_DATATYPE_FLOAT_32</Datatype>
<Shape>
<Rank>1D</Rank>
<Layout></Layout>
<Text></Text>
</Shape>
<Repeated></Repeated>
</Output>
<!--Parameters are optional-->
<Parameter>
<Name>param</Name>
<Mandatory>false</Mandatory>
<Constraint id="" Type=""></Constraint>
<Datatype>QNN_DATATYPE_INT_32</Datatype>
<Shape>
<Rank>1D</Rank>
<Layout></Layout>
<Text></Text>
</Shape>
<Default></Default>
<Enumeration>
<Enum></Enum>
</Enumeration>
</Parameter>
<UseDefaultTranslation></UseDefaultTranslation>
<SupportedBackend>HTP</SupportedBackend>
</OpDef>
Name: The name of the operation.
Description: Optional field that describes the operation through sequences of content and code:
Content: String describing the operation.
Code: String that represents code describing the operation. e.g. output_height = input_height - crop_top - crop_bottom
Reference: Optional field(s) that define one or more references for the operation.
Source: Attribute for the source of operation. E.g. Tensorflow, ONNX, etc.
Url: Attribute for the URL of the source.
Input: Define one or more inputs to the operation. Inputs are extensions of Tensors that have the following addition field(s):
IsStaticTensor: Optional boolean flag that if set to True, indicates that an input tensor is a parameter which contains or references static data. If unset, the tensor is treated as a dynamic input.
Repeated: Optional boolean that specifies whether this input is repeated. Used for operations which have variadic inputs, such as Concat 1.
Output: Define one or more outputs to the operation. Outputs are extensions of Tensors that have the following additional field(s):
Repeated: Optional boolean that specifies whether this output is repeated. Used for operations which have variadic outputs, such as MultiClassNms 1.
Parameter: Optionally, define one or more parameters for the operation. Parameters are extensions of Tensors that additionally define
Enumeration: Optional field for enumerated params. Enumerations are composed of subfields called Enum whose content gives the name of the enum representing a given value. Values are assigned in the order in which the Enum is specified.
SupportedBackend: Field(s) that specify one or more backends on which this operation is supported. Used when backends share a common definition of an operation. If fields vary across backend for the same operation use SupplementalOpDef and mark the field with BACKEND_SPECIFIC.
UseDefaultTranslation: Boolean field that if set to true, indicates that a custom operation overrides a QNN native operation. The custom operation type must match the type of the QNN native operation for accurate conversion. When set to false, a custom operation is converted as a generic user defined operation. In the false mode, the custom op type must match the source framework type.
A key component of creating an OpDef are Inputs, Outputs, and Parameters, all of which are extensions of Tensors. A Tensor element is defined as follows:
Name: The name of the tensor.
Description: Optional field that describes the tensor through sequences of content and code:
Content: String describing the tensor.
Code: String that represents code describing the tensor. e.g. output_height = input_height - crop_top - crop_bottom
Constraint: Optional field that defines one or more constraints on the given tensor. The constraint is given as a string in the body of the element 2.
id: A number specifying the id of the constraint. Used to override constraints for supplemental operation definitions.
Type: The type of the constraint. Valid types are:
Number: a constraint characterized by cardinality. e.g. Number of inputs >= 1
Shape: a constraint characterized by a restriction on dimension of the tensor. e.g. Rank >= 1
Value: a constraint characterized by the value of the tensor. e.g. Tensor is only positive
Datatype: a constraint characterized by a restriction on datatype. Typically used for tensors which can be of several datatypes but must be of the same datatype as another tensor.
Description: a constraint which does not conform to any other category of constraint.
Mandatory: A boolean indicating whether the tensor is must be provided/defined 3.
Datatype: Define the allowable datatypes for the tensor. Must be one of:
QNN_DATATYPE_INT_8
QNN_DATATYPE_INT_16
QNN_DATATYPE_INT_32
QNN_DATATYPE_INT_64
QNN_DATATYPE_UINT_8
QNN_DATATYPE_UINT_16
QNN_DATATYPE_UINT_32
QNN_DATATYPE_UINT_64
QNN_DATATYPE_SFIXED_POINT_4
QNN_DATATYPE_SFIXED_POINT_8
QNN_DATATYPE_SFIXED_POINT_16
QNN_DATATYPE_SFIXED_POINT_32
QNN_DATATYPE_UFIXED_POINT_4
QNN_DATATYPE_UFIXED_POINT_8
QNN_DATATYPE_UFIXED_POINT_16
QNN_DATATYPE_UFIXED_POINT_32
QNN_DATATYPE_BOOL_8
QNN_DATATYPE_FLOAT_16
QNN_DATATYPE_FLOAT_32
QNN_DATATYPE_FLOAT_64
BACKEND_SPECIFIC used to indicate that the datatype is dependent on the Backend. Must be used in conjunction with a SupplementalOpDef to specify a concrete datatype.
Shape: Specify the shape of the tensor.
Rank: The rank of the tensor as an enumeration with the following values:
SCALAR: Scalar
1D: Vector
2D: Matrix
3D: 3D Tensor or Image
4D: 4D Tensor or Batched Image
ND: Generic N-D Tensor N >= 0
Layout: Optional field that specifies the layout of the tensor. Must be one of:
NHWC
NHCW
UNDEFINED used to identify that the layout is neither NHCW or NHWC
BACKEND_SPECIFIC used to indicate that the layout is dependent on the Backend. Must be used in conjunction with a SupplementalOpDef to specify a concrete layout.
Text: Optionally provide text description for the shape of the tensor.
Default: Optionally provide text representing the default value for the tensor. Can be one of
Tensor: use braces or brackets to create list e.g. [[1, 2], [3, 4]]
Scalar: provide scalar value e.g. 1, 1.1, -1
Boolean: provide either 0 (false) or 1 (true)
String: Any other string. If text cannot be resolved into one of above categories it will be stored as string.
Note
Inputs and Outputs of Custom op must have rank >= 1.
Custom op must have at least one input and output.
Both IsDefaultTranslation and IsStaticTensor are only relevant for custom op conversion use cases.
SupplementalOpDef¶
SupplementalOpDef’s are XML elements which define content that is variable across backend(s). SupplementalOpDef’s extend the content defined in OpDef’s, but limits the fields that can be overridden. The SupplementalOpDef is structured as follows. Elements with content are required, and elements that are empty are optional.
<SupplementalOpDef>
<Name>OpName</Name>
<!--Only supplemented Inputs are required-->
<Input>
<Name>in[0]</Name>
<Constraint id="" Type=""></Constraint>
<Datatype></Datatype>
<Shape>
<Layout></Layout>
<Text></Text>
</Shape>
<OnlyDefaultSupported></OnlyDefaultSupported>
</Input>
<!--Only supplemented Outputs are required-->
<Output>
<Name>out[0]</Name>
<Constraint id="" Type=""></Constraint>
<Datatype></Datatype>
<Shape>
<Layout></Layout>
<Text></Text>
</Shape>
<OnlyDefaultSupported></OnlyDefaultSupported>
</Output>
<!--Only supplemented Params are required-->
<Parameter>
<Name>param</Name>
<Constraint id="" Type=""></Constraint>
<Datatype></Datatype>
<Shape>
<Layout></Layout>
<Text></Text>
</Shape>
<OnlyDefaultSupported></OnlyDefaultSupported>
</Parameter>
</SupplementalOpDef>
Name: The name of the operation. Must correspond to an OpDef defined in the same config.
Input: Optionally, extend one or more inputs to the operation. Supplemental inputs are Supplemental Tensors.
Output: Optionally, extend one or more outputs to the operation. Supplemental outputs are Supplemental Tensors.
Parameter: Optionally, extend one or more parameters for the operation. Supplemental parameters are Supplemental Tensors.
Inputs, outputs, and parameters are all Supplemental Tensors, which can only specify certain fields. All fields, except name, in a supplemental tensor are optional.
Name: The name of the tensor. Must correspond to the name of the tensor in the original OpDef that is being extended.
Constraint: Optional field that defines one or more constraints on the given tensor. The constraint is given as a string in the body of the element 2.
id: A number specifying the id of the constraint. Used to override constraints for supplemental operation definitions.
Type: The type of the constraint. Valid types are:
Number: a constraint characterized by cardinality. e.g. Number of inputs >= 1
Shape: a constraint characterized by a restriction on dimension of the tensor. e.g. Rank >= 1
Value: a constraint characterized by the value of the tensor. e.g. Tensor is only positive
Datatype: a constraint characterized by a restriction on datatype. Typically used for tensors which can be of several datatypes but must be of the same datatype as another tensor.
Description: a constraint which does not conform to any other category of constraint.
Datatype: Define the allowable datatypes for the tensor. Must be one of
QNN_DATATYPE_INT_8
QNN_DATATYPE_INT_16
QNN_DATATYPE_INT_32
QNN_DATATYPE_INT_64
QNN_DATATYPE_UINT_8
QNN_DATATYPE_UINT_16
QNN_DATATYPE_UINT_32
QNN_DATATYPE_UINT_64
QNN_DATATYPE_SFIXED_POINT_4
QNN_DATATYPE_SFIXED_POINT_8
QNN_DATATYPE_SFIXED_POINT_16
QNN_DATATYPE_SFIXED_POINT_32
QNN_DATATYPE_UFIXED_POINT_4
QNN_DATATYPE_UFIXED_POINT_8
QNN_DATATYPE_UFIXED_POINT_16
QNN_DATATYPE_UFIXED_POINT_32
QNN_DATATYPE_BOOL_8
QNN_DATATYPE_FLOAT_16
QNN_DATATYPE_FLOAT_32
QNN_DATATYPE_FLOAT_64
Shape: Specify the shape of the tensor.
Layout: Optional field that specifies the layout of the tensor. Must be one of:
NHWC
NHCW
UNDEFINED used to identify that the layout is neither NHCW or NHWC
BACKEND_SPECIFIC used to indicate that the layout is dependent on the Backend. Used in conjunction with a SupplementalOpDef to specify a concrete layout.
Text: Optionally provide text description for the shape of the tensor.
OnlyDefaultSupported: Optional boolean to indicate that the backend only supports the default value defined in the corresponding OpDef for this tensor.
OpDefList¶
An OpDefList is an XML element composed of a sequence of OpDef elements. OpDefLists are agnostic to backend, and serve only as a wrapper around multiple OpDefs.
<OpDefList>
<!--One or more OpDef-->
<OpDef>
<!--OpDef defined above -->
</OpDef>
</OpDefList>
SupplementalOpDefList¶
A SupplementalOpDefList is an XML element composed of a sequence of SupplementalOpDef elements. In addition, SupplementalOpDefLists contain the following fields.
<SupplementalOpDefList Backend="HTP">
<SupportedOps>
<OpName></OpName>
</SupportedOps>
<SupplementalOpDef>
<!--SupplementalOpDef defined above-->
</SupplementalOpDef>
</SupplementalOpDefList>
Backend: An attribute specifiying for which Backend the SupplementalOpDef’s are supplementing.
SupportedOps: An element containing a sequence of OpName elements. Each OpName corresponds to an operation defined in the corresponding OpDefList, and indicates the backend supports the Op. This information may be redundant with the SupportedBackend field of the OpDef element.
OpDefCollection¶
The OpDefCollection is the root XML element of the configuration file meant to be used with
the qnn-op-package-generator. It contains all the information needed to specify all of a users packages.
The OpDefCollection contains the following
<OpDefCollection
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xs:noNamespaceSchemaLocation="OpDef.xsd"
PackageName="ExamplePackage"
Domain="example"
Version="1.0"
>
<!--One OpDefList-->
<OpDefList>
<!--OpDefList defined above-->
</OpDefList>
<!--SupplementalOpDefLists are not required-->
<SupplementalOpDefList Backend="HTP">
<!--SupplementalOpDefList defined above-->
</SupplementalOpDefList>
</OpDefCollection>
PackageName: Attribute specifying the name for the user’s OpPackage. Because packages are per backend, the actual package name will be the value specified here appended with <Backend> e.g. MyPackageNameHtp
Domain: Attribute specifying the domain of the package.
Version: Attribute specifying the version of the package.
OpDefList: One OpDefList specifying all operations for the package(s).
SupplementalOpDefList: Optionally specify one or more SupplementalOpDefList to specify per-backend information.
One OpDefCollection element can be used to produce multiple per-backend packages.
Notes
- 1(1,2)
Operations with an unknown number of inputs/outputs are currently not supported on the HTP backend.
- 2(1,2)
Constraints are purely a descriptive field and do not require mathematical expression. Constraints are currently not enforced.
- 3
Outputs are not mandatory, but cannot have default values and are assumed to be NULL if not provided.