Example XML OpDef Configs

This page outlines several example configuration files to be used with the qnn-op-package-generator. All examples are alterations of the example located at:

${QNN_SDK_ROOT}/examples/QNN/OpPackageGenerator/ExampleOpPackageHtp.xml

Note

the example configs use one Backend, while the qnn-op-package-generator currently supports only the HTP, DSP and CPU backends.

Example Config #1

The following example defines an op package for HTP, which contains one operation named Softmax. This example config represents a case in which an operation is identical across backend and does not need any supplemental information.

<?xml version="1.0" encoding="UTF-8"?>
<OpDefCollection
     PackageName="ExamplePackage"
     Domain="aisw"
     Version="1.0"
>
 <OpDefList>

     <OpDef>
         <Name>Softmax</Name>
         <Description>
             <Content>
                 Computes data normalization exponentially on an input tensor given an optional positive
                 scaling factor, beta. The computation is done element-wise per batch along the last dimension.

                 See Softmax backend definition for supported datatypes and constraints per backend
             </Content>
         </Description>

         <Reference Source="Android NDK NeuralNetworks"
                    Url="ANEURALNETWORKS_SOFTMAX &lt;https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2bfbb83a537701e2843a3d5004250c2c&gt;"/>

         <Input>
             <Name>in[0]</Name>
             <Description>
                 <Content>input activation</Content>
             </Description>
             <Mandatory>true</Mandatory>
             <Datatype>QNN_DATATYPE_UFIXED_POINT_8</Datatype>
             <Datatype>QNN_DATATYPE_UFIXED_POINT_16</Datatype>
             <Shape>
                 <Rank>1</Rank>
                 <Text>a tensor of N dimension</Text>
             </Shape>
         </Input>

         <Output>
             <Name>out[0]</Name>
             <Description>
                 <Content>output activation</Content>
             </Description>
             <Mandatory>true</Mandatory>
             <Datatype>QNN_DATATYPE_UFIXED_POINT_8</Datatype>
             <Datatype>QNN_DATATYPE_UFIXED_POINT_16</Datatype>
             <Shape>
                 <Rank>1</Rank>
                 <Text>a tensor of N dimension</Text>
             </Shape>
         </Output>

         <Parameter>
             <Name>beta</Name>
             <Mandatory>false</Mandatory>
             <Datatype>QNN_DATATYPE_FLOAT_32</Datatype>
             <Shape>
                 <Rank>0</Rank>
             </Shape>
             <Default>1.0</Default>
         </Parameter>

         <!--This Op is implemented on these Backends-->
         <SupportedBackend>HTP</SupportedBackend>
     </OpDef>

</OpDefList>

Here, the Softmax operation defined has one input and one output. Each input and output is a vector, as indicated by the rank being 1, and is mandatory, meaning that a non-null value must be supplied for each input and output. Additionally, each input and output support QNN_DATATYPE_UFIXED_POINT_8 and QNN_DATATYPE_UFIXED_POINT_16 as datatypes for the HTP package. Note that multiple datatypes can be listed for the same tensor.

In addition to the input and output, the Softmax operation has a parameter named beta. Beta is a scalar value, and unlike the activation tensors, it is not mandatory. If unspecified the default used for beta is 1.0. Beta is supported with a datatype of QNN_DATATYPE_FLOAT_32.

Example Config #2

The following example once again defines an op package for HTP, which contains one operation named Softmax. However, this example config represents a case in which an operation has some information that varies between the backends and requires supplemental information.

 <OpDefCollection
      xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
      xs:noNamespaceSchemaLocation="../OpDef.xsd"
      PackageName="ExamplePackage"
      Domain="example"
      Version="1.0"
 >
  <OpDefList>
  <!--Example Op Def Collection, mirroring an OpPackage with supplemental info-->
  <OpDef>
      <Name>Softmax</Name>
      <Description>
          <Content>
              Computes data normalization exponentially on an input tensor given an optional positive
              scaling factor, beta. The computation is done element-wise per batch along the last dimension.

              See Softmax backend definition for supported datatypes and constraints per backend
          </Content>
      </Description>

      <Reference Source="Android NDK NeuralNetworks"
      Url="ANEURALNETWORKS_SOFTMAX &lt;https://developer.android.com/ndk/reference/group/neural-networks#group___neural_networks_1ggaabbe492c60331b13038e39d4207940e0a2bfbb83a537701e2843a3d5004250c2c&gt;"></Reference>

      <Input>
          <Name>in[0]</Name>
          <Description><Content>input activation</Content></Description>
          <Mandatory>true</Mandatory>
          <Datatype>BACKEND_SPECIFIC</Datatype>
          <Shape>
              <Rank>1</Rank>
              <Text>a tensor of N dimension</Text>
          </Shape>
      </Input>

      <Output>
          <Name>out[0]</Name>
          <Description><Content>output activation</Content></Description>
          <Mandatory>true</Mandatory>
          <Datatype>BACKEND_SPECIFIC</Datatype>
          <Shape>
              <Rank>1</Rank>
              <Text>a tensor of N dimension</Text>
          </Shape>
      </Output>

      <Parameter>
          <Name>beta</Name>
          <Mandatory>false</Mandatory>
          <Datatype>QNN_DATATYPE_FLOAT_32</Datatype>
          <Shape>
              <Rank>0</Rank>
          </Shape>
          <Default>1.0</Default>
      </Parameter>

      <SupportedBackend>HTP</SupportedBackend>

  </OpDef>



 </OpDefList>

  <SupplementalOpDefList Backend="HTP">
     <SupportedOps>
          <OpName>Softmax</OpName>
      </SupportedOps>

      <SupplementalOpDef>
          <Name>Softmax</Name>

          <Input>
              <Name>in[0]</Name>
              <Datatype>QNN_DATATYPE_UFIXED_POINT_8</Datatype>
              <Datatype>QNN_DATATYPE_UFIXED_POINT_16</Datatype>
          </Input>

          <Output>
              <Name>out[0]</Name>
              <Datatype>QNN_DATATYPE_UFIXED_POINT_8</Datatype>
              <Datatype>QNN_DATATYPE_UFIXED_POINT_16</Datatype>
          </Output>

          <!--Nothing to do for beta, it remains the same-->

      </SupplementalOpDef>

  </SupplementalOpDefList>

</OpDefCollection>

In this example, the abstract definition of the operation is the same as in the previous example. The Softmax operation contains one input, one output, and one parameter. This is different than the first example in that the datatype of the activation tensors varies across backend, as indicated with the BACKEND_SPECIFIC field. This information is specified with the SupplementalOpDef’s.

Note

Backend specific information must be supplemented to use the qnn-op-package-generator.

Here there is one SupplementalOpDefList for HTP as indicated by the Backend attribute of the SupplementalOpDefList.

The HTP package supports both QNN_DATATYPE_UFIXED_POINT_8 and QNN_DATATYPE_UFIXED_POINT_16 datatypes for the activation tensors. Note that the HTP supplemental information does not specialize the parameter beta. This indicates that the HTP’s specification of beta is identical to that defined in the corresponding OpDef.

Note that the backend support in this example is indicated with the SupportedOp list in the SupplementalOpDefList, instead of the BackendSupport element of the OpDef itself.