QNN HTP Optimization Utility Functions Usage Examples

• OP_ITER

• WITH_SIZE

OP_ITER

Create a multi-output Op by iterating over expression

Usage Hint

Often used with multi-outputs ops such as “Concat”

Syntax

/*
 * op_base         - 'Reference' Op supplying the name and fixed inputs
 * tag             - A string indentifying the split context
 * lo_index        - the first input index
 * hi_index        - the last input index+1
 * f               - The subexpression to iterate
 */
 OP_ITER( op_base, "tag", lo_index, hi_index, f)

Example

// Original Sequence
// In[1,16,8,32] -> Relu[1,16,8,32]

// Tiled on height by 8
//               /-> SlicePad_shape(start=[0,0,0,0])[1,8,8,32] -> Relu[1,8,8,32] \
// In[1,16,8,32]                                                                   --> Concat(axis=1)[1,16,8,32]
//               \-> SlicePad_shape(start=[0,8,0,0])[1,8,8,32] -> Relu[1,8,8,32] /
DEF_OPT(TILING,
    Op("Relu", "In"),
    OK,
    OP_ITER(
        Op(FROM_DEFAULT_PACKAGE("Concat"), gen_ConstScalar_i32(1)),   // Height is axis 1, that's the axis we want to Concat back on
        "I",                                    // Iterator I
        0,                                      // Iterator initialize to 0
        2,                                      // Iteration termination
        WITH_SIZE(              // Here the size is the tiled Op output size
            gen_Shape(1,8,8,32),
            Op("Relu",
                WITH_SIZE(      // Here the size is the shape of the tiled input to the Op
                    gen_Shape(1,8,8,32),    // WITH_SIZE is needed because the default size is the shape of the original pattern output
                ),
                Op(FROM_DEFAULT_PACKAGE("SlicePad_shape"),
                    "In",
                    gen_Shape(0,0,0,0),                     // no padding before
                    gen_Shape(
                        0, MUL(SPLIT_START("I"),8), 0, 0    // Start slicing at (0,0,0,0) for thefirst tile,
                                                            // and (0,8,0,0,) for the second tile
                    ),
                    gen_Shape(1,8,8,32)                     // Size of each In tile is [1,8,8,32]
                    gen_ConstScalar_i32(0)                  // Pad val of 0
                )
            )
        )
    )
)

WITH_SIZE

Specify the reference output size

Usage Hint

Often used with gen_Shape

Syntax

/*
 * ref                 -  reference OpDef with some rank and shape
 * target              -  evaluate 'target' with ref as output size
 */
 WITH_SIZE(ref, target)

Example

// Same as above OP_ITER example