QNN HTP Op Support Revision History

Introduced in QNN SDK Version

Runtime

Description

2.29.0

Quant

  • Dynamic tensor ops have been enabled in OpValidator

  • TopK (Activation type: All)

    • Added support to largest parameter that largest can be equal to 0

  • ElementWiseNeuron (Activation type: INT8, INT16)

    • Added Softplus support with rank 4d

  • BatchNorm, LayerNorm (Activation type: INT8)

    • Added QNN_DATATYPE_SFIXED_POINT_8 support for BatchNorm and LayerNorm

  • Convert (Activation type: INT16)

    • Added updateable quantization support for in[0] and out[0] when in[0] and out[0] both are QNN_DATATYPE_UFIXED_POINT_16 datatype

  • RmsNorm (Activation type: INT16)

    • Added updateable quantization support for in[0], in[1] and in[2] when in[0] is QNN_DATATYPE_UFIXED_POINT_16, in[1] is QNN_DATATYPE_UFIXED_POINT_8 and in[2] is QNN_DATATYPE_SFIXED_POINT_32

  • ElementWiseBinary Equal (Activation type: FP16)

    • Added rank 5d support for in[0], in[1] and out[0] for ElementWiseBinary Equal

  • ReduceSum (Activation type: FP16, FP32)

    • Added rank 5d support for in[0] and out [0]

2.28.0

Quant

  • Axes (Activation type: All)

    • Added support to normalization on final dimension or last 3 diminsions of 4D inputs

  • Prelu (Activation type: All)

    • Added rank 5d support for in[0], in[1] and out[0]

  • NonZero (Activation type: INT16, FP16)

    • Added NonZero op support

  • Tile (Activation type: FP16, FP32)

    • Added rank 5d support for in[0] and out [0]

  • RmsNorm (Activation type: INT16, INT8)

    • Added updateable quantization support for in[0], in[2] and out[0]

  • Conv2d (Activation type:INT8)

    • Added updateable quantization support for in[0], in[1] and out[0]

  • ElementwiseNeuron Relu (Activation type: INT8, INT16)

    • Added QNN_DATATYPE_SFIXED_POINT_16 support for in[0] and out[0]

  • ExtractGlimpse (Activation type: INT16)

    • Added ExtractGlimpse op support

  • Convert (Activation type: INT16)

    • Added updateable quantization support for in[0] and out[0]

  • ElementWiseBinary (Activation type: FP16)

    • Added rank 5d support for in[0], in[1] and out[0] for Add, Sub and Pow

  • ElementWiseUnary Sqrt (Activation type: FP16)

    • Added rank 5d support for in[0] and out[0] for Sqrt

  • ReduceMax, ReduceMin, ReduceMean (Activation type: FP16)

    • Added rank 5d support for in[0]

  • ElementWiseNeuron SoftPlus (Activation type: FP16)

    • Added QNN_DATATYPE_FLOAT_16, QNN_DATATYPE_FLOAT_32 datatype support for in[0] and out[0]

  • ReduceMin (Activation type: INT32)

    • Added ReduceMin op support with 5d rank and QNN_DATATYPE_INT_32 datatype for in [0] and out[0]

2.27.0

Quant

  • ElementWiseNeuron, Sigmoid (Activation type: All)

    • Added constraint that dynamic dimensions is not supported currently

  • Pack (Activation type: All)

    • Added rank 4d support for in[0] and 5d support for out[0]

  • CumulativeSum (Activation type: INT8)

    • Added rank 5d support for in[0] and out [0]

  • HardSigmoid (Activation type: INT8, INT16)

    • Added rank 5d support for in[0] and out [0]

  • Sigmoid, Softmax (Activation type: INT16)

    • Added dynamic dimensions support on width and channel for in[0] and out[0]

  • ReduceMean (Activation type: INT16)

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0] and out[0]

  • ReduceSum (Activation type: FP16, FP32)

    • Added rank 5d support for in[0] and out [0]

2.26.0

Quant

  • RmsNorm

    • Added QNN_DATATYPE_UFIXED_POINT_8 datatype support for in[0], in[1], in[2] and out[0]

  • UnPack

    • Added rank 5d support for in[0]

  • Conv2d, TransposeConv2d, Convert

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0] and out[0]

  • MaMul

    • Added rank 5d support for in[0], in[1] and out[0]

  • ElementWiseSelect

    • Added rank 5d support for in[0], in[1], in[2] and out[0]

  • Pad

    • Enabled EDGE scheme (QNN_OP_PAD_SCHEME_EDGE) to scheme parameter for FP16 and FP32

  • Transpose

    • Added constraint that dynamic dimensions are not supported for in[0] and out[0]

    • Added QNN_DATATYPE_BOOL_8 datatype support for in[0] to Transpose 5d

  • Tile

    • Added QNN_DATATYPE_BOOL_8 datatype support for in[0] and out[0]

  • Softmax

    • Added constraint that dynamic dimensions are not supported for in[0] and out[0]

2.25.0

Quant

  • Gather

    • Added constraint that input/output quant info must match for quantized models

  • GatherElements

    • Added QNN_DATATYPE_INT_32 datatype support for in[1]

  • ElementWiseMultiply, ElementWiseAdd, ElementWiseSub, ElementWisePow

    • Added rank 5d support for in[0], in[1] and out[0]

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for ElementWiseAdd, ElementWiseMultiply

  • ElementwiseRsqrt

    • Added rank 5d support for in[0] and out[0] to both float and quant Rsqrt

  • TopK

    • Added “largest” parameter and only support default value (true) currently

  • Relu

    • Added support to updateable quantization for in[0]

  • Conv2d, Matmul, FullyConnected

    • Added support to block quant weight tensors on V79 devices with the constraint that the block axis should be the second last axis of the weights

  • Prelu

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0], in[1] and out[0]

  • ResizeBilinear

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0] and out[0]

  • PoolMax2d

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0] and out[0]

  • ExtractPatches

    • Added QNN_DATATYPE_FLOAT_16 datatype support for in[0] and out[0]

2.24.0

Quant

  • HTP Backend Op Definition Supplement has been updated to enhance clarity on each supported datatype combination. Please refer to HTP Backend Op Definition Supplement for details.

  • Reshape

    • Added constraint that HTP currently does not support dynamic input/output

2.23.0

Quant

  • Cast

    • Added QNN_DATATYPE_INT_64 datatype support for in[0] and out[0]

  • Transpose

    • Added QNN_DATATYPE_BOOL_8 datatype support for in[0]

  • Relu

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0] and out[0]

  • Gelu

    • Added QNN_DATATYPE_SFIXED_POINT_8 datatype support for in[0] and out[0]

  • ElementwiseUnary

    • Added rank 5d support to Abs op for in[0] and out[0]

    • Added QNN_DATATYPE_SFIXED_POINT_8 datatype support to Abs op for in[0] and out[0]

2.22.0

Quant

  • ElementwiseUnary

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for in[0] and out[0] in RSQRT op

  • Conv3d, TransposeConv3d

    • Added QNN_QUANTIZATION_ENCODING_AXIS_SCALE_OFFSET support with only channel axis and the weights are expected to be signed and symmetrically quantized

  • Reshape

    • Added constraint that 0D tensors are not supported for in[0] and out[0]

  • ElementWiseAdd, ElementWiseAnd, ElementWiseDivide, ElementWiseNotEqual, ElementWiseMaximum, ElementWiseMinimum, ElementWiseMultiply, ElementWisePower, ElementWiseSquaredDifference, ElementWiseSubtract, ElementWiseEqual, ElementWiseGreater, ElementWiseGreaterEqual, ElementWiseLess, ElementWiseLessEqual, ElementWiseBinary, ElementWiseFloorDiv

    • Added constraint that 0D tensors are not supported for inputs and outputs

  • ElementWiseAbs

    • Added QNN_DATATYPE_SFIXED_POINT_8 datatype support for in[0] and out[0]

  • TopK

    • Removed the constraint that for UFIXED_POINT_16 inputs, only k less than or equal to 64 is supported

  • Transpose

    • Added constraint for in[0] of Tranpose 4D: QNN_DATATYPE_UFIXED_POINT_8, QNN_DATATYPE_SFIXED_POINT_8, QNN_DATATYPE_UFIXED_POINT_16, QNN_DATATYPE_SFIXED_POINT_16, QNN_DATATYPE_INT_32, QNN_DATATYPE_UINT_32 are supported

    • Added constraint for in[0] of Transpose 5D: QNN_DATATYPE_UFIXED_POINT_8, QNN_DATATYPE_SFIXED_POINT_8, QNN_DATATYPE_UFIXED_POINT_16, QNN_DATATYPE_FLOAT_16, QNN_DATATYPE_FLOAT_32 are supported

2.21.0

Quant

  • Dequantize

    • Added QNN_DATATYPE_FLOAT_16 datatype support for out[0]

  • ElementWiseBinary

    • Added QNN_DATATYPE_BOOL_8 support for all ElementWiseBinary:comparison ops for in[0]

    • Added QNN_DATATYPE_BOOL_8 support for all ElementWiseBinary:comparison ops for out[0]

2.20.0

Quant

  • ElementWiseXor, CreateSparse, GetSparseIndices, GetSparseValues, SparseToDense, ElementWiseNeuron

    • Added support

  • ElementWiseSin, ElementWiseCos

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for in[0], out[0]

  • Conv3d

    • Added default handling to ignore reuse_sparse_indicies parameter as HTP doesn’t support sparsity

  • Resize

    • Fixed constraint on nearest_mode parameter to match the behaviour in HTP core

  • Convert

    • Added QNN_DATATYPE_UINT_8 datatype support for in[0]

    • Added QNN_DATATYPE_BOOL_8 datatype support for out[0]

2.19.0

Quant

  • Gather

    • Added constraint to comunicate that HTP does not support negative indices

  • GatherElements

    • Added QNN_DATATYPE_INT_32 support for in[0], out[0]

  • BatchNorm, LayerNorm

    • Added constraint to support 16 bit data types for v73 or beyond architecture only

  • Convert

    • Added max supported rank to 5d for in[0] and out[0]

2.18.0

Quant

  • Conv2d, DepthWiseConv2d, TransposeConv2d, FullyConnected, MatMul

    • Added constraint to support 16 bit data types for v73 or beyond architecture only

  • GridSample

    • Added max supported rank to 5d for in[0], in[1] and out[0]

  • Lstm

    • Added rest input for in[24]

    • Added input rank constraint of 2 for in[0]

    • Added description of 2d input not applicable for time_major parameter

2.17.0

Quant

  • Quantize, Dequantize

    • Added max supported rank to 5d for in[0] and out[0]

  • EltwiseMul

    • Added overflow detect for input scales

2.16.0

Quant

  • Matmul

    • Corrected supported rank constraints back to 4D

  • Resize

    • Added constraint to match nearest_mode only supporting default value

  • ElementwiseOr support added

  • ElementWiseBinary

    • Enabled OR

    • Added QNN_DATATYPE_BOOL_8 support for in[0], in[1], out[0]

2.15.0

Quant

  • Split, ReduceMax, ReduceMean, ReduceMin, Convert, ElementWiseAbs

    • Added 5D constraints for inputs and outputs

  • ElementWiseBinary support added

  • Convert

    • Added QNN_DATATYPE_SFIXED_POINT_16 support for out[0]

  • Tile

    • Added QNN_DATATYPE_FLOAT_32, QNN_DATATYPE_INT_32 support for input and output

  • Batchnorm

    • Added QNN_DATATYPE_SFIXED_POINT_16 support for in[0] and in[1]

    • Added QNN_DATATYPE_UFIXED_POINT_16 support for in[1]

  • Conv2d, DepthWiseConv2d, TransposeConv2

    • Added QNN_DATATYPE_UFIXED_POINT_16 support for in[1]

  • ScatterNd

    • Added QNN_DATATYPE_BOOL_8 support for in[0], in[2] and out[2]

2.14.0

Quant

  • SpaceToDepth

    • Added operations, enabled QNN_DATATYPE_UINT_32 mode

  • ReduceSum

    • Added 5D constraints for inputs and outputs

  • LayerNorm

    • Added QNN_DATATYPE_SFIXED_POINT_16 support for in[1] and in[2]

  • ElementWiseSquaredDifference

    • Added QNN_DATATYPE_UFIXED_POINT_16, QNN_DATATYPE_SFIXED_POINT_16 support for in[0], in[1] and out[0]

  • FullyConnected, MatMul

    • Added QNN_DATATYPE_UFIXED_POINT_16 support for in[1]

  • ElementWiseRsqrt

    • Added QNN_DATATYPE_UFIXED_POINT_16 support for in[0] and out[0]

2.13.0

Quant

  • Convert

    • Added QNN_DATATYPE_BOOL_8 support for inputs

  • GroupNorm support added

  • ElementWiseRsqrt

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for in[0] and out[0]

2.12.0

Quant

  • ElementwiseUnary support added

  • Conv2d, DepthWiseConv2d, FullyConnected, MatMul, TransposeConv2d

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0]

    • Constraint added for in[1]: QNN_DATATYPE_SFIXED_POINT_16 Weight must have QNN_DATATYPE_UFIXED_POINT_16 Activation and must be symmetric quantized

  • RoiAlign

    • Added default support for new params: aligned and allow_invalid_roi

2.11.0

Quant

  • ElementWiseAsin, ExtractPatches, RoiAlign

    • Added operations

  • Resize

    • Removed transformation_mode parameter constraint to support Asymmetric Resize Mode

  • NonMaxSuppression

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for in[0] and in[1]

  • DetectionOutput, MultiClassNms

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for all inputs and out[0]

2.10.0

Quant

  • ElementWiseSin, ElementWiseCos, NonMaxSuppression

    • Added operations

  • Conv2d, DepthWiseConv2d, FullyConnected, MatMul, TransposeConv2d

    • Added QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[1]

  • TopK

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for in[0] and out[0]

  • Transpose

    • Added QNN_DATATYPE_BOOL_8 datatype support for in[0] and out[0]

  • Fully-connected, MatMul

    • Fixed Axis quantization validation

  • Conv2d, DepthWiseConv2d, FullyConnected, MatMul, TransposeConv2d

    • Removed static tensor check on INT8 axis quantized weights for in[1]

  • Conv2d, DepthWiseConv2d, TransposeConv2d

    • Allowed Non-Zero Bias Encoding with Per-Channel quantization parameters for in[2]

2.9.0

Quant

  • L2Norm, LogSoftmax

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for in[0] and out[0]

  • ElementWiseEqual, ElementWiseGreater, ElementWiseGreaterEqual, ElementWiseLess ElementWiseLessEqual, ElementWiseNotEqual

    • Added QNN_DATATYPE_UFIXED_POINT_16 and QNN_DATATYPE_SFIXED_POINT_16 datatype support for in[0] and in[1]

  • Conv2d, DepthWiseConv2d, FullyConnected, MatMul, TransposeConv2d

    • Added static tensor constraint for in[1]

  • MatMul

    • Added channel axis constraint for quantization parameters for in[1]

  • ElementWiseAdd, ElementWiseDivide, ElementWiseMaximum, ElementWiseMinimum, ElementWiseMultiply, ElementWiseSquaredDifference, ElementWiseSubtract, ElementWiseEqual, ElementWiseGreater, ElementWiseGreaterEqual, ElementWiseLess, ElementWiseLessEqual, ElementWiseNotEqual, ElementWiseSelect, Gather, GatherNd, MatMul, ScatterNd

    • Fixed rank constraint on incorrect input for in[1]

  • ElementWiseSelect, ScatterNd

    • Fixed rank constraint on incorrect input for in[2]

  • ElementWisePower, ExpandDims

    • Added 5D rank constraint for in[1]

  • OneHot

    • Added max rank constraint of 2 for in[0] and 3 for out[0]

  • ReluMinMax

    • Added max rank constraint of 5 for in[0] and out[0]

2.8.0

Quant

  • Cast, ElementWiseLog

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for inputs and outputs

  • ElementWiseGreaterEqual, ElementWiseLessEqual, ElementWiseNotEqual

    • Added QNN_DATATYPE_INT_32 datatype support for inputs

  • ReduceSum, TopK

    • Added QNN_DATATYPE_INT_32 datatype support for inputs and outputs

  • ElementWiseGreater, ElementWiseGreaterEqual, ElementWiseLess, ElementWiseLessEqual, ElementWiseNotEqual, ElementWisePower, ElementWiseSelect, GatherNd, Softmax

    • Added 5D constraints for inputs and outputs

  • ElementWiseSelect

    • Added QNN_DATATYPE_UFIXED_POINT_8, QNN_DATATYPE_SFIXED_POINT_8, QNN_DATATYPE_SFIXED_POINT_16, QNN_DATATYPE_UFIXED_POINT_16, QNN_DATATYPE_INT_32 datatype support for in[1]

2.7.0

Quant

  • Argmax

    • Added QNN_DATATYPE_UINT_32 datatype support for out[0]

  • Transpose

    • Added QNN_DATATYPE_UINT_32 datatype support for in[0] and out[0]

  • DepthWiseConv2d

    • Added support for dilation parameter with additional stride and kernel size

  • ElementWiseAdd, ElementWiseDivide, ElementWiseExp, ElementWiseMaximum, ElementWiseMinimum, ElementWiseMultiply, ElementWiseSquaredDifference, ElementWiseSubtract, MatMul, Pad, Relu, Sigmoid, Transpose

    • Fixed 5D constraints for out[0]

  • ElementWiseExp, ElementWiseFloor, ReduceMax, ReduceMin, ScatterNd, LayerNorm

    • Added QNN_DATATYPE_UFIXED_POINT_16 datatype support for all inputs and outputs

  • ElementWiseEqual, ElementWiseLess, ElementWiseSelect

    • Added QNN_DATATYPE_INT_32 datatype support for all inputs and outputs

  • ElementWiseGreater

    • Added QNN_DATATYPE_INT_32 datatype support for in[0] and in[1]

  • Reshape

    • Added QNN_DATATYPE_BOOL_8 datatype support for in[0] and out[0]

  • ScatterNd

    • Added 5D constraints for in[0], in[2] and out[0], and 6D constraint for in[1]

  • Softmax

    • Added QNN_DATATYPE_FLOAT_32 datatype support for out[0]

  • InstanceNorm

    • Added constraint to support in[0] with rank of less than 4D

  • Gather, GatherNd

    • Added support for in[1]

  • GatherNd

    • Added constraint for out[0] for input and output quantization check

  • FullyConnected, MatMul

    • Added constraints to support per-channel tensors

2.6.0

Quant

  • Cast

    • Added QNN_DATATYPE_BOOL_8 datatype support for out[0]

2.4.0

Quant

  • Resize support added

  • Gather

    • Constraint added for in[0] and out[0]: “Max Supported rank is 5”

2.3.0

Quant

  • Cast, ExpandDims, Squeeze

    • Added QNN_DATATYPE_UINT_8 support for in[0] and out[0]

  • GatherNd

    • Added QNN_DATATYPE_INT_32 support for in[0] and out[0]

  • MatMul

    • Added QNN_DATATYPE_UFIXED_POINT_8 and QNN_DATATYPE_SFIXED_POINT_32 support for in[2]

  • Pad, Relu

    • constraint added for in[0] and out[0]: “Max Supported rank is 5”

2.2.0

Quant

  • Gelu

    • Added QNN_DATATYPE_UFIXED_POINT_16 support for in[0] and out[0]

  • Reshape

    • Added QNN_DATATYPE_UINT_8 support for in[0] and out[0]

2.1.0

Quant

  • Concat, ElementWiseAdd, ElementWiseDivide, ElementWiseMaximum, ElementWiseMinimum, ElementWiseMultiply, ElementWiseSubtract, ElementWiseSquaredDifference, MatMul, Sigmoid, StridedSlice

    • support for rank 5 added

  • Gather

    • constraint added for out[0]: “Input quantization must be equal to output quantization”