File QnnHtpGraph.h¶

↰ Parent directory (include/QNN/HTP)

QNN HTP component Graph API.

Contents

Definition (include/QNN/HTP/QnnHtpGraph.h)
Detailed Description
Includes
Classes
Enums
Defines
Typedefs
Full File Listing

Definition (`include/QNN/HTP/QnnHtpGraph.h`)¶

Program Listing for File QnnHtpGraph.h

Detailed Description¶

The interfaces in this file work with the top level QNN API and supplements QnnGraph.h for HTP backend

Includes¶

QnnGraph.h (File QnnGraph.h)

Classes¶

Enums¶

Defines¶

Typedefs¶

Typedef QnnHtp_HmxBoundingInfo_t

Full File Listing¶

QNN HTP component Graph API.

   The interfaces in this file work with the top level QNN
   API and supplements QnnGraph.h for HTP backend

Defines

QNN_HTP_GRAPH_CONFIG_OPTION_MAX 0

QnnHtpGraph config value macro. Represents to use the maximum available number of the resource.

Currently only applicable for QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE.

QNN_HTP_GRAPH_OPTIMIZATION_OPTION_INIT {                                                         \     QNN_HTP_GRAPH_OPTIMIZATION_TYPE_UNKNOWN , /*type*/       \ 0.0f                                     /*floatValue*/ \ }: QnnHtpGraph_OptimizationOption_t initializer macro.

QNN_HTP_GRAPH_CUSTOM_CONFIG_INIT {                                                                 \     QNN_HTP_GRAPH_CONFIG_OPTION_UNKNOWN , /*option*/                 \ {                                                               \ QNN_HTP_GRAPH_OPTIMIZATION_OPTION_INIT /*optimizationOption*/ \ }                                                               \ }: QnnHtpGraph_CustomConfig_t initializer macro.

Typedefs

typedef struct QnnHtp_HmxBoundingInfo QnnHtp_HmxBoundingInfo_t¶

The settings in this struct is only applicable for DSP architectures >= V81. Use on other SOCs will return an error.

Values will be defaulted to their SOC’s TURBO frequency (SOC as identified by Qnn_DeviceHandle_t).

On automotive SDKs HMX OP Bounding will be enabled by default.

On non-automotive SDKs using this setting will enable HMX OP Bounding. It is off by default.

Enums

enum QnnHtpGraph_OptimizationType_t¶

This enum provides different HTP graph optimization options that can be used to finalize the graph for optimum performance.

Values:

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_SCHEDULE_THRESHOLD = 1¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_FINALIZE_RETRIES = 2¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_FINALIZE_OPTIMIZATION_FLAG = 3¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_DLBC = 4¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_DLBC_WEIGHTS = 5¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_SPARSE_WEIGHTS_COMPRESSION = 6¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_SLC_ALLOCATOR = 7¶

enumerator QNN_HTP_GRAPH_OPTIMIZATION_TYPE_UNKNOWN = 0x7fffffff¶

enum QnnHtpGraph_ConcurrencyOption_t¶

This enum defines whether graph concurrency (i.e. multiple graphs running concurrently) is possible, and how to behave when circumstances for concurrency aren’t possible.

Values:

enumerator QNN_HTP_GRAPH_CONCURRENCY_OPTION_NONE = 0¶: This graph will not be able to run concurrently with other graphs.

enumerator QNN_HTP_GRAPH_CONCURRENCY_OPTION_DEFAULT = QNN_HTP_GRAPH_CONCURRENCY_OPTION_NONE ¶

enumerator QNN_HTP_GRAPH_CONCURRENCY_OPTION_ALL_SHARED = 1¶: Graph will try to run concurrently, sharing all resources on the DSP (VTCM, HMX, HVX, etc).

enumerator QNN_HTP_GRAPH_CONCURRENCY_OPTION_UNKNOWN = 0x7fffffff¶

enum QnnHtpGraph_ConfigOption_t¶

This enum provides different HTP graph configuration options associated with QnnGraph.

Values:

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_OPTIMIZATION = 1¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_PRECISION = 2¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE_IN_MB = 3¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE = QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE_IN_MB ¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_FOLD_RELU_ACTIVATION_INTO_CONV_OFF = 4¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_SHORT_DEPTH_CONV_ON_HMX_OFF = 5¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_NUM_HVX_THREADS = 6¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_FINALIZE_CONFIG = 7¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORES = 8¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_PARALLEL_GRAPH_EXECUTION_CONFIG = 9¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE_IN_BYTES = 10¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_HMX_BOUNDING = 11¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_WEIGHTS_PACKING = 12¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_ASSUME_SAME_QUANT = 13¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_SHARE_IO_BUFFER = 14¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_RESERVED = 0x7fff0000¶

enumerator QNN_HTP_GRAPH_CONFIG_OPTION_UNKNOWN = 0x7fffffff¶

struct QnnHtpGraph_OptimizationOption_t¶

#include <QnnHtpGraph.h>

Struct describing the set of optimization types and the values associated with each optimization type.

Below is the Map between QnnHtpGraph_OptimizationType_t and allowable values:

#	OptimizationType option	Allowable values
1	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_SCHEDULE_THRESHOLD	Reserved
2	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_FINALIZE_RETRIES	Reserved
3	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_FINALIZE_OPTIMIZATION_FLAG	Defines the optimization strategy used by the HTP backend
4	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_DLBC	Reserved
5	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_DLBC_WEIGHTS	Enables DLBC weights compression
6	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_SPARSE_WEIGHTS_COMPRESSION	Enables Weight Sparsity Compression
7	QNN_HTP_GRAPH_OPTIMIZATION_TYPE_ENABLE_SLC_ALLOCATOR	Enables System Level Cache Allocator usage

Public Members

QnnHtpGraph_OptimizationType_t type¶

float floatValue¶

struct QnnHtpGraph_VtcmConfig_t¶

#include <QnnHtpGraph.h>

This struct encapsulates all the VTCM configurations for parallel graph execution.

|<--           (1) 8MB Total Hardware VTCM           -->|
       |<--            (2) 7MB Addressable           -->|
+------+------+------+------+------+------+------+------+
|  CV  |      |      |      |      |      |      |      |
+------+------+------+------+------+------+------+------+
       |<-- (4) Graph A  -->|<--     (4) Graph B     -->|

     A |> 0 MB      (3) Graph Offset
     B |-------------------> 3 MB

Public Members

uint32_t sizeInBytes¶: (4) above, the amount of VTCM used by a graph

uint32_t offsetInBytes¶: (3) above, where in the addressable region to start VTCM. Note: (3) + (4) <= (2)

uint32_t sizeTotalInBytes¶: (2) Addressable portion of VTCM. Set to less than hardware size so Graph(s) can coexist with other VTCM clients.

uint32_t reserved[3]¶

struct QnnHtpGraph_ParallelGraphExecutionConfig_t¶

#include <QnnHtpGraph.h>

This struct encapsulates all the configurations for parallel graph execution.

Public Members

QnnHtpGraph_ConcurrencyOption_t concurrency¶

QnnHtpGraph_VtcmConfig_t vtcmConfig¶

uint32_t reserved[4]¶

struct QnnHtp_HmxBoundingInfo¶

#include <QnnHtpGraph.h>

The settings in this struct is only applicable for DSP architectures >= V81. Use on other SOCs will return an error.

Values will be defaulted to their SOC’s TURBO frequency (SOC as identified by Qnn_DeviceHandle_t).

On automotive SDKs HMX OP Bounding will be enabled by default.

On non-automotive SDKs using this setting will enable HMX OP Bounding. It is off by default.

Public Members

float targetHmxFreqHz¶: Target HMX freq in Hz. Can be derived from sysMonApp (HexagonSDK) or QProfiler.

float targetDspCoreFreq¶: Target DSP Core freq in Hz. Can be derived from sysMonApp (HexagonSDK) or QProfiler.

struct QnnHtpGraph_FinalizeConfig_t¶

#include <QnnHtpGraph.h>

A struct for different config parameters in a key value format.

Public Members

const char *key¶

Qnn_Scalar_t value¶

struct QnnHtpGraph_CustomConfig_t¶

#include <QnnHtpGraph.h>

Structure describing the set of configurations supported by graph. Objects of this type are to be referenced through QnnGraph_CustomConfig_t.

The struct has two fields - option and a union of corresponding config values Based on the option corresponding item in the union can be used to specify config.

Below is the Map between QnnHtpGraph_ConfigOption_t and config value

+—-+————————————————————————————-+————————————————+ | 2 | QNN_HTP_GRAPH_CONFIG_OPTION_PRECISION | Qnn_Precision_t | +—-+————————————————————————————-+————————————————+ | 3 |

QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE_IN_MB/QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE | uint32_t |

8 | QNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORES | uint32_t |

QnnHtpGraph_ParallelGraphExecutionConfig_t |

reserved for internal purposes |

NOTE: Option #6 (i.e. QNN_HTP_GRAPH_CONFIG_OPTION_NUM_HVX_THREADS), can only be set prior to the first execution of the graph. Proceeding executions will not use the updated value if user does change it after the first execution.

Public Members