Struct TfLiteQnnDelegateHtpBackendOptions

Struct Documentation

struct TfLiteQnnDelegateHtpBackendOptions

Specifies the backend options for the HTP backend. To be used when selecting TfLiteQnnDelegateBackendType::kGpuBackend for the TfLiteQnnDelegateOptions::backend_type.

Public Members

TfLiteQnnDelegateHtpPerformanceMode performance_mode

The default performance mode sets no configurations on the HTP.

TfLiteQnnDelegateHtpPerfCtrlStrategy perf_ctrl_strategy

The default performance control strategy is Manual.

TfLiteQnnDelegateHtpPrecision precision

The default precision mode supports quantized networks. Other precision modes may only be supported on certain SoCs.

TfLiteQnnDelegateHtpPdSession pd_session

Signed or unsigned HTP PD session. The default PD session is unsigned.

TfLiteQnnDelegateHtpOptimizationStrategy optimization_strategy

The default optimization strategy will optimize the graph for inference.

bool useConvHmx

When using short conv hmx, one might have better performance, but convolution that have short depth and/or weights that are not symmetric could exhibit inaccurate results.

bool useFoldRelu

When using fold relu, one might have better performance. This optimization is correct when quantization ranges for convolution are equal to or are subset of the Relu operation.

uint32_t vtcm_size

Option to set VTCM size in MB. This is directly mapped to QNN_HTP_GRAPH_CONFIG_OPTION_VTCM_SIZE under QnnHtpGraph_ConfigOption_t. If VTCM size is set to 0, the default VTCM size will be used. If VTCM size is greater than VTCM size available for this device, it will be set to the maximum VTCM size for this device.

uint32_t num_hvx_threads

Option to set number of HVX threads. This is directly mapped to QNN_HTP_GRAPH_CONFIG_OPTION_NUM_HVX_THREADS under QnnHtpGraph_ConfigOption_t. If this this option is set to 0, the default number of HVX threads will be used. If input exceeds the max number of HVX threads, the maximum number of threads supported will be used.

uint32_t device_id

Some SoCs come with more than 1 HTP device. You can set which HTP device you want to run the model on by this attribute. But in most cases, you can just use the default device_id.