Struct TfLiteQnnDelegateOptions

Struct Documentation

struct TfLiteQnnDelegateOptions

Public Members

TfLiteQnnDelegateBackendType backend_type

The backend QNN library to open and execute the graph with. This is a required argument and will error out if kUndefinedBackend is supplied.

const char *library_path

Optional parameter to override the QNN backend library.

const char *skel_library_dir

Optional parameter specifying the directory of QNN Skel library. Only useful for backends which have a Skel library.

TfLiteQnnDelegateGpuBackendOptions gpu_options

Optional backend specific options for the GPU backend. Only used when selecting TfLiteQnnDelegateBackendType::kGpuBackend, otherwise will be ignored.

TfLiteQnnDelegateHtpBackendOptions htp_options

Optional backend specific options for the HTP backend. Only used when selecting TfLiteQnnDelegateBackendType::kHtpBackend, otherwise will be ignored.

TfLiteQnnDelegateDspBackendOptions dsp_options

Optional backend specific options for the DSP backend. Only used when selecting TfLiteQnnDelegateBackendType::kDspBackend, otherwise will be ignored.

TfLiteQnnDelegateIrBackendOptions ir_options

Optional backend specific options for the IR backend. Only used when selecting TfLiteQnnDelegateBackendType::kIrBackend, otherwise will be ignored.

TfLiteQnnDelegateLogLevel log_level

Logging level of the delegate and the backend. Default is off.

TfLiteQnnDelegateProfilingOptions profiling

Option to enable profiling with the delegate. Default is off.

TfLiteQnnDelegateOpPackageOptions op_package_options

Optional structure to specify op packages loaded and used by the backend.

const char *tensor_dump_output_path

Tensor dump output path. If a path is given, Delegate will write outputs of each OP there. We don’t recommend using this option. It exists only for debugging accuracy issues.

const char *cache_dir

Specifies the directory of a compiled model. Signals intent to either:

  • Save the model if the file doesn’t exist, or

  • Restore model from the file.

Model Cache specific options. Only used when setting model_token, otherwise will be ignored.

We don’t recommend that delegate instances with/without cache be mixed in same process, unless an instance without cache is initialized, invoked, and terminated before an instance with cache is used in order to make sure all resources are prepared correctly.

TfLiteDelegate* delegate_wo_cache =
TfLiteQnnDelegateCreate(&options_wo_cache);
interpreter_0->ModifyGraphWithDelegate(delegate_wo_cache);

// Perform inference with interpreter_0

TfLiteQnnDelegateDelete(delegate_wo_cache);

// after this, another delegate_with_cache can be used in the same
// process, though not recommended at this moment.
TfLiteDelegate* delegate_with_cache =
TfLiteQnnDelegateCreate(&options_with_cache);

// another interpreter
interpreter_1->ModifyGraphWithDelegate(delegate_with_cache);

// more delegates...etc.

const char *model_token

The unique null-terminated token string that acts as a ‘namespace’ for all serialization entries. Should be unique to a particular model (graph & constants). For an example of how to generate this from a TFLite model, see StrFingerprint() in lite/delegates/serialization.h.

Model Cache specific options. Only used when setting cache_dir, otherwise will be ignored.

TfLiteQnnDelegateSkipOption skip_options

Option to skip node by specifying node types or node ids.

TfLiteQnnDelegateGraphPriority graph_priority

Option to set graph priority.