File QnnContext.h¶

    Requires Backend to be initialized.
    Graphs and Tensors are created within Context.
    Context content once created can be cached into a binary form.

Defines

QNN_CONTEXT_ASYNC_EXECUTION_QUEUE_DEPTH_INIT {                                                                  \     QNN_CONTEXT_ASYNC_EXECUTION_QUEUE_DEPTH_TYPE_UNDEFINED , /*type*/ \ {                                                                \ 0 /*depth*/                                                    \ }                                                                \ }: QnnContext_AsyncExecutionQueueDepth_t initializer macro.

QNN_CONTEXT_CONFIG_INIT {                                          \     QNN_CONTEXT_CONFIG_UNDEFINED , /*option*/ \ {                                        \ NULL /*customConfig*/                  \ }                                        \ }: QnnContext_Config_t initializer macro.

QNN_CONTEXT_PROPERTY_INIT {                                                   \     QNN_CONTEXT_PROPERTY_OPTION_UNDEFINED , /*option*/ \ {                                                 \ NULL /*customProperty*/                         \ }                                                 \ }: QnnContext_Property_t initializer macro.

Typedefs

typedef void *QnnContext_CustomConfig_t¶

Context specific object for custom configuration.

Please refer to documentation provided by the backend for usage information

typedef void (*QnnContext_createFromBinaryNotifyFn_t)(Qnn_ContextHandle_t context, Qnn_GraphHandle_t graph, const char *graphName, QnnContext_createFromBinaryAsyncNotifyType_t notifyType, void *notifyParam, Qnn_ErrorHandle_t status)¶

A client-defined callback function.

Param context: [in] handle to a created context
Param graph: [in] handle to a created graph
Param graphName: [in] created graph’s name
Param notifyType: [in] enum type indicating whether a context or a graph init is complete
Param notifyParam: [in] Client supplied data object which may be used to identify which function this callback applies to.
Param status: [in] graph or context initialization result
Return: None

typedef void *QnnContext_CustomProperty_t¶

Context specific object for custom property.

Please refer to documentation provided by the backend for usage information

Enums

enum QnnContext_Error_t

QNN Context API result / error codes.

Values:

enumerator QNN_CONTEXT_MIN_ERROR = 5000

enumerator QNN_CONTEXT_NO_ERROR = 0: Qnn context success.

enumerator QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE = QNN_COMMON_ERROR_NOT_SUPPORTED: There is optional API component that is not supported yet. See QnnProperty.

enumerator QNN_CONTEXT_ERROR_MEM_ALLOC = QNN_COMMON_ERROR_MEM_ALLOC: Context-specific memory allocation/deallocation failure.

enumerator QNN_CONTEXT_ERROR_INVALID_ARGUMENT = 5000 + 0: An argument to QNN context API is deemed invalid by a backend.

enumerator QNN_CONTEXT_ERROR_CTX_DOES_NOT_EXIST = 5000 + 1: A QNN context has not yet been created in the backend.

enumerator QNN_CONTEXT_ERROR_INVALID_HANDLE = 5000 + 2: Invalid/NULL QNN context handle.

enumerator QNN_CONTEXT_ERROR_NOT_FINALIZED = 5000 + 3: Attempting an operation when graphs in a context haven’t been finalized.

enumerator QNN_CONTEXT_ERROR_BINARY_VERSION = 5000 + 4: Attempt to access context binary with an incompatible version.

enumerator QNN_CONTEXT_ERROR_CREATE_FROM_BINARY = 5000 + 5: Failure to create context from binary.

enumerator QNN_CONTEXT_ERROR_GET_BINARY_SIZE_FAILED = 5000 + 6: Failure to get size of a QNN serialized context.

enumerator QNN_CONTEXT_ERROR_GET_BINARY_FAILED = 5000 + 7: Failure to generate a QNN serialized context.

enumerator QNN_CONTEXT_ERROR_BINARY_CONFIGURATION = 5000 + 8: Invalid context binary configuration.

enumerator QNN_CONTEXT_ERROR_SET_PROFILE = 5000 + 9: Failure to set profile.

enumerator QNN_CONTEXT_ERROR_INVALID_CONFIG = 5000 + 10: Invalid config.

enumerator QNN_CONTEXT_ERROR_BINARY_SUBOPTIMAL = 5000 + 11: Attempt to create a context from suboptimal binary.

enumerator QNN_CONTEXT_ERROR_ABORTED = 5000 + 12: Call aborted early due to a QnnSignal_trigger call issued to the observed signal object.

enumerator QNN_CONTEXT_ERROR_TIMED_OUT = 5000 + 13: Call aborted early due to a QnnSignal timeout.

enumerator QNN_CONTEXT_MAX_ERROR = 5999

enumerator QNN_CONTEXT_ERROR_UNDEFINED = 0x7FFFFFFF

enum QnnContext_ConfigOption_t¶

This enum defines context config options.

Values:

enumerator QNN_CONTEXT_CONFIG_OPTION_CUSTOM = 0¶: Sets context custom options via QnnContext_CustomConfig_t.

enumerator QNN_CONTEXT_CONFIG_OPTION_PRIORITY = 1¶: Sets the default priority for graphs in this context. QNN_GRAPH_CONFIG_OPTION_PRIORITY can be used to override this default.

enumerator QNN_CONTEXT_CONFIG_OPTION_ERROR_REPORTING = 2¶: Sets the error reporting level.

enumerator QNN_CONTEXT_CONFIG_OPTION_OEM_STRING = 3¶: Sets the string used for custom oem functionality. This config option is DEPRECATED.

enumerator QNN_CONTEXT_CONFIG_ASYNC_EXECUTION_QUEUE_DEPTH = 4¶: Sets async execution queue depth for all graphs in this context. This option represents the number of executions that can be in the queue at a given time before QnnGraph_executeAsync() will start blocking until a new spot is available. Queue depth is subject to a maximum limit determined by the backend and available system resources. The default depth is backend-specific, refer to SDK documentation.

enumerator QNN_CONTEXT_CONFIG_ENABLE_GRAPHS = 5¶: Null terminated array of null terminated strings listing the names of the graphs to deserialize from a context binary. All graphs are enabled by default. An error is generated if an invalid graph name is provided.

enumerator QNN_CONTEXT_CONFIG_MEMORY_LIMIT_HINT = 6¶: Sets the peak memory limit hint of a deserialized context in megabytes.

enumerator QNN_CONTEXT_CONFIG_PERSISTENT_BINARY = 7¶: Indicates that the context binary pointer is available during QnnContext_createFromBinary and until QnnContext_free is called.

enumerator QNN_CONTEXT_CONFIG_BINARY_COMPATIBILITY = 8¶: Sets the context binary check type when reading binary caches.

enumerator QNN_CONTEXT_CONFIG_UNDEFINED = 0x7FFFFFFF¶

enum QnnContext_BinaryCompatibilityType_t¶

Values:

enumerator QNN_CONTEXT_BINARY_COMPATIBILITY_PERMISSIVE = 0¶: A binary cache is compatible if it could run on the device. This is the default.

enumerator QNN_CONTEXT_BINARY_COMPATIBILITY_STRICT = 1¶: A binary cache is compatible if it could run on the device and fully utilize hardware capability, otherwise QnnContext_CreateFromBinary may return QNN_CONTEXT_ERROR_BINARY_SUBOPTIMAL.

enumerator QNN_CONTEXT_BINARY_COMPATIBILITY_TYPE_UNDEFINED = 0x7FFFFFF¶

enum QnnContext_AsyncExecutionQueueDepthType_t¶

Values:

enumerator QNN_CONTEXT_ASYNC_EXECUTION_QUEUE_DEPTH_TYPE_NUMERIC = 0¶: Sets a numeric value for the maximum queue depth.

enumerator QNN_CONTEXT_ASYNC_EXECUTION_QUEUE_DEPTH_TYPE_UNDEFINED = 0x7FFFFFF¶

enum QnnContext_createFromBinaryAsyncNotifyType_t¶

Enum to distinguish notify type.

Values:

enumerator QNN_CONTEXT_NOTIFY_TYPE_GRAPH_INIT = 0¶

enumerator QNN_CONTEXT_NOTIFY_TYPE_CONTEXT_INIT = 1¶

enumerator QNN_CONTEXT_NOTIFY_TYPE_UNDEFINED = 0x7FFFFFF¶: Unused, present to ensure 32 bits.

enum QnnContext_ParamsVersion_t¶

Enum to distinguish various context params definitions.

Values:

enumerator QNN_CONTEXT_PARAMS_VERSION_1 = 1¶

enumerator QNN_CONTEXT_PARAMS_VERSION_UNDEFINED = 0x7FFFFFFF¶: Unused, present to ensure 32 bits.

enum QnnContext_SectionType_t¶

Enum to distinguish type of binary section to retrieve.

Values:

enumerator QNN_CONTEXT_SECTION_UPDATABLE = 1¶: Portion of the context binary containing recent updates applied through QnnTensor_updateGraphTensors() or QnnTensor_updateContextTensors()

enumerator QNN_CONTEXT_SECTION_UNDEFINED = 0x7FFFFFFF¶: Unused, present to ensure 32 bits.

enum QnnContext_MemType_t¶

An enum specifying memory types of context binary data.

Values:

enumerator QNN_CONTEXTMEMTYPE_RAW = 0¶: Raw memory pointer.

enumerator QNN_CONTEXTMEMTYPE_MEMHANDLE = 1¶: Memory object, provide capability for memory sharing in between QNN accelerator backends.

enumerator QNN_CONTEXTMEMTYPE_UNDEFINED = 0x7FFFFFFF¶

enum QnnContext_BufferVersion_t¶

Enum to distinguish various context params definitions.

Values:

enumerator QNN_CONTEXT_BUFFER_VERSION_1 = 1¶

enumerator QNN_CONTEXT_BUFFER_VERSION_UNDEFINED = 0x7FFFFFFF¶: Unused, present to ensure 32 bits.

enum QnnContext_PropertyOption_t¶

This enum defines context property options.

Values:

enumerator QNN_CONTEXT_PROPERTY_OPTION_CUSTOM = 0¶: Gets context custom properties, see backend specific documentation.

enumerator QNN_CONTEXT_PROPERTY_OPTION_UNDEFINED = 0x7FFFFFFF¶: Value selected to ensure 32 bits.

Functions

Qnn_ErrorHandle_t QnnContext_create(Qnn_BackendHandle_t backend, Qnn_DeviceHandle_t device, const QnnContext_Config_t **config, Qnn_ContextHandle_t *context)¶

A function to create a context. Context holds graphs, operations and tensors.

Parameters

backend – [in] A backend handle.
device – [in] A device handle to set hardware affinity for the created context. NULL value can be supplied for device handle and it is equivalent to calling QnnDevice_create() with NULL config.
config – [in] Pointer to a NULL terminated array of config option pointers. NULL is allowed and indicates no config options are provided. All config options have default value, in case not provided. If same config option type is provided multiple times, the last option value will be used.
context – [out] A handle to the created context.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: at least one argument is invalid
QNN_CONTEXT_ERROR_MEM_ALLOC: failure in allocating memory when creating context
QNN_CONTEXT_ERROR_INVALID_HANDLE: backend or device is not a valid handle
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: an optional feature is not supported
QNN_CONTEXT_ERROR_INVALID_CONFIG: one or more config values is invalid
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION: SSR occurence (successful recovery)
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION_FATAL: SSR occurence (unsuccessful recovery)

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_setConfig(Qnn_ContextHandle_t context, const QnnContext_Config_t **config)¶

A function to set/modify configuration options on an already generated context. Backends are not required to support this API.

Parameters

context – [in] A context handle.
config – [in] Pointer to a NULL terminated array of config option pointers. NULL is allowed and indicates no config options are provided. All config options have default value, in case not provided. If same config option type is provided multiple times, the last option value will be used. If a backend cannot support all provided configs it will fail.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_INVALID_HANDLE: context is not a valid handle
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: at least one config option is invalid
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: an optional feature is not supported
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION: SSR occurence (successful recovery)
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION_FATAL: SSR occurence (unsuccessful recovery)

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_getBinarySize(Qnn_ContextHandle_t context, Qnn_ContextBinarySize_t *binaryBufferSize)¶

A function to get the size of memory to be allocated to hold the context content in binary (serialized) form. This function must be called after all entities in the context have been finalized.

Parameters

context – [in] A context handle.
binaryBufferSize – [out] The amount of memory in bytes a client will need to allocate to hold context content in binary form.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_HANDLE: context is not a valid handle
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: binaryBufferSize is NULL
QNN_CONTEXT_ERROR_NOT_FINALIZED: if there were any non-finalized entities in the context
QNN_CONTEXT_ERROR_GET_BINARY_SIZE_FAILED: Operation failure due to other factors
QNN_COMMON_ERROR_OPERATION_NOT_PERMITTED: Attempting to get binary size for a context re-created from a cached binary.
QNN_CONTEXT_ERROR_MEM_ALLOC: Not enough memory is available to retrieve the context content.

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_getBinary(Qnn_ContextHandle_t context, void *binaryBuffer, Qnn_ContextBinarySize_t binaryBufferSize, Qnn_ContextBinarySize_t *writtenBufferSize)¶

A function to get the context content in binary (serialized) form. The binary can be used to re-create context by using QnnContext_createFromBinary(). This function must be called after all entities in the context have been finalized. Unconsumed tensors are not included in the binary. Client is responsible for allocating sufficient and valid memory to hold serialized context content produced by this method. It is recommended the user calls QnnContext_getBinarySize() to allocate a buffer of sufficient space to hold the binary.

Parameters

context – [in] A context handle.
binaryBuffer – [in] Pointer to the user-allocated context binary memory.
binaryBufferSize – [in] Size of binaryBuffer to populate context binary with, in bytes.
writtenBufferSize – [out] Amount of memory actually written into binaryBuffer, in bytes.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_HANDLE: context is not a valid handle
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: one of the arguments to the API is invalid/NULL
QNN_CONTEXT_ERROR_NOT_FINALIZED: if there were any non-finalized entities in the context
QNN_CONTEXT_ERROR_GET_BINARY_FAILED: Operation failure due to other factors
QNN_COMMON_ERROR_OPERATION_NOT_PERMITTED: Attempting to get binary for a context re-created from a cached binary.
QNN_CONTEXT_ERROR_MEM_ALLOC: Not enough memory is available to retrieve the context content.

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_validateBinary(Qnn_BackendHandle_t backend, Qnn_DeviceHandle_t device, const QnnContext_Config_t **config, const void *binaryBuffer, Qnn_ContextBinarySize_t binaryBufferSize)¶

A function to validate a stored binary. The binary was previously obtained via QnnContext_getBinary() and stored by a client.

Parameters

backend – [in] A backend handle.
device – [in] A device handle to set hardware affinity for the created context. NULL value can be supplied for device handle and it is equivalent to calling QnnDevice_create() with NULL config.
config – [in] Pointer to a NULL terminated array of config option pointers. NULL is allowed and indicates no config options are provided. In case they are not provided, all config options have a default value in accordance with the serialized context. If the same config option type is provided multiple times, the last option value will be used.
binaryBuffer – [in] A pointer to the context binary.
binaryBufferSize – [in] Holds the size of the context binary.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: binaryBuffer is NULL
QNN_CONTEXT_ERROR_MEM_ALLOC: memory allocation error while validating binary cache
QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: failed to validate binary cache
QNN_CONTEXT_ERROR_BINARY_VERSION: incompatible version of the binary
QNN_CONTEXT_ERROR_BINARY_CONFIGURATION: binary is not configured for this device
QNN_CONTEXT_ERROR_BINARY_SUBOPTIMAL: suboptimal binary is used when QNN_CONTEXT_BINARY_COMPATIBILITY_STRICT is specified in the config option
QNN_CONTEXT_ERROR_INVALID_HANDLE: backend, or device is not a valid handle
QNN_CONTEXT_ERROR_INVALID_CONFIG: one or more config values is invalid

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_createFromBinary(Qnn_BackendHandle_t backend, Qnn_DeviceHandle_t device, const QnnContext_Config_t **config, const void *binaryBuffer, Qnn_ContextBinarySize_t binaryBufferSize, Qnn_ContextHandle_t *context, Qnn_ProfileHandle_t profile)¶

A function to create a context from a stored binary. The binary was previously obtained via QnnContext_getBinary() and stored by a client. The content of a context created in this way cannot be further altered, meaning no new nodes or tensors can be added to the context. Creating context by deserializing provided binary is meant for fast content creation, ready to execute on.

Parameters

backend – [in] A backend handle.
device – [in] A device handle to set hardware affinity for the created context. NULL value can be supplied for device handle and it is equivalent to calling QnnDevice_create() with NULL config.
config – [in] Pointer to a NULL terminated array of config option pointers. NULL is allowed and indicates no config options are provided. In case they are not provided, all config options have a default value in accordance with the serialized context. If the same config option type is provided multiple times, the last option value will be used.
binaryBuffer – [in] A pointer to the context binary.
binaryBufferSize – [in] Holds the size of the context binary.
context – [out] A handle to the created context.
profile – [in] The profile handle on which metrics are populated and can be queried. Use NULL handle to disable profile collection. A handle being re-used would reset and is populated with values from the current call.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: binaryBuffer or context is NULL
QNN_CONTEXT_ERROR_MEM_ALLOC: memory allocation error while creating context
QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: failed to deserialize binary and create context from it
QNN_CONTEXT_ERROR_BINARY_VERSION: incompatible version of the binary
QNN_CONTEXT_ERROR_BINARY_CONFIGURATION: binary is not configured for this device
QNN_CONTEXT_ERROR_BINARY_SUBOPTIMAL: suboptimal binary is used when QNN_CONTEXT_BINARY_COMPATIBILITY_STRICT is specified in the config option
QNN_CONTEXT_ERROR_SET_PROFILE: failed to set profiling info
QNN_CONTEXT_ERROR_INVALID_HANDLE: backend, __profile_, or device is not a valid handle
QNN_CONTEXT_ERROR_INVALID_CONFIG: one or more config values is invalid
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION: SSR occurence (successful recovery)
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION_FATAL: SSR occurence (unsuccessful recovery)

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_createFromBinaryWithSignal(Qnn_BackendHandle_t backend, Qnn_DeviceHandle_t device, const QnnContext_Config_t **config, const void *binaryBuffer, Qnn_ContextBinarySize_t binaryBufferSize, Qnn_ContextHandle_t *context, Qnn_ProfileHandle_t profile, Qnn_SignalHandle_t signal)¶

A function to create a context from a stored binary, which supports control signals. The binary was previously obtained via QnnContext_getBinary() and stored by a client. The content of a context created in this way cannot be further altered, meaning no new nodes or tensors can be added to the context. Creating context by deserializing provided binary is meant for fast content creation, ready to execute on.

Parameters

backend – [in] A backend handle.
device – [in] A device handle to set hardware affinity for the created context. NULL value can be supplied for device handle and it is equivalent to calling QnnDevice_create() with NULL config.
config – [in] Pointer to a NULL terminated array of config option pointers. NULL is allowed and indicates no config options are provided. In case they are not provided, all config options have a default value in accordance with the serialized context. If the same config option type is provided multiple times, the last option value will be used.
binaryBuffer – [in] A pointer to the context binary.
binaryBufferSize – [in] Holds the size of the context binary.
context – [out] A handle to the created context.
profile – [in] The profile handle on which metrics are populated and can be queried. Use NULL handle to disable profile collection. A handle being re-used would reset and is populated with values from the current call.
signal – [in] Signal object to control the execution of the create context from binary process. NULL may be passed to indicate that no execution control is requested, and the create operation should continue to completion uninterrupted. The signal object, if not NULL, is considered to be in-use for the duration of the call.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: binaryBuffer or context is NULL
QNN_CONTEXT_ERROR_MEM_ALLOC: memory allocation error while creating context
QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: failed to deserialize binary and create context from it
QNN_CONTEXT_ERROR_BINARY_VERSION: incompatible version of the binary
QNN_CONTEXT_ERROR_BINARY_CONFIGURATION: binary is not configured for this device
QNN_CONTEXT_ERROR_BINARY_SUBOPTIMAL: suboptimal binary is used when QNN_CONTEXT_BINARY_COMPATIBILITY_STRICT is specified in the config option
QNN_CONTEXT_ERROR_SET_PROFILE: failed to set profiling info
QNN_CONTEXT_ERROR_INVALID_HANDLE: backend, __profile_, or device is not a valid handle
QNN_CONTEXT_ERROR_INVALID_CONFIG: one or more config values is invalid
QNN_CONTEXT_ERROR_ABORTED: the call is aborted before completion due to user cancellation
QNN_CONTEXT_ERROR_TIMED_OUT: the call is aborted before completion due to a timeout

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_createFromBinaryListAsync(Qnn_BackendHandle_t backend, Qnn_DeviceHandle_t device, const QnnContext_Params_t **contextParams, const QnnContext_Config_t **listConfig, Qnn_SignalHandle_t signal)¶

The purpose of this function is to asynchronously create multiple contexts from binaries in a single API call. The API can be used with QnnSignal. When the function is invoked, the deserialization/initialization of each context will occur in the background. As soon as a graph reaches an executable state (i.e., initialization is completed), the client will receive a notification via the specified notification function. Once notification is received, the client can proceed to execute the graph. If a context contains multiple graphs, there will be multiple callbacks through the notification function.

Parameters

backend – [in] A backend handle.
device – [in] A device handle used to set hardware affinity for the created context. A NULL value can be supplied for the device handle, which is equivalent to calling QnnDevice_create() with a NULL config.
contextParams – [in] Pointer to a NULL-terminated array of context parameters.
listConfig – [in] Config pointer to a NULL-terminated array of config option pointers that apply to all contexts in the list. NULL is allowed and indicates that no config options are provided. If not provided, all config options have default values consistent with the serialized context. If the same config option type is provided multiple times, the last option value will be used. listConfig will override options also specified in contextParams.
signal – [in] Signal object to control the create context from binary process. NULL may be passed to indicate that no execution control is requested, and the create operation should continue to completion uninterrupted. The signal object, if not NULL, is considered to be in-use for the duration of the call including the asynchronous deserialization/initialization.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: contextParams is empty or any individual contextParam’s binaryBuffer or notifyFunc is NULL or binaryBufferSize is 0
QNN_CONTEXT_ERROR_MEM_ALLOC: memory allocation error while creating any context
QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: failed to deserialize any binary and create context from it
QNN_CONTEXT_ERROR_BINARY_VERSION: incompatible version of the binary
QNN_CONTEXT_ERROR_BINARY_CONFIGURATION: binary is not configured for this device
QNN_CONTEXT_ERROR_BINARY_SUBOPTIMAL: suboptimal binary is used when QNN_CONTEXT_BINARY_COMPATIBILITY_STRICT is specified in the config option
QNN_CONTEXT_ERROR_SET_PROFILE: failed to set profiling info
QNN_CONTEXT_ERROR_INVALID_HANDLE: backend, device, signal or any individual contextParam’s profile is not a valid handle
QNN_CONTEXT_ERROR_INVALID_CONFIG: one or more config values in either listConfig or any individual contextParam’s config is invalid
QNN_CONTEXT_ERROR_ABORTED: the call is aborted before completion due to user cancellation including during any individual asynchronous deserialization/initialization
QNN_CONTEXT_ERROR_TIMED_OUT: the call is aborted before completion due to a timeout including during any individual asynchronous deserialization/initialization

Note

: Until a notification is received indicating that at least one graph or context is in an executable state, other context or graph-based functions cannot be called.

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_getBinarySectionSize(Qnn_ContextHandle_t context, Qnn_GraphHandle_t graph, QnnContext_SectionType_t section, Qnn_ContextBinarySize_t *binaryBufferSize)¶

Retrieve a section of the binary as specified by section. The size of this section depends on the type of section requested. For example, for QNN_CONTEXT_SECTION_UPDATABLE sections, this will have all the updatable tensor information.

Parameters

context – [in] A context handle.
graph – [in] A graph handle. This argument is optional. When supplied the return size only applies to the size of the context binary section pertaining to this graph. When excluded the returned binary contains associated updates to all graphs in the context. Some backends may require graph as an argument. Support is determined by QNN_PROPERTY_CONTEXT_SUPPORT_BINARY_SECTION_FULL_CONTEXT.
section – [in] The section of the binary to retrieve.
binaryBufferSize – [out] The amount of memory in bytes a client will need to allocate to hold context content updates in binary form.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_HANDLE: context is not a valid handle
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: binaryBufferSize is NULL
QNN_CONTEXT_ERROR_NOT_FINALIZED: if there were any non-finalized entities in the context
QNN_CONTEXT_ERROR_GET_BINARY_SIZE_FAILED: Operation failure due to other factors
QNN_COMMON_ERROR_OPERATION_NOT_PERMITTED: Attempting to get binary size for a context re-created from a cached binary.
QNN_CONTEXT_ERROR_MEM_ALLOC: Not enough memory is available to retrieve the context content.

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_getBinarySection(Qnn_ContextHandle_t context, Qnn_GraphHandle_t graph, QnnContext_SectionType_t section, const QnnContext_Buffer_t *binaryBuffer, Qnn_ContextBinarySize_t *writtenBufferSize, Qnn_ProfileHandle_t profile, Qnn_SignalHandle_t signal)¶

Retrieve section of the context binary. Content of the section is specified by section. The size of the section is retrieved from QnnContext_getBinarySectionSize().

Parameters

context – [in] A context handle.
binaryBuffer – [in] Pointer to the user-allocated context binary memory.
graph – [in] A graph handle. This argument is optional. When supplied the returned binary only contains the context binary section pertaining to this graph. When excluded the returned binary contains associated updates to all graphs in the context. Some backends may require graph as an argument. Support is determined by QNN_PROPERTY_CONTEXT_SUPPORT_BINARY_SECTION_FULL_CONTEXT.
section – [in] The section of the binary to retrieve. When section is QNN_CONTEXT_SECTION_UPDATABLE the returned binary will contain all of the updatable tensors associated with the context and graph combination. Binary sections of type QNN_CONTEXT_SECTION_UPDATABLE have Qnn System Context metadata containing information about any modified input and output tensors, and therefore may be used with QnnSystemContext_getMetadata() and QnnSystemContext_getBinaryInfo().
profile – [in] The profile handle on which metrics are populated and can be queried. Use NULL handle to disable profile collection. A handle being re-used would reset and is populated with values from the current call.
signal – [in] Signal object to control the execution of the create context from binary process. NULL may be passed to indicate that no execution control is requested, and the create operation should continue to completion uninterrupted. The signal object, if not NULL, is considered to be in-use for the duration of the call.
writtenBufferSize – [out] Amount of memory actually written into binaryBuffer, in bytes.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_HANDLE: context is not a valid handle
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: one of the arguments to the API is invalid/NULL
QNN_CONTEXT_ERROR_NOT_FINALIZED: if there were any non-finalized entities in the context
QNN_CONTEXT_ERROR_GET_BINARY_FAILED: Operation failure due to other factors
QNN_COMMON_ERROR_OPERATION_NOT_PERMITTED: Attempting to get binary update for a context re-created from a cached binary.
QNN_CONTEXT_ERROR_MEM_ALLOC: Not enough memory is available to retrieve the context content.

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_applyBinarySection(Qnn_ContextHandle_t context, Qnn_GraphHandle_t graph, QnnContext_SectionType_t section, const QnnContext_Buffer_t *binaryBuffer, Qnn_ProfileHandle_t profile, Qnn_SignalHandle_t signal)¶

Apply a section to the contextBinary produced by a prior QnnContext_getBinarySection() call. If successful, this section overwrites previously applied sections. If the call to applyBinarySection() fails, it indicates the changes were not applied, and that the context retains its prior state. In this case the context is still valid and may be used for subsequent inferences.

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: a feature is not supported
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: binaryBuffer or context is NULL
QNN_CONTEXT_ERROR_MEM_ALLOC: memory allocation error while creating context update
QNN_CONTEXT_ERROR_SET_PROFILE: failed to set profiling info
QNN_CONTEXT_ERROR_INVALID_HANDLE: backend, __profile_, or signal is not a valid handle

Parameters

context – [in] A context handle.
graph – [in] A graph handle. This argument is optional. When supplied the returned binary only contains the context binary section pertaining to this graph. When excluded the returned binary contains associated updates to all graphs in the context.
section – [in] The section of the binary to retrieve. When section is QNN_CONTEXT_SECTION_UPDATABLE the returned binary will contain all of the updatable tensors associated with the context and graph combination.
binaryBuffer – [in] Pointer to the user-allocated context binary memory.
profile – [in] The profile handle on which metrics are populated and can be queried. Use NULL handle to disable profile collection. A handle being re-used would reset and is populated with values from the current call.
signal – [in] Signal object to control the execution of the create context from binary process. NULL may be passed to indicate that no execution control is requested, and the create operation should continue to completion uninterrupted. The signal object, if not NULL, is considered to be in-use for the duration of the call.

Note

Use corresponding API through QnnInterface_t.

Note

When using this API with QNN_CONTEXT_CONFIG_PERSISTENT_BINARY enabled, binaryBuffer should be available and persistent from first call to QnnContext_applyBinarySection until QnnContext_free.

Qnn_ErrorHandle_t QnnContext_getProperty(Qnn_ContextHandle_t contextHandle, QnnContext_Property_t **properties)¶

A function to get a list of context properties. Backends are not required to support this API.

Parameters

contextHandle – [in] A context handle.
[in/out] – properties Pointer to a null terminated array of pointers containing the properties associated with the passed contextHandle. Memory for this information is owned and managed by the client. Client needs to populate the property options being requested. If contextHandle is not recognized, the pointer properties points to is set to nullptr.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_INVALID_HANDLE: contextHandle is not a valid handle
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: properties is NULL or at least one property option is invalid
QNN_CONTEXT_ERROR_UNSUPPORTED_FEATURE: at least one valid property option is not supported

Note

Use corresponding API through QnnInterface_t.

Qnn_ErrorHandle_t QnnContext_free(Qnn_ContextHandle_t context, Qnn_ProfileHandle_t profile)¶

A function to free the context and all associated graphs, operations & tensors.

Parameters

context – [in] A context handle.
profile – [in] The profile handle on which metrics are populated and can be queried. Use NULL handle to disable profile collection. A handle being re-used would reset and is populated with values from the current call.

Returns

Error code:

QNN_SUCCESS: no error is encountered
QNN_CONTEXT_ERROR_INVALID_ARGUMENT: profile is not a valid handle
QNN_CONTEXT_ERROR_INVALID_HANDLE: context is not a valid handle
QNN_CONTEXT_ERROR_MEM_ALLOC: an error is encountered with de-allocation of associated memory
QNN_CONTEXT_ERROR_SET_PROFILE: failed to set profiling info
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION: SSR occurence (successful recovery)
QNN_COMMON_ERROR_SYSTEM_COMMUNICATION_FATAL: SSR occurence (unsuccessful recovery)

Note

Use corresponding API through QnnInterface_t.

struct QnnContext_AsyncExecutionQueueDepth_t¶

#include <QnnContext.h>

This struct provides async execution queue depth.

Public Members

QnnContext_AsyncExecutionQueueDepthType_t type¶

union unnamed¶

#include <QnnContext.h>

Public Members

uint32_t depth¶

struct QnnContext_Config_t¶

#include <QnnContext.h>

This struct provides context configuration.

Public Members

QnnContext_ConfigOption_t option¶

union unnamed¶

#include <QnnContext.h>

Public Members

QnnContext_CustomConfig_t customConfig¶: Used with QNN_CONTEXT_CONFIG_OPTION_CUSTOM.

Qnn_Priority_t priority¶: Used with QNN_CONTEXT_CONFIG_OPTION_PRIORITY.

Qnn_ErrorReportingConfig_t errorConfig¶: Used with QNN_CONTEXT_CONFIG_OPTION_ERROR_REPORTING.

const char *oemString¶: DEPRECATED. Used with QNN_CONTEXT_CONFIG_OPTION_OEM_STRING.

QnnContext_AsyncExecutionQueueDepth_t asyncExeQueueDepth¶: Used with QNN_CONTEXT_CONFIG_ASYNC_EXECUTION_QUEUE_DEPTH.

const char *const *enableGraphs¶: Used with QNN_CONTEXT_CONFIG_ENABLE_GRAPHS.

uint64_t memoryLimitHint¶: Used with QNN_CONTEXT_CONFIG_MEMORY_LIMIT_HINT.

uint8_t isPersistentBinary¶: Used with QNN_CONTEXT_CONFIG_PERSISTENT_BINARY.

QnnContext_BinaryCompatibilityType_t binaryCompatibilityType¶: Used with QNN_CONTEXT_CONFIG_BINARY_COMPATIBILITY.

struct QnnContext_ParamsV1_t¶

#include <QnnContext.h>

This structure serves as a consolidated representation of context-related parameters. QnnContext_createFromBinaryListAsync API takes a list of these parameters for initializing multiple context binaries.

Public Members

const QnnContext_Config_t **config¶: Config pointer to a NULL-terminated array of config option pointers for one context. NULL is allowed and indicates that no config options are provided. If not provided, all config options have default values consistent with the serialized context. If the same config option type is provided multiple times, the last option value will be used.

const void *binaryBuffer¶: A pointer to the context binary.

const Qnn_ContextBinarySize_t binaryBufferSize¶: Holds the size of the context binary.

Qnn_ProfileHandle_t profile¶: The profile handle on which metrics are populated and can be queried. Use a NULL handle to disable profile collection. If a handle is reused, it will reset and be populated with values from the current call.

QnnContext_createFromBinaryNotifyFn_t notifyFunc¶: Pointer to a notification function, cannot be NULL.

void *notifyParam¶: Client-supplied data object which will be passed back via notifyFn and can be used to identify which context’s asynchronous initialization instance the notifyFn applies to. Can be NULL if client does not need it.

struct QnnContext_Params_t¶

#include <QnnContext.h>

Structure which provides various versions of context params.

Public Members

QnnContext_ParamsVersion_t version¶

union unnamed

#include <QnnContext.h>

Public Members

QnnContext_ParamsV1_t v1

struct Qnn_BinaryBuffer_t

#include <QnnContext.h>

A struct which defines a memory buffer.

Public Members

void *data: app-accessible data pointer, provided by app.

Qnn_ContextBinarySize_t dataSize: size of buffer, in bytes, pointed to by data.

struct QnnContext_BufferV1_t¶

#include <QnnContext.h>

This structure defines a client created binary buffer containing the context binary.

Public Members

QnnContext_MemType_t memType¶

union unnamed¶

#include <QnnContext.h>

Actual data contained in the context binary.

Public Members

Qnn_BinaryBuffer_t binaryBuf¶: Context binary data provided by client as a pointer to raw memory (see QNN_CONTEXTMEMTYPE_RAW).

Qnn_MemHandle_t memHandle¶: Context binary data shared via a memory handle (see QNN_CONTEXTMEMTYPE_MEMHANDLE).

struct QnnContext_Buffer_t¶

#include <QnnContext.h>

Structure which provides various versions of context params.

Public Members

QnnContext_BufferVersion_t version¶

union unnamed¶

#include <QnnContext.h>

Public Members

QnnContext_BufferV1_t v1¶

struct QnnContext_Property_t¶

#include <QnnContext.h>

This struct provides context property. Option is specified by the client. Everything else is written by the backend.

Public Members

QnnContext_PropertyOption_t option¶

union unnamed

#include <QnnContext.h>

Public Members

QnnContext_CustomProperty_t customProperty

File QnnContext.h¶

Definition (include/QNN/QnnContext.h)¶

Detailed Description¶

Includes¶

Included By¶

Classes¶

Enums¶

Functions¶

Defines¶

Typedefs¶

Unions¶

Full File Listing¶

Definition (`include/QNN/QnnContext.h`)¶