Saver

This section provides information specific to QNN Saver Backend.

Overview

The QNN Saver is a backend provided by the QNN SDK primarily intended for debugging purposes. It is a backend in the sense that it is a shared library that implements all QNN APIs, but unlike all other backends, Saver does not execute any graphs. Instead, it records all QNN API calls and their arguments into files which can be replayed on any QNN backend.

Here are some details on the files that Saver produces - saver_output.c and params.bin:

• saver_output.c is a C file containing all QNN API calls.

• params.bin is an unstructured binary file that contains all input/output/parameter tensor data provided during tensor creation, op config validation, and graph execution.

saver_output.c can be used to visually inspect all QNN API calls to ensure they are being called in the correct order, the arguments are well-formed, etc. It can also be compiled and linked against a QNN backend library to be replayed.

At runtime, the tensor data will be read from params.bin and the recorded QNN APIs will be called in the same order as they were invoked on the Saver backend. Any error codes returned from QNN APIs during replay will be printed to stdout.

For instructions on how to generate and replay saver_output.c, refer to Save execution sequence with Saver and replay on a backend.

API Specializations

This section contains information related to API specialization for the Saver backend. All QNN Saver backend specialization is available under <QNN_SDK_ROOT>/include/QNN/Saver/ directory.

QNN Saver API

In addition to all other QNN APIs, the Saver backend also implements QnnSaver_initialize(). This function can be used to supply Saver with custom configuration. This function is optional, and is only needed if custom configuration is desired. If this function is used, it must be called before QnnBackend_initialize().

This function and its corresponding datatypes are located in File QnnSaver.h.

Usage

Similar to other QNN APIs that support custom configurations (e.g. QnnBackend_initialize()), QnnSaver_initialize() also takes as input a NULL-terminated array of pointers to config objects. The appropriate config object for this API is Struct QnnSaver_Config_t.

A description of each config option as well as code examples are shown below.

Output directory

The default output directory for Saver outputs (saver_output.c and params.bin) is ./saver_output/. This can be configured to any directory. If a relative path is given, the output directory will be relative to the current working directory. If the specified directory does not exist, it will be created.

Output directory configuration

QnnSaver_Config_t outputDirCfg;
outputDirCfg.option = QNN_SAVER_CONFIG_OPTION_OUTPUT_DIRECTORY;
outputDirCfg.outputDirectory = "./saver_output_config_example/"; //can be absolute e.g. "/data/local/tmp/saver/"

const QnnSaver_Config_t *saverCfg[] = {&outputDirCfg, NULL};
QnnSaver_initialize(saverCfg);

Output file timestamp

By default, Saver outputs are overwritten between consecutive runs. To prevent overwriting, the filenames can be made unique by appending a timestamp. This option may be useful if Saver is being used in an automated environment, as all outputs will be saved for later debugging. Any nonzero value will enable this option.

Note

Keep in mind that because params.bin contains all input tensor data, enabling this setting will result in duplication of this data, and may take up significant storage over time.

Timestamp configuration

QnnSaver_Config_t appendTimestampCfg;
appendTimestampCfg.option = QNN_SAVER_CONFIG_OPTION_APPEND_TIMESTAMP;
appendTimestampCfg.appendTimestamp = 1;

const QnnSaver_Config_t *saverCfg[] = {&appendTimestampCfg, NULL};
QnnSaver_initialize(saverCfg);

Backend ID

By default, Saver does not record backend-specific custom configs when they are passed to supported APIs (e.g. QnnBackend_initialize()). To enable these backend-specific configs to be recorded, Saver must be supplied with the desired backend ID of the backend to which these configs apply.

IDs for the backends exposed by QNN are defined in <QNN_SDK_ROOT>/include/QNN/<backend>/Qnn<backend>Common.h.

Backend ID configuration

QnnSaver_Config_t backendIdCfg;
backendIdCfg.option = QNN_SAVER_CONFIG_OPTION_BACKEND_ID;
backendIdCfg.backendId = QNN_BACKEND_ID_GPU;

const QnnSaver_Config_t *saverCfg[] = {&backendIdCfg, NULL};
QnnSaver_initialize(saverCfg)

// Can now pass GPU custom configs and have them recorded by Saver in saver_output.c
QnnGpuBackend_CustomConfig_t gpuConfig;
gpuConfig.kernelRepoDir = "/data/local/tmp/example/";
gpuConfig.invalidateKernelRepo = 1;

QnnBackend_Config_t backendConfigGpu;
backendConfigGpu.option = QNN_BACKEND_CONFIG_OPTION_CUSTOM;
backendConfigGpu.customConfig = &gpuConfig;

const QnnBackend_Config_t *backendConfig[] = {&backendConfigGpu, NULL};
QnnBackend_initialize(backendConfig);