File SNPEBuilder.h

↰ Parent directory (SNPE)

Definition (SNPE/SNPEBuilder.h)

Includes

Full File Listing

Typedefs

typedef void *Snpe_SNPEBuilder_Handle_t

A typedef to indicate a SNPEBuilder handle

Functions

Snpe_SNPEBuilder_Handle_t Snpe_SNPEBuilder_Create(Snpe_DlContainer_Handle_t containerHandle)

Constructor of NeuralNetwork Builder ith a supplied model.

The builder class for creating SNPE objects. Not meant to be extended.

Parameters

containerHandle – [in] A DlContainer holding the model.

Returns

A new instance of a SNPEBuilder object that can be used to configure and build an instance of SNPE.

Snpe_ErrorCode_t Snpe_SNPEBuilder_Delete(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle)

Destroys/frees a SNPEBuilder object

Parameters

snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

Returns

SNPE_SUCCESS if Delete operation successful.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetPerformanceProfile(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_PerformanceProfile_t performanceProfile)

Requests a performance profile.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • performanceProfile – [in] The target performance profile.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetProfilingLevel(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_ProfilingLevel_t profilingLevel)

Sets the profiling level. Default profiling level for SNPEBuilder is off. Off and basic only applies to DSP runtime.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • profilingLevel – [in] The target profiling level.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetExecutionPriorityHint(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_ExecutionPriorityHint_t priority)

Sets a preference for execution priority.

This allows the caller to give coarse hint to SNPE runtime about the priority of the network. SNPE runtime is free to use this information to co-ordinate between different workloads that may or may not extend beyond SNPE.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • priority – [in] The target performance profile.

Snpe_ErrorCode_t Snpe_SNPEBuilder_NetworkInit(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_StringList_Handle_t snpeNetworkNames)

Sets the graphs that will be initialized in a multi-graph DLC.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • snpeNetworkNames – [in] List of network names to to be initialized.

Note

The networkName is specified in snpe-dlc-info and defaults to the name of the first graph in the DLC.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetOutputLayers(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_StringList_Handle_t outputLayerNames)

Sets the layers that will generate output.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • outputLayerNames – [in] List of layer names to output. An empty list will result in only the final layer of the model being the output layer. The list will be copied.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetOutputLayersForNetwork(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, const char *networkName, Snpe_StringList_Handle_t outputLayerNames)

Sets the layers that will generate output.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • networkName – [in] specifies network name on which the output layer names are to be set

  • outputLayerNames – [in] List of layer names to output. An empty list will result in only the final layer of the model being the output layer. The list will be copied.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetOutputTensors(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_StringList_Handle_t outputTensorNames)

Sets the output tensor names.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • outputTensorNames – [in] List of tensor names to output. An empty list will result in producing output for the final output tensor of the model. The list will be copied.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetOutputTensorsForNetwork(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, const char *networkName, Snpe_StringList_Handle_t outputTensorNames)

Sets the output tensor names.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • networkName – [in] specifies network name on which the output tensor names are to be set

  • outputTensorNames – [in] List of tensor names to output. An empty list will result in producing output for the final output tensor of the model. The list will be copied.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetUseUserSuppliedBuffers(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, int bufferMode)

Sets whether this neural network will perform inference with input from user-supplied buffers, and write output to user-supplied buffers. Default behaviour is to use tensors created by ITensorFactory.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • bufferMode – [in] Boolean whether to use user-supplied buffer or not.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetDebugMode(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, int debugMode)

Sets the debug mode of the runtime.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • debugMode – [in] This enables debug mode for the runtime. It does two things. For an empty outputLayerNames list, all layers will be output. It might also disable some internal runtime optimizations (e.g., some networks might be optimized by combining layers, etc.).

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetInputDimensions(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_TensorShapeMap_Handle_t inputDimensionsMapHandle)

Sets network’s input dimensions to enable resizing of the spatial dimensions of each layer for fully convolutional networks, and the batch dimension for all networks.

Parameters

tensorShapeMapHandle – [in] : Handle to the map of input names and their new dimensions. The new dimensions overwrite the input dimensions embedded in the model and then resize each layer of the model. If the model contains layers whose dimensions cannot be resized e.g FullyConnected, exception will be thrown when SNPE instance is actually built. In general the batch dimension is always resizable. After resizing of layers’ dimensions in model based on new input dimensions, the new model is revalidated against all runtime constraints, whose failures may result in cpu fallback situation.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetInputDimensionsForNetwork(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, const char *networkName, Snpe_TensorShapeMap_Handle_t inputDimensionsMapHandle)

Sets network’s input dimensions to enable resizing of the spatial dimensions of each layer for fully convolutional networks, and the batch dimension for all networks.

Parameters

  • networkName – [in] : specifies network name on which the input dims are to be set

  • tensorShapeMapHandle – [in] : Handle to the map of input names and their new dimensions. The new dimensions overwrite the input dimensions embedded in the model and then resize each layer of the model. If the model contains layers whose dimensions cannot be resized e.g FullyConnected, exception will be thrown when SNPE instance is actually built. In general the batch dimension is always resizable. After resizing of layers’ dimensions in model based on new input dimensions, the new model is revalidated against all runtime constraints, whose failures may result in cpu fallback situation.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetInitCacheMode(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, int cacheMode)

Sets the mode of init caching functionality.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • mode – [in] Boolean. This flag enables/disables the functionality of init caching. When init caching functionality is enabled, a set of init caches will be created during network building/initialization process and will be added to DLC container. If such DLC container is saved by the user, in subsequent network building/initialization processes these init caches will be loaded from the DLC so as to reduce initialization time. In disable mode, no init caches will be added to DLC container.

Snpe_SNPE_Handle_t Snpe_SNPEBuilder_Build(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle)

Returns an instance of SNPE based on the current parameters.

Parameters

snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

Returns

A new instance of a SNPE object that can be used to execute models or null if any errors occur.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetPlatformConfig(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_PlatformConfig_Handle_t platformConfigHandle)

Sets the platform configuration.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • platformConfig – [in] The platform configuration.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetRuntimeProcessorOrder(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_RuntimeList_Handle_t runtimeListHandle)

Sets network’s runtime order of precedence. Example: CPU_FLOAT32, GPU_FLOAT16, AIP_FIXED8_TF.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • runtimeListHandle – [in] The list of runtime in order of precedence

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetUnconsumedTensorsAsOutputs(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, int setOutput)

Sets the unconsumed tensors as output.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • setOutput – [in] Boolean. This enables unconsumed tensors (i.e) outputs which are not inputs to any layer (basically dead ends) to be marked for output

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetTimeOut(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, uint64_t timeoutMicroSec)

Execution terminated when exceeding time limit. Only valid for dsp runtime currently.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • timeout – [in] Time limit value in microseconds

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetBufferDataType(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_IOBufferDataTypeMap_Handle_t dataTypeMapHandle)

Sets the datatype of the buffer. Only valid for dsp runtime currently.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • dataTypeMapHandle – [in] Map of the buffer names and the datatype that needs to be set.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetSingleThreadedInit(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, int singleThreadedInit)

Sets up the entire initialization callflow to happen on the user’s thread.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • singleThreadedInit – [in] Flag indicating user’s intent to perform initialization callflow on caller’s thread. When set to 1, initialization will happen on the user’s thread When set to 0, initialization will happen on a new thread. This is the default behavior (analogous to not calling this API)

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetCpuFixedPointMode(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, bool cpuFxpMode)

Sets the fixed point execution mode for CPU runtime. If a floating point DLC is executed with this option set, the program will be terminated with an exception. If a quantized DLC is executed without this option set, the execution will be in floating point mode in CPU.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • cpuFxpMode – [in] Boolean If set to true, enables the fixed point mode.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetAcceleratedInit(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, bool acceleratedInit)

Sets the accelerated init mode for HTP runtime for execution from an offline prepared cache. If DLC without HTP cache is called with this mode, the program terminates Init acceleration requires cache prepared with current SDK; older cache falls back to regular initialization. If an offline prepared model with an invalid cache is discovered, the program continues normal execution with a warning.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • acceleratedInit – [in] Boolean If set to true, enables HTP Init accelerate.

Note

Init time is reduced by using more resources. This would cause execution to slow down if executes are done concurrently with initialization. This should only be used for an offline prepared DLC, for DSP (HTP) runtime. Default value is false.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetMemoryLimitHint(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, uint64_t memoryLimitHintInMb)

Sets memoryLimitHint for DSP Based on the memory limit provided graphs will be switched from cache blob when execute is invoked.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • memoryLimitHintInMb – [in] Sets the peak memory limit hint of a cache in megabytes.

Note

Memory footprint is reduced at the cost of slower first execution after graph switching. Default is set to 0, set to any non zero value to enter low memory mode with multi graph init. User must first load the largest graph inside the cache to avoid undefined behaviors. User can strategically specify Snpe_SNPEBuilder_NetworkInit to ensure the first graph is the largest graph if it has not been serialized as the first graph in the cache. Currently any non zero value would enable graph switching.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetModelName(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, const char *modelName)

Sets model name for logging.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • modelName – [in] String Model name for logging.

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetCacheCompatibilityMode(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_CacheCompatibility_t mode)

Sets compatibility check mode.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • mode – [in] Mode of checking compatibility

Snpe_ErrorCode_t Snpe_SNPEBuilder_ValidateCache(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle)

Validates the cache records available in the DLC for compatibility with the current hardware depending on the set cache compatibility mode.

Parameters

snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

Returns

One of the following Snpe_ErrorCode_t SNPE_SUCCESS: The DLC contains a valid and usable cache record SNPE_ERRORCODE_DLCACHING_INCOMPATIBLE: None of the existing caches are compatible on the current hardware SNPE_ERRORCODE_DLCACHING_SUBOPTIMAL_CACHE: Compatibility mode is strict and the best available cache is suboptimal SNPE_ERRORCODE_DLCONTAINER_MISSING_RECORDS: The DLC contains no cache records SNPE_ERRORCODE_DLCONTAINER_INVALID_RECORD: Failed to read DLC records SNPE_ERRORCODE_CONFIG_INVALID_PARAM: An invalid builder configurations is set at the time of invocation SNPE_ERRORCODE_UNKNOWN_EXCEPTION: Encountered an unexpected miscellaneous error

Note

The intended usage of this function is to validate the cache ahead of time to determine the need for setting the init cache option before calling the build function. It is expected to be called after all other builder options have been set and before the build function is invoked. In particular, this function is sensitive to the resized input dimensions, output op and tensor names, and cache compatibility mode settings

Note

Validation will always fail with SNPE_ERRORCODE_CONFIG_INVALID_PARAM if coupled with cache compatibility mode ALWAYS_GENERATE_NEW_CACHE

Snpe_ErrorCode_t Snpe_SNPEBuilder_SetCustomPerfProfile(Snpe_SNPEBuilder_Handle_t snpeBuilderHandle, Snpe_SNPEPerfProfile_Handle_t perfProfileHandle)

Set the custom performance profile.

Parameters

  • snpeBuilderHandle – [in] Handle to access the SNPEBuilder object

  • perfProfileHandle – [in] Handle to access the perfprofile object

Returns

true on successfully applied