Class SNPEBuilder¶

Defined in File SNPEBuilder.hpp

Class Documentation¶

class SNPEBuilder¶

Public Functions

explicit SNPEBuilder(DlContainer::IDlContainer *container)¶

Constructor of NeuralNetwork Builder ith a supplied model.

Parameters: container – [in] A DlContainer holding the model.
Returns: A new instance of a SNPEBuilder object that can be used to configure and build an instance of SNPE.

SNPEBuilder &setPerformanceProfile(DlSystem::PerformanceProfile_t performanceProfile)¶

Requests a performance profile.

Parameters: performanceProfile – [in] The target performance profile.

SNPEBuilder &setProfilingLevel(DlSystem::ProfilingLevel_t profilingLevel)¶

Sets the profiling level. Default profiling level for SNPEBuilder is off. Off and basic only applies to DSP runtime.

Parameters: profilingLevel – [in] The target profiling level.

SNPEBuilder &setExecutionPriorityHint(DlSystem::ExecutionPriorityHint_t priority)¶

Sets a preference for execution priority. This allows the caller to give coarse hint to SNPE runtime about the priority of the network. SNPE runtime is free to use this information to co-ordinate between different workloads that may or may not extend beyond SNPE.

Parameters: priority – [in] The target performance profile.

SNPEBuilder &networkInit(const DlSystem::StringList &networkNames)¶

Sets the graphs that will be initialized in a multi-graph DLC.

Parameters: networkNames – [in] List of network names to be initialized.

Note

The networkName is specified in snpe-dlc-info and defaults to the name of the first graph in the DLC.

SNPEBuilder &setOutputLayers(const char *networkName, const DlSystem::StringList &outputLayerNames)¶

Sets the layers that will generate output.

Parameters

networkName – [in] specifies network name on which the output layer names are to be set
outputLayerNames – [in] List of layer names to output. An empty list will result in only the final layer of the model being the output layer. The list will be copied.

SNPEBuilder &setOutputLayers(const DlSystem::StringList &outputLayerNames)¶

Sets the layers that will generate output.

Parameters: outputLayerNames – [in] List of layer names to output. An empty list will result in only the final layer of the model being the output layer. The list will be copied.

SNPEBuilder &setOutputTensors(const DlSystem::StringList &outputTensorNames)¶

Sets the output tensor names.

Parameters: outputTensorNames – [in] List of tensor names to output. An empty list will result in producing output for the final output tensor of the model. The list will be copied.

SNPEBuilder &setOutputTensors(const char *networkName, const DlSystem::StringList &outputTensorNames)¶

Sets the output tensor names.

Parameters

networkName – [in] specifies network name on which the output tensor names are to be set
outputTensorNames – [in] List of tensor names to output. An empty list will result in producing output for the final output tensor of the model. The list will be copied.

SNPEBuilder &setUseUserSuppliedBuffers(int bufferMode)¶

Sets whether this neural network will perform inference with input from user-supplied buffers, and write output to user-supplied buffers. Default behaviour is to use tensors created by ITensorFactory.

Parameters: bufferMode – [in] Boolean whether to use user-supplied buffer or not.

SNPEBuilder &setDebugMode(int debugMode)¶

Sets the debug mode of the runtime.

Parameters: debugMode – [in] This enables debug mode for the runtime. It does two things. For an empty outputLayerNames list, all layers will be output. It might also disable some internal runtime optimizations (e.g., some networks might be optimized by combining layers etc.).

SNPEBuilder &setInputDimensions(const DlSystem::TensorShapeMap &inputDimensionsMap)¶

Sets network’s input dimensions to enable resizing of the spatial dimensions of each layer for fully convolutional networks, and the batch dimension for all networks.

Parameters: inputDimensionsMap – [in] : Handle to the map of input names and their new dimensions. The new dimensions overwrite the input dimensions embedded in the model and then resize each layer of the model. If the model contains layers whose dimensions cannot be resized e.g FullyConnected, exception will be thrown when SNPE instance is actually built. In general the batch dimension is always resizable. After resizing of layers’ dimensions in model based on new input dimensions, the new model is revalidated against all runtime constraints, whose failures may result in cpu fallback situation.

SNPEBuilder &setInputDimensions(const char *networkName, const DlSystem::TensorShapeMap &inputDimensionsMap)¶

Sets network’s input dimensions to enable resizing of the spatial dimensions of each layer for fully convolutional networks, and the batch dimension for all networks.

Parameters

networkName – [in] : specifies network name on which the input dims are to be set
inputDimensionsMap – [in] : Handle to the map of input names and their new dimensions. The new dimensions overwrite the input dimensions embedded in the model and then resize each layer of the model. If the model contains layers whose dimensions cannot be resized e.g FullyConnected, exception will be thrown when SNPE instance is actually built. In general the batch dimension is always resizable. After resizing of layers’ dimensions in model based on new input dimensions, the new model is revalidated against all runtime constraints, whose failures may result in cpu fallback situation.

SNPEBuilder &setInitCacheMode(int cacheMode)¶

Sets the mode of init caching functionality.

Parameters: mode – [in] Boolean. This flag enables/disables the functionality of init caching. When init caching functionality is enabled, a set of init caches will be created during network building/initialization process and will be added to DLC container. If such DLC is saved by the user, in subsequent network building/initialization processes these init caches will be loaded from the DLC so as to reduce initialization time. In disable mode, no init caches will be added to DLC container.
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setPlatformConfig(const DlSystem::PlatformConfig &platformConfigHandle)¶

Sets the platform configuration.

Parameters: platformConfig – [in] The platform configuration.
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setRuntimeProcessorOrder(const DlSystem::RuntimeList &runtimeList)¶

Sets network’s runtime order of precedence. Example: CPU_FLOAT32, GPU_FLOAT16, AIP_FIXED8_TF.

Parameters: runtimeListHandle – [in] The list of runtime in order of precedence
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setUnconsumedTensorsAsOutputs(int setOutput)¶

Sets the unconsumed tensors as output.

Parameters: setOutput – [in] Boolean. This enables unconsumed tensors (i.e) outputs which are not inputs to any layer (basically dead ends) to be marked for output
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setTimeOut(uint64_t timeoutMicroSec)¶

Execution terminated when exceeding time limit. Only valid for dsp runtime currently.

Parameters: timeout – [in] Time limit value in microseconds
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setBufferDataType(const DlSystem::IOBufferDataTypeMap &dataTypeMap)¶

Sets the datatype of the buffer. Only valid for dsp runtime currently.

Parameters: dataTypeMapHandle – [in] Map of the buffer names and the datatype that needs to be set.
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setSingleThreadedInit(int singleThreadedInit)¶

Sets up the entire initialization callflow to happen on the user’s thread.

Parameters: singleThreadedInit – [in] Flag indicating user’s intent to perform initialization callflow on caller’s thread. When set to 1, initialization will happen on the user’s thread. When set to 0, initialization will happen on a new thread. This is the default behaviour(analogous to not calling this API)
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setCpuFixedPointMode(bool cpuFxpMode)¶

Sets the fixed point execution mode for CPU runtime. If a floating point DLC is executed with this option set, the program will be terminated with an exception. If a quantized DLC is executed without this option set, the execution will be in floating point mode in CPU.

Parameters: cpuFxpMode – [in] Boolean If set to true, enables the fixed point mode.
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setAcceleratedInit(bool acceleratedInit)¶

Sets the accelerated init mode for HTP runtime for execution from an offline prepared cache. If DLC without HTP cache is called with this mode, the program terminates Init acceleration requires cache prepared with current SDK; older cache falls back to regular initialization. If an offline prepared model with an invalid cache is discovered, the program continues normal execution with a warning.

Parameters: acceleratedInit – [in] Boolean If set to true, enables HTP init accelerate.
Returns: The current instance of SNPEBuilder.

Note

Init time is reduced by using more resources. This would cause execution to slow down if executes are done concurrently with initialization. This should only be used for an offline prepared DLC, for DSP (HTP) runtime. Default value is false.

SNPEBuilder &setMemoryLimitHint(uint64_t memoryLimitHintInMb)¶

Sets memoryLimitHint for DSP Based on the memory limit provided graphs will be switched from cache blob when execute is invoked.

Parameters: memoryLimitHintInMb – [in] Sets the peak memory limit hint of a cache in megabytes.

Note

Memory footprint is reduced at the cost of slower first execution after graph switching. Default is set to 0, set to any non zero value to enter low memory mode with multi graph init. User must first load the largest graph inside the cache to avoid undefined behaviors. User can strategically specify networkInit to ensure the first graph is the largest graph if it has not been serialized as the first graph in the cache. Currently any non zero value would enable graph switching.

SNPEBuilder &setModelName(DlSystem::String modelName)¶

Sets model name for logging.

Parameters: modelName – [in] String Model name for logging.
Returns: The current instance of SNPEBuilder.

SNPEBuilder &setCacheCompatibilityMode(DlSystem::CacheCompatibility_t mode)¶

Sets compatibility check mode.

Parameters: mode – [in] Mode of checking compatibility
Returns: The current instance of SNPEBuilder.

Snpe_ErrorCode_t validateCache()¶

Validates the cache records available in the DLC for compatibility with the current hardware depending on the set cache compatibility mode.

Returns: One of the following Snpe_ErrorCode_t SNPE_SUCCESS: The DLC contains a valid and usable cache record SNPE_ERRORCODE_DLCACHING_INCOMPATIBLE: None of the existing caches are compatible on the current hardware SNPE_ERRORCODE_DLCACHING_SUBOPTIMAL_CACHE: Compatibility mode is strict and the best available cache is suboptimal SNPE_ERRORCODE_DLCONTAINER_MISSING_RECORDS: The DLC contains no cache records SNPE_ERRORCODE_DLCONTAINER_INVALID_RECORD: Failed to read DLC records SNPE_ERRORCODE_CONFIG_INVALID_PARAM: An invalid builder configurations is set at the time of invocation SNPE_ERRORCODE_UNKNOWN_EXCEPTION: Encountered an unexpected miscellaneous error

Note

The intended usage of this function is to validate the cache ahead of time to determine the need for setting the init cache option before calling the build function. It is expected to be called after all other builder options have been set and before the build function is invoked. In particular, this function is sensitive to the resized input dimensions, output op and tensor names, and cache compatibility mode settings

Note

Validation will always fail with SNPE_ERRORCODE_CONFIG_INVALID_PARAM if coupled with cache compatibility mode ALWAYS_GENERATE_NEW_CACHE

Snpe_ErrorCode_t setCustomPerfProfile(DlSystem::SNPEPerfProfile perfProfile)¶

Sets custom performance profile.

Parameters: perfProfile, performance – [in] profile level
Returns: SNPE_SUCCESS upon successful setting of performance profile

std::unique_ptr<SNPE> build() noexcept¶

Returns an instance of SNPE based on the current parameters.

Returns: A new instance of a SNPE object that can be used to execute models or null if any errors occur.