File PSNPE.hpp

↰ Parent directory (SNPE)

Definition (SNPE/PSNPE.hpp)

Includes

Namespaces

Full File Listing

namespace PSNPE

Typedefs

using OutputAsyncCallbackFunc = std::function<void(OutputAsyncCallbackParam)>

This callback is called when the output data is ready, only use for Output Async mode.

using InputOutputAsyncCallbackFunc = std::function<void(InputOutputAsyncCallbackParam)>

This callback is called when the output data is ready, only use for Output-Input Async mode.

using InputOutputAsyncInputCallback = std::function<std::shared_ptr<ApplicationBufferMap>(InputOutputInputAsyncCallbackParam)>

This callback is called when the input data is ready,only use for Output-Input Async mode.

Enums

enum BuildMode

build snpe instance in serial or parallel

Values:

enumerator SERIAL = 0
enumerator PARALLEL = 1
enum InputOutputTransmissionMode

Input and output transmission mode.

Values:

enumerator sync = 0
enumerator outputAsync = 1
enumerator inputOutputAsync = 2
struct OutputAsyncCallbackParam
#include <PSNPE.hpp>

A structure representing parameters of callback function of Async Output mode.

Public Functions

OutputAsyncCallbackParam() = delete
OutputAsyncCallbackParam(OutputAsyncCallbackParam &&other) noexcept

Public Members

DataIndexReference<size_t> dataIndex = {*this}
ExecuteStatusReference<bool> executeStatus = {*this}
ErrorMsgReference<std::string> errorMsg = {*this}
CallbackIDReference<size_t> callbackID = {*this}
struct InputOutputInputAsyncCallbackParam
#include <PSNPE.hpp>

A structure representing parameters of callback function of Async Input/Output mode.

Public Functions

InputOutputInputAsyncCallbackParam() = delete
InputOutputInputAsyncCallbackParam(InputOutputInputAsyncCallbackParam &&other) noexcept

Public Members

InputsReference<std::vector<std::string>> inputs = {*this}
InputNamesReference<DlSystem::StringList> inputNames = {*this}
CallbackIDReference<size_t> callbackID = {*this}
struct InputOutputAsyncCallbackParam
#include <PSNPE.hpp>

Public Functions

InputOutputAsyncCallbackParam(InputOutputAsyncCallbackParam &&other) noexcept

Public Members

DataIndexReference<size_t> dataIndex = {*this}
OutputMapReference<ApplicationBufferMap> outputMap = {*this}
ExecuteStatusReference<bool> executeStatus = {*this}

OOOH, this will be super tricky to not have a copy every time.

ErrorMsgReference<std::string> errorMsg = {*this}
CallbackIDReference<size_t> callbackID = {*this}
struct BuildConfig
#include <PSNPE.hpp>

Public Members

BuildMode buildMode = BuildMode::SERIAL

Specify build in serial mode or parallel mode.

zdl::DlContainer::IDlContainer *container

The opened container ptr.

zdl::DlSystem::StringList outputBufferNames

Specify the output layer name.

zdl::DlSystem::StringList outputTensors

Specify the output layer name.

RuntimeConfigList runtimeConfigList

The runtime config list for PSNPE,.

See also

RuntimeConfig

size_t inputThreadNumbers = 1

Specify the number of threads used in the execution phase to process input data, only used in inputOutputAsync mode.

size_t outputThreadNumbers = 1

Specify the number of threads used in the execution phase to process output data, only used in inputOutputAsync and outputAsync mode.

OutputAsyncCallbackFunc outputCallback

The callback to deal with output data ,only used in outputAsync mode.

InputOutputAsyncCallbackFunc inputOutputCallback

The callback to deal with output data ,only used in inputOutputAsync mode.

InputOutputAsyncInputCallback inputOutputInputCallback

The callback to deal with input data ,only used in inputOutputAsync mode.

InputOutputTransmissionMode inputOutputTransmissionMode = InputOutputTransmissionMode::sync

Specify execution mode.

zdl::DlSystem::ProfilingLevel_t profilingLevel = zdl::DlSystem::ProfilingLevel_t::OFF

Specify profiling level for Diaglog.

uint64_t encode[2] = {0, 0}
bool enableInitCache = false
bool acceleratedInit = false
std::string platformOptions
std::string diaglogOutputDir = "./diaglogs/"

Specify a diaglog output directory to save the generated Diaglog files.

size_t callbackID = {}
class PSNPE
#include <PSNPE.hpp>

The class for executing SNPE instances in parallel.

Public Functions

PSNPE()
bool build(BuildConfig &buildConfig) noexcept

Build snpe instances.

bool execute(UserBufferList &inputBufferList, UserBufferList &outputBufferList) noexcept

Execute snpe instances in Async Output mode and Sync mode.

Parameters

  • inputBufferList – [in] A list of user buffers that contains the input data

  • outputBufferList – [inout] A list of user buffers that will hold the output data

Returns

True in case of successful execution, false otherwise

bool executeInputOutputAsync(const DlSystem::StringList &inputMap, size_t dataIndex, bool isTF8buff, bool isTF8Outputbuff) noexcept

Execute snpe instances in Async Input/Output mode.

Parameters

  • inputMap – [in] A map of input buffers that contains input data. The names of buffers need to be matched with names retrived through getInputTensorNames()

  • dataIndex – [in] Index of the input data

  • isTF8buff – [in] Whether prefer to using 8 bit quantized element for inference

Returns

True if executed successfully; flase, otherwise.

bool executeInputOutputAsync(const std::vector<std::string> &inputMap, size_t dataIndex, bool isTF8buff, bool isTF8Outputbuff) noexcept

Execute snpe instances in Async Input/Output mode.

Parameters

  • inputMap – [in] A map of input buffers that contains input data. The names of buffers need to be matched with names retrived through getInputTensorNames()

  • dataIndex – [in] Index of the input data

  • isTF8buff – [in] Whether prefer to using 8 bit quantized element for inference

Returns

True if executed successfully; flase, otherwise.

bool executeInputOutputAsync(const DlSystem::StringList &inputMap, size_t dataIndex, bool isTF8buff) noexcept

Execute snpe instances in Async Input/Output mode.

Parameters

  • inputMap – [in] A map of input buffers that contains input data. The names of buffers need to be matched with names retrived through getInputTensorNames()

  • dataIndex – [in] Index of the input data

  • isTF8buff – [in] Whether prefer to using 8 bit quantized element for inference

Returns

True if executed successfully; flase, otherwise.

bool executeInputOutputAsync(const std::vector<std::string> &inputMap, size_t dataIndex, bool isTF8buff) noexcept

Execute snpe instances in Async Input/Output mode.

Param

const DlSystem::StringList getInputTensorNames() const noexcept

Get the input tensor names for PSNPE object.

Returns

The string list of input tensor names

const DlSystem::StringList getOutputTensorNames() const noexcept

Get the output tensor names for PSNPE object.

Returns

The string list of output tensor names

const DlSystem::TensorShape getInputDimensions() const noexcept

Get the input dimension shape for PSNPE object.

Returns

The tensor shape of input dimension

const zdl::DlSystem::TensorShape getInputDimensions(const char *name) const noexcept

Get the input dimension shape for the specific input name for PSNPE object.

Parameters

name – [in] The name of input data

Returns

The tensor shape of a specific input name

zdl::DlSystem::TensorShape getBufferAttributesDims(const char *name) const noexcept

Get the number of elements in each dimension for input and output buffer.

Parameters

name – [in] The name of input and output buffer

Returns

Dimension size

DlSystem::Optional<DlSystem::IBufferAttributes*> getInputOutputBufferAttributes(const char *name) const noexcept

Get the handle of IBufferAttributes.

Parameters

name – [in] The name of attribute buffer

Returns

Handle to access IBufferAttributes

DlSystem::Optional<DlSystem::IBufferAttributes*> getInputOutputBufferAttributesForNetwork(const char *networkName, const char *name) const noexcept

Get the handle of IBufferAttributes.

Parameters

  • networkName – [in] The name of network

  • name – [in] The name of attribute buffer

Returns

Handle to access IBufferAttributes

bool registerIonBuffers(const DlSystem::UserMemoryMap &ionBufferMap) const noexcept
bool deregisterIonBuffers(const DlSystem::StringList &ionBufferNames) const noexcept
bool registerMemoryMappedBuffers(const DlSystem::UserMemoryMap &memoryMappedBufferMap) noexcept

Register Client Memory-Mapped Buffers (Example ION buffers in Android)

Parameters

memmoryMappedBufferMap – [in] A UserMemoryMap of virtual addresses

Returns

SNPE_SUCCESS upon successful memory mapped buffer registration

Note

UserBuffer type passed for registration must match the data type of the tensor in the dlc For regular UserBuffers SNPE performs an online data conversion (quantization or dequantization etc). This is not possible for memory mapped buffers hence can lead to issues during execution or accuracy degradation

bool deregisterMemoryMappedBuffers(const DlSystem::StringList &bufferNames) noexcept

Deregister Client Memory-Mapped Buffers (Example ION buffers in Android)

Parameters

bufferNamesHandle – [in] A StringList of memory mapped buffer names

Returns

SNPE_SUCCESS upon successful memory mapped buffer deregistration

bool setPerformanceProfile(size_t instanceIdx, DlSystem::PerformanceProfile_t performanceProfile) noexcept

Set the performnace profile to the Psnpe.

Parameters

  • instanceIdx – [in] index of psnpe object

  • performanceProfile – [in] performance profile level

Returns

SNPE_SUCCESS upon successful setting of performance profile

bool setCustomPerfProfile(size_t instanceIdx, DlSystem::SNPEPerfProfile perfProfile) noexcept

Set the custom performnace profile to the Psnpe.

Parameters

  • instanceIdx – [in] index of psnpe object

  • perfProfile – [in] custom performance profile object of SNPEPerfProfile

Returns

SNPE_SUCCESS upon successful setting of performance profile

const char *getLastErrorString()

Get the error message during the failed execution.

Returns

The error message