File IUserBuffer.hpp

↰ Parent directory (DlSystem)

Definition (DlSystem/IUserBuffer.hpp)

Includes

Full File Listing

namespace DlSystem
class UserBufferEncoding
#include <IUserBuffer.hpp>

Subclassed by DlSystem::UserBufferEncodingBool, DlSystem::UserBufferEncodingFloat, DlSystem::UserBufferEncodingFloatN, DlSystem::UserBufferEncodingIntN, DlSystem::UserBufferEncodingTfN, DlSystem::UserBufferEncodingUintN, DlSystem::UserBufferEncodingUnsigned8Bit

Public Types

enum class ElementType_t

An enum class of all supported element types in a IUserBuffer

Values:

enumerator UNKNOWN = 0

Unknown element type.

enumerator FLOAT = 1

Each element is presented by 32-bit float.

enumerator UNSIGNED8BIT = 2

Each element is presented by an unsigned int.

enumerator FLOAT16 = 3

Each element is presented by 16-bit float.

enumerator TF8 = 10

Each element is presented by an 8-bit quantized value.

enumerator TF16 = 11

Each element is presented by an 16-bit quantized value.

enumerator INT32 = 12

Each element is presented by Int32.

enumerator UINT32 = 13

Each element is presented by UInt32.

enumerator INT8 = 14

Each element is presented by Int8.

enumerator UINT8 = 15

Each element is presented by UInt8.

enumerator INT16 = 16

Each element is presented by Int16.

enumerator UINT16 = 17

Each element is presented by UInt16.

enumerator BOOL8 = 18
enumerator INT64 = 19
enumerator UINT64 = 20

Public Functions

virtual ~UserBufferEncoding() = default
UserBufferEncoding(UserBufferEncoding &&other) noexcept
ElementType_t getElementType() const noexcept

Retrieves the element type.

Returns

Element type

size_t getElementSize() const noexcept

Retrieves the size of the element, in bytes.

Returns

Size of the element, in bytes.

class UserBufferSource
#include <IUserBuffer.hpp>

Subclassed by DlSystem::UserBufferSourceGLBuffer

Public Types

enum class SourceType_t

Values:

enumerator UNKNOWN = 0

Unknown buffer source type.

enumerator CPU = 1

The network inputs are from CPU buffer.

enumerator GLBUFFER = 2

The network inputs are from OpenGL buffer.

Public Functions

SourceType_t getSourceType() const noexcept

Retrieves the source type.

Returns

Source type

Protected Functions

UserBufferSource(HandleType handle)
class UserBufferSourceGLBuffer : public DlSystem::UserBufferSource
#include <IUserBuffer.hpp>

Public Functions

UserBufferSourceGLBuffer()

An source type where input data is delivered from OpenGL buffer

class UserBufferEncodingUnsigned8Bit : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingUnsigned8Bit()

Copy Constructor for UserBufferEncodingUnsigned8Bit.

An encoding type where each element is represented by an unsigned int.

Userbuffer size assumes uint8 encoding for each element. (i.e., a tensor with dimensions (2,3) will be represented by (2 * 3) * 1 = 6 bytes in memory).

Returns

a handle to the UserBufferEncodingUnsigned8Bit

UserBufferEncoding(UserBufferEncoding &&other) noexcept
class UserBufferEncodingFloatN : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingFloatN(uint8_t bWidth = 32)

An encoding type where each element is represented by a float N.

Userbuffer size assumes float N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

Parameters

bWidth – [in] is bitWidth N.

UserBufferEncodingFloatN(const UserBufferEncodingFloatN &other)

Copy Constructor for UserBufferEncodingFloatN.

An encoding type where each element is represented by a float N

Userbuffer size assumes float N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

Parameters

other – [in] : Source UserBufferEncodingFloatN to copy from

Returns

a handle to the constructed UserBufferEncodingFloatN

UserBufferEncoding(UserBufferEncoding &&other) noexcept

Public Static Functions

static ElementType_t getTypeFromWidth(uint8_t width)
class UserBufferEncodingFloat : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingFloat()

An encoding type where each element is represented by a float. Userbuffer size assumes float encoding for each element. (i.e., a tensor with dimensions (2,3) will be represented by (2 * 3) * 4 = 24 bytes in memory).

UserBufferEncodingFloat(const UserBufferEncodingFloat &other)

Copy Constructor for UserBufferEncodingFloat.

An encoding type where each element is represented by a float.

Userbuffer size assumes float encoding for each element. (i.e., a tensor with dimensions (2,3) will be represented by (2 * 3) * 4 = 24 bytes in memory).

Parameters

otherHandle – [in] : a handle to another UserBufferEncodingFloat to copy

Returns

a handle to the constructed UserBufferEncodingFloat

UserBufferEncodingFloat(UserBufferEncodingFloat &&other) noexcept

Copy Constructor for UserBufferEncodingFloat.

An encoding type where each element is represented by a float.

Userbuffer size assumes float encoding for each element. (i.e., a tensor with dimensions (2,3) will be represented by (2 * 3) * 4 = 24 bytes in memory).

Parameters

otherHandle – [in] : a handle to another UserBufferEncodingFloat to copy

Returns

a handle to the constructed UserBufferEncodingFloat

UserBufferEncoding(UserBufferEncoding &&other) noexcept
class UserBufferEncodingTfN : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Subclassed by DlSystem::UserBufferEncodingTf8

Public Functions

template<typename T, typename U, typename std::enable_if<std::is_integral<T>::value && std::is_floating_point<U>::value, int>::type = 0>
UserBufferEncodingTfN(T stepFor0, U stepSize, uint8_t bWidth = 8)

An encoding type where each element is represented by tfN, which is an N-bit quantized value, which has an exact representation of 0.0.

Userbuffer size assumes tf N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

UserBufferEncodingTfN(const UserBufferEncoding &ubEncoding)

Copy Constructor for UserBufferEncodingTfN.

An encoding type where each element is represented by tfN, which is an N-bit quantized value, which has an exact representation of 0.0

Userbuffer size assumes tf N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

Parameters

otherHandle – the UserBufferEncodingTfN to copy

Returns

a handle to a newly constructed UserBufferEncodingTfN

UserBufferEncodingTfN(const UserBufferEncodingTfN &ubEncoding)

Copy Constructor for UserBufferEncodingTfN.

An encoding type where each element is represented by tfN, which is an N-bit quantized value, which has an exact representation of 0.0

Userbuffer size assumes tf N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

Parameters

otherHandle – the UserBufferEncodingTfN to copy

Returns

a handle to a newly constructed UserBufferEncodingTfN

void setStepExactly0(uint64_t stepExactly0)

Sets the step value that represents 0.

Parameters

  • userBufferEncodingHandle – [in] : Handle to access the encoding

  • stepExactly0 – [in] : The step value that represents 0

void setQuantizedStepSize(const float quantizedStepSize)

Sets the float value that each step represents.

Parameters

  • userBufferEncodingHandle – [in] : Handle to access the encoding

  • quantizedStepSize – [in] : The float value of each step size

uint64_t getStepExactly0() const

Retrieves the step that represents 0.0.

Parameters

userBufferEncodingHandle – [in] : Handle to access the encoding

Returns

Step value

float getMin() const

Calculates the minimum floating point value that can be represented with this encoding.

Parameters

userBufferEncodingHandle – [in] : Handle to access the encoding

Returns

Minimum representable floating point value

float getMax() const

Calculates the maximum floating point value that can be represented with this encoding.

Parameters

userBufferEncodingHandle – [in] : Handle to access the encoding

Returns

Maximum representable floating point value

float getQuantizedStepSize() const

Retrieves the step size.

Parameters

userBufferEncodingHandle – [in] : Handle to access the encoding

Returns

Step size

UserBufferEncoding(UserBufferEncoding &&other) noexcept

Public Static Functions

static ElementType_t getTypeFromWidth(uint8_t width)

Get the tfN type corresponding to a given bitwidth.

Parameters

width – bitwidth of tfN type

Returns

ElementType corresponding to a tfN of width bits

class UserBufferEncodingIntN : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingIntN(uint8_t bWidth = 32)

An encoding type where each element is represented by a Int.

Userbuffer size assumes int N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 32 will be represented by (2 * 3) * 4 = 24 bytes in memory).

UserBufferEncodingIntN(const UserBufferEncoding &ubEncoding)

Copy Constructor for UserBufferEncodingIntN.

An encoding type where each element is represented by a Int

Userbuffer size assumes int N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 32 will be represented by (2 * 3) * 4 = 24 bytes in memory).

Parameters

otherHandle – the UserBufferEncodingIntN to copy

Returns

a handle to a newly constructed UserBufferEncodingIntN

Public Static Functions

static ElementType_t getTypeFromWidth(uint8_t width)

Get the int type corresponding to a given bitwidth.

Parameters

width – bitwidth of int type

Returns

ElementType corresponding to a int of width bits

class UserBufferEncodingUintN : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingUintN(uint8_t bWidth = 32)

An encoding type where each element is represented by a Uint.

Userbuffer size assumes uint N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 32 will be represented by (2 * 3) * 4 = 24 bytes in memory).

UserBufferEncodingUintN(const UserBufferEncoding &ubEncoding)

Copy Constructor for UserBufferEncodingUintN.

An encoding type where each element is represented by a Uint

Userbuffer size assumes uint N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 32 will be represented by (2 * 3) * 4 = 24 bytes in memory).

Parameters

otherHandle – the UserBufferEncodingUintN to copy

Returns

a handle to a newly constructed UserBufferEncodingUintN

Public Static Functions

static ElementType_t getTypeFromWidth(uint8_t width)

Get the uint type corresponding to a given bitwidth.

Parameters

width – bitwidth of uint type

Returns

ElementType corresponding to a uint of width bits

class UserBufferEncodingTf8 : public DlSystem::UserBufferEncodingTfN
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingTf8() = delete
template<typename T, typename U, typename std::enable_if<std::is_integral<T>::value && std::is_floating_point<U>::value, int>::type = 0>
UserBufferEncodingTf8(T stepFor0, U stepSize)
UserBufferEncodingTf8(const UserBufferEncoding &ubEncoding)
template<typename T, typename U, typename std::enable_if<std::is_integral<T>::value && std::is_floating_point<U>::value, int>::type = 0>
UserBufferEncodingTfN(T stepFor0, U stepSize, uint8_t bWidth = 8)

An encoding type where each element is represented by tfN, which is an N-bit quantized value, which has an exact representation of 0.0.

Userbuffer size assumes tf N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

UserBufferEncodingTfN(const UserBufferEncoding &ubEncoding)

Copy Constructor for UserBufferEncodingTfN.

An encoding type where each element is represented by tfN, which is an N-bit quantized value, which has an exact representation of 0.0

Userbuffer size assumes tf N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

Parameters

otherHandle – the UserBufferEncodingTfN to copy

Returns

a handle to a newly constructed UserBufferEncodingTfN

UserBufferEncodingTfN(const UserBufferEncodingTfN &ubEncoding)

Copy Constructor for UserBufferEncodingTfN.

An encoding type where each element is represented by tfN, which is an N-bit quantized value, which has an exact representation of 0.0

Userbuffer size assumes tf N encoding for each element. (i.e., a tensor with dimensions (2,3) with a provided bitwidth of 16 will be represented by (2 * 3) * 2 = 12 bytes in memory).

Parameters

otherHandle – the UserBufferEncodingTfN to copy

Returns

a handle to a newly constructed UserBufferEncodingTfN

class UserBufferEncodingBool : public DlSystem::UserBufferEncoding
#include <IUserBuffer.hpp>

Public Functions

UserBufferEncodingBool(uint8_t bWidth = 8)
UserBufferEncodingBool(const UserBufferEncoding &ubEncoding)
class IUserBuffer : public Wrapper<IUserBuffer, Snpe_IUserBuffer_Handle_t, true>
#include <IUserBuffer.hpp>

Public Functions

const TensorShape &getStrides() const

Retrieves the total number of bytes between elements in each dimension if the buffer were to be interpreted as a multi-dimensional array.

Returns

A const reference to the number of bytes between elements in each dimension. e.g. A tightly packed tensor of floats with dimensions [4, 3, 2] would return strides of [24, 8, 4].

Warning

Do not modify the TensorShape returned by reference. Treat it as a const reference.

size_t getSize() const

Retrieves the size of the buffer, in bytes.

Returns

Size of the underlying buffer, in bytes.

size_t getOutputSize() const

Retrieves the size of the inference data in the buffer, in bytes.

The inference results from a dynamic-sized model may not be exactly the same size as the UserBuffer provided to SNPE. This function can be used to get the amount of output inference data, which may be less or greater than the size of the UserBuffer.

If the inference results fit in the UserBuffer, getOutputSize() would be less than or equal to getSize(). But if the inference results were more than the capacity of the provided UserBuffer, the results would be truncated to fit the UserBuffer. But, getOutputSize() would be greater than getSize(), which indicates a bigger buffer needs to be provided to SNPE to hold all of the inference results.

Returns

Size required for the buffer to hold all inference results, which can be less or more than the size of the buffer, in bytes.

uint64_t getAddrOffset() const

Retrieves the byte offset to the starting address of the buffer.

Returns

byte offset of the buffer from total allocated memory address.

bool setBufferAddress(void *buffer) noexcept

Changes the underlying memory that backs the UserBuffer.

This can be used to avoid creating multiple UserBuffer objects when the only thing that differs is the memory location.

Parameters

buffer – [in] : Pointer to the memory location

Returns

Whether the set succeeds.

bool setBufferAddressOffset(uint64_t offset) noexcept

Updates the address offset of the UserBuffer.

Parameters

offset – [in] : byte offset to the starting address of the UserBuffer.

Returns

Whether the set succeeds.

const UserBufferEncoding &getEncoding() const noexcept

Gets a reference to the data encoding object of the underlying buffer.

This is necessary when the UserBuffer is re-used, and the encoding parameters can change. For example, each input can be quantized with different step sizes.

Returns

Data encoding meta-data

UserBufferEncoding &getEncoding() noexcept

Gets a reference to the data encoding object of the underlying buffer.

This is necessary when the UserBuffer is re-used, and the encoding parameters can change. For example, each input can be quantized with different step sizes.

Returns

Data encoding meta-data

Private Members

friend BaseType

Private Static Attributes

static constexpr DeleteFunctionType DeleteFunction = {Snpe_IUserBuffer_Delete}