File IUserBufferFactory.hpp

↰ Parent directory (DlSystem)

Definition (DlSystem/IUserBufferFactory.hpp)

Includes

Full File Listing

namespace DlSystem
class IUserBufferFactory
#include <IUserBufferFactory.hpp>

Public Functions

IUserBufferFactory()
std::unique_ptr<IUserBuffer> createUserBuffer(void *buffer, size_t bufSize, const TensorShape &strides, UserBufferEncoding *userBufferEncoding) noexcept

Creates a UserBuffer.

Parameters

  • buffer – [in] Pointer to the buffer that the caller supplies

  • bufSize – [in] Buffer size, in bytes

  • strides – [in] Total number of bytes between elements in each dimension. E.g. A tightly packed tensor of floats with dimensions [4, 3, 2] would have strides of [24, 8, 4].

  • userBufferEncoding – [in] Reference to an UserBufferEncoding object

Note

Caller has to ensure that memory pointed to by buffer stays accessible for the lifetime of the object created

std::unique_ptr<IUserBuffer> createUserBufferShared(void *buffer, size_t bufSize, uint64_t addrOffset, const TensorShape &strides, UserBufferEncoding *userBufferEncoding) noexcept

Creates a UserBuffer.

Parameters

  • buffer – [in] Pointer to the buffer that the caller supplies.

  • bufSize – [in] Buffer size, in bytes.

  • addrOffset – [in] byte offset for sharing a buffer amongst tensors.

  • strides – [in] Total number of bytes between elements in each dimension. E.g. A tightly packed tensor of floats with dimensions [4, 3, 2] would have strides of [24, 8, 4].

  • userBufferEncoding – [in] Reference to an UserBufferEncoding object

Note

Caller has to ensure that memory pointed to by buffer stays accessible for the lifetime of the object created

std::unique_ptr<IUserBuffer> createUserBuffer(void *buffer, size_t bufSize, const TensorShape &strides, UserBufferEncoding *userBufferEncoding, UserBufferSource *userBufferSource) noexcept

Creates a UserBuffer.

Parameters

  • buffer – [in] Pointer to the buffer that the caller supplies

  • bufSize – [in] Buffer size, in bytes

  • strides – [in] Total number of bytes between elements in each dimension. E.g. A tightly packed tensor of floats with dimensions [4, 3, 2] would have strides of [24, 8, 4].

  • userBufferEncoding – [in] Reference to an UserBufferEncoding object

  • userBufferSource – [in] Reference to an UserBufferSource object

Note

Caller has to ensure that memory pointed to by buffer stays accessible for the lifetime of the object created

bool Float32ToTfN(const float *inputBuffer, size_t inputBufferSizeBytes, bool isDynamicEncoding, float &scale, uint64_t &offset, void *outputBuffer, size_t outputBufferSizeBytes, unsigned bitWidth) noexcept

Converts/Quantizes a float32 buffer to an unsigned fixed point 8/16 bit buffer.

Parameters

  • inputBuffer – [in] Pointer to the float32 buffer that the caller supplies

  • inputBufferSizeBytes – [in] Input buffer size, in bytes. Must be at least 4 bytes

  • isDynamicEncoding – [in] When set to false, scale and offset provided (next 2 argumets) will be used to qunatize. When set to true, scale and offset provided will be ignored and min/max will be computed from the inputBuffer to derrive the scale and offset for quantization. The computed scale and offset will be populated back to the caller in the next two arguments

  • [in/out] – scale Qunatization encoding scale to be passed in by the caller when isDynamicEncoding is true. When isDynamicEncoding is false this argument is ignored and is populated as an outparam.

  • [in/out] – offset Qunatization encoding offset to be passed in by the caller when isDynamicEncoding is true. When isDynamicEncoding is false this argument is ignored and is populated as an outparam.

  • outputBuffer – [in] Pointer to the unsigned fixed point 8/16 bit buffer that the caller supplies

  • outputBufferSizeBytes – [in] Output buffer size, in bytes. Must be large enough for all the inputs

  • bitWidth – [in] Quantization bitwidth (8 or 16)

Note

This API can do in-place quantization i.e. when outputBuffer equals inputBuffer pointer. But the output buffer CANNOT start at an offset from input buffer which will lead to corrupting inputs at sbsequent indexes.

bool TfNToFloat32(const void *inputBuffer, size_t inputBufferSizeBytes, float scale, uint64_t offset, float *outputBuffer, size_t outputBufferSizeBytes, unsigned bitWidth) noexcept

Converts/De-Quantizes an unsigned fixed point 8/16 bit buffer to a float32 buffer.

Parameters

  • inputBuffer – [in] Pointer to the unsigned fixed point 8/16 bit buffer that the caller supplies

  • inputBufferSizeBytes – [in] Input buffer size, in bytes

  • [in/out] – scale Qunatization encoding scale to be passed in by the caller

  • [in/out] – offset Qunatization encoding offset to be passed in by the caller

  • outputBuffer – [in] Pointer to the float32 buffer that the caller supplies

  • outputBufferSizeBytes – [in] Output buffer size, in bytes. Must be large enough for all the inputs

  • bitWidth – [in] Quantization bitwidth (8 or 16)

Note

This API CANNOT in place de-quantization. The input and output buffers must have no overlap