Sampler Updates

Note

Please refer to ${SDK_ROOT}/examples/Genie/configs/sampler.json for the parameters that can be updated

Genie provides the flexibility of updating a single parameter and multiple params in one API call

The APIs used for this exercise are:

GenieSamplerConfig_createFromJson

GenieDialog_getSampler

GenieSamplerConfig_setParam

GenieDialogSampler_applyConfig

Example on how to update sampler parameters in between queries

# Create dialog config
GenieDialogConfig_Handle_t dialogConfigHandle = NULL;
GenieDialogConfig_createFromJson(dialogConfigStr, &dialogConfigHandle);

# Create dialog
GenieDialog_Handle_t dialogHandle = NULL;
GenieDialog_create(dialogConfigHandle, &dialogHandle);

# Query with original config
GenieDialog_query(dialogHandle, promptStr, GenieDialog_SentenceCode_t::GENIE_DIALOG_SENTENCE_COMPLETE, queryCallback)

# Get dialog sampler handle
GenieDialogSampler_Handle_t samplerHandle = NULL;
GenieDialog_getSampler(dialogHandle, &samplerHandle);

# Create sampler config with a new sampler config
GenieSamplerConfig_Handle_t samplerConfigHandle = NULL;
GenieSamplerConfig_createFromJson(samplerConfigStr, &samplerConfigHandle);

# Apply the new sampler config
GenieDialogSampler_applyConfig(samplerHandle, samplerConfigHandle);

# Query with updated config
GenieDialog_query(dialogHandle, promptStr, GenieDialog_SentenceCode_t::GENIE_DIALOG_SENTENCE_COMPLETE, queryCallback)

# Update single parameter
GenieSamplerConfig_setParam(samplerConfigHandle, "top-p", "0.8");
GenieSamplerConfig_setParam(samplerConfigHandle, "top-k", "30");

# Apply the new sampler config
GenieDialogSampler_applyConfig(samplerHandle, samplerConfigHandle);

# Query with updated config
GenieDialog_query(dialogHandle, promptStr, GenieDialog_SentenceCode_t::GENIE_DIALOG_SENTENCE_COMPLETE, queryCallback)

# Update multiple parameters(top-k and top-p)
std::string valueStr = "\"sampler\" : {\n      \"top-k\" : 20,\n      \"top-p\" : 0.75\n } "
GenieSamplerConfig_setParam(samplerConfigHandle, "", valueStr.c_str());

# Apply the new sampler config
GenieDialogSampler_applyConfig(samplerHandle, samplerConfigHandle);

# Query with updated config
GenieDialog_query(dialogHandle, promptStr, GenieDialog_SentenceCode_t::GENIE_DIALOG_SENTENCE_COMPLETE, queryCallback)