Genie Dialog Engine Sharing

This feature allows an engine to be shared across multiple dialogs which might be of different dialog types and have different features like LoRA Config for the same context binaries.

Note

  • Engine sharing allows multiple dialogs to share the same instance of an engine, with their own KV Manager, IO Buffer, and LoRA Config.

  • This will reduce memory footprint.

  • Supported Dialogs: Eaglet, SSD, Basic

  • Supported Features: IO Buffers/KVManager and LoRA Config

  • Supported Backends: Qnn-HTP

Comparison: Without and With Engine Sharing

The following diagram illustrates the difference between without engine sharing and with engine sharing:

Engine Sharing Comparison - Without and With Engine Sharing

The diagram shows how engine sharing allows multiple dialogs to share the same engine instance while maintaining their individual KV Manager, IO Buffer, and LoRA Config components.