Introduction

Note

Qualcomm® Gen AI Inference Extensions is also referred to as Genie in the source code and documentation.

Qualcomm® Genie enables running generative transformer models on the Qualcomm AI Runtime stack, for example, on Qualcomm AI Engine Direct (QNN). Support is currently limited to large language models. Genie queries support execution on the QNN HTP backend as well as on Snapdragon CPU with the Genie provided QNN GenAiTransformer backend.

Conventions

Function declarations, function names, type declarations, filenames, directory names and library names are shown in a different font. For example: #include

Commands and code samples appear in a specially formatted code section. For example:

output = (input - offset) * scale.

Mathematical expressions appear in a specially formatted math section. For example:

\[y = x + 1\]

Environment variables appear preceded by $, for example ${SDK_ROOT}.