Windows

Basic Dialog

This section contains tutorials that pertain to running the Llama 2 7B model on HTP within a basic dialog.

Dialog JSON Configuration

See Genie Dialog JSON configuration string for details on the fields and what they mean. An example model_config can be found at ${QNN_SDK_ROOT}/examples/Genie/configs/llama2-7b/llama2-7b-htp-windows.json. Note that the tokenizer path and context binary fields will need to be updated based on your actual preparation steps.

Inference

Open Developer PowerShell for VS2022 on Windows on Snapdragon host and run:

# Make sure environment is setup as per instructions, or can cd into bin folder on Windows host
cd <QNN_SDK_ROOT>\bin\aarch64-windows-msvc
.\genie-t2t-run.exe -c <path to llama2-7b-htp.json>
                    -p "Tell me about Qualcomm"