QNN HTP

See Genie Embedding JSON configuration string for details on the fields and what they mean. An example model_config can be found at ${QNN_SDK_ROOT}/examples/Genie/configs/bge/bge-large-htp.json. Note that the tokenizer path and context binary fields will need to be updated based on your actual preparation steps.

Note

Use regular QNN flow to get required context binaries of BGE model.

To run on QNN HTP backend, open a command shell on android and run the following. This assumes that the HTP architecture has been set (e.g., ARCH=79). Please use the steps mentioned above for libraries, binaries, tokenizer and backend_ext_config.

Note

Results will be saved in output.raw file in working directory.

adb shell mkdir -p /data/local/tmp/
adb push <path to bge-large-htp.json> /data/local/tmp/

# open adb shell
adb shell
export LD_LIBRARY_PATH=/data/local/tmp/
export PATH=$LD_LIBRARY_PATH:$PATH

cd $LD_LIBRARY_PATH
./genie-t2e-run -c <path to bge-large-htp.json>
                -p "What is the most popular cookie in the world?"