Genie Pipeline

This section contains tutorials that pertain to running GLM-4v on HTP using GeniePipeline and GenieNode APIs. The API invocations are constructed using genie-app tool.

HTP Backend Example Model Config and Backend Extension Config

See Genie Node JSON configuration string for details on the fields and what they mean. Example model_configs and script to tie together Node and Pipeline calls can be found at ${QNN_SDK_ROOT}/examples/Genie/configs/lmm/glm-4v/. Note that the tokenizer path and context binary fields will need to be updated based on your actual preparation steps.

GLM-4V model inference on Android

To run on QNN HTP backend, open a command shell on android and run the following. This assumes that the HTP architecture has been set (e.g., ARCH=75).

adb shell mkdir -p /data/local/tmp/
adb push ${QNN_SDK_ROOT}/bin/aarch64-android/genie-app /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libGenie.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtp.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnSystem.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtpPrepare.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtpNetRunExtensions.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtpV${ARCH}Stub.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/hexagon-v${ARCH}/unsigned/libQnnHtpV${ARCH}Skel.so /data/local/tmp/
adb push <path to htp_backend_ext_config.json> /data/local/tmp/
adb push <path to siglip.json> /data/local/tmp/
adb push <path to text-encoder.json> /data/local/tmp/
adb push <path to glm-4v.json> /data/local/tmp/
adb push <path to tokenizer.json> /data/local/tmp/
adb push <path to model bin files> /data/local/tmp/
adb push <path to LMMScript> /data/local/tmp/

# open adb shell
adb shell

export LD_LIBRARY_PATH=/data/local/tmp/
export PATH=$LD_LIBRARY_PATH:$PATH

cd $LD_LIBRARY_PATH
./genie-app -s <path to LMMScript>