Genie Pipeline

This section contains tutorials that pertain to running wt19-en-de on GenAiTransformer using GeniePipeline and GenieNode APIs. The API invocations are constructed using genie-app tool.

GenAiTransformer Backend Example

See Genie Node JSON configuration string for details on the fields and what they mean. Example model_configs and script to tie together Node and Pipeline calls can be found at ${QNN_SDK_ROOT}/examples/Genie/configs/wt19-en-de/.

Model Download

Download the model from: https://huggingface.co/facebook/wmt19-en-de

Model Preparation

Open a command shell on Linux host and run:

# Make sure environment is setup as per instructions, or can cd into bin folder on Linux host
cd ${QNN_SDK_ROOT}/bin/x86_64-linux-clang/
./qnn-genai-transformer-composer --model <path to model> --outfile <output filename with complete path> --export_tokenizer_json --dump_lut

Eg:

qnn-genai-transformer-composer --model wt19_en_de_model --outfile wt19_en_de_model_artifacts/model.bin --export_tokenizer_json --dump_lut

This will generate model_encoder.bin, model_decoder.bin, LUT.bin and tokenizer.json inside wt19_en_de_model_artifacts.

Model Inference on Android

To run on QNN GenAiTransformer backend, open a command shell on android and run the following.

adb shell mkdir -p /data/local/tmp/
adb push ${QNN_SDK_ROOT}/bin/aarch64-android/genie-app /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libGenie.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnGenAiTransformer.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnGenAiTransformerModel.so /data/local/tmp/
adb push ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnGenAiTransformerCpuOpPkg.so /data/local/tmp/
adb push <path to text-encoder.json> /data/local/tmp/
adb push <path to text-generator.json> /data/local/tmp/
adb push <path to LUT.bin> /data/local/tmp/
adb push <path to tokenizer.json> /data/local/tmp/
adb push <path to model bin files> /data/local/tmp/
adb push <path to LMMScript> /data/local/tmp/

# open adb shell
adb shell

export LD_LIBRARY_PATH=/data/local/tmp/
export PATH=$LD_LIBRARY_PATH:$PATH

cd $LD_LIBRARY_PATH
./genie-app -s <path to LMMScript>