Offline Flow : Conversion (WIP)

Convert the Base Model

  • Same Converter/Quantizer for LoRA and non-LoRA usages

  • Converter has a new “–lora_weight_list” parameter to pass LoRA Weights Names, to identify as “updatable tensors” in the graph

../../_static/resources/lora/qnn_tutorial_lora_offline_conversion_base_model.png

Import the LoRA Adapters via. qairt-lora-importer

Note: This feature is currently supported only for ONNX models

../../_static/resources/lora/qnn_tutorial_lora_offline_conversion_lora_adapters.png

What is the format of “LoRA Weights Names” provided to qairt-converter tool with –lora_weight_list option?

  • A text file contains LoRA adapter weight tensor names with newline as delimiter

  • Refer to the below sample code that generates a text file with LoRA adapter weight tensor names from LoRA adapter .safetensors file

from safetensors.numpy import load_file

def save_tensor_names(safetensor_path, save_path="./tensor_names.txt"):
    tensor_name_to_data_map = load_file(safetensor_path)
    with open(save_path, 'w') as text_file:
        for tensorAtt_name in tensor_name_to_data_map:
            text_file.write(tensor_name + '\n')