Overview

Contents

The TFLite Delegate provides several ways to support running TFLite models on target devices. This library extends the QNN SDK, resulting in nearly identical workflows to prepare and execute a model, as well as for implementing an application which uses the model on the target device.

To see exactly what is different about those workflows, you will need to complete both of these tutorials:

1. Preparing and executing a model with TFLite Delegate.

2. Buliding an application that uses TFLite Delegate:

   1. C/C++ (for Linux)

   2. Java (for Android)

Navigating These Docs

Section	Description
Tutorials	These explain how to use this API. This is where you should start. The rest of these docs primarily explain how each feature works in case you need additional context for your specific use case.
Acceleration Support	Additional context on what specifically is supported for each backend (e.g., operators supported on GPUs).
Tools	Reference docs for CLI tools like qtld-net-run.
Custom Operator Support	Detailed instructions for writing your own operations (e.g., Abs) if you want to optimize the implementation or add support for a missing operator.
Frequently Asked Questions	Common errors and what to do about them. Useful during troubleshooting.
API	Reference docs for the C, Java, and External APIs. See the tutorials section for example code using these APIs.
Qualcomm® AI Engine Direct Backend Library	This page explains which library files from the QNN SDK are relevant based on your chosen backend (e.g., libQnnGpu.so for GPU backends).
Release Notes	QNN TFLite Delegate release notes log and details.

TFLite Delegate Files

These are the files that you may need as you use the TFLite Delegate:

<QNN_SDK_ROOT>
├── bin
│   ├── aarch64-android
│   │   └── qtld-net-run
│   ├── aarch64-oe-linux-gcc9.3
│   │   └── qtld-net-run
│   ├── aarch64-oe-linux-gcc11.2
│   │   └── qtld-net-run
│   └── aarch64-ubuntu-gcc9.4
│       └── qtld-net-run
├── docs
├── examples
├── include
│   └── QNN
│       └── TFLiteDelegate
│           └── QnnTFLiteDelegate.h
├── lib
│   ├── aarch64-android
│   │   └── libQnnTFLiteDelegate.so
│   ├── aarch64-oe-linux-gcc9.3
│   │   └── libQnnTFLiteDelegate.so
│   ├── aarch64-oe-linux-gcc11.2
│   │   └── libQnnTFLiteDelegate.so
│   ├── aarch64-ubuntu-gcc9.4
│   │   └── libQnnTFLiteDelegate.so
│   └── android
│       └── qtld-release.aar
├── LICENSE.pdf
├── QNN_TFLITE_DELEGATE_NOTICE.txt
├── QNN_TFLITE_DELEGATE_README.txt
└── QNN_TFLITE_DELEGATE_ReleaseNotes.txt

Each folder under <QNN_SDK_ROOT> has the following artifacts:

Path / File	Description
docs/QNN/TFLiteDelegate	A folder of HTML files that contain documentation for the Qualcomm® AI Engine Direct Delegate.
examples/QNN/TFLiteDelegate/Models	Scripts that can generate model.
include/QNN/TFLiteDelegate/QnnTFLiteDelegate.h	Qualcomm® AI Engine Direct Delegate header file containing the C interface for the delegate.
QNN_TFLITE_DELEGATE_README.txt	A text file acting as the starting point for this package.
QNN_TFLITE_DELEGATE_ReleaseNotes.txt	The release notes for the current package.
bin/<architecture>/qtld-net-run	An example application used to perform inference with the Qualcomm® AI Engine Direct Delegate. See the Tools page for more information.
lib/<architecture>/libQnnTFLiteDelegate.so	The Qualcomm® AI Engine Direct Delegate dynamic library.
lib/<architecture>/qtld-release.aar	The Qualcomm® AI Engine Direct Delegate AAR.

Next Steps

Learn how to use QNN TFLite Delegate by following the Preparing and executing a model with TFLite Delegate tutorial.