Tutorial - Using TFLite Delegate With a Java Application

This page will outline the setup required for using the Qualcomm® AI Engine Direct Delegate. Currently, only C APIs are provided. Aarch64-Android is supported. For other platforms or architectures, please check libraries under lib directory.

Prerequisites

1. Qualcomm® AI Engine Direct libraries, which are in Qualcomm® AI Engine Direct SDK, lib directory. We use $QNN_SDK_ROOT for the root of unzipped Qualcomm® AI Engine Direct SDK.

2. The TensorFlow Lite Android C API integrated into a C/C++ Android application. Check release notes for the supported TensorFlow version.

3. A Qualcomm device with Android Debug Bridge (ADB) connected.

4. Qualcomm® AI Engine Direct Delegate artifacts.

Android Java Application

The Qualcomm® AI Engine Direct Delegate Java Interface provides an API for Android applications through an AAR in the SDK at $QNN_SDK_ROOT/lib/android/qtld-release.aar.

Users can also obtain packages via Maven Central. Taking QNN 2.29.0 as an example, please include these packages as dependencies: pkg:maven/com.qualcomm.qti/qnn-runtime@2.29.0 (https://central.sonatype.com/artifact/com.qualcomm.qti/qnn-runtime), pkg:maven/com.qualcomm.qti/qnn-litert-delegate@2.29.0 (https://central.sonatype.com/artifact/com.qualcomm.qti/qnn-litert-delegate)

Following are the steps to create the delegate:

• Step 1: Setting up the Qualcomm® AI Engine Direct Delegate libraries:

If you are using AAR in the SDK, open the top build.gradle and add a library directory as follows:

allprojects {
   repositories {
      flatDir {
            dirs 'libs'
      }
   }
}

Now create the libs folder under app and place the AAR there.

Next, include the AAR into the app’s build.gradle with the following:

implementation(name:'qtld-release', ext:'aar')

Then, add Qualcomm® AI Engine Direct libraries from SDK to your Android APK. Please notice that this is different from Qualcomm® AI Engine Direct Delegate libraries, which are automatically included in the APK by way of the qtld-release.aar module. E.g. Create app/src/main/jniLibs/arm64-v8a directory and add the following Qualcomm® AI Engine Direct libraries to the directory. Please replace * below with the desired DSP/HTP version.

$QNN_SDK_ROOT/lib/aarch64-android/libQnnDsp.so
$QNN_SDK_ROOT/lib/aarch64-android/libQnnDspV*Stub.so
$QNN_SDK_ROOT/lib/hexagon-v*/unsigned/libQnnDspV*Skel.so
$QNN_SDK_ROOT/lib/aarch64-android/libQnnGpu.so
$QNN_SDK_ROOT/lib/aarch64-android/libQnnHtp.so
$QNN_SDK_ROOT/lib/aarch64-android/libQnnHtpPrepare.so
$QNN_SDK_ROOT/lib/aarch64-android/libQnnHtpV*Stub.so
$QNN_SDK_ROOT/lib/hexagon-v*/unsigned/libQnnHtpV*Skel.so
$QNN_SDK_ROOT/lib/aarch64-android/libQnnSystem.so

If you are using Maven Central, open the top build.gradle and add mavenCentral() to the repositories block as follows:

allprojects {
   repositories {
      mavenCentral()
   }
}

Next, include the packages under app’s build.gradle. Taking QNN 2.26.0 as an example:

dependencies {
   implementation 'com.qualcomm.qti:qnn-runtime:2.26.0'
   implementation 'com.qualcomm.qti:qnn-tflite-delegate:2.26.0'
}

Notes: Whether using a manually-packaged QNN or the qnn-runtime package from Maven, you can reduce your APK size by removing unused backends. E.g. If you are not targeting the GPU backend, you don’t need libQnnGpu.so. Under the app’s build.gradle, exclude libQnnGpu.so as follows:

packagingOptions {
   exclude 'lib/arm64-v8a/libQnnGpu.so'
}

• Step 2: Additional Configuration

Under app’s build.gradle, please configure to use the legacy packaging as follows:

packagingOptions {
     jniLibs.useLegacyPackaging true
}

Grant the app permission to load libcdsprpc.so, a requirement for QNN. Mark it as not required in order to maintain compatibility on non-Qualcomm devices. Please configure app’s AndroidManifest.xml as follows:

<uses-native-library
    android:name="libcdsprpc.so"
    android:required="false" />

• Step 3: Prevent ProGuard from eliminating unused function

Users should only be performing this step if ProGuard is set on Qualcomm® AI Engine Direct libraries. ProGuard will eliminate code identified as unused but actually used by native code, which will cause runtime errors. To prevent this from happening, it is recommended to add the following to the app’s proguard-rules.pro:

-keep class com.qualcomm.qti.** { *; }

• Step 4: Create a delegate and initialize a TensorFlow Lite Interpreter

An example of using HTP backend is below:

import com.qualcomm.qti.QnnDelegate;
private QnnDelegate qnnDelegate = null;
try {
   // Created a default Options
   QnnDelegate.Options options = new QnnDelegate.Options();
   // Set the backend and library path
   options.setBackendType(QnnDelegate.Options.BackendType.HTP_BACKEND);
   options.setSkelLibraryDir(activity.getApplicationInfo().nativeLibraryDir);
   // Create the Delegate instance.
   qnnDelegate = new QnnDelegate(options);
   tfliteOptions.addDelegate(qnnDelegate);
}
catch (UnsupportedOperationException e) {
   // Delegate creation failed
}
tfliteInterpreter = new Interpreter(tfliteModel, tfliteOptions);

Note that the $ADSP_LIBRARY_PATH needs to be set only if the Skel library is required. The delegate provides the Options.setSkelLibraryDir() public method for setting $ADSP_LIBRARY_PATH variable for the user.