Running the Word-RNN Model¶

Overview

The example C++ application in this tutorial is called snpe-net-run. It is a command line executable that executes a neural network using Qualcomm® Neural Processing SDK APIs.

The required arguments to snpe-net-run are:

A neural network model in the DLC file format
An input list file with paths to the input data.

Optional arguments to snpe-net-run are:

Choice of GPU or DSP runtime (default is CPU)
Output directory (default is ./output)
Show help description

snpe-net-run creates and populates an output directory with the results of executing the neural network on the input data.

The Qualcomm® Neural Processing SDK provides Linux and Android binaries of snpe-net-run under

$SNPE_ROOT/bin/x86_64-linux-clang
$SNPE_ROOT/bin/aarch64-android
$SNPE_ROOT/bin/aarch64-oe-linux-gcc8.2
$SNPE_ROOT/bin/aarch64-oe-linux-gcc9.3

Introduction

Recurrent Neural Network (RNN) architectures are widely used in Machine Learning Applications for processing sequential input data. This chapter will show a simple Word-RNN example for predicting the next word in an embedding using Long Short-Term Memory (LSTM). The step-by-step example will create, train, convert, and execute a Word-RNN model with Qualcomm® Neural Processing SDK.

The external python3 packages needed by this example are:

numpy
pandas
sklearn
tensorflow (1.6 or 1.11)

There are six files in $SNPE_ROOT/examples/Models/word_rnn folder

inference.py
input_list.txt
belling_the_cat.txt
word_rnn.py
word_rnn_adb.sh
NOTICE.txt

The word_rnn.py python3 script creates and trains an RNN model with one LSTM layer. After RNN training is done, the corresponding frozen protobuf file will be generated.

The inference.py prompts the user to enter several words, at which point snpe-net-run will be called in a loop to generate subsequent words.

Prerequisites

The Qualcomm® Neural Processing SDK has been set up following the Qualcomm (R) Neural Processing SDK Setup chapter.
The Tutorials Setup has been completed.
TensorFlow is installed (see TensorFlow Setup)

Create, Train, and Convert Word-RNN Model

Run word_rnn.py to create and train the Word-RNN model.

cd $SNPE_ROOT/examples/Models/word_rnn
python3 word_rnn.py

The terminal will show the following messages.

Training will be logged in word_rnn_log.
Load training file belling_the_cat.txt.
Embedding created.
Iter= 1000
Iter= 2000
Iter= 3000
Iter= 4000
Iter= 5000
Optimization done.
Converted 2 variables to const ops.
Save frozen graph in word_rnn.pb.

Then, convert the frozen graph model with snpe-tensorflow-to-dlc.

snpe-tensorflow-to-dlc --input_network word_rnn.pb \
                       --input_dim Placeholder "1, 4, 1" \
                       --out_node "rnn/lstm_cell/mul_11" \
                       --output_path word_rnn.dlc

After dlc conversion, we can view the converted dlc architecture with snpe-dlc-info and snpe-dlc-viewer as follows:

snpe-dlc-info -i word_rnn.dlc
snpe-dlc-viewer -i word_rnn.dlc

Run on Linux Host

Go to the base location for the model and run the python3 script including snpe-net-run

python3 inference.py

After running inference.py, you will see a list of word embedding keys and user input prompt as follows:

Load training file belling_the_cat.txt.
Embedding created.
Use host cpu.
Display word embedding keys:
dict_keys(['long', 'is', 'up', 'it', 'i', 'chief', 'our', 'procured', 'her', 'in', 'mouse', 'council', 'treacherous', 'meet', 'manner', 'approaches', 'with', 'propose', 'which', 'consider', 'thought', 'know', 'bell', 'signal', 'always', 'by', 'small', 'old', 'could', 'about', 'neck', 'of', 'approach', 'well', 'easy', 'take', 'all', 'outwit', 'met', 'they', 'this', 'who', 'cat', 'what', '.', 'will', 'attached', 'their', 'when', 'receive', 'agree', 'applause', 'and', 'if', 'now', 'to', 'a', 'round', 'enemy', 'was', 'ribbon', 'us', 'had', 'general', 'ago', 'means', 'last', 'venture', 'got', 'sly', 'measures', 'young', 'she', 'very', 'impossible', 'therefore', 'we', 'should', 'one', 'mice', 'case', '?', 'make', 'nobody', 'he', 'that', 'consists', 'spoke', 'from', 'easily', 'at', 'neighbourhood', 'the', 'looked', 'then', 'until', 'an', 'common', 'but', 'be', 'would', 'danger', 'retire', 'proposal', 'another', 'you', ',', 'while', 'escape', 'some', 'remedies', 'said'])
Please input 4 words:

User can input embedded words and see the results. For example: long ago , the

...
...
-------------------------------------------------------------------------------
Model String: N/A
SNPE vX.Y.Z.dev
-------------------------------------------------------------------------------
Processing DNN input(s):
./input.raw
-------------------------------------------------------------------------------
Model String: N/A
SNPE vX.Y.Z.dev
-------------------------------------------------------------------------------
Processing DNN input(s):
./input.raw
Inference result: long ago , the said she a , , and the could could , , and the could could , , and the could could , , and the could could , , and the could

The inference.py will call snpe-net-run several times to generate subsequent words with trained LSTM model.

Binary data input

Note that the Word-RNN model does not accept pure text files as input. The model expects its input tensor dimension to be 1x4x1 as a float array. The create_embedding function in inference.py will parse, collect, encode, and build the word embedding. User inputs will then be transformed into a 1x4x1 vector and sent into the LSTM model. Afterwards the LSTM output will be also transformed into the corresponding embedded word.

Run on Target Platform ( Android/LE/UBUN )

Select target architecture

Qualcomm® Neural Processing SDK provides binaries for different target platforms. Android binaries are compiled with clang using libc++ STL implementation. Below are examples for aarch64-android (Android platform) and aarch64-oe-linux-gcc11.2 toolchain (LE platform). Similarly other toolchains for different platforms can be set as SNPE_TARGET_ARCH

# For Android targets: architecture: arm64-v8a - compiler: clang - STL: libc++
export SNPE_TARGET_ARCH=aarch64-android

# Example for LE targets
export SNPE_TARGET_ARCH=aarch64-oe-linux-gcc11.2

For simplicity, this tutorial sets the target binaries to aarch64-android.

Push libraries and binaries to target

Push Qualcomm® Neural Processing SDK libraries and the prebuilt snpe-net-run executable to /data/local/tmp/snpeexample on the Android target. Set SNPE_TARGET_DSPARCH to the DSP architecture of the target Android device.

export SNPE_TARGET_ARCH=aarch64-android
export SNPE_TARGET_DSPARCH=hexagon-v73

adb -s $DEVICE_ID shell "mkdir -p /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin"
adb -s $DEVICE_ID shell "mkdir -p /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib"
adb -s $DEVICE_ID shell "mkdir -p /data/local/tmp/snpeexample/dsp/lib"

adb -s $DEVICE_ID push $SNPE_ROOT/lib/$SNPE_TARGET_ARCH/*.so \
      /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib
adb -s $DEVICE_ID push $SNPE_ROOT/lib/$SNPE_TARGET_DSPARCH/unsigned/*.so \
      /data/local/tmp/snpeexample/dsp/lib
adb -s $DEVICE_ID push $SNPE_ROOT/bin/$SNPE_TARGET_ARCH/snpe-net-run \
      /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin

Set up enviroment variables

Set up the library path, the path variable, and the target architecture in adb shell to run the executable with the -h argument to see its description.

adb -s $DEVICE_ID shell
export SNPE_TARGET_ARCH=aarch64-android
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib
export PATH=$PATH:/data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin
snpe-net-run -h
exit

Push model data to Android target

To execute the Word-RNN model on your Android target follow these steps:

adb -s $DEVICE_ID shell "mkdir -p /data/local/tmp/word_rnn"
adb -s $DEVICE_ID push input_list.txt /data/local/tmp/word_rnn
adb -s $DEVICE_ID push input.raw /data/local/tmp/word_rnn
adb -s $DEVICE_ID push word_rnn.dlc /data/local/tmp/word_rnn
adb -s $DEVICE_ID push word_rnn_adb.sh /data/local/tmp/word_rnn

Note: It may take some time to push the word_rnn dlc file to your target.

Running on Android using CPU Runtime

Run the Android C++ executable with the following commands:

cd /data/local/tmp/word_rnn
snpe-net-run --container word_rnn.dlc --input_list input_list.txt

We will get the same result as when we Run on Linux Host.