Qualcomm® Gen AI Inference Extensions
v2.39.0
Introduction
Tutorials
Library
Genie C API
Thread safety
Components
GeniePipeline
GenieNode
GenieDialog
GenieEmbedding
GenieProfile
GenieSampler
GenieEngine
GenieTokenizer
Tools
Qualcomm® Gen AI Inference Extensions
»
Library
»
GenieTokenizer
GenieTokenizer
¶
The GenieTokenizer feature provides a user with APIs to encode text and decode tokens.