NVIDIA's Tripy Provides a "Debuggable Pythonic Frontend" to TensorRT for Deep Learning Projects

NVIDIA has launched a tool that, it hopes, will make machine learning and artificial intelligence on its graphics processors more accessible: Tripy, a Pythonic frontend to the company's TensorRT ecosystem.

"Tripy is a debuggable, Pythonic frontend for TensorRT, a deep learning inference compiler," the company writes of its early release. "What you can expect: high performance by leveraging TensorRT’s optimization capabilities; an intuitive API [Application Programming Interface] that follows conventions of the ecosystem; debuggability with features like eager mode to interactively debug mistakes; excellent error messages that are informative and actionable; friendly documentation that is comprehensive but concise, with code examples."

NVIDIA's last few generations of graphics processors have been moving away from generating pictures and towards machine learning and artificial intelligence (ML and AI) workloads — adding dedicated Tensor Cores, alongside the general-purpose CUDA cores and raytracing-specific RT Cores. To make best use of these, NVIDIA launched TensorRT — an ecosystem that includes a compiler designed to optimize neural network models for best performance on NVIDIA GPUs.

Now, those same advantages are available to developers used to working in Python via Tripy — a frontend for TensorRT. Using Tripy, developers can use Python 3 to define models, initialize them, execute them, and compile them to improve performance. The frontend also includes support for post-training quantization, a process by which a model is scaled to a lower precision in order to boost performance or add compatibility for lower-end hardware.

Available on GitHub now as part of the experimental TensorRT Incubator repository, Tripy is at the time of writing tagged as version 0.0.10 — suggesting it may not be ready for production use quite yet. The source code is made available under the permissive Apache 2.0 license.