NVIDIA brings Python-based inference to Jetson’s DLA

NVIDIA has introduced the cuda.bindings.cudla module, which lets applications control the Deep Learning Accelerator (DLA) on Jetson AGX Orin directly from Python. It connects the DLA to the GPU through shared CUDA streams: an application can send neural-network inference operations to the DLA while leaving preprocessing and postprocessing to the GPU. NVIDIA’s article walks through the full process, from loading a compiled model to reading its output, in nine steps that include setting up the device, allocating memory and synchronizing operations. The same interface also works on NVIDIA RTX Spark under Windows.

Figure 1. Software stack: from a Python application to DLA hardware on Jetson Orin through cuda.bindings.cudla

Source: developer.nvidia.com

Figure 2. Compile a standalone DLA loadable once and run inference in Python with JetPack libcudla.so

Source: developer.nvidia.com

=== HEADLINE ===

NVIDIA brings Python-based inference to Jetson’s DLA