ONNC on NVDLA AI
Exploits the full power of the underlying AI chips
ONNCx NVDLA Release 1.4.0
- Version: r1.4.0
- Release Date: 2019/8/30
ONNC on NVDLA AI starter kits provides software development kit (SDK) and board support package (BSP) for IC design houses who plan to develop its own deep learning accelerator (DLA) based on NVDLA.
It’s a comprehensive set of FPGA-based prototyping tools including AI-related hardware and software components:
- compiler (ONNC)
- drivers (KMD/UMD)
- Linux kernel
- virtual platform(GreenSocs)
- FPGA netlist
- NVDLA verilog RTL code
AI Starter Kits optimize your design process and ensures that your software runs and exploits the full power of the underlying AI chips. Skymizer aims at providing a unique solution tailored to your needs to help you build a strong hardware/software co-design team and efficiently save your time ahead of AI chip fabrication.
AI Starter Kits version r1.4.0 brings you native calibration interface for NVDLA INT8 Quantization.
The demand for AI model computing power capacity doubles every 100 days. The surging demand forces developers to exploit more performance and save more power from hardware and AI model. Quantizing AI model is a popular and effective approach. This approach reduces the model size, the amount of calculation and power consumption simultaneously.
Open Neural Network Compiler (ONNC) provides a native calibration interface allows the user to directly operate the NVDLA’s quantization register. There are many quantization tools providing quantization supports only the layer level. Layer-level calibration usually causes unacceptable precision loss when DLA compilers transform a model-level layer into multiple hardware-level layers. The transformation will cause large precision loss if developers don’t adjust scaling factors for each hardware-level layer. To resolve this issue, ONNC r1.4.0 provides native calibration interfaces. Users can adjust scaling factors and offsets for each hardware layers.
In our upcoming release, we will release a hardware calibrator to minimize the precision loss during post-training quantization process on NVDLA.
Workflow of Native calibration interface
This figures bellow shows the workflow of the native calibration interface. Users can assign scaling factors and offset to each quantization register inside NVDLA by a simple JSON file. This JSON file is also refered to as the Calibration Table (Ctable). In the next process, ONNC will used the content of the Ctable to do calibration.
For more information about the native calibration interface specification please refer to the description here.
- Add 3 more quantization json examples: inception_v1, vgg19, zfnet512
- Add image mode quantization json examples for 7 models: alexnet, googlenet, caffenet, rcnn_ilsvrc13, inception_v1, vgg19, zfnet512
- New loadable version
- Using L2 threshold searching algorithm for all quantization json example
- Fix compiler error in specific STL implementation