ONNC Compiler Porting and Optimization for NVDLA-Based Neural Network Inference Engines

Date:
8:00am-11:00am (Taipei Time) Saturday, May 30
8:00pm-11:00pm (New York Time) Friday, May 29

Location: Virtual Conference
Join ISCA event Whova Portal: https://whova.com/portal/webapp/aisca_202004/

Abstract

The NVIDIA Deep Learning Accelerator provides free intellectual property licensing to anyone wanting to build a chip that uses deep neural networks for inference applications. With extensive documentation and tools, many business proposals and research projects choose NVDLA as their inference engine design. However, lack of extensible compiler support becomes the major bottleneck for supporting more AI models and optimizations. This tutorial presents the first open source compiler that supports NVDLA-based designs. The ONNC compiler has more support than the official NVDLA compiler and relieves programmers from manually specifying the low-level details of models that are not supported by the official NVDLA compiler. It also enables opportunities for hardware customization and proprietary optimization. We will cover the overview, porting, and optimizations in three subsections. In each subsection, we will have hands-on labs to demonstrate how to run and customize the NVDLA backend in ONNC for product development and research projects.

Intended Audience

Researchers and practitioners in academia or industry looking for an open-source AI compiler for NVDLA-based neural network inference engines.

Prerequisite

Since this tutorial involves hands-on exercises and labs, please pre-install the docker, follow the instruction in Lab 1. ONNC Working Environment Setup to set up your working environment, and bring your laptop to the sessions.

Tutorial Outline

This year, ISCA tutorials go online as virtual sessions. There are audiences from different time zones. It is difficult for us to choose a time slot that serves everyone. To broadcast the program to friends in the worldwide research community, we have recorded some videos for the majority of the presentation materials for reference. During the live broadcasting,  we will have some interactive sections for a recent update on ONNC and Q & A sections. Welcome to join us online.

Section I – ONNC Open Source Project Overview

ONNC Release Update
Working Environment Setup  (Cheng-Chi Wang) Video  PDF
C-Backend (Po-Yen Chen)  Video PDF

Section II – ONNC NVDLA Backend

ONNC Software Architecture Overview (Po-Yen Chen)  Video   PDF
NVDLA Overview (Ing-June Lu)   Video    PDF
NVDLA Programming Tips (Ing-June Lu) Video    PDF
Porting ONNC to NVDLA (Po-Yen Chen)  Video PDF

Section III – Compiler Optimizations

Graph-Level optimizations (Yang-Ge Ma)  Video   PDF
Hardware-Dependent optimizations (Cheng-Yi Chou)  Video  PDF

Section IV – Joint Projects

ONNC-Wasm Project – A joint project with Second State (Hung Ying Tai) Video   PDF
ONNC-CIM Project – A joint project with ECL Lab@NTU CSIE  (Yu-Jie Wang and Hsiang-Yun Cheng)  Video  PDF

Download

ONNC Tutorial on GitHub
ONNC on GitHub

Organizer & Host

Dr. Wei-Fen Lin is the VP of Engineering at Skymizer Taiwan Inc., where she leads the R&D teams and oversees the development of Skymizer products. She received B.S.E in Electrical Engineering from National Taiwan University, M.S. E. in System Science Engineering  and Ph.D. in Computer Science Engineering from University of Michigan, Ann Arbor. Prior to joining Skymizer, she was a computer architect at high-tech companies in silicon valley and Taiwan. Her research interests include high-performance computing, software/hardware co-design, performance modeling, and optimization. She founded Mijotech Inc and Play Lab to promote STEAM education in her spare time.

Video Speakers

Po-Yen Chen (Skymizer)
Hung Ying Tai (Second State)
Yu-Jie Wang (Skymizer)
Cheng-Chi Wang (Skymizer)
Hsiang-Yun Cheng (Sinica Taiwan)
Ing-June Lu (Skymizer)
Cheng-Yi Chou (Skymizer)
Yang-Ge Ma (Skymizer)