GTC2020 Tutorial - Dive into Deep Learning

Instructors: Rachel Hu (AWS AI), Aston Zhang (AWS AI)

Deep learning is transforming the world nowadays. However, realizing deep learning presents unique challenges because any single application brings together various disciplines. Applying deep learning requires simultaneously understanding:

1. the engineering required to train models efficiently, navigating the pitfalls of numerical computing and getting the most out of available hardware;

2. the mathematics of a given modeling approach;

3. the optimization algorithms for fitting the models to data;

4. and the experience of choosing proper hyperparameters for the solution.

To fulfill the strong wishes of simpler but more practical deep learning materials, Dive into Deep Learning, a unified resource of deep learning was born to achieve the following goals:

• Offering depth theory and runnable code, showing readers how to solve problems in practice;

• Allow for rapid updates, both by us, and also by the community at large;

• Be complemented by a forum for interactive discussions of technical details and to answer questions;

• Be freely available for everyone.

Prerequisites

GPU Fundamentals

• Installations with CUDA

• Basic Operations on GPUs

• Hardware for deep learning

Deep Learning Fundamentals

Here are a few concepts that will be the prerequistes for this lecture. Take a look if some of them are not familiar to you! :)

title	notes	slides
Data Manipulation with Ndarray	D2L	nbviewer
Multilayer Perceptron (MLP)	D2L	nbviewer
Softmax Regression	D2L	nbviewer

Syllabus

In this training, we are going to provide an overview of the in-depth convolutional neural networks (CNN) theory and handy python code. What is more important, the audience would be able to train a simple CNN model on our pre-setup cloud-computing instances for free. Here are the detailed schedule:

Topics	Slides
Dive into Deep Learning	Slides
Fundamental of Convolutional Neural Networks	Slides, Jupyter Notebook
LeNet & AlexNet	Slides, Jupyter Notebook
Intro to Natural Language Processing	Slides
TextCNN on Sentiment Analysis	Jupyter Notebook
Resources and Q&A	Links

Resources and Q&A

• AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on deep learning and real-world applications spanning image, text, or tabular data;

• GluonNLP offers state-of-the-art pretrained NLP models, easy text preprocessing, datasets loading and neural models building;

• GluonCV provides state-of-the-art deep learning models in computer vision and carefully designed APIs that greatly reduce the implementation complexity;

• GluonTS supports deep learning based probabilistic time series modeling;

• Deep Graph Libray develops easy-to-use, high performance and scalable Python package for deep learning on graphs;

• TVM: automatic generates and optimizes tensor operators on more backend with better performance for CPUs, GPUs and specialized accelerators.

If you have any question, please leave us a message at our discussion forum. Have fun diving into deep learning!