GTC2020 Tutorial - Dive into Deep Learning

Instructors: Rachel Hu (AWS AI), Aston Zhang (AWS AI)

Deep learning is transforming the world nowadays. However, realizing deep learning presents unique challenges because any single application brings together various disciplines. Applying deep learning requires simultaneously understanding:

  1. the engineering required to train models efficiently, navigating the pitfalls of numerical computing and getting the most out of available hardware;

  2. the mathematics of a given modeling approach;

  3. the optimization algorithms for fitting the models to data;

  4. and the experience of choosing proper hyperparameters for the solution.

To fulfill the strong wishes of simpler but more practical deep learning materials, Dive into Deep Learning, a unified resource of deep learning was born to achieve the following goals:

  • Offering depth theory and runnable code, showing readers how to solve problems in practice;

  • Allow for rapid updates, both by us, and also by the community at large;

  • Be complemented by a forum for interactive discussions of technical details and to answer questions;

  • Be freely available for everyone.

Prerequisites

Deep Learning Fundamentals

Here are a few concepts that will be the prerequistes for this lecture. Take a look if some of them are not familiar to you! :)

title

notes

slides

Data Manipulation with Ndarray

D2L

nbviewer

Multilayer Perceptron (MLP)

D2L

nbviewer

Softmax Regression

D2L

nbviewer

Syllabus

In this training, we are going to provide an overview of the in-depth convolutional neural networks (CNN) theory and handy python code. What is more important, the audience would be able to train a simple CNN model on our pre-setup cloud-computing instances for free. Here are the detailed schedule:

Topics

Slides

Dive into Deep Learning

Slides

Fundamental of Convolutional Neural Networks

Slides, Jupyter Notebook

LeNet & AlexNet

Slides, Jupyter Notebook

Intro to Natural Language Processing

Slides

TextCNN on Sentiment Analysis

Jupyter Notebook

Resources and Q&A

Links

Resources and Q&A

  • AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on deep learning and real-world applications spanning image, text, or tabular data;

  • GluonNLP offers state-of-the-art pretrained NLP models, easy text preprocessing, datasets loading and neural models building;

  • GluonCV provides state-of-the-art deep learning models in computer vision and carefully designed APIs that greatly reduce the implementation complexity;

  • GluonTS supports deep learning based probabilistic time series modeling;

  • Deep Graph Libray develops easy-to-use, high performance and scalable Python package for deep learning on graphs;

  • TVM: automatic generates and optimizes tensor operators on more backend with better performance for CPUs, GPUs and specialized accelerators.

If you have any question, please leave us a message at our discussion forum. Have fun diving into deep learning!