Dive into Deep Learning in 1 Day
================================

Last updated：|today|

Information
-----------

-  Speaker： `Alex Smola <https://alex.smola.org/>`__

Overview
--------

Did you ever want to find out about deep learning but didn’t have time
to spend months? New to machine learning? Do you want to build image
classifiers, NLP apps, train on many GPUs or even on many machines? If
you’re an engineer or data scientist, this course is for you. This is
about the equivalent of a Coursera course, all packed into one day. The
course consists of four segments of 90 minutes each.

1. Deep Learning Basics
2. Convolutional Neural Networks for computer vision
3. Best practices (GPUs, Parallelization, Fine Tuning, Transfer
   Learning)
4. Recurrent Neural Networks for natural language (RNN, LSTM)

Prerequisites
-------------

You should have some basic knowledge of `Linear
Algebra <http://numpy.d2l.ai/chapter_preliminaries/scalar-tensor.html>`__,
`Calculus <http://numpy.d2l.ai/chapter_preliminaries/calculus.html>`__,
`Probability <http://numpy.d2l.ai/chapter_preliminaries/probability.html>`__,
and `Python <https://learnpythonthehardway.org/>`__ (here’s `another
book <https://www.diveinto.org/python3/table-of-contents.html>`__ to
learn Python). Moreover, you should have some experience with
`Jupyter <https://jupyter.org/>`__ notebooks, or with
`SageMaker <http://aws.amazon.com/sagemaker>`__ notebooks. To run things
on (multiple) GPUs you need access to a GPU server, such as the
`P2 <https://aws.amazon.com/ec2/instance-types/p2/>`__,
`G3 <https://aws.amazon.com/ec2/instance-types/g3/>`__, or
`P3 <https://aws.amazon.com/ec2/instance-types/p3/>`__ instances.

Syllabus
--------

-  This course relies heavily on the `Dive into Deep
   Learning <http://numpy.d2l.ai>`__ book. There’s a lot more detail in
   the book (notebooks, examples, math, applications).
-  The crash course will get you started. For more information also see
   `other courses and tutorials <http://courses.d2l.ai>`__ based on the
   book.
-  All notebooks below are availabe at
   `d2l-ai/1day-notebooks <https://github.com/d2l-ai/1day-notebooks>`__,
   which contains instructions how to setup the running environments.

=========== =================================================================================
Time        Topics
=========== =================================================================================
9:00—10:00  `Part 1: Deep learning basic <#part-1-deep-learning-basic>`__
10:00—11:00 `Part 2: Convolutional neural networks <#part-2-convolutional-neural-networks>`__
11:00—12:00 `Part 3: Performance <#part-3-performance>`__
12:00—1:00  `Part 4: Recurrent neural networks <#part-4-recurrent-neural-networks>`__
=========== =================================================================================

Part 1: Deep Learning Basic
~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Slides**: `[keynote] <slides/Part-1.key>`__,
`[pdf] <slides/Part-1.pdf>`__

**Notebooks**:

1. Data Manipulation with Numpy
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-1/1-ndarray.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-1/1-ndarray.ipynb#/>`__
2. Automatic Differentiation
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-1/2-autograd.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-1/2-autograd.ipynb#/>`__
3. Linear Regression
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-1/4-linear-regression-gluon.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-1/4-linear-regression-gluon.ipynb#/>`__
4. Image Classification Data (Fashion-MNIST)
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-1/5-fashion-mnist.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-1/5-fashion-mnist.ipynb#/>`__
5. Softmax Regression
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-1/7-softmax-regression-gluon.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-1/7-softmax-regression-gluon.ipynb#/>`__
6. Multilayer Perceptrons
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-1/9-mlp-gluon.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-1/9-mlp-gluon.ipynb#/>`__

Part 2: Convolutional neural networks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Slides**: `[keynote] <slides/Part-2.key>`__,
`[pdf] <slides/Part-2.pdf>`__

**Notebooks**:

1. GPUs
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/1-use-gpu.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/1-use-gpu.ipynb#/>`__
2. Convolutions
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/2-conv-layer.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/2-conv-layer.ipynb#/>`__
3. Pooling
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/3-pooling.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/3-pooling.ipynb#/>`__
4. Convolutional Neural Networks (LeNet)
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/4-lenet.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/4-lenet.ipynb#/>`__
5. Deep Convolutional Neural Networks (AlexNet)
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/5-alexnet.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/5-alexnet.ipynb#/>`__
6. Inception Networks (GoogLeNet)
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/7-googlenet.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/7-googlenet.ipynb#/>`__
7. Residual Networks (ResNet)
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-2/8-resnet.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-2/8-resnet.ipynb#/>`__

Part 3: Performance
~~~~~~~~~~~~~~~~~~~

**Slides**: `[keynote] <slides/Part-3.key>`__,
`[pdf] <slides/Part-3.pdf>`__

**Notebooks**:

1. A Hybrid of Imperative and Symbolic Programming
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-3/1-hybridize.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-3/1-hybridize.ipynb#/>`__
2. Multi-GPU Computation Implementation from Scratch
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-3/2-multiple-gpus.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-3/2-multiple-gpus.ipynb#/>`__
3. Concise Implementation of Multi-GPU Computation
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-3/3-multiple-gpus-gluon.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-3/3-multiple-gpus-gluon.ipynb#/>`__
4. Fine Tuning
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-3/4-fine-tuning.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-3/4-fine-tuning.ipynb#/>`__

Part 4: Recurrent neural networks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Slides**: `[keynote] <slides/Part-4.key>`__,
`[pdf] <slides/Part-4.pdf>`__

**Notebooks**:

1. Text Preprocessing
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-4/1-text-preprocessing.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-4/1-text-preprocessing.ipynb#/>`__
2. Concise Implementation of Recurrent Neural Networks
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-4/3-rnn-gluon.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-4/3-rnn-gluon.ipynb#/>`__
3. Long Short Term Memory (LSTM)
   `[ipynb] <https://github.com/mli/d2l-1day-notebooks/blob/master/notebooks-4/5-lstm.ipynb>`__
   `[slides] <https://nbviewer.jupyter.org/format/slides/github/mli/d2l-1day-notebooks/blob/master/notebooks-4/5-lstm.ipynb#/>`__