Syllabus¶
1 - Data Collection¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Logistics | |||
| Course introduction | 1. Challenges in Deploying Machine Learning | ||
| Data acquisition | 1. Data Collection for Machine Learning | ||
| Web scraping | |||
| Data labeling |
2 - Data Preprocessing¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Exploratory data analysis | |||
| Data cleaning | |||
| Data transformation | |||
| Feature engineering | |||
| Data summary |
3 - ML model recap¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| ML overview | |||
| Tree methods | |||
| Linear methods | 1. Ch3 in D2L | ||
| Neural networks | 1. Ch4, Ch6, Ch8 in D2L |
4 - Model Validation¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Evaluation metrics | |||
| Underfitting and overfitting | |||
| Model validation |
5 - Model Combination¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Bias and variance | |||
| Bagging | |||
| Boosting | |||
| Stacking |
6 - Covariate Shift¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Generalization performance recap | |||
| Covariate shift | |||
| Covariate shift with more math | |||
| adversarial data and invariants |
7 - Label Shift¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Two sample test | |||
| Label shift | |||
| Drift detection | |||
| Covariate drift estimation |
8 - Data beyond IID¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Dependent random variables | |||
| Time series | |||
| Graphs | |||
| Graph neural networks |
9 - Model Tuning¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Hyperparameters | |||
| Beyond grid search | |||
| Neural network architecture search |
10 - Deep Network Tuning¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Batch and layer norms | |||
| Residual connections | |||
| Attention | |||
| Efficient training |
11 - Transfer Learning¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Fine-tuning for CV | |||
| Embeddings for NLP | |||
| Fine tuning for NLP | |||
| Zero/few shot | |||
| Prompt generation |
12 - Distillation¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Function approximation | |||
| Image data | |||
| Alternative criteria | |||
| Text data | |||
| Targeted distillation | |||
| Tabular data |
13 - Multimodal data¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Feature generation | |||
| Multiple towers | |||
| Stacking | |||
| Multimodal with graphs | |||
| Multimodal with time series |
14 - Model Deployment¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Efficiency on CPU/GPUs/edge devices | |||
| Fusion | |||
| Serialization | |||
| Model serving |
15 - Fairness (Criteria)¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Simpson's paradox | |||
| Compass study | |||
| Legal definitions of fairness | |||
| Fairness gone wrong | |||
| 50 years fairness survey | |||
| Special cases |
16 - Fairness (Fixes) and Explainability¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| Data sources | |||
| Synthetic data | |||
| Granger causality | |||
| Shapley value | |||
| Shapley Implementation |
17 - Guest Lecture¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| TBD |
18 - Guest Lecture¶
| Topic | Slides | Video | Optional Materials |
|---|---|---|---|
| TBD |