Practical Machine Learning

Practical information

Course Info:CS 329P, 2021 Autumn, Stanford
Instructors: Qingqing Huang, Mu Li, Alex Smola
Lectures:Wed, Fri 9:45 AM - 11:15 AM
Room: 200-205
Course Forum: Ed Discussion (enrolled student only)
Grading Policy:Midterm (10%), Homework (40%), Project (50%)

Overview

Applying Machine Learning (ML) to solve real problems accurately and robustly requires more than just training the latest ML model. First, you will learn practical techniques to deal with data. This matters since real data is often not independently and identically distributed. It includes detecting covariate, concept, and label shifts, and modeling dependent random variables such as the ones in time series and graphs. Next, you will learn how to efficiently train ML models, such as tuning hyper-parameters, model combination, and transfer learning. Last, you will learn about fairness and model explainability, and how to efficiently deploy models. This class will teach both statistics, algorithms and code implementations. Homeworks and the final project emphasize solving real problems.

Prerequisites

Python programing and machine learning (CS 229), basic statistics. Eqivalent knowledge is fine, and we will try to make the class as self-contained as possible. This is a class where you need to get your hands dirty with programming.

Instructors

Qingqing Huang

Sr. Research Scientist, Google Brain

Mu Li

Sr. Principal Scientist, AWS

Alex Smola

VP/Distinguished Scientist, AWS

Lectures

The tentative schedule is listed as follows. Note that italic topics are optional, namely we may either remove them or provide self-study vidoes.

Part I: Basic ML Modeling
Date Lecture Topics
9/22 1. Data Collection Logistics, course introduction, data acquisition, web scraping, data labeling
9/24 2. Data Preprocessing Exploratory data analysis, data cleaning, data transformation, feature engineering, data summary
9/29 3. ML model recap ML overview, linear methods, tree methods, neural networks
10/1 4. Model Validation Metrics, statistical guarantees, underfit/overfit, overfitting in practice, hold out validation, cross validation
Assignment 1 due in
10/6 5. Model Combination Bias/variance, bagging, backfitting, boosting, stacking
Part II: Broken Assumptions
Date Lecture Topics
10/8 6. Covariate and Concept Shift IID revisited, two sample tests, concept shift, covariate shift
10/13 7. Label Shift and Drift Detection Label shift, drift detection, covariate drift estimation
10/15 Midterm Presentation
Assignment 2 due in
10/20 8. Data beyond IID Dependent random variables, time series, graphs, graph neural networks
Part III: Performance Tuning
Date Lecture Topics
10/22 9. Model Tuning Hyperparameters, beyond grid search, neural network architecture search
10/27 Midterm Exam
10/29 10. Deep Network Tuning Batch and layer norms, residual connections, attention, efficient training
Assignment 3 due in
11/3 11. Transfer Learning Fine-tuning for CV, embeddings for NLP, fine tuning for NLP, zero/few shot, prompt generation
11/5 12. Distillation Function approximation, image data, alternative criteria, text data, targeted distillation, tabular data
11/10 13. Multimodal data Feature generation, multiple towers, stacking, multimodal with graphs, multimodal with time series
Assignment 4 due in
Part IV: Beyond the Model
Date Lecture Topics
11/12 14. Model Deployment Efficiency on CPU/GPUs/edge devices, fusion, serialization, model serving
11/17 15. Fairness (Criteria) Simpson's paradox, compass study, legal definitions of fairness, fairness gone wrong, 50 years fairness survey, special cases
11/19 16. Fairness (Fixes) and Explainability Data sources, synthetic data, Granger causality, Shapley value, Shapley Implementation
11/24 Thanksgiving Recess
11/26 Thanksgiving Recess
12/1 17. Guest Lecture TBD
12/3 18. Guest Lecture TBD
12/8 Final Presentation
12/10 Final Presentation

Course Format

The evaluation is as follows: midterm exam (10%), homework (40%), and project (50%). In the midterm exam, we will ask some theory questions, let you spot the mistakes in code examples, and describe modeling challenges with solutions.

There are 4 assignments. They contain questions similar to the midterm exam. But more importantly, we will ask you to write code to solve real problems with ML based on the baseline implementations we provided.

These assignments may inspire you to choose your course projects. The course project will have two presentations. In the midterm, each group will provide a 1 page summary of project progress and execution plan, and prepare 3 slides for a 5 min presentation. The final presentation will be 10min in length, and the final report is up to 6 pages in ICML style.