Practical Machine Learning¶
|Course Info:||CS 329P, 2021 Autumn, Stanford|
|Instructors:||Qingqing Huang, Mu Li, Alex Smola|
|Lectures:||Wed, Fri 9:45 AM - 11:15 AM|
|Course Forum:||Ed Discussion (enrolled student only)|
|Grading Policy:||Midterm (10%), Homework (40%), Project (50%)|
Applying Machine Learning (ML) to solve real problems accurately and robustly requires more than just training the latest ML model. First, you will learn practical techniques to deal with data. This matters since real data is often not independently and identically distributed. It includes detecting covariate, concept, and label shifts, and modeling dependent random variables such as the ones in time series and graphs. Next, you will learn how to efficiently train ML models, such as tuning hyper-parameters, model combination, and transfer learning. Last, you will learn about fairness and model explainability, and how to efficiently deploy models. This class will teach both statistics, algorithms and code implementations. Homeworks and the final project emphasize solving real problems.
Python programing and machine learning (CS 229), basic statistics. Eqivalent knowledge is fine, and we will try to make the class as self-contained as possible. This is a class where you need to get your hands dirty with programming.
Sr. Research Scientist, Google Brain
Sr. Principal Scientist, AWS
VP/Distinguished Scientist, AWS
The tentative schedule is listed as follows. Note that italic topics are optional, namely we may either remove them or provide self-study vidoes.
|Part I: Basic ML Modeling|
|9/22||1. Data Collection||Logistics, course introduction, data acquisition, web scraping, data labeling|
|9/24||2. Data Preprocessing||Exploratory data analysis, data cleaning, data transformation, feature engineering, data summary|
|9/29||3. ML model recap||ML overview, linear methods, tree methods, neural networks|
|10/1||4. Model Validation||Metrics, statistical guarantees, underfit/overfit, overfitting in practice, hold out validation, cross validation|
|Assignment 1 due in|
|10/6||5. Model Combination||Bias/variance, bagging, backfitting, boosting, stacking|
|Part II: Broken Assumptions|
|10/8||6. Covariate and Concept Shift||IID revisited, two sample tests, concept shift, covariate shift|
|10/13||7. Label Shift and Drift Detection||Label shift, drift detection, covariate drift estimation|
|Assignment 2 due in|
|10/20||8. Data beyond IID||Dependent random variables, time series, graphs, graph neural networks|
|Part III: Performance Tuning|
|10/22||9. Model Tuning||Hyperparameters, beyond grid search, neural network architecture search|
|10/29||10. Deep Network Tuning||Batch and layer norms, residual connections, attention, efficient training|
|Assignment 3 due in|
|11/3||11. Transfer Learning||Fine-tuning for CV, embeddings for NLP, fine tuning for NLP, zero/few shot, prompt generation|
|11/5||12. Distillation||Function approximation, image data, alternative criteria, text data, targeted distillation, tabular data|
|11/10||13. Multimodal data||Feature generation, multiple towers, stacking, multimodal with graphs, multimodal with time series|
|Assignment 4 due in|
|Part IV: Beyond the Model|
|11/12||14. Model Deployment||Efficiency on CPU/GPUs/edge devices, fusion, serialization, model serving|
|11/17||15. Fairness (Criteria)||Simpson's paradox, compass study, legal definitions of fairness, fairness gone wrong, 50 years fairness survey, special cases|
|11/19||16. Fairness (Fixes) and Explainability||Data sources, synthetic data, Granger causality, Shapley value, Shapley Implementation|
|12/1||17. Guest Lecture||TBD|
|12/3||18. Guest Lecture||TBD|
The evaluation is as follows: midterm exam (10%), homework (40%), and project (50%). In the midterm exam, we will ask some theory questions, let you spot the mistakes in code examples, and describe modeling challenges with solutions.
There are 4 assignments. They contain questions similar to the midterm exam. But more importantly, we will ask you to write code to solve real problems with ML based on the baseline implementations we provided.
These assignments may inspire you to choose your course projects. The course project will have two presentations. In the midterm, each group will provide a 1 page summary of project progress and execution plan, and prepare 3 slides for a 5 min presentation. The final presentation will be 10min in length, and the final report is up to 6 pages in ICML style.