E2 236 Foundations of Machine Learning (FoML)
About E2 236
The main objective of E2 236 is to provide an introduction to the theory, methods, and concepts essential for developing programs that learn from data. As a graduate-level course, it is designed to establish a rigorous foundation in the underlying mathematics and algorithms of machine learning. We will cover a wide number of techniques, from the simplest (linear regression) to advanced deep learning techniques, alongside theoretical concepts such as the PAC learning framework.
This is 3:1 credit course.
Prerequisite
Basic linear algebra, probability, and knowledge of Python.
Syllabus
Machine learning landscape, classification, regression, optimization, PAC learning framework, kernel methods, Gaussian processes, SVM, clustering, EM methods, Ensemble learning, neural networks, graph machine learning.
Textbooks
Christopher Bishop, Pattern Recognition and Machine Learning
Christopher Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts
Kevin Murphy, Machine Learning: A Probabilistic Perspective
Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms
Aurelien Geron, Hands-On Machine Learning with Scikit-Learn and PyTorch: Concepts, Tools, and Techniques to Build Intelligent Systems
Course requirements and grading
Mid-term exam: 25%
Surprise tests (note the attendance policy): 5%
Project: 20%
Final assessment: 50%
Syllabus (Schedule pdf)
| Lecture number | Topic | Reading | Material | Lab |
| 0 | Introduction and logistics | Slides | ||
| 1 | Maching learning landscape | Slides | ||
| 2 | Classification | Notes | ||
| 3 | Classification | Notes | ||
| 4 | Classification | Notes | Lab 1 (.ipynb file) | |
| 5 | Regression | |||
| 6 | Regression | |||
| 7 | Regression | Notes | Lab 2 (.ipynb file) | |
| 8 | Optimization | Lab 3a (.ipynb file) | ||
| 9 | Optimization | Notes | Lab 3b (.ipynb file) | |
| 10 | PAC learning framework | Notes | ||
| 11 | PAC learning framework | |||
| 12 | Kernel methods | Lab 4 (.ipynb file) | ||
| 13 | Gaussian Processes | Lab 5 (.ipynb file) | ||
| 14 | SVMs | Lab 6 (.ipynb file) | ||
| 15 | PCA and CCA | Lab | ||
| 16 | Clustering | |||
| 17 | Clustering | Lab | ||
| 18 | EM method | Project | ||
| 19 | Ensemble learning | |||
| 20 | Neural networks: Multilayer perceptron | |||
| 21 | Training multilayer perceptrons | Lab | ||
| 22 | RNNs and its variants | Lab | ||
| 23 | CNNs | Lab 9 | ||
| 24 | Autoencoders | Lab | ||
| 25 | GANs | Lab | ||
| 26 | Transformers | Lab | ||
| 27 | Graph neural networks | Lab |