E2 236 Foundations of Machine Learning (FoML)

About E2 236

The main objective of E2 236 is to provide an introduction to the theory, methods, and concepts essential for developing programs that learn from data. As a graduate-level course, it is designed to establish a rigorous foundation in the underlying mathematics and algorithms of machine learning. We will cover a wide number of techniques, from the simplest (linear regression) to advanced deep learning techniques, alongside theoretical concepts such as the PAC learning framework.

This is 3:1 credit course.

Prerequisite

Basic linear algebra, probability, and knowledge of Python.

Syllabus

Machine learning landscape, classification, regression, optimization, PAC learning framework, kernel methods, Gaussian processes, SVM, clustering, EM methods, Ensemble learning, neural networks, graph machine learning.

Textbooks

Christopher Bishop, Pattern Recognition and Machine Learning
Christopher Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts
Kevin Murphy, Machine Learning: A Probabilistic Perspective
Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms
Aurelien Geron, Hands-On Machine Learning with Scikit-Learn and PyTorch: Concepts, Tools, and Techniques to Build Intelligent Systems

Course requirements and grading

Three assignments: 10% each, i.e., 30% in total
Project: 20% in total
Final assessment: 50%

Syllabus (Schedule pdf)

Lecture number	Topic	Reading	Material	Lab
0	Introduction and logistics		Slides
1	Maching learning landscape		Slides
2	Classification
3	Classification			Lab 1
4	Classification
5	Regression
6	Regression			Lab 2
7	Regression
8	Optimization
9	Optimization			Lab 3
10	PAC learning framework			Assignment 1
11	PAC learning framework
12	Kernel methods
13	Gaussian Processes			Lab 3
14	SVMs			Lab 4
15	PCA and CCA			Lab 5
16	Clustering			Lab 6
17	Clustering			Assignment 2
18	EM method			Project
19	Ensemble learning
20	Neural networks: Multilayer perceptron
21	Training multilayer perceptrons			Lab 7
22	RNNs and its variants			Lab 8
23	CNNs			Lab 9
24	Autoencoders			Assignment 3/Lab 10
25	GANs			Lab 11
26	Transformers			Lab 12
27	Graph neural networks
28	Graph neural networks			Lab 13