E2 237, Fall 2025

Statistical Learning Theory

Lectures

Homework

Course Syllabus

  • Binary classification: SVM, kernel methods
  • Complexity bounds: bias complexity trade-off, Rademacher complexity, VC-dimension
  • Multiclass classification: decision trees, nearest neighbours
  • Estimation: parameter estimation, nonparametric regression
  • Optimization: stochastic gradient descent, minimax
  • Decision theory: statistical decision theory, large-sample asymptotics
  • Information theoretic bounds: mutual information method, lower bound via hypothesis testing, entropic bounds for statistical estimation, strong data processing inequality

Prerequisite

Instructor’s approval is required for crediting this course. Course requires a background in the first graduate course in probability theory and random processes.

Description

The aim of this course is to provide performance guarantees on various data driven algorithms for classification, estimation, and decision problems under uncertainty. The guarantees are provided by the upper and lower bounds on the algorithm accuracy as a function of the number of samples. The upper bounds are derived from the classical complexity results and the lower bounds follow from information theoretic techniques.

Teams/GitHub/Overleaf Information

Teams

We will use Microsoft Teams for all the course related communication.
Please do not send any email regarding the course.
You can signup for the course team Statistical-Learning-2025 using the following code qga6c3b.

Instructor

Parimal Parag
Office: EC 2.17
Hours: By appointment.

Time and Location

Classroom: EC 1.07, ECE main building
Hours: Tue/Thu 10:00-11:30.

Tests and grading policy

Mid Term Hours: 10:00-11:30
Mid Term Venue: EC 1.08, ECE main building

Final Hours: 14:00-17:00
Final Venue: EC 1.07, ECE main building

Teaching Assistants

Bishal Jaiswal
Email: bishalj@iisc
Hours: By appointment.

Alok Kumar Komal
Email: alokkomal@iisc
Hours: By appointment.

Pooja Subramaniam
Email: poojas@iisc
Hours: By appointment.

References

Foundations of Machine Learning, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, 2nd edition, MIT Press, 2018.
Information Theory: From Coding to Learning, Yury Polyanskiy and Yihong Wu, Cambridge University Press, 2023.
Information-theoretic Methods for High-dimensional Statistics, Yihong Wu, Lecture notes.
High-Dimensional Statistics: A Non-asymptotic Viewpoint, Martin Wainwright, Cambridge University Press, 2019.
Introduction to Statistical Learning Theory, Olivier Bousquet, Stephane Boucheron, and Gabor Lugosi, Lecture notes.