About the course
The primary aim of E9 211 is to develop a mathematical theory of linear adaptive filters. Adaptation is accomplished by adjusting the free parameters of a filter according to the input data to achieve the desired output. Such adaptive algorithms are frequently encountered in many signal processing and machine learning algorithms. The adaptive signal processing course provides a comprehensive treatment of mathematical signal processing algorithms for designing optimum and linear filters; designing, implementing, and analyzing adaptive filters applied to system identification, inverse modeling (deconvolution), adaptive control, and interference cancellation; and some selected emerging topics in signal processing.
Prerequisites
Undergraduate courses on probability theory, digital signal processing, and linear algebra. First few lectures provide the required background in linear algebra and random processes.
Syllabus
Review of linear algebra and random processes. Optimal estimation. Linear estimation. Steepest-descent algorithms. Stochastic-gradient algorithms. Least squares and recursive least squares. Kalman filtering. Particle filtering. Blind deconvolution and beamforming. Subspace tracking. Robust adaptive filters. Iterative solvers of large-scale linear systems. Selected emerging topics.
Textbooks
- T1. Adaptive Filters, by Ali H. Sayed, John Wiley & Sons, NJ, 2008
- T1_1. Fundamentals of Adaptive Filtering, by Ali H. Sayed, Wiley student edition,2016
- T2. Adaptive Signal Processing, by Bernard Widrow, Pearson, 2002
- Other useful resources
- Adaptive Signal Processing: Next-Generation Solutions, Tulay Adali and Simon Haykin, Wiley-India edition, 2010.
- Adaptive Filter Theory, Simon Haykin, fourth edition, Pearson India, 2002.
Course requirements and grading
- Three homeworks (programming): 10% each, i.e., 30% in total
- Mandatory to participate in the final exam and to pass the course.
- Prepare reports using LaTeX.
- Submit only pdf files. Include Matlab scripts as appendices. Word documents will not be graded.
- Late submissions are allowed, but will not be graded.
- Midterm assessment on December 3, 2020: 20%
- Open book written exam. 24 hour turn in time.
- Syllabus covered till 19-11-2020, i.e., till and including Steepest-descent method.
- Project: 30%
- Prepare reports using LaTeX.
- Submit only pdf files. Include Matlab scripts as appendices. Word documents will not be graded.
- Final assessment on January 21, 2021: 20%
- Open book written exam. 24 hour turn in time.
- Needless to say, includes the entire syllabus.
Lecture schedule
| # | Date | Topic | Reference | Slides |
|---|---|---|---|---|
| 1 | 08-10-2020 | Introduction | slides | |
| 2 | 13-10-2020 | Linear algebra | slides | |
| 3 | 15-10-2020 | Linear algebra - contd. | slides | |
| 4 | 20-10-2020 | Optimization theory and random processes | slides | |
| 5 | 22-10-2020 | Exercise session | ||
| 6 | 27-10-2020 | Optimal estimation | T1 - Ch. 1, Ch. 2.1-2.2 | slides |
| 7 | 29-10-2020 | Linear estimation | T1 - Ch. 3 and Ch. 4 | slides |
| 8 | 03-11-2020 | Linear models | T1 - Ch. 5 | slides |
| 9 | 05-11-2020 | Constrained estimation and beamforming | T1. Ch.6.1-6.4 and 6.5 | slides |
| 10 | 10-11-2020 | Exercise session | T1: Examples 4.1,4.2, 5.5; Problems I.14, II.7, II.17, II.29 | |
| 11 | 12-11-2020 | Direction estimation and beamforming | ||
| 12 | 17-11-2020 | Steepest gradient descent | T1. Ch 8 (T1_1. Ch.4) | slides |
| 13 | 19-11-2020 | Steepest gradient descent contd. | T1. Ch 9 (except 9.8) (T1_1. Ch.4) | slides |
| 14 | 24-11-2020 | Newton's method and LMS | T1. Ch. 9.8, 10 (except 10.7); T1_1 Ch. 5.1-5.4 | slides slides |
| 15 | 26-11-2020 | LMS contd. | T1. Ch. 11 and Ch. 13 (except 10.7);Ch 5.6 and Ch. 5.8 | slides |
| 16 | 01-12-2020 | Exercise session | T1: Problems III.1, III.8 | |
| 17 | 03-12-2020 | Mid-term exam | ||
| 18 | 08-12-2020 | Mid-term solutions, Leaky and Sign LMS | T1. Ch 12.1 ; T1_1. Ch. 5.7.1 | |
| 19 | 10-12-2020 | Constant modulus algorithm | T1. Ch. 12.2; T1_1. Ch. 5.7.2 | |
| 20 | 15-12-2020 | Recursive least squares | T1. Ch 14 ; T1_1. Ch. 12 | |
| 21 | 17-12-2020 | Kalman filter and weighted least squares | T1. Ch 7; T1_1. Ch. 2.C | slides |
| 22 | 22-12-2020 | Kalman filter and recursive least squares | T1. Ch 31; T1_1. Ch. 12.A | |
| 23 | 24-12-2020 | Adaptive linearly constrained beamforming | paper | |
| 24 | 29-12-2020 | No class | ||
| 25 | 31-12-2020 | No class | ||
| 26 | 05-01-2021 | Total least squares and total least mean squares | T1_1. Ch. 17.B | |
| 27 | 07-01-2021 | Course project discussion | ||
| 28 | 12-01-2021 | Covariance estimation | special topic | |
| 29 | 14-01-2021 | Graph signal processing | special topic |
Project
This project consists of two parts on implementing and studying adaptive filters for adaptive noise cancellation: (a) using single-channel microphone recordings and (b) using multi-channel microphone recordings.
Description and data (available on request)
Demo by Prasobh Sankar