E9 211 Adaptive Signal Processing

About E9 211 Adaptive Signal Processing

The primary aim of E9 211 is to develop a mathematical theory of linear adaptive filters. Adaptation is accomplished by adjusting the free parameters of a filter according to the input data to achieve the desired output. Such adaptive algorithms are frequently encountered in many signal processing and machine learning algorithms. The adaptive signal processing course provides a comprehensive treatment of mathematical signal processing algorithms for designing optimum and linear filters; designing, implementing, and analyzing adaptive filters applied to system identification, inverse modeling (deconvolution), adaptive control, and interference cancellation; and some selected emerging topics in signal processing.

Prerequisite

Undergraduate courses on probability theory, digital signal processing, and linear algebra. First few lectures provide the required background in linear algebra and random processes.

Lectures

Tuesdays and Thursdays 11:30am-1:00pm, Online via MS Teams.

Syllabus

Review of linear algebra and random processes. Optimal estimation. Linear estimation. Steepest-descent algorithms. Stochastic-gradient algorithms. Least squares and recursive least squares. Kalman filtering. Particle filtering. Blind deconvolution and beamforming. Subspace tracking. Robust adaptive filters. Iterative solvers of large-scale linear systems. Selected emerging topics.

Textbooks

  • T1. Adaptive Filters, by Ali H. Sayed, John Wiley & Sons, NJ, 2008

  • T1_1. Fundamentals of Adaptive Filtering, by Ali H. Sayed, Wiley student edition,2016

  • T2. Adaptive Signal Processing, by Bernard Widrow, Pearson, 2002

  • Other useful resources

    • Adaptive Signal Processing: Next-Generation Solutions, Tulay Adali and Simon Haykin, Wiley-India edition, 2010.

    • Adaptive Filter Theory, Simon Haykin, fourth edition, Pearson India, 2002.

Course requirements and grading

  • Three homeworks (programming): 10% each, i.e., 30% in total

    • Mandatory to participate in the final exam and to pass the course.

    • Prepare reports using LaTeX.

    • Submit only pdf files. Include Matlab scripts as appendices. Word documents will not be graded.

    • Late submissions are allowed, but will not be graded.

  • Midterm assessment on December 3, 2020: 20%

    • Open book written exam. 24 hour turn in time.

    • Syllabus covered till 19-11-2020, i.e., till and including Steepest-descent method.

  • Project: 30%

    • Prepare reports using LaTeX.

    • Submit only pdf files. Include Matlab scripts as appendices. Word documents will not be graded.

  • Final assessment on January 21, 2021: 20%

    • Open book written exam. 24 hour turn in time.

    • Needless to say, includes the entire syllabus.

Schedule

Lecture numberDate Topic Reference Slides
1 08-10-2020 Introduction slides
2 13-10-2020 Linear algebra slides
315-10-2020 Linear algebra - contd. slides
420-10-2020 Optimization theory and random processes slides
522-10-2020 Exercise session
627-10-2020 Optimal estimation T1 - Ch. 1, Ch. 2.1-2.2 slides
729-10-2020 Linear estimation T1 - Ch. 3 and Ch. 4 slides
803-11-2020 Linear models T1 - Ch. 5 slides
905-11-2020 Constrained estimation and beamforming T1. Ch.6.1-6.4 and 6.5 slides
1010-11-2020 Exercise session T1: Examples 4.1,4.2, 5.5; Problems I.14, II.7, II.17, II.29
1112-11-2020 Direction estimation and beamforming
1217-11-2020 Steepest gradient descent T1. Ch 8 (T1_1. Ch.4) slides
1319-11-2020 Steepest gradient descent contd. T1. Ch 9 (except 9.8) (T1_1. Ch.4) slides
1424-11-2020 Newton's method and LMS T1. Ch. 9.8, 10 (except 10.7); T1_1 Ch. 5.1-5.4 slides
slides
1526-11-2020 LMS contd. T1. Ch. 11 and Ch. 13 (except 10.7);Ch 5.6 and Ch. 5.8 slides
1601-12-2020 Exercise session T1: Problems III.1, III.8
1703-12-2020 Mid-term exam
1808-12-2020 Mid-term solutions, Leaky and Sign LMS T1. Ch 12.1 ; T1_1. Ch. 5.7.1
1910-12-2020 Recursive least squares T1. Ch 14 ; T1_1. Ch. 12
2015-12-2020 Kalman filter and weighted least squares T1. Ch 7; T1_1. Ch. 2.C slides
2117-12-2020 Kalman filter and recursive least squares T1. Ch 31; T1_1. Ch. 12.A
2222-12-2020 Factor analysis and covariance estimation from subsampling not included for exam
2324-12-2020 Graph signal processing not included for exam
2429-12-2020 No class
2531-12-2020 No class
2605-01-2021 Adaptive linearly constrained beamforming paper
2707-01-2021 Finite precision effects of LMS and RLS
2812-01-2021 Finite precision effects of LMS and RLS
2914-01-2021 Recap

Previous exams