DSP Lecture 21: Gradient descent and LMS

ECSE-4530 Digital Signal Processing
Rich Radke, Rensselaer Polytechnic Institute
Lecture 21: Gradient descent and LMS (11/13/14)

0:00:22 Recap: the Wiener filter
0:04:42 Estimating R and p in practice
0:06:45 The filter taps change over time based on data
0:08:40 How does the filter converge?
0:12:07 Steepest descent (gradient descent)
0:15:03 Basic equation
0:17:05 Step size considerations
0:19:30 Steepest descent for the Wiener filter
0:22:10 The final result
0:23:41 Comments on convergence
0:24:57 Convergence is related to the eigenvalues of R
0:27:01 Revisiting convergence problems
0:31:11 These kinds of optimization problems are common throughout engineering
0:36:05 The LMS (least-mean-square) algorithm
0:37:54 Estimating R from data
0:41:30 The LMS equations
0:44:16 Comments on convergence
0:45:49 Tap-input power
0:46:44 Adaptive step sizes

Follows Section 13.2 of the textbook (Proakis and Manolakis, 4th ed.).