(2)Quadratic cost functions o Optimal filter design ● Filters are optimized according to a certain cost function. A FIR Filter is characterized by a coefficient vector,which in the transversal filter case is the tap-weight vector w. Reduce to picking the coefficient vector that minimizes the chosen cost function. o Naturally,there is no unique definition for a cost function. 2020-01-18 11
2020-01-18 11 (2) Quadratic cost functions Optimal filter design Filters are optimized according to a certain cost function. A FIR Filter is characterized by a coefficient vector, which in the transversal filter case is the tap-weight vector w. Reduce to picking the coefficient vector that minimizes the chosen cost function. Naturally, there is no unique definition for a cost function
MSE criterion o A popular cost function ● the mean-square value of the difference between the desired response and the actual filter output o For FIR filters and linear combiners that is as(w)=Eemf）=Eam）-wunf） o It has the highly desirable property of being quadratic with a (mostly)unique minimum. 2020-01-18 12
2020-01-18 12 MSE criterion A popular cost function the mean-square value of the difference between the desired response and the actual filter output For FIR filters and linear combiners that is It has the highly desirable property of being quadratic with a (mostly) unique minimum. 2 2 ( ) ( ) ( ) ( ) H MSE J E e n E d n n w w u
Remark (1) o WF Probabilistic framework:all signals are viewed as realizations of stochastic processes. ● Certain statistical information (correlations and cross- correlations)has to be available in order to design the filter. o LMS ● Only data sequences are given,i.e.only one realization of the processes involved. If instantaneous estimates are substituted for the statistical parameters,this leads to the so-called LMS algorithm 2020-01-18 13
2020-01-18 13 Remark (1) WF Probabilistic framework: all signals are viewed as realizations of stochastic processes. Certain statistical information (correlations and crosscorrelations) has to be available in order to design the filter. LMS Only data sequences are given, i.e. only one realization of the processes involved. If instantaneous estimates are substituted for the statistical parameters, this leads to the so-called LMS algorithm
Remark(2) o RLS ● Stationarity and ergodicity are invoked ● time averaging is used to obtain more reliable estimates of the process statistics. This in particular represents one way of deriving to the so-called RLS procedure 2020-01-18 14
2020-01-18 14 Remark (2) RLS Stationarity and ergodicity are invoked time averaging is used to obtain more reliable estimates of the process statistics. This in particular represents one way of deriving to the so-called RLS procedure
Remark (3) o Least Squares error Replace the mean-squared error cost function by a time averaged squared error Js(w)）=ld)-w"u(k) k=1 It is also quadratic with a(mostly)unique minimum. This 'direct'approach indeed represents a valid alternative to the probabilistic 'detour'and leads to comparable results without having to rely on all the probabilistic machinery (stationarity,ergodicity,etc.) In particular,this again leads to the RLS procedure 2020-01-18 15
2020-01-18 15 Remark (3) Least Squares error Replace the mean-squared error cost function by a time averaged squared error It is also quadratic with a (mostly) unique minimum. This ‘direct’ approach indeed represents a valid alternative to the probabilistic ‘detour’ and leads to comparable results without having to rely on all the probabilistic machinery (stationarity, ergodicity, etc.). In particular, this again leads to the RLS procedure 2 1 ( ) ( ) ( ) L H LS k J d k k w w u