Adaptive filters MNw

Adaptive filtering is a topic of immense practical and theoretical value, having applications in areas ranging from digital and wireless communications to biomedical systems. This book enables readers to gain a gradual and solid introduction to the subject, its applications to a variety of topical problems, existing limitations, and extensions of current theories. The book consists of eleven parts, each part containing a series of focused lectures and ending with bibliographic comments, problems, and computer projects with MATLAB solutions. Adaptive Filters......Page 3 Contents......Page 9 Preface......Page 19 Notation......Page 27 Acknowledgments......Page 32 BACKGROUND MATERIAL......Page 33 A.1 Variance of a Random Variable......Page 35 A.2 Dependent Random Variables......Page 37 A.3 Complex-Valued Random Variables......Page 38 A.4 Vector-Valued Random Variables......Page 40 A.5 Gaussian Random Vectors......Page 41 B.1 Hermitian and Positive-Definite Matrices......Page 46 B.2 Range Spaces and Nullspaces of Matrices......Page 48 B.3 Schur Complements......Page 50 B.4 Cholesky Factorization......Page 51 B.5 QR Decomposition......Page 53 B.6 Singular Value Decomposition......Page 54 B.7 Kronecker Products......Page 57 C.1 Cauchy-Riemann Conditions......Page 59 C.3 Vector Arguments......Page 60 PART I: OPTIMAL ESTIMATION......Page 62 1.1 Estimation Without Observations......Page 63 1.2 Estimation Given Dependent Observations......Page 65 1.3 Orthogonality Principle......Page 70 1.4 Gaussian Random Variables......Page 72 2.1 Optimal Estimator in the Vector Case......Page 76 2.2 Spherically Invariant Gaussian Variables......Page 80 2.3 Equivalent Optimization Criterion......Page 83 Summary and Notes......Page 85 Problems and Computer Projects......Page 88 PART II: LINEAR ESTIMATION......Page 93 3 Normal Equations......Page 94 3.1 Mean-Square Error Criterion......Page 95 3.3 Minimization by Completion-of-Squares......Page 97 3.4 Minimization of the Error Covariance Matrix......Page 99 3.5 Optimal Linear Estimator......Page 100 4.1 Design Examples......Page 101 4.2 Orthogonality Condition......Page 106 4.3 Existence of Solutions......Page 108 4.4 Nonzero-Mean Variables......Page 110 5.1 Estimation using Linear Relations......Page 112 5.2 Application: Channel Estimation......Page 114 5.3 Application: Block Data Estimation......Page 115 5.4 Application: Linear Channel Equalization......Page 116 5.5 Application: Multiple-Antenna Receivers......Page 119 6 Constrained Estimation......Page 121 6.1 Minimum-Variance Unbiased Estimation......Page 122 6.2 Example: Mean Estimation......Page 124 6.3 Application: Channel and Noise Estimation......Page 125 6.4 Application: Decision Feedback Equalization......Page 127 6.5 Application: Antenna Beamforming......Page 135 7.1 Innovations Process......Page 138 7.2 State-Space Model......Page 140 7.3 Recursion for the State Estimator......Page 141 7.4 Computing the Gain Matrix......Page 142 7.6 Covariance Form......Page 143 7.7 Measurement and Time-Update Form......Page 144 Summary and Notes......Page 145 Problems and Computer Projects......Page 149 PART III: STOCHASTIC GRADIENT ALGORITHMS......Page 172 8 Steepest–Descent Technique......Page 173 8.1 Linear Estimation Problem......Page 174 8.2 Steepest-Descent Method......Page 176 8.3 More General Cost Functions......Page 181 9.1 Modes of Convergence......Page 182 9.2 Optimal Step-Size......Page 183 9.3 Weight-Error Vector Convergence......Page 185 9.4 Time Constants......Page 187 9.5 Learning Curve......Page 188 9.6 Contour Curves of the Error Surface......Page 189 9.7 Iteration-Dependent Step-Sizes......Page 191 9.8 Newton’s Method......Page 194 10.1 Motivation......Page 197 10.2 Instantaneous Approximation......Page 199 10.3 Computational Cost......Page 200 10.4 Least-Perturbation Property......Page 201 10.5 Application: Adaptive Channel Estimation......Page 202 10.6 Application: Adaptive Channel Equalization......Page 205 10.7 Application: Decision-Feedback Equalization......Page 206 10.8 Ensemble-Average Learning Curves......Page 208 11.1 Instantaneous Approximation......Page 212 11.2 Computational Cost......Page 213 11.3 Power Normalization......Page 214 11.4 Least-Perturbation Property......Page 216 12.1 Non-Blind Algorithms......Page 217 12.2 Blind Algorithms......Page 220 12.3 Some Properties......Page 222 13.1 Instantaneous Approximation......Page 225 13.3 Least-Perturbation Property......Page 227 13.4 Affine Projection Interpretation......Page 228 14.1 Instantaneous Approximation......Page 232 14.2 Computational Cost......Page 234 Summary and Notes......Page 236 Problems and Computer Projects......Page 243 PART IV: MEAN-SQUARE PERFORMANCE......Page 261 15.1 Performance Measure......Page 262 15.2 Stationary Data Model......Page 264 15.3 Energy Conservation Relation......Page 268 15.4 Variance Relation......Page 271 15.A Interpretations of the Energy Relation......Page 273 16.1 Variance Relation......Page 278 16.3 Separation Principle......Page 279 16.4 White Gaussian Input......Page 280 16.5 Statement of Results......Page 283 16.6 Simulation Results......Page 284 17.1 Separation Principle......Page 286 17.A Relating NLMS to LMS......Page 288 18.1 Real-Valued Data......Page 291 18.2 Complex-Valued Data......Page 293 18.3 Simulation Results......Page 294 19.1 Performance of RLS......Page 296 19.2 Performance of Other Filters......Page 300 19.3 Performance Table for Small Step-Sizes......Page 303 20.1 Motivation......Page 304 20.2 Nonstationary Data Model......Page 305 20.3 Energy Conservation Relation......Page 310 20.4 Variance Relation......Page 311 21.1 Performance of LMS......Page 314 21.2 Performance of NLMS......Page 318 21.3 Performance of Sign-Error LMS......Page 319 21.4 Performance of RLS......Page 321 21.5 Comparison of Tracking Performance......Page 323 21.6 Comparing RLS and LMS......Page 326 21.7 Performance of Other Filters......Page 327 21.8 Performance Table for Small Step-Sizes......Page 329 Summary and Notes......Page 330 Problems and Computer Projects......Page 338 PART V: TRANSIENT PERFORMANCE......Page 362 22.1 Data Model......Page 363 22.3 Weighted Energy Conservation Relation......Page 364 22.4 Weighted Variance Relation......Page 367 23.1 Mean and Variance Relations......Page 374 23.3 Mean-Square Behavior......Page 377 23.4 Mean-Square Stability......Page 380 23.5 Steady-State Performance......Page 384 23.A Convergence Time......Page 387 24.1 Mean and Variance Relations......Page 391 24.2 Mean-Square Stability and Performance......Page 394 24.3 Small Step-Size Approximations......Page 396 24.A Independence and Averaging Analysis......Page 397 25.1 NLMS Filter......Page 405 25.2 Data-Normalized Filters......Page 408 25.A Stability Bound......Page 411 25.B Stability of NLMS......Page 412 Summary and Notes......Page 414 Problems and Computer Projects......Page 422 PART VI: BLOCK ADAPTIVE FILTERS......Page 446 26.1 Transform-Domain Filters......Page 447 26.2 DFT-Domain LMS......Page 455 26.3 DCT-Domain LMS......Page 457 26.A DCT-Transformed Regressors......Page 458 27.1 Motivation......Page 460 27.2 Block Data Formulation......Page 462 27.3 Block Convolution......Page 465 28.1 DFT Block Adaptive Filters......Page 474 28.2 Subband Adaptive Filters......Page 481 28.A Another Constrained DFT Block Filter......Page 487 28.B Overlap-Add Block Adaptive Filters......Page 489 Summary and Notes......Page 499 Problems and Computer Projects......Page 502 PART VII: LEAST-SQUARES METHODS......Page 509 29 Least-Squares Criterion......Page 510 29.1 Least-Squares Problem......Page 511 29.2 Geometric Argument......Page 512 29.3 Algebraic Arguments......Page 514 29.4 Properties of Least-Squares Solution......Page 516 29.5 Projection Matrices......Page 518 29.6 Weighted Least-Squares......Page 519 29.7 Regularized Least-Squares......Page 521 29.8 Weighted Regularized Least-Squares......Page 523 30.1 Motivation......Page 526 30.2 RLS Algorithm......Page 527 30.3 Regularization......Page 529 30.4 Conversion Factor......Page 530 30.5 Time-Update of the Minimum Cost......Page 531 30.6 Exponentially-Weighted RLS Algorithm......Page 532 31.1 Equivalence in Linear Estimation......Page 535 31.2 Kalman Filtering and Recursive Least-Squares......Page 536 31.A Extended RLS Algorithms......Page 542 32.1 Backward Order-Update Relations......Page 549 32.2 Forward Order-Update Relations......Page 559 32.3 Time-Update Relation......Page 563 Summary and Notes......Page 568 Problems and Computer Projects......Page 575 PART VIII: ARRAY ALGORITHMS......Page 594 33.1 Some Difficulties......Page 595 33.2 Square-Root Factors......Page 596 33.3 Preservation Properties......Page 598 33.4 Motivation for Array Methods......Page 600 34.1 Givens Rotations......Page 605 34.2 Householder Transformations......Page 610 35 QR and Inverse QR Algorithms......Page 614 35.1 Inverse QR Algorithm......Page 615 35.2 QR Algorithm......Page 618 35.3 Extended QR Algorithm......Page 622 35.A Array Algorithms for Kalman Filtering......Page 623 Summary and Notes......Page 627 Problems and Computer Projects......Page 629 PART IX: FAST RLS ALGORITHMS......Page 635 36.1 Hyperbolic Givens Rotations......Page 636 36.2 Hyperbolic Householder Transformations......Page 639 36.3 Hyperbolic Basis Rotations......Page 642 37 Fast Array Algorithm......Page 644 37.1 Time-Update of the Gain Vector......Page 645 37.2 Time-Update of the Conversion Factor......Page 646 37.3 Initial Conditions......Page 647 37.4 Array Algorithm......Page 648 37.A Chandrasekhar Filter......Page 652 38 Regularized Prediction Problems......Page 655 38.1 Regularized Backward Prediction......Page 656 38.2 Regularized Forward Prediction......Page 658 38.3 Low-Rank Factorization......Page 661 39.1 Fast Transversal Filter......Page 662 39.2 FAEST Filter......Page 664 39.3 Fast Kalman Filter......Page 665 39.4 Stability Issues......Page 667 Summary and Notes......Page 673 Problems and Computer Projects......Page 676 PART X: LATTICE FILTERS......Page 686 40 Three Basic Estimation Problems......Page 687 40.1 Motivation for Lattice Filters......Page 688 40.2 Joint Process Estimation......Page 690 40.3 Backward Estimation Problem......Page 693 40.4 Forward Estimation Problem......Page 696 40.5 Time and Order-Update Relations......Page 698 41.1 Significance of Data Structure......Page 703 41.2 A Posteriori-Based Lattice Filter......Page 706 41.3 A Priori-Based Lattice Filter......Page 707 42.1 A Priori Error-Feedback Lattice Filter......Page 710 42.2 A Posteriori Error-Feedback Lattice Filter......Page 714 42.3 Normalized Lattice Filter......Page 716 43 Array Lattice Filters......Page 722 43.1 Order-Update of Output Estimation Errors......Page 723 43.2 Order-Update of Backward Estimation Errors......Page 724 43.3 Order-Update of Forward Estimation Errors......Page 725 43.4 Significance of Data Structure......Page 727 Summary and Notes......Page 729 Problems and Computer Projects......Page 732 PART XI: ROBUST FILTERS......Page 738 44.1 Indefinite Least-Squares Formulation......Page 739 44.2 Recursive Minimization Algorithm......Page 744 44.3 Time-Update of the Minimum Cost......Page 747 44.4 Singular Weighting Matrices......Page 748 44.B Inertia Conditions......Page 750 45.1 A Posteriori-Based Robust Filters......Page 752 45.2 ε-NLMS Algorithm......Page 758 45.3 A Priori-Based Robust Filters......Page 760 45.4 LMS Algorithm......Page 764 45.A H∞ Filters......Page 766 46.1 Robustness of LMS......Page 769 46.2 Robustness of ε--NLMS......Page 773 46.3 Robustness of RLS......Page 774 Summary and Notes......Page 777 Problems and Computer Projects......Page 781 REFERENCES AND INDICES......Page 791 References......Page 792 Author Index......Page 809 Subject Index......Page 814 Adaptive Filters 3 Contents 9 Preface 19 Notation 27 Acknowledgments 32 BACKGROUND MATERIAL 33 A Random Variables 35 A.1 Variance of a Random Variable 35 A.2 Dependent Random Variables 37 A.3 Complex-Valued Random Variables 38 A.4 Vector-Valued Random Variables 40 A.5 Gaussian Random Vectors 41 B Linear Algebra 46 B.1 Hermitian and Positive-Definite Matrices 46 B.2 Range Spaces and Nullspaces of Matrices 48 B.3 Schur Complements 50 B.4 Cholesky Factorization 51 B.5 QR Decomposition 53 B.6 Singular Value Decomposition 54 B.7 Kronecker Products 57 C Complex Gradients 59 C.1 Cauchy-Riemann Conditions 59 C.2 Scalar Arguments 60 C.3 Vector Arguments 60 PART I: OPTIMAL ESTIMATION 62 1 Scalar-Valued Data 63 1.1 Estimation Without Observations 63 1.2 Estimation Given Dependent Observations 65 1.3 Orthogonality Principle 70 1.4 Gaussian Random Variables 72 2 Vector-Valued Data 76 2.1 Optimal Estimator in the Vector Case 76 2.2 Spherically Invariant Gaussian Variables 80 2.3 Equivalent Optimization Criterion 83 Summary and Notes 85 Problems and Computer Projects 88 PART II: LINEAR ESTIMATION 93 3 Normal Equations 94 3.1 Mean-Square Error Criterion 95 3.2 Minimization by Differentiation 97 3.3 Minimization by Completion-of-Squares 97 3.4 Minimization of the Error Covariance Matrix 99 3.5 Optimal Linear Estimator 100 4 Orthogonality Principle 101 4.1 Design Examples 101 4.2 Orthogonality Condition 106 4.3 Existence of Solutions 108 4.4 Nonzero-Mean Variables 110 5 Linear Models 112 5.1 Estimation using Linear Relations 112 5.2 Application: Channel Estimation 114 5.3 Application: Block Data Estimation 115 5.4 Application: Linear Channel Equalization 116 5.5 Application: Multiple-Antenna Receivers 119 6 Constrained Estimation 121 6.1 Minimum-Variance Unbiased Estimation 122 6.2 Example: Mean Estimation 124 6.3 Application: Channel and Noise Estimation 125 6.4 Application: Decision Feedback Equalization 127 6.5 Application: Antenna Beamforming 135 7 Kalman Filter 138 7.1 Innovations Process 138 7.2 State-Space Model 140 7.3 Recursion for the State Estimator 141 7.4 Computing the Gain Matrix 142 7.5 Riccati Recursion 143 7.6 Covariance Form 143 7.7 Measurement and Time-Update Form 144 Summary and Notes 145 Problems and Computer Projects 149 PART III: STOCHASTIC GRADIENT ALGORITHMS 172 8 Steepest–Descent Technique 173 8.1 Linear Estimation Problem 174 8.2 Steepest-Descent Method 176 8.3 More General Cost Functions 181 9 Transient Behavior 182 9.1 Modes of Convergence 182 9.2 Optimal Step-Size 183 9.3 Weight-Error Vector Convergence 185 9.4 Time Constants 187 9.5 Learning Curve 188 9.6 Contour Curves of the Error Surface 189 9.7 Iteration-Dependent Step-Sizes 191 9.8 Newton’s Method 194 10 LMS Algorithm 197 10.1 Motivation 197 10.2 Instantaneous Approximation 199 10.3 Computational Cost 200 10.4 Least-Perturbation Property 201 10.5 Application: Adaptive Channel Estimation 202 10.6 Application: Adaptive Channel Equalization 205 10.7 Application: Decision-Feedback Equalization 206 10.8 Ensemble-Average Learning Curves 208 11 Normalized LMS Algorithm 212 11.1 Instantaneous Approximation 212 11.2 Computational Cost 213 11.3 Power Normalization 214 11.4 Least-Perturbation Property 216 12 Other LMS-Type Algorithms 217 12.1 Non-Blind Algorithms 217 12.2 Blind Algorithms 220 12.3 Some Properties 222 13 Affine Projection Algorithm 225 13.1 Instantaneous Approximation 225 13.2 Computational Cost 227 13.3 Least-Perturbation Property 227 13.4 Affine Projection Interpretation 228 14 RLS Algorithm 232 14.1 Instantaneous Approximation 232 14.2 Computational Cost 234 Summary and Notes 236 Problems and Computer Projects 243 PART IV: MEAN-SQUARE PERFORMANCE 261 15 Energy Conservation 262 15.1 Performance Measure 262 15.2 Stationary Data Model 264 15.3 Energy Conservation Relation 268 15.4 Variance Relation 271 15.A Interpretations of the Energy Relation 273 16 Performance of LMS 278 16.1 Variance Relation 278 16.2 Small Step-Sizes 279 16.3 Separation Principle 279 16.4 White Gaussian Input 280 16.5 Statement of Results 283 16.6 Simulation Results 284 17 Performance of NLMS 286 17.1 Separation Principle 286 17.2 Simulation Results 288 17.A Relating NLMS to LMS 288 18 Performance of Sign-Error LMS 291 18.1 Real-Valued Data 291 18.2 Complex-Valued Data 293 18.3 Simulation Results 294 19 Performance of RLS and Other Filters 296 19.1 Performance of RLS 296 19.2 Performance of Other Filters 300 19.3 Performance Table for Small Step-Sizes 303 20 Nonstationary Environments 304 20.1 Motivation 304 20.2 Nonstationary Data Model 305 20.3 Energy Conservation Relation 310 20.4 Variance Relation 311 21 Tracking Performance 314 21.1 Performance of LMS 314 21.2 Performance of NLMS 318 21.3 Performance of Sign-Error LMS 319 21.4 Performance of RLS 321 21.5 Comparison of Tracking Performance 323 21.6 Comparing RLS and LMS 326 21.7 Performance of Other Filters 327 21.8 Performance Table for Small Step-Sizes 329 Summary and Notes 330 Problems and Computer Projects 338 PART V: TRANSIENT PERFORMANCE 362 22 Weighted Energy Conservation 363 22.1 Data Model 363 22.2 Data-Normalized Adaptive Filters 364 22.3 Weighted Energy Conservation Relation 364 22.4 Weighted Variance Relation 367 23 LMS with Gaussian Regressors 374 23.1 Mean and Variance Relations 374 23.2 Mean Behavior 377 23.3 Mean-Square Behavior 377 23.4 Mean-Square Stability 380 23.5 Steady-State Performance 384 23.6 Small Step-Size Approximations 387 23.A Convergence Time 387 24 LMS with non-Gaussian Regressors 391 24.1 Mean and Variance Relations 391 24.2 Mean-Square Stability and Performance 394 24.3 Small Step-Size Approximations 396 24.A Independence and Averaging Analysis 397 25 Data-Normalized Filters 405 25.1 NLMS Filter 405 25.2 Data-Normalized Filters 408 25.A Stability Bound 411 25.B Stability of NLMS 412 Summary and Notes 414 Problems and Computer Projects 422 PART VI: BLOCK ADAPTIVE FILTERS 446 26 Transform Domain Adaptive Filters 447 26.1 Transform-Domain Filters 447 26.2 DFT-Domain LMS 455 26.3 DCT-Domain LMS 457 26.A DCT-Transformed Regressors 458 27 Efficient Block Convolution 460 27.1 Motivation 460 27.2 Block Data Formulation 462 27.3 Block Convolution 465 28 Block and Subband Adaptive Filters 474 28.1 DFT Block Adaptive Filters 474 28.2 Subband Adaptive Filters 481 28.A Another Constrained DFT Block Filter 487 28.B Overlap-Add Block Adaptive Filters 489 Summary and Notes 499 Problems and Computer Projects 502 PART VII: LEAST-SQUARES METHODS 509 29 Least-Squares Criterion 510 29.1 Least-Squares Problem 511 29.2 Geometric Argument 512 29.3 Algebraic Arguments 514 29.4 Properties of Least-Squares Solution 516 29.5 Projection Matrices 518 29.6 Weighted Least-Squares 519 29.7 Regularized Least-Squares 521 29.8 Weighted Regularized Least-Squares 523 30 Recursive Least-Squares 526 30.1 Motivation 526 30.2 RLS Algorithm 527 30.3 Regularization 529 30.4 Conversion Factor 530 30.5 Time-Update of the Minimum Cost 531 30.6 Exponentially-Weighted RLS Algorithm 532 31 Kalman Filtering and RLS 535 31.1 Equivalence in Linear Estimation 535 31.2 Kalman Filtering and Recursive Least-Squares 536 31.A Extended RLS Algorithms 542 32 Order and Time-Update Relations 549 32.1 Backward Order-Update Relations 549 32.2 Forward Order-Update Relations 559 32.3 Time-Update Relation 563 Summary and Notes 568 Problems and Computer Projects 575 PART VIII: ARRAY ALGORITHMS 594 33 Norm and Angle Preservation 595 33.1 Some Difficulties 595 33.2 Square-Root Factors 596 33.3 Preservation Properties 598 33.4 Motivation for Array Methods 600 34 Unitary Transformations 605 34.1 Givens Rotations 605 34.2 Householder Transformations 610 35 QR and Inverse QR Algorithms 614 35.1 Inverse QR Algorithm 615 35.2 QR Algorithm 618 35.3 Extended QR Algorithm 622 35.A Array Algorithms for Kalman Filtering 623 Summary and Notes 627 Problems and Computer Projects 629 PART IX: FAST RLS ALGORITHMS 635 36 Hyperbolic Rotations 636 36.1 Hyperbolic Givens Rotations 636 36.2 Hyperbolic Householder Transformations 639 36.3 Hyperbolic Basis Rotations 642 37 Fast Array Algorithm 644 37.1 Time-Update of the Gain Vector 645 37.2 Time-Update of the Conversion Factor 646 37.3 Initial Conditions 647 37.4 Array Algorithm 648 37.A Chandrasekhar Filter 652 38 Regularized Prediction Problems 655 38.1 Regularized Backward Prediction 656 38.2 Regularized Forward Prediction 658 38.3 Low-Rank Factorization 661 39 Fast Fixed-Order Filters 662 39.1 Fast Transversal Filter 662 39.2 FAEST Filter 664 39.3 Fast Kalman Filter 665 39.4 Stability Issues 667 Summary and Notes 673 Problems and Computer Projects 676 PART X: LATTICE FILTERS 686 40 Three Basic Estimation Problems 687 40.1 Motivation for Lattice Filters 688 40.2 Joint Process Estimation 690 40.3 Backward Estimation Problem 693 40.4 Forward Estimation Problem 696 40.5 Time and Order-Update Relations 698 41 Lattice Filter Algorithms 703 41.1 Significance of Data Structure 703 41.2 A Posteriori-Based Lattice Filter 706 41.3 A Priori-Based Lattice Filter 707 42 Error-Feedback Lattice Filters 710 42.1 A Priori Error-Feedback Lattice Filter 710 42.2 A Posteriori Error-Feedback Lattice Filter 714 42.3 Normalized Lattice Filter 716 43 Array Lattice Filters 722 43.1 Order-Update of Output Estimation Errors 723 43.2 Order-Update of Backward Estimation Errors 724 43.3 Order-Update of Forward Estimation Errors 725 43.4 Significance of Data Structure 727 Summary and Notes 729 Problems and Computer Projects 732 PART XI: ROBUST FILTERS 738 44 Indefinite Least-Squares 739 44.1 Indefinite Least-Squares Formulation 739 44.2 Recursive Minimization Algorithm 744 44.3 Time-Update of the Minimum Cost 747 44.4 Singular Weighting Matrices 748 44.A Stationary Points 750 44.B Inertia Conditions 750 45 Robust Adaptive Filters 752 45.1 A Posteriori-Based Robust Filters 752 45.2 ε-NLMS Algorithm 758 45.3 A Priori-Based Robust Filters 760 45.4 LMS Algorithm 764 45.A H∞ Filters 766 46 Robustness Properties 769 46.1 Robustness of LMS 769 46.2 Robustness of ε--NLMS 773 46.3 Robustness of RLS 774 Summary and Notes 777 Problems and Computer Projects 781 REFERENCES AND INDICES 791 References 792 Author Index 809 Subject Index 814 Adaptive Filtering Is A Topic Of Immense Practical And Theoretical Value, Having Applications In Areas Ranging From Digital And Wireless Communications To Biomedical Systems. Now, Preserving The Style And Main Features Of The Earlier Award-winning Publication, Fundamentals Of Adaptive Filtering (2005 Terman Award), The Author Offers Readers And Instructors A Concentrated, Systematic, And Up-to-date Treatment Of The Subject In This Valuable New Book. Adaptive Filters Allows Readers To Gain A Gradual And Solid Introduction To The Subject, Its Applications To A Variety Of Topical Problems, Existing Limitations, And Extensions Of Current Theories. This Book Consists Of Eleven Parts - Each Part Containing A Series Of Focused Lectures And Ending With Bibliographic Comments, Problems, And Computer Projects With Matlab[registered] Solutions Available To All Readers. Additional Features Include: Numerous Tables, Figures, And Projects; Special Focus On Geometric Constructions, Physical Intuition, Linear-algebraic Concepts, And Vector Notation; Background Material On Random Variables, Linear Algebra, And Complex Gradients Collected In Three Introductory Chapters; And, Complete Solutions Manual Available For Instructors Matlab[registered] Solutions Available For All Computer Projects. Adaptive Filters Offers A Fresh, Focused Look At The Subject In A Manner That Will Entice Students, Challenge Experts, And Appeal To Practitioners And Instructors. Cover13; -- Title13; -- Copyright13; -- Dedication13; -- Contents13; -- Preface -- Notation -- Acknowledgments -- Background Material 13; -- A Random Variables -- A.1 Variance Of A Random Variable -- A.2 Dependent Random Variables -- A.3 Complex-valued Random Variables -- A.4 Vector-valued Random Variables -- A.5 Gaussian Random Vectors -- B Linear Algebra -- B.1 Hermitian And Positive-definite Matrices -- B.2 Range Spaces And Nullspaces Of Matrices -- B.3 Schur Complements -- B.4 Cholesky Factorization -- B.5 Qr Decomposition -- B.6 Singular Value Decomposition -- B.7 Kronecker Products -- C Complex Gradients -- C.1 Cauchy-riemann Conditions -- C.2 Scalar Arguments -- C.3 Vector Arguments -- Part I: Optimal Estimation13; -- 113;scalar-valued Data -- 1.1 Estimation Without Observations -- 1.2 Estimation Given Dependent Observations -- 1.3 Orthogonality Principle -- 1.4 Gaussian Random Variables -- 213;vector-valued Data -- 2.1 Optimal Estimator In The Vector Case --^ 2.2 Spherically Invariant Gaussian Variables -- 2.3 Equivalent Optimization Criterion -- Summary And Notes -- Problems And Computer Projects -- Part Ii: Linear Estimation13; -- 3 Normal Equations13; -- 3.1 Mean-square Error Criterion -- 3.2 Minimization By Differentiation -- 3.3 Minimization By Completion-of-squares -- 3.4 Minimization Of The Error Covariance Matrix -- 3.5 Optimal Linear Estimator -- 4 Orthogonality Principle13; -- 4.1 Design Examples -- 4.2 Orthogonality Condition -- 4.3 Existence Of Solutions -- 4.4 Nonzero-mean Variables -- 5 Linear Models13; -- 5.1 Estimation Using Linear Relations -- 5.2 Application: Channel Estimation -- 5.3 Application: Block Data Estimation -- 5.4 Application: Linear Channel Equalization -- 5.5 Application: Multiple-antenna Receivers -- 6 Constrained Estimation13; -- 6.1 Minimum-variance Unbiased Estimation -- 6.2 Example: Mean Estimation -- 6.3 Application: Channel And Noise Estimation -- 6.4 Application: Decision Feedback Equalization --^ 6.5 Application: Antenna Beamforming -- 7 Kalman Filter13; -- 7.1 Innovations Process -- 7.2 State-space Model -- 7.3 Recursion For The State Estimator -- 7.4 Computing The Gain Matrix -- 7.5 Riccati Recursion -- 7.6 Covariance Form -- 7.7 Measurement And Time-update Form -- Summary And Notes -- Problems And Computer Projects -- Part Iii: Stochastic Gradient Algorithms13; -- 8 Steepest8211;descent Technique13; -- 8.1 Linear Estimation Problem -- 8.2 Steepest-descent Method -- 8.3 More General Cost Functions -- 9 Transient Behavior13; -- 9.1 Modes Of Convergence -- 9.2 Optimal Step-size -- 9.3 Weight-error Vector Convergence -- 9.4 Time Constants -- 9.5 Learning Curve -- 9.6 Contour Curves Of The Error Surface -- 9.7 Iteration-dependent Step-sizes -- 9.8 Newton8217;s Method -- 10 Lms Algorithm13; -- 10.1 Motivation -- 10.2 Instantaneous Approximation -- 10.3 Computational Cost -- 10.4 Least-perturbation Property -- 10.5 Application: Adaptive Channel Estimation --^ 10.6 Application: Adaptive Channel Equalization -- 10.7 Application: Decision-feedback Equalization -- 10.8 Ensemble-average Learning Curves -- 11 Normalized Lms Algorithm13; -- T$137. Ali H. Sayed. Includes Bibliographical References (p. 758-774) And Indexes. Adaptive Filtering: Algorithms and Practical Implementation, Third Edition, presents basic concepts of adaptive signal processing and filtering in a concise and straightforward manner. It concentrates on on-line algorithms whose adaptation occurs whenever a new sample of each environment signal is available. The material also illustrates block algorithms using a sub-band filtering framework whose adaptation occurs when a new block of data is available. Highlights of the new edition include: Expanded treatment of complex algorithms throughout the book New chapters on Data-Selective and Blind Adaptive Filtering An enlarged discussion of linear-constrained Wiener filters Detailed analysis of the affine projection algorithm Updated derivations and many new examples A primer on Kalman filtering in Appendix D as a complement to RLS algorithms. Algorithms are presented in a unified framework using a consistent notation that facilitates their actual implementation. The main algorithms are summarized and described in tables. Many examples address problems drawn from actual applications. The family of LMS and RLS algorithms as well as set-membership, sub-band, blind, nonlinear and IIR adaptive filtering, are covered. Problems are included at the end of chapters. Adaptive Filtering: Algorithms and Practical Implementation, Third Edition, is intended for advanced undergraduate and graduate students studying adaptive filtering and will also serve as an up-to-date and useful reference for professional engineers working in the field