Data-Driven Modeling & Scientific Computation : Methods for Complex Systems & Big Data

The burgeoning field of data analysis is expanding at an incredible pace due to the proliferation of data collection in almost every area of science. The enormous data sets now routinely encountered in the sciences provide an incentive to develop mathematical techniques and computational algorithms that help synthesize, interpret and give meaning to the data in the context of its scientific setting. A specific aim of this book is to integrate standard scientific computing methods with data analysis. By doing so, it brings together, in a self-consistent fashion, the key ideas from: DT statistics, DT time-frequency analysis, and DT low-dimensional reductions The blend of these ideas provides meaningful insight into the data sets one is faced with in every scientific subject today, including those generated from complex dynamical systems. This is a particularly exciting field and much of the final part of the book is driven by intuitive examples from it, showing how the three areas can be used in combination to give critical insight into the fundamental workings of various problems. Data-Driven Modeling and Scientific Computation is a survey of practical numerical solution techniques for ordinary and partial differential equations as well as algorithms for data manipulation and analysis. Emphasis is on the implementation of numerical schemes to practical problems in the engineering, biological and physical sciences. An accessible introductory-to-advanced text, this book fully integrates MATLAB and its versatile and high-level programming functionality, while bringing together computational and data skills for both undergraduate and graduate students in scientific computing. Cover 1 Contents 8 Prolegomenon 14 How to Use This Book 16 About MATLAB 19 PART I: Basic Computations and Visualization 20 1 MATLAB Introduction 22 1.1 Vectors and Matrices 22 1.2 Logic, Loops and Iterations 28 1.3 Iteration: The Newton–Raphson Method 32 1.4 Function Calls, Input/Output Interactions and Debugging 37 1.5 Plotting and Importing/Exporting Data 42 2 Linear Systems 50 2.1 Direct Solution Methods for Ax = b 50 2.2 Iterative Solution Methods for Ax = b 54 2.3 Gradient (Steepest) Descent for Ax = b 58 2.4 Eigenvalues, Eigenvectors and Solvability 63 2.5 Eigenvalues and Eigenvectors for Face Recognition 68 2.6 Nonlinear Systems 75 3 Curve Fitting 80 3.1 Least-Square Fitting Methods 80 3.2 Polynomial Fits and Splines 84 3.3 Data Fitting with MATLAB 88 4 Numerical Differentiation and Integration 96 4.1 Numerical Differentiation 96 4.2 Numerical Integration 102 4.3 Implementation of Differentiation and Integration 106 5 Basic Optimization 112 5.1 Unconstrained Optimization (Derivative-Free Methods) 112 5.2 Unconstrained Optimization (Derivative Methods) 118 5.3 Linear Programming 124 5.4 Simplex Method 129 5.5 Genetic Algorithms 132 6 Visualization 138 6.1 Customizing Plots and Basic 2D Plotting 138 6.2 More 2D and 3D Plotting 144 6.3 Movies and Animations 150 PART II: Differential and Partial Differential Equations 154 7 Initial and Boundary Value Problems of Differential Equations 156 7.1 Initial Value Problems: Euler, Runge–Kutta and Adams Methods 156 7.2 Error Analysis for Time-Stepping Routines 163 7.3 Advanced Time-Stepping Algorithms 168 7.4 Boundary Value Problems: The Shooting Method 172 7.5 Implementation of Shooting and Convergence Studies 179 7.6 Boundary Value Problems: Direct Solve and Relaxation 183 7.7 Implementing MATLAB for Boundary Value Problems 186 7.8 Linear Operators and Computing Spectra 191 8 Finite Difference Methods 199 8.1 Finite Difference Discretization 199 8.2 Advanced Iterative Solution Methods for Ax = b 205 8.3 Fast Poisson Solvers: The Fourier Transform 205 8.4 Comparison of Solution Techniques for Ax = b: Rules of Thumb 209 8.5 Overcoming Computational Difficulties 214 9 Time and Space Stepping Schemes: Method of Lines 219 9.1 Basic Time-Stepping Schemes 219 9.2 Time-Stepping Schemes: Explicit and Implicit Methods 224 9.3 Stability Analysis 228 9.4 Comparison of Time-Stepping Schemes 232 9.5 Operator Splitting Techniques 235 9.6 Optimizing Computational Performance: Rules of Thumb 238 10 Spectral Methods 244 10.1 Fast Fourier Transforms and Cosine/Sine Transform 244 10.2 Chebychev Polynomials and Transform 248 10.3 Spectral Method Implementation 252 10.4 Pseudo-Spectral Techniques with Filtering 254 10.5 Boundary Conditions and the Chebychev Transform 259 10.6 Implementing the Chebychev Transform 263 10.7 Computing Spectra: The Floquet–Fourier–Hill Method 268 11 Finite Element Methods 275 11.1 Finite Element Basis 275 11.2 Discretizing with Finite Elements and Boundaries 280 11.3 MATLAB for Partial Differential Equations 285 11.4 MATLAB Partial Differential Equations Toolbox 290 PART III: Computational Methods for Data Analysis 296 12 Statistical Methods and Their Applications 298 12.1 Basic Probability Concepts 298 12.2 Random Variables and Statistical Concepts 305 12.3 Hypothesis Testing and Statistical Significance 313 13 Time–Frequency Analysis: Fourier Transforms and Wavelets 320 13.1 Basics of Fourier Series and the Fourier Transform 320 13.2 FFT Application: Radar Detection and Filtering 327 13.3 FFT Application: Radar Detection and Averaging 335 13.4 Time–Frequency Analysis: Windowed Fourier Transforms 341 13.5 Time–Frequency Analysis and Wavelets 347 13.6 Multi-Resolution Analysis and the Wavelet Basis 354 13.7 Spectrograms and the Gábor Transform in MATLAB 359 13.8 MATLAB Filter Design and Wavelet Toolboxes 365 14 Image Processing and Analysis 377 14.1 Basic Concepts and Analysis of Images 377 14.2 Linear Filtering for Image Denoising 383 14.3 Diffusion and Image Processing 388 15 Linear Algebra and Singular Value Decomposition 395 15.1 Basics of the Singular Value Decomposition (SVD) 395 15.2 The SVD in Broader Context 400 15.3 Introduction to Principal Component Analysis (PCA) 406 15.4 Principal Components, Diagonalization and SVD 410 15.5 Principal Components and Proper Orthogonal Modes 414 15.6 Robust PCA 422 16 Independent Component Analysis 431 16.1 The Concept of Independent Components 431 16.2 Image Separation Problem 438 16.3 Image Separation and MATLAB 443 17 Image Recognition: Basics of Machine Learning 450 17.1 Recognizing Dogs and Cats 450 17.2 The SVD and Linear Discrimination Analysis 455 17.3 Implementing Cat/Dog Recognition in MATLAB 464 18 Basics of Compressed Sensing 468 18.1 Beyond Least-Square Fitting: The L[sup(1)] Norm 468 18.2 Signal Reconstruction and Circumventing Nyquist 475 18.3 Data (Image) Reconstruction from Sparse Sampling 483 19 Dimensionality Reduction for Partial Differential Equations 491 19.1 Modal Expansion Techniques for PDEs 491 19.2 PDE Dynamics in the Right (Best) Basis 497 19.3 Global Normal Forms of Bifurcation Structures in PDEs 501 19.4 The POD Method and Symmetries/Invariances 511 19.5 POD Using Robust PCA 518 20 Dynamic Mode Decomposition 525 20.1 Theory of Dynamic Mode Decomposition (DMD) 525 20.2 Dynamics of DMD Versus POD 529 20.3 Applications of DMD 534 21 Data Assimilation Methods 540 21.1 Theory of Data Assimilation 540 21.2 Data Assimilation, Sampling and Kalman Filtering 545 21.3 Data Assimilation for the Lorenz Equation 548 22 Equation-Free Modeling 556 22.1 Multi-Scale Physics: An Equation-Free Approach 556 22.2 Lifting and Restricting in Equation-Free Computing 561 22.3 Equation-Free Space–Time Dynamics 566 23 Complex Dynamical Systems: Combining Dimensionality Reduction, Compressive Sensing and Machine Learning 570 23.1 Combining Data Methods for Complex Systems 570 23.2 Implementing a Dynamical Systems Library 575 23.3 Flow Around a Cylinder: A Prototypical Example 583 PART IV: Scientific Applications 590 24 Applications of Differential Equations and Boundary Value Problems 592 24.1 Neuroscience and the Hodgkin–Huxley Model 592 24.2 Celestial Mechanics and the Three-Body Problem 596 24.3 Atmospheric Motion and the Lorenz Equations 600 24.4 Quantum Mechanics 604 24.5 Electromagnetic Waveguides 607 25 Applications of Partial Differential Equations 609 25.1 The Wave Equation 609 25.2 Mode-Locked Lasers 612 25.3 Bose–Einstein Condensates 619 25.4 Advection–Diffusion and Atmospheric Dynamics 623 25.5 Introduction to Reaction–Diffusion Systems 630 25.6 Steady State Flow Over an Airfoil 635 26 Applications of Data Analysis 639 26.1 Analyzing Music Scores and the Gábor Transform 639 26.2 Image Denoising through Filtering and Diffusion 641 26.3 Oscillating Mass and Dimensionality Reduction 644 26.4 Music Genre Identification 645 References 648 Index of MATLAB Commands 653 A 653 B 653 C 653 D 653 E 653 F 653 G 653 H 653 I 653 L 653 M 653 N 653 O 653 P 653 Q 653 R 653 S 653 T 653 U 653 V 653 W 653 X 654 Y 654 Index 655 A 655 B 655 C 655 D 655 E 655 F 655 G 655 H 656 I 656 J 656 K 656 L 656 M 656 N 656 O 656 P 657 Q 657 R 657 S 657 T 657 U 657 V 657 W 657 ISBN,9780199660339,(hbk.),ISBN,978–0–19–966034–6,(pbk.) ISBN 9780199660339 (hbk.),ISBN 978–0–19–966034–6 (pbk.) Machine Generated Contents Note: Pt. I Basic Computations And Visualization -- 1.matlab Introduction -- 1.1.vectors And Matrices -- 1.2.logic, Loops And Iterations -- 1.3.iteration: The Newton-raphson Method -- 1.4.function Calls, Input/output Interactions And Debugging -- 1.5.plotting And Importing/exporting Data -- 2.linear Systems -- 2.1.direct Solution Methods For Ax = B -- 2.2.iterative Solution Methods For Ax = B -- 2.3.gradient (steepest) Descent For Ax = B -- 2.4.eigenvalues, Eigenvectors And Solvability -- 2.5.eigenvalues And Eigenvectors For Face Recognition -- 2.6.nonlinear Systems -- 3.curve Fitting -- 3.1.least-square Fitting Methods -- 3.2.polynomial Fits And Splines -- 3.3.data Fitting With Matlab -- 4.numerical Differentiation And Integration -- 4.1.numerical Differentiation -- 4.2.numerical Integration -- 4.3.implementation Of Differentiation And Integration -- 5.basic Optimization -- 5.1.unconstrained Optimization (derivative-free Methods) -- Contents Note Continued: 5.2.unconstrained Optimization (derivative Methods) -- 5.3.linear Programming -- 5.4.simplex Method -- 5.5.genetic Algorithms -- 6.visualization -- 6.1.customizing Plots And Basic 2d Plotting -- 6.2.more 2d And 3d Plotting -- 6.3.movies And Animations -- Pt. Ii Differential And Partial Differential Equations -- 7.initial And Boundary Value Problems Of Differential Equations -- 7.1.initial Value Problems: Euler, Runge-kutta And Adams Methods -- 7.2.error Analysis For Time-stepping Routines -- 7.3.advanced Time-stepping Algorithms -- 7.4.boundary Value Problems: The Shooting Method -- 7.5.implementation Of Shooting And Convergence Studies -- 7.6.boundary Value Problems: Direct Solve And Relaxation -- 7.7.implementing Matlab For Boundary Value Problems -- 7.8.linear Operators And Computing Spectra -- 8.finite Difference Methods -- 8.1.finite Difference Discretization -- 8.2.advanced Iterative Solution Methods For Ax = B -- Contents Note Continued: 8.3.fast Poisson Solvers: The Fourier Transform -- 8.4.comparison Of Solution Techniques For Ax = B: Rules Of Thumb -- 8.5.overcoming Computational Difficulties -- 9.time And Space Stepping Schemes: Method Of Lines -- 9.1.basic Time-stepping Schemes -- 9.2.time-stepping Schemes: Explicit And Implicit Methods -- 9.3.stability Analysis -- 9.4.comparison Of Time-stepping Schemes -- 9.5.operator Splitting Techniques -- 9.6.optimizing Computational Performance: Rules Of Thumb -- 10.spectral Methods -- 10.1.fast Fourier Transforms And Cosine/sine Transform -- 10.2.chebychev Polynomials And Transform -- 10.3.spectral Method Implementation -- 10.4.pseudo-spectral Techniques With Filtering -- 10.5.boundary Conditions And The Chebychev Transform -- 10.6.implementing The Chebychev Transform -- 10.7.computing Spectra: The Floquet-fourier-hill Method -- 11.finite Element Methods -- 11.1.finite Element Basis -- 11.2.discretizing With Finite Elements And Boundaries -- Contents Note Continued: 11.3.matlab For Partial Differential Equations -- 11.4.matlab Partial Differential Equations Toolbox -- Pt. Iii Computational Methods For Data Analysis -- 12.statistical Methods And Their Applications -- 12.1.basic Probability Concepts -- 12.2.random Variables And Statistical Concepts -- 12.3.hypothesis Testing And Statistical Significance -- 13.time-frequency Analysis: Fourier Transforms And Wavelets -- 13.1.basics Of Fourier Series And The Fourier Transform -- 13.2.fft Application: Radar Detection And Filtering -- 13.3.fft Application: Radar Detection And Averaging -- 13.4.time-frequency Analysis: Windowed Fourier Transforms -- 13.5.time-frequency Analysis And Wavelets -- 13.6.multi-resolution Analysis And The Wavelet Basis -- 13.7.spectrograms And The Gabor Transform In Matlab -- 13.8.matlab Filter Design And Wavelet Toolboxes -- 14.image Processing And Analysis -- 14.1.basic Concepts And Analysis Of Images -- Contents Note Continued: 14.2.linear Filtering For Image Denoising -- 14.3.diffusion And Image Processing -- 15.linear Algebra And Singular Value Decomposition -- 15.1.basics Of The Singular Value Decomposition (svd) -- 15.2.the Svd In Broader Context -- 15.3.introduction To Principal Component Analysis (pca) -- 15.4.principal Components, Diagonalization And Svd -- 15.5.principal Components And Proper Orthogonal Modes -- 15.6.robust Pca -- 16.independent Component Analysis -- 16.1.the Concept Of Independent Components -- 16.2.image Separation Problem -- 16.3.image Separation And Matlab -- 17.image Recognition: Basics Of Machine Learning -- 17.1.recognizing Dogs And Cats -- 17.2.the Svd And Linear Discrimination Analysis -- 17.3.implementing Cat/dog Recognition In Matlab -- 18.basics Of Compressed Sensing -- 18.1.beyond Least-square Fitting: The L1 Norm -- 18.2.signal Reconstruction And Circumventing Nyquist -- 18.3.data (image) Reconstruction From Sparse Sampling -- Contents Note Continued: 19.dimensionality Reduction For Partial Differential Equations -- 19.1.modal Expansion Techniques For Pdes -- 19.2.pde Dynamics In The Right (best) Basis -- 19.3.global Normal Forms Of Bifurcation Structures In Pdes -- 19.4.the Pod Method And Symmetries/invariances -- 19.5.pod Using Robust Pca -- 20.dynamic Mode Decomposition -- 20.1.theory Of Dynamic Mode Decomposition (dmd) -- 20.2.dynamics Of Dmd Versus Pod -- 20.3.applications Of Dmd -- 21.data Assimilation Methods -- 21.1.theory Of Data Assimilation -- 21.2.data Assimilation, Sampling And Kalman Filtering -- 21.3.data Assimilation For The Lorenz Equation -- 22.equation-free Modeling -- 22.1.multi-scale Physics: An Equation-free Approach -- 22.2.lifting And Restricting In Equation-free Computing -- 22.3.equation-free Space-time Dynamics -- 23.complex Dynamical Systems: Combining Dimensionality Reduction, Compressive Sensing And Machine Learning -- 23.1.combining Data Methods For Complex Systems -- Contents Note Continued: 23.2.implementing A Dynamical Systems Library -- 23.3.flow Around A Cylinder: A Prototypical Example -- Pt. Iv Scientific Applications -- 24.applications Of Differential Equations And Boundary Value Problems -- 24.1.neuroscience And The Hodgkin-huxley Model -- 24.2.celestial Mechanics And The Three-body Problem -- 24.3.atmospheric Motion And The Lorenz Equations -- 24.4.quantum Mechanics -- 24.5.electromagnetic Waveguides -- 25.applications Of Partial Differential Equations -- 25.1.the Wave Equation -- 25.2.mode-locked Lasers -- 25.3.bose-elnstein Condensates -- 25.4.advection-diffusion And Atmospheric Dynamics -- 25.5.introduction To Reaction-diffusion Systems -- 25.6.steady State Flow Over An Airfoil -- 26.applications Of Data Analysis -- 26.1.analyzing Music Scores And The Gabor Transform -- 26.2.image Denoising Through Filtering And Diffusion -- 26.3.oscillating Mass And Dimensionality Reduction -- 26.4.music Genre Identification. J. Nathan Kutz, Department Of Applied Mathematics, University Of Washington. Includes Bibliographical References (pages 629-633) And Index. Combining scientific computing methods and algorithms with modern data analysis techniques, including basic applications of compressive sensing and machine learning, this book develops techniques that allow for the integration of the dynamics of complex systems and big data. MATLAB is used throughout for mathematical solution strategies. The burgeoning field of data analysis is expanding at an incredible pace due to the proliferation of data collection in almost every area of science. The enormous data sets now routinely encountered in the sciences provide an incentive to develop mathematical techniques and computational algorithms that help synthesize, interpret and give meaning to the data in the context of its scientific setting. A specific aim of this book is to integrate standard scientific computing methodswith data analysis. By doing so, it brings together, in a self-consistent fashion, the key ideas from:. statistics,. time-frequency analysis, and . low-dimensional reductions The blend of these ideas provides meaningful insight into the data sets one is faced with in every scientific subject today, including those generated from complex dynamical systems. This is a particularly exciting field and much of the final part of the book is driven by intuitive examples from it, showing how the three areas can be used in combination to give critical insight into the fundamental workings of various problems. Data-Driven Modeling and Scientific Computation is a survey of practical numerical solution techniques for ordinary and partial differential equations as well as algorithms for data manipulation and analysis. Emphasis is on the implementation of numerical schemes to practical problems in the engineering, biological and physical sciences. An accessible introductory-to-advanced text, this book fully integrates MATLAB and its versatile and high-level programming functionality, while bringing together computational and data skills for both undergraduate and graduate students in scientific computing