Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition (Adaptation, Learning, and Optimization, 15)

For many engineering problems we require optimization processes with dynamic adaptation as we aim to establish the dimension of the search space where the optimum solution resides and develop robust techniques to avoid the local optima usually associated with multimodal problems. This book explores multidimensional particle swarm optimization, a technique developed by the authors that addresses these requirements in a well-defined algorithmic approach. After an introduction to the key optimization techniques, the authors introduce their unified framework and demonstrate its advantages in challenging application domains, focusing on the state of the art of multidimensional extensions such as global convergence in particle swarm optimization, dynamic data clustering, evolutionary neural networks, biomedical applications and personalized ECG classification, content-based image classification and retrieval, and evolutionary feature synthesis. The content is characterized by strong practical considerations, and the book is supported with fully documented source code for all applications presented, as well as many sample datasets. The book will be of benefit to researchers and practitioners working in the areas of machine intelligence, signal processing, pattern recognition, and data mining, or using principles from these areas in their application domains. It may also be used as a reference text for graduate courses on swarm optimization, data clustering and classification, content-based multimedia search, and biomedical signal processing applications. Preface......Page 5 Abstract......Page 6 Contents......Page 8 Acronyms......Page 12 Tables......Page 15 Figures......Page 20 1 Introduction......Page 26 1.1...Optimization Era......Page 27 1.2...Key Issues......Page 29 1.3...Synopsis of the Book......Page 32 References......Page 35 2.1...History of Optimization......Page 37 2.2.1 Gradient Descent Method......Page 53 2.2.2 Newton--Raphson Method......Page 54 2.2.3 Nelder--Mead Search Method......Page 56 2.3.1 Simulated Annealing......Page 57 2.3.2 Stochastic Approximation......Page 59 2.4.1 Genetic Algorithms......Page 61 2.4.2 Differential Evolution......Page 65 References......Page 67 3.1...Introduction......Page 69 3.2...Basic PSO Algorithm......Page 70 3.3...Some PSO Variants......Page 73 3.3.1 Tribes......Page 75 3.3.2 Multiswarms......Page 77 3.4.1 Nonlinear Function Minimization......Page 79 3.4.2 Data Clustering......Page 81 3.4.3.1 An Overview......Page 85 3.4.3.2 BP versus PSO: Comparative Performance Evaluation Over Medical Datasets......Page 89 3.5...Programming Remarks and Software Packages......Page 98 References......Page 104 4.1...The Need for Multi-dimensionality......Page 107 4.2...The Basic Idea......Page 109 4.3...The MD PSO Algorithm......Page 111 4.4.1 MD PSO Operation in PSO_MDlib Application......Page 116 4.4.2 MD PSO Operation in PSOTestApp Application......Page 118 References......Page 123 5 Improving Global Convergence......Page 124 5.1.2 PSO with FGBF......Page 125 5.1.4 Nonlinear Function Minimization......Page 127 5.2.1 Dynamic Environments: The Test Bed......Page 139 5.2.2 Multiswarm PSO......Page 140 5.2.3 FGBF for the Moving Peak Benchmark for MPB......Page 141 5.2.4 Optimization over Multidimensional MPB......Page 142 5.2.5 Performance Evaluation on Conventional MPB......Page 143 5.2.6 Performance Evaluation on Multidimensional MPB......Page 147 5.3...Who Will Guide the Guide?......Page 151 5.3.1 SPSA Overview......Page 153 5.3.2.1 First SA-Driven PSO Approach: gbest Update by SPSA......Page 154 5.3.2.2 Second SA-driven PSO Approach: aGB Formation by SPSA......Page 156 5.3.3 Applications to Non-linear Function Minimization......Page 157 5.4...Summary and Conclusions......Page 164 5.5...Programming Remarks and Software Packages......Page 165 5.5.1 FGBF Operation in PSO_MDlib Application......Page 166 5.5.2 MD PSO with FGBF Application Over MPB......Page 167 References......Page 170 6 Dynamic Data Clustering......Page 173 6.1.1 The Theory......Page 174 6.1.2 Results on 2D Synthetic Datasets......Page 177 6.2.1 Motivation......Page 182 6.2.2 Fuzzy Model over HSV-HSL Color Domains......Page 185 6.2.3 DC Extraction Results......Page 186 6.2.3.1 Comparative Evaluations Against MPEG-7 DCD......Page 187 6.2.3.2 Robustness and Parameter Insensitivity......Page 189 6.2.3.3 Computational Complexity Analysis......Page 190 6.2.4 Summary and Conclusions......Page 192 6.3.1 SA-Driven MD PSO-Based Dynamic Clustering in 2D Datasets......Page 193 6.3.2 Summary and Conclusions......Page 196 6.4.1 FGBF Operation in 2D Clustering......Page 198 6.4.2 DC Extraction in PSOTestApp Application......Page 201 6.4.3 SA-DRIVEN Operation in PSOTestApp Application......Page 205 References......Page 207 7 Evolutionary Artificial Neural Networks......Page 209 7.1...Search for the Optimal Artificial Neural Networks: An Overview......Page 210 7.2.1 PSO for Artificial Neural Networks: The Early Attempts......Page 212 7.2.2 MD PSO-Based Evolutionary Neural Networks......Page 213 7.2.3 Classification Results on Synthetic Problems......Page 215 7.2.4 Classification Results on Medical Diagnosis Problems......Page 222 7.2.5 Parameter Sensitivity and Computational Complexity Analysis......Page 225 7.3...Evolutionary RBF Classifiers for Polarimetric SAR Images......Page 227 7.3.1 Polarimetric SAR Data Processing......Page 229 7.3.2 SAR Classification Framework......Page 231 7.3.3 Polarimetric SAR Classification Results......Page 233 7.4...Summary and Conclusions......Page 239 7.5...Programming Remarks and Software Packages......Page 240 References......Page 249 8 Personalized ECG Classification......Page 253 8.1.1 Introduction and Motivation......Page 255 8.1.2.2 Feature Extraction Methodology......Page 257 8.1.2.3 Preprocessing by Principal Component Analysis......Page 260 8.1.3.1 MD PSO Optimality Evaluation......Page 261 8.1.3.2 Classification Performance......Page 263 8.1.3.3 Robustness......Page 265 8.2...Classification of Holter Registers......Page 266 8.2.1 The Related Work......Page 267 8.2.2 Personalized Long-Term ECG Classification: A Systematic Approach......Page 268 8.2.3 Experimental Results......Page 272 8.3...Summary and Conclusions......Page 275 8.4...Programming Remarks and Software Packages......Page 277 References......Page 279 9 Image Classification and Retrieval by Collective Network of Binary Classifiers......Page 281 9.1...The Era of CBIR......Page 282 9.2...Content-Based Image Classification and Retrieval Framework......Page 284 9.2.1 Overview of the Framework......Page 285 9.2.2 Evolutionary Update in the Architecture Space......Page 286 9.2.3.1 The Topology......Page 287 9.2.3.2 Evolution of the CNBC......Page 289 9.2.3.3 Incremental Evolution of the CNBC......Page 291 9.3...Results and Discussions......Page 292 9.3.1 Database Creation and Feature Extraction......Page 293 9.3.2.1 Feature Scalability and Comparative Evaluations......Page 294 9.3.2.2 Incremental CNBC Evolutions......Page 296 9.3.2.3 CNBC Class Scalability......Page 298 9.3.3 CBIR Results......Page 299 9.4...Summary and Conclusions......Page 302 9.5...Programming Remarks and Software Packages......Page 303 References......Page 315 10.1...Introduction......Page 317 10.2...Feature Synthesis and Selection: An Overview......Page 319 10.3.1 Motivation......Page 321 10.3.2.2 Encoding of the MD PSO Particles......Page 323 10.3.2.3 The Fitness Function......Page 325 10.4...Simulation Results and Discussions......Page 328 10.4.1 Performance Evaluations with Respect to Discrimination and Classification......Page 329 10.4.2 Comparative Performance Evaluations on Content-Based Image Retrieval......Page 331 10.5...Programming Remarks and Software Packages......Page 336 References......Page 343 This book explores multidimensional particle swarm optimization, a technique developed by the authors and presented in a well-defined algorithmic approach. All featured applications are supported with fully documented source code as well as sample datasets.