Wireless Multimedia Computational Communications

This book discusses the evolving designs and applications of multimedia content delivery and focuses on computing-based methods. It offers readers an in-depth understanding of how computational resources at both the source and the destination of the networking continuum can be exploited. This enhances the overall performance of multimedia data networking. This book also presents novel designs and applications focusing on information delivery based on computing. It starts with an overview of the multimedia computational communications as well as spanning topics. The topics range in experience evaluation using electroencephalography, semantic knowledge bases with the next generation of multiple access, end-to-end semantic communication framework and cloud-edge-end intelligent coordination computing. The authors believe this book offers readers a clear picture of the current state and the next steps in multimedia computational communication networks. Graduate students majoring in the areas of communication networks, computer science and engineering and electrical engineering will find this book useful as a secondary text or reference book. Professionals and researchers working in computational transmission solutions for multimedia communication networks will find this book to be a valuable resource as well. Foreword Preface Acknowledgments Contents Acronyms 1 Wireless Multimedia Computational Communications 1.1 Introduction 1.2 Architecture of Multimedia Computational Communications 1.2.1 Evaluation Criterion: QoS → QoE 1.2.2 Prior Knowledge Base and Structural Coding: Pixel → Spatial-Temporal Structure 1.2.3 Semantic Information Transmission: Rate-Distortion → Rate-Semantic-Distortion 1.2.4 Cloud-Edge-End Intelligent Collaboration and Computing References 2 QoE Evaluation Model Based on EEG 2.1 Data-Driven QoE Evaluation Model: From Network Parameters to QoE 2.1.1 Introduction 2.1.2 Related Work 2.1.3 Database Establishment and Analysis 2.1.3.1 Data Collection 2.1.3.2 Feature Selection 2.1.3.3 Data Cleaning 2.1.4 Model Formulation 2.1.5 Result and Analysis 2.1.5.1 Results of Feature Selection 2.1.5.2 Results of Data Cleaning 2.1.5.3 Results of QoE Prediction 2.1.6 Conclusion 2.2 EEG-Based Image Perceptual Quality Assessment 2.2.1 Introduction 2.2.2 Database Preparation 2.2.2.1 Stimuli and Participants 2.2.2.2 Experiment Design 2.2.2.3 EEG Data Pretreatment 2.2.3 Proposed Method 2.2.3.1 Database Analysis 2.2.3.2 Overview of the Network 2.2.3.3 EEG Feature Extraction Network 2.2.3.4 EEG Score Prediction Network 2.2.4 Experiment Results and Discussions 2.2.4.1 Data Analysis 2.2.4.2 Evaluation of the EEG Feature Extraction Network 2.2.4.3 Evaluation of the EEG Score Prediction Network 2.2.5 Conclusion References 3 Prior Knowledge Base 3.1 Framework of Hierarchical Semantic Knowledge Base 3.2 OAR-Based Semantic Knowledge Base 3.2.1 Introduction 3.2.2 Related Works 3.2.2.1 Object-Attribute-Relation Model 3.2.2.2 Soccer Match Analysis 3.2.2.3 Graph Neural Networks 3.2.2.4 Attention Mechanisms 3.2.3 The Proposed Method 3.2.3.1 Multimodal Representation 3.2.3.2 Graph Reasoning 3.2.3.3 Optimization Objective 3.2.4 Experiments and Analyses 3.2.4.1 Datasets 3.2.4.2 Experimental Setup 3.2.4.3 Effectiveness of Involving Object-Attribute-Relation Graphs in Soccer Event Recognition and Prediction 3.2.4.4 Contributions of Visual Information 3.2.4.5 Effectiveness of Attention Mechanisms 3.2.4.6 Impact of Message Passing Functions 3.2.4.7 Impact of the Length of the Data Fragment 3.2.4.8 Impact of Initial Edge Selection Methods 3.2.4.9 Qualitative Results 3.2.5 Conclusion References 4 Structural Coding 4.1 Sketch Graph Representation for Multimedia Computational Communications: A Learning-Based Method 4.1.1 Introduction 4.1.2 Related Works 4.1.2.1 End-to-End Multimedia Computational Communications 4.1.2.2 Image Semantic Representations 4.1.2.3 Edge Detection and Sketch Graph 4.1.3 Learning-Based Sketch Graph Model 4.1.3.1 Overall Framework 4.1.3.2 Data Preparation 4.1.3.3 Network Design and Inference 4.1.3.4 Sketch Graph Modification and Compression Encoding 4.1.4 Experiments, Results, and Analysis 4.1.4.1 Loss Functions and Training Settings 4.1.4.2 Comparison Algorithms and Metrics 4.1.4.3 Evaluation Dataset and Metrics 4.1.4.4 Performance and Analysis 4.1.5 Conclusion, Discussion, and Future Work 4.2 Panoramic Image Generation: From 2D Sketch toSpherical Image 4.2.1 Introduction 4.2.2 Related Work 4.2.2.1 Transform Coding–Based Reconstruction/Generation 4.2.2.2 Compressed Sensing–Based Reconstruction/Generation 4.2.2.3 Deep Learning–Based Reconstruction/Generation 4.2.3 Proposed Model 4.2.3.1 2D Sketch 4.2.3.2 Spherical Convolution 4.2.3.3 Panoramic Image Generation Model 4.2.3.4 Network Architecture 4.2.4 Result and Analysis 4.2.4.1 Results on AOI Dataset 4.2.4.2 Results on SYNTHIA Dataset 4.2.4.3 Quantitative Evaluations 4.2.4.4 Ablation Experiments 4.2.5 Conclusion References 5 End-to-End Semantic Information Transmission 5.1 Joint Semantic-Channel Coding for Multimodal Data 5.1.1 System Model 5.1.1.1 Semantic Transmitter 5.1.1.2 Semantic Receiver 5.1.2 Multimodal Multiuser Semantic Communications 5.1.2.1 Model Description 5.1.2.2 Training Algorithm 5.1.3 Simulation Results 5.1.3.1 Implementation Details 5.1.3.2 Benchmarks and Performance Metrics 5.1.3.3 Multimodal Multiuser Semantic Communications 5.1.3.4 Latency and Computational Complexity Analysis 5.1.4 Conclusions 5.2 Semantic-Aware Network Resource Allocation 5.2.1 Resource Allocation for a Single-Task Network 5.2.1.1 System Model 5.2.1.2 Resource Allocation Scheme 5.2.1.3 Simulation Results and Comparison 5.2.2 Resource Allocation for a Multitask Network 5.2.2.1 System Model 5.2.2.2 Resource Allocation Problem 5.2.2.3 A Matching Theory-Based Solution 5.2.2.4 Simulation Results References 6 QoE Optimization for Wireless Multimedia Communications 6.1 Rate-Semantic-Distortion-Based Image Compression 6.1.1 The RL-Based Adaptive Semantic Coding 6.1.1.1 Representation Unit: From Pixel to Semantic Concept 6.1.1.2 Reconstruction Metric: From Pixel Loss to Semantic-Perceptual Loss 6.1.1.3 The RL-Based Semantic Bit Allocation Model 6.1.2 Semantic Decoder and Training Details 6.1.2.1 Soft Quantization and Entropy Coding 6.1.2.2 Generative Semantic Decoder 6.1.3 Experimental Results 6.1.3.1 Simulation Setup 6.1.3.2 Evaluation Metrics 6.1.3.3 Semantic Performance 6.1.3.4 Perceptual Performance 6.2 Rebuffering Optimization for DASH via Pricing and EEG-Based QoE Modeling 6.2.1 System Model and Rebuffering Length Calculation 6.2.2 Calibration of the Effect of Video Rebuffering on QoE Using EEG 6.2.2.1 Methods 6.2.2.2 Results 6.2.2.3 Calibration of the Effect of Video Rebuffering on QoE 6.2.3 Game Formulation 6.2.3.1 Game Formulation 6.2.4 Stackelberg Equilibrium 6.2.5 Algorithm to Solve the Stackelberg Equilibrium 6.2.6 Experimental Results 6.2.6.1 Validating the Results of the Distribution of User Time Limit via Parallel Experimentation 6.2.6.2 Performance of the Proposed Bandwidth Allocation Algorithm References 7 Cloud-Edge-End Intelligent Coordination and Computing 7.1 Overview of Cloud Edge Collaboration 7.1.1 Cloud Computing and Edge Computing 7.1.2 Research Status of Cloud-Edge Collaboration 7.1.2.1 Cloud-Edge Collaboration at Infrastructure Level 7.1.2.2 Cloud-Edge Collaboration at Platform Level 7.1.2.3 Cloud-Edge Collaboration at Software Level 7.1.3 Conclusions and Research Challenges 7.2 End-Side Active Sensing and Computing 7.2.1 Significance of Loading the Calculation on the End Side 7.2.1.1 Prosperity and Challenge of Cloud Intelligence 7.2.1.2 Growing Importance of Terminal Intelligence 7.2.2 Active Visual Camera System 7.2.2.1 Framework of Active Visual Camera System 7.2.2.2 Intelligent Sensing: A Solution for RGB-IR Sensors 7.2.2.3 Advantages and Cases of Spectral Extension 7.2.2.4 The ISP Technology Inside the Camera 7.2.2.5 End-Side Computing: More Efficient Identification Method References 8 Future Prospects 8.1 Brain-Inspired Multimedia Computational Communications 8.2 Semantic Communications