Versatile Video Coding (VVC) : Machine Learning and Heuristics

translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Foreword Acknowledgements Contents 1 Introduction References 2 Versatile Video Coding (VVC) 2.1 Basic Video Coding Concepts 2.2 VVC: A Hybrid Video Encoder 2.3 VVC Frames Organization and Block Partitioning 2.4 VVC Encoding Tools 2.4.1 VVC Prediction Tools 2.4.2 VVC Residual Coding and Entropy Coding 2.4.3 VVC In-Loop Filters 2.5 VVC Common Test Conditions References 3 VVC Intra-frame Prediction 3.1 Angular Intra-prediction 3.2 Multiple Reference Line Prediction 3.3 Matrix-Based Intra-prediction 3.4 Intra-sub-partition 3.5 Encoding of Chrominance CBs 3.6 Transform Coding References 4 State-of-the-Art Overview References 5 Performance Analysis of VVC Intra-frame Prediction 5.1 Methodology 5.2 VVC Versus HEVC: Intra-frame Compression Performance and Computational Effort Evaluation 5.3 VVC Intra-frame Computational Effort Distribution of Luminance and Chrominance Channels 5.4 VVC Intra-frame Block Size Analysis 5.5 VVC Intra-frame Encoding Mode Analysis 5.6 VVC Intra-frame Encoding Transform Analysis 5.7 Rate-Distortion and Computational Effort of VVC Intra-frame Coding Tools 5.8 General Discussion References 6 Heuristic-Based Fast Multi-type Tree Decision Scheme for Luminance 6.1 Initial Analysis 6.2 Designed Scheme 6.3 Results and Discussion References 7 Light Gradient Boosting Machine Configurable Fast Block Partitioning for Luminance 7.1 Background on LGBM Classifiers 7.2 Methodology 7.3 Features Analysis and Selection 7.4 Classifiers Training and Performance 7.5 Classifiers Integration 7.6 Results and Discussion References 8 Learning-Based Fast Decision for Intra-frame Prediction Mode Selection for Luminance 8.1 Fast Planar/DC Decision Based on Decision Tree Classifier 8.2 Fast MIP Decision based on Decision Tree Classifier 8.3 Fast ISP Decision Based on the Block Variance 8.4 Learning-Based Fast Decision Design 8.5 Results and Discussion References 9 Fast Intra-frame Prediction Transform for Luminance Using Decision Trees 9.1 Fast MTS Decision Based on Decision Tree Classifier 9.2 Fast LFNST Decision Based on Decision Tree Classifier 9.3 Fast Transform Decision Design 9.4 Results and Discussion References 10 Heuristic-Based Fast Block Partitioning Scheme for Chrominance 10.1 Chrominance CB Splitting Early Termination Based on Luminance QTMT 10.2 Fast Chrominance Split Decision Based on Variance of Sub-blocks 10.3 Fast Block Partitioning Scheme for Chrominance Coding Design 10.4 Results and Discussion References 11 Conclusions and Open Research Possibilities Index This book discusses the Versatile Video Coding (VVC), the ISO and ITU state-of-the-art video coding standard. VVC reaches a compression efficiency significantly higher than its predecessor standard (HEVC) and it has a high versatility for efficient use in a broad range of applications and different types of video content, including Ultra-High Definition (UHD), High-Dynamic Range (HDR), screen content, 360 videos, and resolution adaptivity. The authors introduce the novel VVC tools for block partitioning, intra-frame and inter-frames predictions, transforms, quantization, entropy coding, and in-loop filtering. The authors also present some solutions exploring VVC encoding behavior at different levels to accelerate the intra-frame prediction, applying statistical-based heuristics and machine learning (ML) techniques. This book includes: A high-level description of the VVC novel encoding tools; A detailed description of the VVC intra-frame prediction; A deep statistical assessment of the VVC intra-frame prediction behavior; Five algorithms to reduce the VVC intra-frame prediction encoding effort This book discusses the Versatile Video Coding (VVC), the ISO and ITU state-of-the-art video coding standard. VVC reaches a compression efficiency significantly higher than its predecessor standard (HEVC) and it has a high versatility for efficient use in a broad range of applications and different types of video content, including Ultra-High Definition (UHD), High-Dynamic Range (HDR), screen content, 360o videos, and resolution adaptivity. The authors introduce the novel VVC tools for block partitioning, intra-frame and inter-frames predictions, transforms, quantization, entropy coding, and in-loop filtering. The authors also present some solutions exploring VVC encoding behavior at different levels to accelerate the intra-frame prediction, applying statistical-based heuristics and machine learning (ML) techniques.