Statistical Learning Tools for Electricity Load Forecasting

This monograph explores a set of statistical and machine learning tools that can be effectively utilized for applied data analysis in the context of electricity load forecasting. Drawing on their substantial research and experience with forecasting electricity demand in industrial settings, the authors guide readers through several modern forecasting methods and tools from both industrial and applied perspectives – generalized additive models (GAMs), probabilistic GAMs, functional time series and wavelets, random forests, aggregation of experts, and mixed effects models. A collection of case studies based on sizable high-resolution datasets, together with relevant R packages, then illustrate the implementation of these techniques. Five real datasets at three different levels of aggregation (nation-wide, region-wide, or individual) from four different countries (UK, France, Ireland, and the USA) are utilized to study five problems: short-term point-wise forecasting, selection of relevant variables for prediction, construction of prediction bands, peak demand prediction, and use of individual consumer data. This text is intended for practitioners, researchers, and post-graduate students working on electricity load forecasting; it may also be of interest to applied academics or scientists wanting to learn about cutting-edge forecasting tools for application in other areas. Readers are assumed to be familiar with standard statistical concepts such as random variables, probability density functions, and expected values, and to possess some minimal modeling experience. Contents 1 Introduction 1.1 Industrial Motivation 1.2 Data Sets 1.2.1 General Considerations 1.2.2 Salient Features of Electricity Demand 1.2.3 Irish Individual Electrical Demand Data 1.2.3.1 Data Presentation 1.2.3.2 Data Processing 1.2.3.3 Getting the Data 1.2.4 French National Demand Data 1.2.4.1 Data Presentation 1.2.4.2 Data Processing 1.2.4.3 Getting the Data 1.2.5 US Regional Demand Data from the GEFCOM 2014 Competition 1.2.5.1 Data Presentation 1.2.5.2 Data Processing 1.2.5.3 Getting the Data 1.3 Problems 1.3.1 Short-Term Point Forecasting 1.3.2 Probabilistic Forecasting 1.3.3 Selection of Relevant Variables for Prediction 1.3.4 Peak Demand Forecasting 1.3.5 Adaptive Forecasting 1.3.6 Bottom-up and Hierarchical Forecasting 1.4 Assessment and Validation 1.4.1 Assessment Scores 1.4.1.1 Pointwise Forecasting 1.4.1.2 Probabilistic Forecasting 1.4.2 Validation Procedures 1.4.2.1 Cross-Validation 1.4.2.2 Bootstrapping Part I A Toolbox of Models 2 Additive Modelling of Electricity Demand with mgcv 2.1 Introducing GAMs 2.1.1 GAM Model Structure 2.1.2 GAM Model Fitting in a Bayesian Framework 2.1.3 Basic Smooth Effects and Penalties 2.1.3.1 Thin Plate Splines and Derivative-Based Penalties 2.1.3.2 Smooth Effects in mgcv 2.1.4 Model Selection 2.1.4.1 Model Selection via Smoothing Parameter Estimation 2.1.4.2 Performing AIC-Based Model Selection Under Penalization 2.1.4.3 Choosing the Type and the Basis Dimension of a Smooth Effect 2.1.5 Example: Modelling Aggregated Irish Smart Meter Data 2.2 More Smooth Effects and Big Data Methods 2.2.1 Tensor-Product and By-variable Smooths 2.2.2 GAM Methods for Large Data Sets 2.2.3 Example: Modelling Aggregate Irish Smart Meter Data (Continued) 2.2.4 Alternatives to mgcv for GAM Modelling 2.2.5 Summary 3 Probabilistic GAMs: Beyond Mean Modelling 3.1 Introduction to GAMLSS Modelling in mgcv 3.1.1 Probabilistic GAM Modelling of GEFCom 2014 Data 3.2 Introducing QGAM Models 3.2.1 QGAM Model Structure 3.2.2 Fitting QGAM Models with qgam 3.2.3 Distribution-Free QGAM Modelling of GEFCom 2014 Data 3.2.4 Alternatives to mgcv and qgam for GAMLSS and QGAM Modelling 3.3 Summary 4 Functional Time Series 4.1 Functional Data 4.2 Wavelets 4.3 KWF: A Nonparametric Regression for Stationary FTS 4.4 Prediction Interval 4.4.1 Bootstrap Generation 4.4.2 Two Variants from the KWF Method 4.5 Clustering Functional Data 4.5.1 Clustering by Feature Extraction 4.5.2 Clustering Using a Dissimilarity Measure 5 Random Forests 5.1 Random Forests: An Ensemble Based Method 5.1.1 CART Trees 5.1.2 Principle of Random Forests 5.1.3 OOB Error 5.2 Variable Importance Measures and Marginal Effects 5.2.1 Permutation Variable Importance 5.2.2 Group Variable Importance 5.2.3 Marginal Effects 5.3 Tuning Meta-Parameters 5.3.1 Tuning for Prediction 5.3.2 Tuning for Computing VI 5.4 Theoretical Results 5.5 A Variant Adapted to Time Series 5.6 Electricity Data Modeling Using Random Forests 5.6.1 CART Trees 5.6.1.1 Nested Sequence of Pruned Subtrees 5.6.1.2 Optimal and Suboptimal CART Trees 5.6.2 Random Forests 6 Aggregation of Experts 6.1 Introduction 6.2 Online Forecasting of Arbitrary Sequence with a Set of Experts 6.3 The Notion of Regret 6.4 Aggregation with Exponential Weights 6.5 Gradient Trick 6.6 Aggregation with Adaptive Learning Rates 6.7 Specialized Experts 6.8 Nonconvex Aggregation 6.8.1 Ridge 6.8.2 Tricks 6.8.2.1 Constant Bias 6.8.2.2 Random Walk 6.9 Dealing with Breaks 6.9.1 Shifting Oracle 6.9.2 Fixed Share 7 Mixed Effects Models for Electricity Load Forecasting 7.1 Introduction 7.2 The Standard Linear Mixed Effects Model 7.2.1 Classical Linear Model 7.2.2 Random Effects 7.2.3 A Simple Example of a LME Model 7.3 Stochastic Linear Mixed Models for Longitudinal Data 7.4 Regression Trees for Mixed Effects Longitudinal Data 7.4.1 The RE-EM Tree Algorithm 7.5 Functional Mixed Effects Models 7.5.1 A Penalized Spline Approach to Functional Mixed Effects Model Analysis 7.6 Predicting Time Series of Electricity Consumption Part II Case Studies: Models in Action on Specific Applications Case Studies Organization 8 Disaggregated Forecasting of the Total Consumption of a Given Subset of Customers 8.1 Data 8.1.1 Original Data Set 8.1.2 Other Data Sets 8.2 Problems 8.3 Modeling and Results 8.3.1 From Individual Curves to a Hierarchy of Partitions for Forecasting 8.3.2 Numerical Experiments 8.4 Validation 8.5 Interpretation 8.6 Complements and Discussion 8.6.1 Upscaling 8.6.2 Discussion 9 Aggregation of Multiscale Experts for Bottom-Up Load Forecasting 9.1 Data 9.2 Problem 9.3 Methods 9.4 Numerical Results 9.5 Discussions 10 Short-Term Electricity Load Forecasting for Fine-Grained Data with PLAM 10.1 Data 10.1.1 Data Transformation 10.1.2 Generation of Aggregates 10.2 Problem 10.3 Modelling 10.3.1 Estimation and Model Selection in PLAMs 10.4 Analysis and Results 10.5 Discussion and Conclusion 11 Functional State Space Models 11.1 Data 11.2 Problems 11.3 Modelling 11.4 Model Construction 11.5 Prediction Performances 11.6 Supplements and Discussion 12 Forecasting Daily Peak Demand Using GAMs 12.1 Forecasting Problem 12.2 Modelling 12.2.1 A High-Resolution Approach 12.2.2 A Multiresolution Approach 12.3 Results on GEFCom 2014 Data 12.4 Conclusion 13 Forecasting During the Lockdown Period 13.1 Data 13.2 Problem 13.3 Methods 13.3.1 GAM and Adaptive GAM 13.3.2 RF and Adaptive RF 13.3.3 Stacking GAM and RF 13.3.4 Aggregation Algorithms 13.4 Numerical Results 13.5 Discussions References