The energy sector has long presented data scientists with complex statistical challenges, such as understanding variations in electricity demand and forecasting household power consumption. Among these, one problem stands out for both technical problem solvers and the general population reliant on a stable energy supply: extreme events. These rare phenomena, often climate-related, can significantly impact energy systems and infrastructure. Extreme events include hazards such as droughts, floods, wildfires, and severe weather, each of which underscores the sector’s vulnerability to climate change. As these events increase in frequency and intensity, resilient forecasting and modelling approaches are needed to help mitigate their effects.

This blog post is inspired by a talk given by Dr Daniela Castro-Camilo at one of our Beyond Energy events. Dr Castro-Camilo shared insights into extreme event modelling work conducted by researchers at the University of Glasgow, including Dr Jethro Browell, in partnership with Scottish Power. Together, they developed a method for probabilistic fault prediction at the district level in Scotland, which combines ensemble numerical weather prediction with methods from quantile and distributional regression. Among the methodologies explored, Dr Castro-Camilo presented results using the quantile regression generalised additive model (QR GAM). This post explores the QR GAM methodology, used to predict extreme events (or faults) and thereby support fault prediction and resilience planning in the energy sector.

This post is divided into several digestible chunks:

1. What is quantile regression (QR)?

2. What is a generalised additive model (GAM)?

3. What is a GAM QR?

This blog aims to provide an intuitive overview of QR GAM and how it addresses the needs of extreme event modelling, while leaving out more detailed discussions on parameter or quantile estimation techniques.

Part 1: What is quantile regression?

When modelling extreme events, one operates in the less likely parts of a probability distribution, known as the tails, rather than the centre, where more common outcomes reside (see the figure below for an illustration using a Gaussian distribution). Traditional statistical modelling often focuses on estimating the mean, which corresponds to the 0.5 quantile (or 50th percentile) of a distribution, whereas extreme event modelling often requires a focus on other quantiles, such as the 0.05 or 0.95 quantiles (5th or 95th percentiles), to capture rare but impactful events.

In general, for any \(\tau\)-th quantile, instead of answering the question:

How does the conditional expectation of a random variable \(Y\) depend on some explanatory variables \(X\)?

We ask:

How does the conditional quantile of a random variable \(Y\) vary with the explanatory variables \(X\)?

As shown in the figure below, instead of modelling and predicting the mean, one may be interested in the extremes represented by the 0.05 (grey) or 0.95 (teal) quantiles.

More formally, a random variable is fully characterised by its distribution function, \(F(y)=\mathbb{P}(Y\leq y)\), which provides the probability that \(Y\) is less than or equal to a given value \(y\). The quantile function \(Q_Y(\tau)\) for any \(\tau \in (0, 1)\) is then defined as:

\(Q_Y(\tau) = \inf{y \in \mathbb{R} \mid F(y) \geq \tau}\), (1)

which gives the smallest value of \(y\) at which the cumulative distribution function \(F(y)\) reaches or exceeds the probability level \(\tau\).

To further illustrate, consider a sample \((y_i, x_1, \dots, x_p)\) and let us draw some parallels between quantile regression and the already familiar linear regression (LR) model:

\(y_i = \beta_0 + \beta_1 x_1 + \dots + \beta_p x_p + \epsilon_i\),

where \(\epsilon_i\) are independent errors identically distributed according to some distribution function \(F\).

The corresponding quantile regression model can be written as:

\(Q_Y(\tau\mid x_i) = \beta_0 + \beta_1 x_1 + \dots + \beta_p x_p + F^{-1}(\tau)\),

where \(F^{-1}(\tau)\) denotes the quantile function corresponding to the distribution function \(F\). Unlike traditional linear regression, which explicitly includes an error term \(\epsilon_i\) to capture residuals, quantile regression includes the error term implicitly in the model through the specification of the conditional quantile function. This focus on different conditional quantiles enables quantile regression to capture behaviours in the tails of the distribution, making it valuable for modelling extreme events.

In summary…

Part 2: What is a generalised additive model?

In a Generalised Additive Model (GAM), the linear model is extended to allow for nonlinear relationships between the response and the predictor variables. Energy systems often exhibit non-linear behaviours, such as the impact of weather patterns on energy demand, the non-linear response of renewable energy sources to environmental conditions, and the complex interactions between various components of the energy infrastructure.

Recall the linear model defined for a sample \((y_i, x_1, \dots, x_p)\) in the previous section:

\(y_i = \beta_0 + \beta_1 x_1 + \dots + \beta_p x_p + \epsilon_i\),

where \(\epsilon_i\) are independent errors identically distributed according to some distribution function \(F\). The corresponding extension to the GAM model can be expressed as:

\(y_i = \beta_0 + f_1(x_1) + \dots + f_p(x_p) + \epsilon_i\),

where \(f_1(x_1), \dots, f_p(x_p)\) are smooth functions of the predictor variables, allowing for nonparametric estimation of the relationship between the response and the predictors. Examples of such smooth functions include polynomial basis functions, cubic splines, and smoothing splines.

In general, instead of answering the question:

How does the conditional expectation of a random variable \(Y\) depend on some explanatory variables \(X\) assuming linear relationships?

We ask:

How does the conditional expectation of a random variable \(Y\) depend on some explanatory variables \(X\) allowing for non-linear relationships?

To further illustrate the flexibility of GAMs, it is important to note that each smooth function \(f_j(x_j)\) can be tailored to capture specific patterns in the data, adapting to different nonlinearities in each predictor variable independently. This allows GAMs to handle a wide range of relationships without imposing a strict parametric form, making them highly suitable for complex systems like energy forecasting, where linear assumptions often fall short. A simple illustration of such a case is shown in the figure below, where GAM (teal fitted line) allows one to capture the non-linearity of the data where linear regression (black line) fails to do so.

For example, in the context of energy demand influenced by temperature, GAMs can capture nonlinear seasonal patterns and threshold effects, where small changes in temperature can lead to abrupt shifts in demand. Additionally, GAMs can incorporate smooth interaction terms between predictors, further enhancing their ability to model dependencies between variables, such as the combined effect of wind speed and solar radiation on renewable energy production. This flexibility makes GAMs a powerful choice for accurately modeling the intricacies of energy systems.

In summary…

Part 3: What is a QR GAM?

The ability to model extreme events and non-linear relationships is indeed crucial in the energy sector. Quantile regression addresses the former by focusing on the estimation of conditional quantiles, while the generalised additive model addresses the latter by allowing for non-linear relationships between the response and the predictor variables.

Combining quantile regression and the generalised additive model results in the quantile regression generalised additive model (QR GAM), which allows for the estimation of conditional quantiles using nonparametric smooth functions instead of working with parametric models. Our question of interest then becomes:

How does the conditional quantile of \(Y\) vary with explanatory variables \(X\) allowing for non-linear relationships?

The definition of the QR GAM model can be written as:

\(Q_Y(\tau \mid x_i) = \beta_0 + f_1(x_1) + \dots + f_p(x_p) + F^{-1}(\tau)\),

where \(F^{-1}(\tau)\) denotes the quantile function corresponding to the distribution function \(F\), and \(f_1(x_1), \dots, f_p(x_p)\) are smooth functions of the predictor variables.

The QR GAM model provides a powerful framework for capturing non-linear relationships and estimating conditional quantiles, making it particularly relevant for analysing extreme events and non-linear dynamics in the energy sector. This adaptability to complex, rare occurrences may just be the reason why researchers at the University of Glasgow selected QR GAM to model faults, working in collaboration with Scottish Power.

In summary…

Food for thought

As we develop increasingly sophisticated models to predict and mitigate extreme events in the energy sector, it’s worth considering how these methodologies might translate to other critical aspects of energy management. Can tools like QR GAM be adapted to forecast extreme fluctuations in household energy consumption, for instance, or predict rapid shifts in renewable generation due to sudden weather changes? Beyond technical capabilities, how can the energy industry balance the complexity of advanced modelling with the need for interpretability and actionable insights for operators and policymakers? Predicting the unpredictable may be more within reach than ever, but applying these insights effectively will require a careful blend of innovation, transparency, and sector-specific knowledge.

Want to learn more about advanced modelling techniques for extreme events in energy? Reach out to us today.