Hypercube Podcast Transcript

Title:  Jethro Browell, University of Glasgow, on How the Academic World Feeds the Energy Sector

Host: Adam Sroka
Guest: Jethro Browell

Intro: Welcome to the Hypercube podcast, where we explore how companies in the energy and utility sector leverage data analytics to make smarter decisions and accelerate business growth. I’m Adam Sroka, founder of Hypercube, a strategic consultancy that supports asset owner-operators, traders, route-to-market providers, and energy services companies to unlock the power of data.

If you’re interested in hearing real-world examples of how data and AI are advancing the energy sector, this is the show for you. 

Adam Sroka: Hello and welcome back to the Hypercube podcast. Today I am joined by a very special guest, Jethro Browell. Jethro, would you like to introduce yourself for the unaware and uninitiated amongst the audience? 

Jethro Browell: Yeah, thanks Adam. I’m Jethro Browell, I’m a professor at the University of Glasgow in the School of Maths and Statistics and yeah, I work on Primarily energy forecasting, but data analytics in general, mostly applied in the energy sector, sometimes other places, but yeah, it’s great to be here.

Adam Sroka: Big brain stuff then, professor of maths and stats and energy forecasting. It’s quite intimidating for the likes of me, um, out in the wilds here, but I guess let’s go right back. Could you talk us through how you got into that? Like that’s, that’s like pinnacle academic career, right? Could you walk us through the journey a bit?

Jethro Browell: Sure. So I was always into maths. As a kid growing up and after an undergrad in maths and physics. Which was quite abstract, um, by the end, I felt I wanted to do something a bit more practical, a bit more applied. Uh, at the time, the University of Strathclyde had a centre for PhDs in wind energy, which really appealed to me, because I was able to do a, like, basically a one year taught course in all things wind energy, before choosing what I would then, uh, spend my three years of PhD research on.

Uh, so that was just what I, uh, was, right, just what I was looking for. And I did, uh, after that one year of training, I ended up doing a PhD in wind power forecasting. At the University of Strathclyde after that stayed on as a postdoc, uh, doing more wind power forecasting, but diversifying into other types of energy forecasting, not just predicting things like production and demand and prices, but also things like, uh, accessibility to offshore wind farms and maintenance, that kind of thing.

Basically, anything that you. Might want to have a perspective on, uh, in the future in the energy sector, we were happy to have a go at forecasting. And that led me into an academic job at Strathclyde in the School of Engineering, well, Department of Electronic and Electrical Engineering. But there came a time when I felt it’d be worth looking for a new home and maybe in a math department or a computer science department, which felt more like where my research was taking me.

And so I moved to the University of Glasgow in 2021. And that’s where I’ve been since. 

Adam Sroka: Interesting story. And yeah, big respects. Stratified PhD myself. So the thing I wanted to get into then a little bit was, um, because that’s an interesting journey, right? A lot of us go to university, do subjects we’re either good at or interested in and without really having a plan or knowing where we’re going.

And I think I had a similar sort of vein. I noticed that I wanted to do something a little bit more concrete and like in industry. Do you think then that the curriculums are the right way round for like having real world impacts? Or do you think the universities or, or courses could, uh, are doing enough or could be doing things better or completely differently to prepare people for like these kinds of roles in, in, especially in the energy sector, I guess?

Jethro Browell: Yeah, I think that’s a really difficult one for me, both reflecting my own experience. As a student, but also now as an academic teaching, you know, undergrads who have come to do a degree in statistics or joint maths and statistics or statistics and something else. I feel like, you know, there’s so much we could teach.

It’s important not to deviate too far from the core of those subjects. But what we try to do, and maybe we could do better, is uh, shine a light on where those skills and knowledge can take people. And we do that to a, to a large degree, you know, through the examples we use in teaching, uh, which is maybe easier in stats than maths.

You know, for example, we do a lot of, uh, environmental statistics examples. I, I’ve started introducing energy sector examples in my own teaching, got colleagues who do a lot of kind of health and, uh, government statistics, which enable us to link, you know, the theoretical concepts we teach to quite practical things.

Um, that people might experience in their own lives or read about in the news. But that’s harder in other subjects, you know, especially, you know, maths and maybe more theoretical aspects of physics. And I think just making sure that students who are asking these questions can access information about career paths is really important.

And especially, um, career paths that can have a real impact on, on the world, right? I mean, I, my motivation for getting into wind energy wasn’t just, you know, You know, something that practical and useful. It was addressing a, you know, a societal challenge and I was really passionate about and lots of our students have that passion and I think we have some responsibility to help.

Help them direct it somewhere, somewhere useful. 

Adam Sroka: Uh, I totally see that every organisation we’ve worked with, with like clever techies and maths and statistics people that have bright prospects, like those that, that really want to be in the energy roles, really One to be the energy roles and you see actually a lot of organisations punching above their weight from a hiring point of view, because they’re, they’re cashing in on some of that, right.

They’re getting people that just actually love it. And they want to be there, even though they could go off and get like much higher paid jobs, optimising shopping baskets for Amazon or something. But, um, no, it’s really interesting to see, I guess. What about the other way around then? Like, do you think the energy sector is to, does enough to education and educational institutions and, and bring people in?

Jethro Browell: I mean, my experience has been, uh, that they do quite a bit. I mean, In Glasgow, for example, you know, um, some of the big, the big utilities that, uh, have bases here, you know, have scholarships off the master’s programs and undergrad programs and summer internships. And I think that’s great. And that’s the way to go.

I think the, the challenges may be keeping, uh, those programs up to date with the skills, the energy sector needs, right? Traditionally, those kinds of things have been directed to, uh, you know, engineering degrees, uh, you know, these companies always need those skills to, to build and maintain physical assets, but it’s that need for digital and more data centric solutions is growing.

Um, maybe there is. scope to, you know, direct more of that, that support and basically recruitment, uh, towards, you know, the likes of math degrees, computer science degrees, um, operations research, uh, if they’re the skills they want and the skills I think they’re going to need over the next 10, 10, 20 years and beyond.

Adam Sroka: Do you think that there’s enough openness with like data sets and potential projects and collaboration? Do you think that happens? Because I find that it’s quite a mixed bag of like super defensive about our data and we don’t want anyone to know, but then there’ll be random pockets of really innovative, like sort of cross organisational collaboration that makes a big impact.

What’s your kind of experience of that landscape? 

Jethro Browell: Yeah, I’ve seen that change a huge amount over the 10 years since, you know, I finished my PhD to where I am now. I think some corners of our sector are still quite like that, especially when the data owner still has this firm belief that there’s value in the data itself or value in being the only person with that extensive data set.

Yeah, I’m thinking about wind turbine condition monitoring, for example, where The OEMs know that, uh, they have a really strong offering if they, they keep all that data to themselves. In other areas like, uh, electricity networks, where the regulator has actually had, you know, quite a, an important role in opening up data that’s improved markedly, um, which has been fantastic for, for research.

So, you know, the amount of data available from network companies themselves has increased, you know, whether that’s now data from substations, um, you know, half hourly metering broken down by embedded generation. That’s all great stuff. The challenge now is every DNO in Great Britain at least is doing it slightly differently at different paces.

Maybe some of the quality control isn’t quite there and documentation to help you understand. What to expect in terms of quality isn’t quite there But we’re on that journey and it’s going the right direction. And I think when it comes to innovation and research actually Once you have a relationship with, with a company and you’ve maybe shown a little bit of what might be possible.

Um, I found striking, uh, some kind of, uh, arrangement where, you know, data for learning is a fairly equitable exchange and I’ve, I’ve, I’ve been fortunate, you know, not to have any issues around publishing academic papers based on proprietary data and that kind of thing. So yeah, it’s certainly going in the right direction.

Our current bugbear though is, uh. Uh, the price tag that some of the electricity market operators place on their data, right? You know, if you look at the market platforms, they’ll ask very large amounts of money for Uh, especially, you know large amounts of historic data, you know order books Uh as well as you know, executed trades and prices which is understandable, right?

Because there are a lot of new entrants into this market wanting to monetize that data So I I understand where they’re coming from, but it would be nice to have a at least a You A discount for researchers, which, which, uh, isn’t always there. So 

Adam Sroka: We did it episode a while back with David LaCour from NSTA. And like that was, that was all the same kind of thing for sizing data in the North sea. Like someone eventually had to just go and buy it all. And then. Kind of make it available, but off the back of that, there’s now loads of collaboration and research and such going forward. And I guess if there’s people in the audience that are like maybe coming towards the end of their PhD or have any like secret nuggets of wisdom or gotchas to watch out for if they, they look at your career and think that’s what I want to do. I want to be there one day. 

Jethro Browell: You know, in academia, the key there. I would take away is about, is about backing, backing yourself. If you think you’ve got a good idea and no one else has had that idea yet, then, uh, I guess there are two, two outcome, two, two reasons for that. Maybe one is, is, you know, you’re onto a winner there and you should really hammer it home or the other is it’s not such a great idea, you know?

And I, and I’ve had both probably more of the latter, right. But, um, if you can, you know, really back yourself and explore those avenues. And, and take your colleagues, uh, students. and funders with you and then that’s, that’s the way I’ve carved a path for myself in this, this, uh, this career. Yeah, 

Adam Sroka: no, I totally agree. And I think I say it a lot, um, internally at Hypercube, a lot of the value in most things is just in the doing of it at all. Like, and sticking to it. Like, if you write, A single blog post you’re in the top like one millionth of a percent of all bloggers because no one else, no one else bothers, do you know what I mean?

Like for every person that blogs there’s a hundred thousand that haven’t bothered to write the technical blog and like actually just the doing of it and improving over time is almost as important as the quality, like the, the absolute quality. And, um, at least I found with the kind of my journey, so now I really agree with that.

Reminds me of, so last year we, I took it a wee hackathon up in Glasgow, which was good fun. And we, uh, yeah, we had some of that learning for some of the teens, which is good, but that was a feed into the hybrid energy forecasting and trading competition that has recently concluded now. Am I right? That’s 

Jethro Browell: Right. Yeah. It finished, um, in the middle of May and actually all the prize winning teams got together at a conference in France last week. So I guess for anyone listening, uh, who, uh, is, is, um, doesn’t know what we’re talking about. And this, uh, competition is kind of like a Kaggle style data science competition, except it ran in real time.

So participants were required to. forecast the production from a combined wind and solar power portfolio, and they were to produce daily forecasts for each half hour of the day ahead. So by 9. 20am, which is gate closure for one of the day ahead auctions in Great Britain, they had to issue these forecasts, and they also had to submit trades, shadow trades if you like, to the The GB wholesale market as well.

They were scored on their forecast skill and they also received a score for revenue after imbalance. Yeah. And they had to do that every day for the three months. Yeah. At the end we had a leaderboard for both forecasting and trading, and then a combined leaderboard with ranks from both. And it was a really hotly contested in the end.

Six step back. The motivation for these competitions originally was, was to address a bit of a crisis in academic publishing, which. You know, sometimes it makes even makes the mainstream media right, you know, about reproducibility of results and so on and energy forecasting wasn’t exempt from this. In fact, the fact that people were, you know, testing methods and showing results on private data, not sharing code meant we were actually maybe more exposed to this than other sectors.

And, you know, even people with the best intention could accidentally. involve a bit of data leakage or the results could be very particular to their data set and not generalised to the other data sets and so on. So there was a real need to establish which methods really were the best, best in class at the time.

And so in 2012, there was a wind and load forecasting competition, which was very successful and ran again in 2014 with wind, load, solar, and price. And now probabilistic forecasting. Which really raised the profile of probabilistic forecasting at that time, which was still a bit of a niche topic, maybe.

And this is as developed now there are. There are actually quite a few energy forecasting specific competitions. Some run by academics like myself, some run by companies as a recruitment exercise, right, where the prize is an internship, uh, that kind of thing. It’s incredibly helpful to, you know, establish or reestablish what was best in class methods after these problems.

So maybe no surprise that gradient boosted trees of varying implementations regularly top the leaderboards, right, for the forecasting task. There’s a bit more diversity in the trading side of this competition. Uh, we had some participants who went all in on trying to forecast every aspect of the market.

So as well as their wind and solar forecasts, they were predicting the day ahead price, the imbalance price, the system length, all of that informed by, you know, national load and national wind forecast, they really went to town on it and, you know, and placed in, you know, among the top positions. And then we had others who just took a completely data driven approach, like fully prescriptive analytics.

And they just said, well, here was my forecast information in the past. Here’s what the optimal trade would have been in the past. And they just learned a model to directly predict what the optimal trade would be. And that also performed extremely well. In fact, that’s the approach that the winning team took, uh, ultimately.

Uh, which might not fly, you know, in a, on a trading floor, right? If you just have some algorithm telling you what the optimal decision is. But for the, for this competition, it showed the strength of that kind of idea. 

Adam Sroka: I think that always happens with these competitions, doesn’t it? That there’s the, you’re always bound by the constraints of the game.

Like these are the rules, but still really interesting learnings. And there’s nothing to say that like a model like that couldn’t be, Be one of the inputs to a lot of trading decisions and help make some bets. I’ve certainly seen traders have those kinds of tools to hand and have the option to go against them or whatever, because ultimately the decision’s on them.

But that’s really cool. Would you, um, are we going to see it again next year, this competition? 

Jethro Browell: I’m not sure if it’ll be an annual thing. I think, you know, daily submissions for three months is a big ask of participants, even though the majority were automating their solution. They were still wanting to check it regularly and, you know, seven days a week, that’s a big ask.

I don’t have any plans to run it next year. And, but the API based platform that it ran on is continuing as a kind of open benchmark with a rolling, a rolling leaderboard. So people can participate in that setup of forecasting winds. Uh, it was holds you on wind farm and all the solar in East England. And then the other trading setup based on GB So you could participate you could start participating now if you want to see how you place on that rolling leaderboard you know this forecasting competition was an evolution of those that have gone before and that we We provided much more weather data than ever before.

So we provided grid weather data from two models. It was a hybrid portfolio as opposed to individual assets. And then we have this trading component, you know, trying to bridge this gap between forecasting and decision making, which is often where a lot of people stop at the forecasting part, right? But the value of any forecast improvement, uh, comes from improved decision making.

And if that’s not there, then, you know, you have to question whether it was worth investing all that effort in the forecast improvement. So I think anything. Next, uh, in this line of competitions would need to do something new and a bit different. Uh, we’ve already had a few chats about what that might be.

Maybe including a bit, uh, of the intraday trading mechanism, which wasn’t part of the current competition, or maybe taking more of a system operation perspective. So not just forecasting supply and demand, but maybe constraints, for example. Could all be really interesting directions to take it. Alright, bye.

Yeah, if any listeners out there have cool ideas along those lines, you’re more than welcome to send me suggestions on a postcard. 

Adam Sroka: So I’ve been in the weeds of doing this for a previous company that we work with, and we got right into the weeds of like building models for every other participant.

Because there’s not that many, right? So you build models per how did this person behave and how like this company behave and all that. It’s, it’s great fun. Um, Yeah, no, I’ll ask the audience and see, see if anyone’s got any suggestions. We’d love to get involved again next year if we can, maybe more so than we did last year.

I think I know the answer, but for, for people out there that are maybe fresh out of their data science MSC, LSTMs, deep learning, LLM based time series stuff. Did that do any, any good or? 

Jethro Browell: So it hadn’t previously. However, the second place team, uh, use an AI based weather model in previous iterations of these competitions, where it’s been like supervised learning set up, you know, weather for weather forecast, input, power, production, output, neural networks haven’t featured at the leaderboards, but these AI based.

Weather models where, you know, taking the, the gridded weather data and using a deep neural network to try and predict the future state of the weather and then using that to predict power, um, performed really, really well. 

Adam Sroka: Okay. Because that operates more like a, yeah, it’s not just like a 1D time series thing. It’s yeah. Okay. That’s really interesting. 

Jethro Browell: Exactly. And we’re just starting to see the emergence of these full AI weather models in the weather forecasting sector, which I suppose is, it’s no surprise that we’re starting to see them in the application domains, uh, like this competition already. Um, but that’s a really exciting direction.

And it’s something I think we’re going to hear a lot more about in the energy space, you know, as. The prospect of getting more frequent weather forecasts, larger number of ensembles in our ensemble weather forecasts, and just faster access to data, right? Today, you might wait several hours to get the weather forecast, you know, initialised at midnight.

You might not get that forecast until 6am, 7am. But with these AI weather models that run incredibly quickly, you could be getting that forecast hours earlier. Um, potentially more updates as well. So, exciting times ahead. 

Adam Sroka: Yeah, well that leads into, yeah, I was going to ask, like, how do you think, what sort of technologies and trends do you think are going to be impactful over the next few years?

Jethro Browell: Oh, that’s a huge question. I can’t speak for the energy sector as a whole, but for my, corner of it, you know, I think we’ll see a lot more data driven decision support. Uh, as we move towards, uh, systems like markets and power systems that have traditionally been kind of demand led, where we’ve scheduled generation to meet demand, we’re now going to see that flip around to a world where we’re, uh, generation led and since most of that generation is, or a large fraction of that generation is going to be weather dependent, we could think of it as a weather led.

energy system, right? Where we’re trying to schedule demand and what schedule, scheduleable generation we have around, uh, around the weather, and that will introduce a lot of new challenges and we’ll have to start working a bit more proactively with the uncertainty that’s inherent in that, that setup, we’re already seeing the adoption of more probabilistic tools, you know, out there, um, I think.

That’s only going to continue to develop as people get more familiar with them and realise that actually there’s, there’s a lot more to it than just, you know, fan plots and value at risk and all this kind of stuff. There’s a huge amount of rich information about. possible ways, you know, that the weather might evolve and that imposes a lot of structure on the way the energy system will, will respond, right?

You know, the uncertainty might be around the timing of a weather front passing over Great Britain and that imposes a huge amount of spatial structure, right? Um, you know, which assets are gonna be affected when, uh, and so on, which at the moment we, we tend to collapse into, You know one big uncertainty envelope for You know the national level wind and demand output for example but we could do a lot more with that and I think this is something that the research community has been looking at for a while and that Uh, I think will be the next, the next thing to, to find its way into, uh, energy companies and power system operators, the next, the next level of uncertainty information.

Adam Sroka: So we, I asked you about how does all technology affect all of the energy sector, which was a big question. Um, what about like emerging technologies and how that plays a role in sort of energy education? How do you think that might have an effect? 

Jethro Browell: One, you know, emergent technology that’s impacting all education, obviously.

energy is, you know, the large language models. I think this is something we’re grappling with, you know, at the University of Glasgow and I think the whole educational sector worldwide. You know, how do we teach and examine in a meaningful way in this new world, but also how do we prepare students to go out into a world where this is a tool that’s going to be available to them.

You know, I mean, I use chat GPT to help me with my coding. Why should I deprive my students of that? Right. But you also need, we also need to ensure that we’re teaching people the fundamentals to use these tools safely and effectively, which I think, uh, we’re still working out those technologies. As well as chatbots, so also offer similar opportunities for time series forecasting.

So there’s been this emergence of foundation models for time series, which is quite interesting. This idea that you could have, you know, zero shot learning where you’ve got this foundation model for time series and you, you know, You can just give it your, your time series and ask for a prediction.

There’s no need to, to train anything new. Um, it’s, that’s quite exciting and appealing and it’s very early days. I’ve yet yet to be convinced that this is, uh, going to produce the most accurate predictions, but if it produces good enough predictions with minimal effort, uh, that could go a long way to kind of democratizing access to this kind of stuff, right?

If we imagine that that future world you were talking about with, You know prosumers with EVs and heat pumps being exposed to Variable prices for example. Surely it’s only fair then that everybody in that situation has accurate price forecasts And these kinds of, uh, methods maybe open the door to that.

Adam Sroka: It’s an interesting space. I think there’s like, I mean, Energy Nerd as a, as a U, but I find this stuff really exciting and the kind of the next 10 years could like, there’s a lot, a lot to do, but. There’s a lot of doing as well, so it’s quite a nice place to try and solve some big country problems. Great conversation as always.

Definitely, uh, good to catch up. One thing I always ask everyone before we wrap up, um, if people want to find more, find out more about you or like your research or things you’re doing, is there anywhere you’d like to send them, or is there anything you’d like to plug or promote to the audience? 

Jethro Browell: Yeah, and you can check out my personal website, uh, jethrobrowell.com, or my university profile. Uh, there you’ll find a bit more about me, a bunch of papers I’ve written if you’re interested in getting right into the nitty gritty, and that’s also where I kind of post updates on activities and opportunities, um, yeah, happy to connect with people there or, or on LinkedIn, uh, I’m quite active as well.

Adam Sroka: Yeah, great. Well, a pleasure as always. Thank you very much for today. And, uh, yeah, another great conversation. 

Jethro Browell: Thanks, Adam. 

Outro: And that’s it for this episode of the Hypercube podcast. Thanks for tuning in today.

If you have any questions about the topics we covered, you can reach out to us on LinkedIn or check out our website at  wearehypercube.com. You can also join Beyond Energy, our Slack community of data leaders from the sector. There’s a link to sign up in the episode description. We’re just getting this show off the ground, so if you like today’s episode, please leave us a rating, review, or subscribe wherever you get your podcasts.

It all really helps. See you next time.