Speaker
Description
Context
In the context of the emerging risks faced by the electrical grid, a number of initiatives have been launched by major players to devise innovative ways of operating the power grid based on optimization and machine learning [1-3].
Among them, RTE, French TSO, is animating the L2RPN competition (Learning to Run a Power Network) [2] to encourage the development of solutions based on Reinforcement Learning approach. In this competition, agents learns to operate a synthetic grid network in real time during one-week scenarios and to deal with forced outages.
A successful agent is able to optimize cost operation while avoiding blackouts and favoring low carbon emission energy source. The competition is based on the GridAlive software ecosystem to model the grid and the interaction of the agent [4].
Multi-agent framework combining Reinforcement Learning, Optimization and Expert Heuristics
For our participation in the 2023 edition, we developed a solution that ranked first on the private leaderboard of the competition. It is based on a multi-agent framework that allows a cooperation between specialized agents. The solution mainly relies on a topology agent that is learned based on a curriculum learning approach [5, 6]. It is build in an iterative way, from greedy agents whose roles are to identify most relevant actions on the grid, to model-based agent trained by reinforcement learning to take into account the dynamic of scenarios. The topology agent is backed by a resdispatch agent when no satisfactory topological configuration could be find and by expert agents to deal with individual case.
References
[1] ARPA-E. Grid Optimization Competition | Challenge 3. url: https://gocompetition.energy.gov/challenges/challenge-3/.
[2] Antoine Marot et al. “Learning to run a power network challenge for training topology controllers”. In: Electric Power Systems Research 189 (Dec. 2020), p. 106635. issn: 0378-7796. doi: 10.1016/J.EPSR.2020.106635.
[3] Jan Viebahn et al. “Potential and challenges of AI-powered decision support for short-term system operations”. In: (2022).
[4] RTE. Grid Alive. url: https://github.com/rte-france/gridAlive.
[5] Yoshua Bengio et al. “Curriculum learning”. In: ICML 09: Proceedings of the 26th Annual International Conference on Machine Learning. Vol. 382. New York, New York, USA: ACM Press, 2009, pp. 1–8. isbn: 9781605585161. doi: 10.1145/1553374.1553380.
[6] Malte Lehna et al. “Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents”. In: Energy and AI 14 (Oct. 2023), p. 100276. issn: 2666-5468. doi: 10.1016/J.EGYAI.2023.100276.
