skip to content
Tree Graph Logo

Research & Projects

Research


1. Hierarchical RL & Tree Search for Power Grid Control

Scalable Tree Search over Graphs with Learned Action Pruning for Power Grid Control

Florence Cloutier, Cyrus Neary, Adriana Hugessen, Viktor Todosijević, Zina Kamel, Glen Berseth

RLC 2025 Workshop on Practical Insights into Reinforcement Learning for Real Systems.

HERO-GT: hierarchical RL and tree search for power grid control

We propose a hierarchical learning and planning framework for large-scale power-grid control. A heterogeneous GNN first identifies the most relevant subgraph to act on, reducing the action and observation space. A tree-search procedure is then applied locally over this region to evaluate topological actions, showing that combining structure-aware graph representations with search improves efficiency and generalization across different grid topologies.

2. Multi-Agent Molecular Generation under Multi-Property Constraints

M4oLGen: Multi-Agent, Multi-Stage Molecular Generation under Precise Multi-Property Constraints

Yizhan Li, Florence Cloutier, Sifan Wu, Ali Parviz, Boris Knyazev, Yan Zhang, Glen Berseth, Bang Liu

Under review at the International Conference on Learning Representations (ICLR), 2026.

We introduce a multi-agent, multi-stage pipeline for controllable molecule generation, where specialized agents collaborate to satisfy multiple structural and chemical constraints. The system progressively refines molecules across stages, achieving both diversity and precise satisfaction of property targets, offering a scalable framework for realistic multi-objective molecular design.

Projects


These are some relevant projects I worked on during my master's and undergraduate degrees.

Bayesian Graph Neural Networks for the Optimal Power Flow Problem

Mila · 2024

Explored uncertainty-aware graph learning methods for power system optimization using Bayesian GNNs on the Optimal Power Flow (OPF) problem.

  • Developed a Bayesian GNN model combining approximate inference techniques such as variational inference, Monte Carlo sampling, Monte Carlo dropout, and ensemble methods to capture predictive uncertainty.
  • Implemented a baseline following the CANOS paper (Piloto et al., 2024) using the OPFDataset to benchmark performance on OPF.
  • Applied constraint-aware learning with Lagrangian relaxation and dual methods in the loss function to enforce physical and engineering feasibility.
  • Demonstrated that the Bayesian approach improves generalization, provides calibrated uncertainty estimates, and enhances robustness for high-stakes power grid decisions.

Autonomous Driving – Duckietown

Université de Montréal · Duckietown Class · 2024

Applied simplified inverse reinforcement learning (IRL) and DrQv2 for autonomous driving control in the Duckietown environment.

  • Implemented a simplified IRL approach combining a predefined reward function with learned residuals to preserve interpretability while improving performance.
  • Used expert trajectories to fine-tune the reward function so the agent learns realistic driving constraints.
  • Showed that the IRL agent learned subtle behaviours such as staying in the right lane, while a pure RL agent tended to drive on the left.

Exploration Drones – Web-Controlled Multi-Drone System

Polytechnique Montréal · Jan 2023 – May 2023 · Python · TypeScript · C++

Built an interactive system to control exploration drones and visualize their mapping of unknown environments.

  • Developed a Vue-based web application to control drones and display their live exploration map.
  • Implemented the exploration algorithm on the embedded system to explore and map unknown terrain while avoiding obstacles in multi-drone scenarios.

Package Delivery Drone System for Montréal

Data Structures Project · Polytechnique Montréal · May 2020 – Aug 2020 · Java

Designed algorithms to reduce energy consumption in a fleet of delivery drones.

  • Assigned packages to drones based on weight, size, availability, and route cost using Dijkstra’s shortest path algorithm.
  • Improved overall fleet efficiency by optimizing routing and load balancing.

Embedded Robot for Covid Distancing

Embedded Systems Project · Polytechnique Montréal · Jan 2020 – May 2020 · C++

Developed an embedded robot that maintains preventive distances from people in a crowd.

  • Programmed a finite state machine defining the robot’s reaction to six different proximity scenarios.
  • Improved development efficiency by scripting automated deployment of the embedded code.