Background

During the last decade, tremendous advancements in Machine Learning (ML) have triggered extraordinary success stories in numerous applications. Modern research shows that the integration of ML methods in network control can be very beneficial in solving complicated problems fast. A specific category of ML, Reinforcement Learning (RL), became exceptionally popular for achieving superhuman skills in complex games like Chess and Go. RL can solve general-purpose problems where a machine only needs to interact with any environment.

In this work, you will use popular RL techniques to solve challenging problems in modern IP-optical networks. These networks consist of multilayer nodes containing an IP router (electrical layer) and an optical switch (optical layer). End-to-end connections require a solution that involves both layers. Such a problem is known as Routing, Modulation, and Spectrum Assignment (RMSA) and is NP-complete, ergo, very hard to solve optimally.

Problem Description

Within this thesis, you are called to model the RMSA problem into a Markov Decision Process (MDP). You must use a simulated environment to find near-optimal policies with popular RL techniques from public Python libraries. You will need to investigate the current literature and benchmark your deployed algorithms.

Acquired Knowledge and Skills

This work will deepen your knowledge of modern networking and RL techniques. You will execute simulations and dive into programming with Python. Finally, you will learn how to evaluate and benchmark your algorithms.