Bild mit Unilogo
home uni uni kontakt contact
unilogo Universität Stuttgart
Institute of Communication Networks and Computer Engineering (IKR)

Project description


Master thesis No. 1037    (Offer)   [pdf]

Solving the Routing, Modulation, and Spectrum Assignment Problem using Reinforcement Learning



Reinforcement Learning

Multi-layer networks
Network control




During the last decade, tremendous advancements in Machine Learning (ML) have triggered extraordinary success stories in numerous applications. Modern research shows that the integration of ML methods in network control can be very beneficial in solving complicated problems fast. A specific category of ML, Reinforcement Learning (RL), became exceptionally popular for achieving superhuman skills in complex games like Chess and Go. RL can solve general-purpose problems where a machine only needs to interact with any environment.

In this work, you will use popular RL techniques to solve challenging problems in modern IP-optical networks. These networks consist of multilayer nodes containing an IP router (electrical layer) and an optical switch (optical layer). End-to-end connections require a solution that involves both layers. Such a problem is known as Routing, Modulation, and Spectrum Assignment (RMSA) and is NP-complete, ergo, very hard to solve optimally.

Problem Description

Within this thesis, you are called to model the RMSA problem into a Markov Decision Process (MDP). You must use a simulated environment to find near-optimal policies with popular RL techniques from public Python libraries. You will need to investigate the current literature and benchmark your deployed algorithms.

Acquired Knowledge and Skills

This work will deepen your knowledge of modern networking and RL techniques. You will execute simulations and dive into programming with Python. Finally, you will learn how to evaluate and benchmark your algorithms.


Desirable knowledge

Programming Experience
Machine Learning Experience

Computer Networks
Communication Networks Architecture and Design


Dipl.-Ing. Filippos Christou, room 1.319 (ETI II), phone 685-67968, [E-Mail]