Title: An Orientation-Based Spatio-Temporal Feature Extraction Architecture for Effective Information Recall in Reinforcement Learning

Brief Abstract: Deep Reinforcement Learning (DRL) has achieved tremendous success in solving various decision making problems. Mobile autonomous systems can be trained using a DRL algorithmm to perform a variety of high-level tasks. These tasks may require the autonomous system to remember the past, e.g., explored regions of the state space. However, creating an exact map of the surrounding environment online is bound to be computationally expensive. To solve this problem, this thesis introduces a novel architecture that extracts long-term temporal features and associates them with spatial directions. This architecture is trained jointly with an actor and a critic architecture, aiming to maximize the expected discounted cumulative long-term reward.

Supervisor: Dr. Adrian Redder

Title: Adaptive Multi-Agent Path Planning and Navigation with A* and Q-Learning

Abstract: The A* (A-star) algorithm is a widely used path planning algorithm, especially in applications such as navigation systems, robotics, and games. It’s wide applicability arises from its strong properties while being easy to implement. However, the application of A* to a multi-agent system (MAS) can become difficult due to the growth of computational costs. The problem is that individual optimal paths generated A* can conflict. Therefore, multi-agent versions of A* must guarantee that two trajectories do not collide. This problem has given rise to conflict-based search methods that generate non-conflicting trajectories for individual agents of a MAS, trading off computational feasibility for optimality. However, these methods, as well as A* itself, have another crucial drawback, which is their inability to deal with dynamic obstacles, such as other robots that are not part of the MAS, or humans moving in the mission area.

This thesis seeks to solve the problem of conflicting trajectories and the presence of dynamic obstacles by using learning based methods, such as Q-learning, to adjust individual A* paths online when facing conflicts or dynamic obstacles. 

Supervisor: Dr. Adrian Redder

Title: Advancing to Model-Free, Data-Driven COCP: The Synergy of Willem’s Fundamental Lemma and SPSA

Abstract: Model-free, data-driven approaches are used since it is often very challenging, or even impossible,
to create precise mathematical models for complex dynamical systems. This is the case in fields such as robotics, autonomous vehicle control, or unstructured environments, including natural disasters and unpredictable market dynamics.

Data-driven control systems are gaining significant attention, particularly because traditional ap-
proaches depend on accurate models that are often unavailable or only partially defined. This raises
the central question: How can data be effectively utilized to develop optimal control strategies that
do not rely on the knowledge of a complete system model?

The research objective of this thesis is to develop a data-driven approach that delivers a Convex Optimization Control Policy (COCP) within a Convex Optimization framework. We propose to integrate Willems’ Fundamental Lemma with the Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm into COCPs, thereby enabling the development of a robust, model-free control methodology grounded in data-driven principles.

Supervisor: Dr. Adrian Redder

Title: Active SLAM for aerial robots in dense environments

Brief Abstract: Active SLAM (A-SLAM) presents a solution that allows for the autonomous exploration of environments without the need for human supervision. This thesis aims to develop a three-layer framework to optimize and influence every step of the exploration. This thesis will seek to implement a 3D A-SLAM where the main information is the amount of entropy observable from a given point of view.

Supervisor: 

Title: Thermal imaging for robot navigation

Brief Abstract: Robot navigation in challenging environments poses significant difficulties, particularly when relying solely on traditional sensors such as cameras and lidars, which may encounter limitations in low light conditions or in the presence of obscurants like smoke or fog.  By incorporating thermal imaging technology, robots can enhance their perception capabilities, enabling them to navigate through environments where traditional sensors struggle. This research aims to provide a comprehensive review of recent advancements in thermal camera technology and its application in robot navigation.

Supervisor: 

Title: Deep Reinforcement Learning to generate critical scenarios to validate autonomous vehicles

Supervisor: Erdal Kayacan

Title: Distributed, Stochastic, Gradient-Based Optimization for Networked Systems with Energy Constraints in Ambient Intelligence

Brief Abstract: Efficient coverage of a two-dimensional area by mobile robots is crucial in various application fields, such as surveillance, environmental monitoring, and rescue missions. Individual robots are often unable to provide sufficient coverage, necessitating a cooperative approach among multiple robots. 
This collaboration is significantly complicated due to stochastic uncertainties that arise from various sources, including environmental variations and sensor noises. It's therefore essential to consider the real conditions where the sensor performance vary stochastically and to optimize the mobile sensor coverage. This is the topic of the present thesis.

Supervisor: Dr. Adrian Redder

Title: Data-driven model-free reference governor design based on differentiable convex optimization models

Brief Abstract: In den letzten Jahre hat das Interesse an modellfreien Referenz Governorn immmer mehr zugenommen. Das heißt, Referenz Governor bei denen ein exakter Modell des zu regelnden Systems nicht bekannt sein muss. In dieser Bachelorarbeit soll ein neuer Ansatz für einen modellfreien Referenz Governor getestet werden. Bei diesem Ansatz soll das konvexe Optimierungsproblem eines Referenz Governors selbst parametrisiert und optimiert wird. Dieser Ansatz beruht auf der Idee, dass man die Ableitung der Lösung eines konvexen Optimierungsproblems effizient annähern kann.

Supervisor: Dr. Adrian Redder

Title: End-to-end tracking of multiple objects using time models

Brief Abstract: This thesis addresses the impact of using time models on the accuracy of multiple object tracking and how it differs from traditional methods in this area.

Partners: dSPACE GmbH

Supervisor: Prof. Dr. Erdal Kayacan

Title: AI-Driven Mixed-Case Palletization with Robots

Brief Abstract: In the field of logistics, organizing boxes of varying sizes on a standard pallet is a tedious and challenging operation. This thesis aims to achieve and solve the challenge of palletizing boxes of various sizes on a pallet using reinforcement learning techniques. The RL agent determines the box's orientation and location on the pallet, while the robot picks and places the boxes on the pallet. For training and simulation purposes, the Nvidia Isaac simulator will be used.

Partners: Unchained Robotics

Supervisor: Prof. Dr. Erdal Kayacan

Title: Nonlinear control for a tilting quadrotor

Brief Abstract: Tilting quadrotors are innovative unmanned aerial vehicles (UAVs) that offer enhanced agility and versatility compared to traditional quadcopters. By integrating tilting mechanisms into their design, these quadrotors can dynamically adjust the orientation of their rotors, enabling rapid shifts in flight direction and improved maneuverability. 

Supervisor:
 

Title: Learning nonlinear model predictive control for quadrotor in dynamic environments

Brief Abstract: The thesis aims to develop a drone and the underlying control system. This will include the following workflow:

• Design, select parts, and construct the drone.
• Research on Gaussian Processes for disturbance estimation.
• Implementation of a nonlinear MPC for position control.
• Utilize Gaussian Process for disturbance estimation caused by wind or changing payloads.
• Use disturbance estimation from GP to adapt NMPC
• Tests in simulation and real-world scenarios e.g. wind turbine inspection which can include wind or changing payloads.

Supervisor: