PhD on AI for Safety-Critical Multi-modal Learning

Introduction

Are you eager to make a difference in the advancement of AI via reliable state-of-the-art deep learning models? This position will explore efficient multi-modal models for safety-critical applications based on robustness guarantees and explainable AI built on insight into learned representations. 

Job Description

We are seeking a highly motivated PhD student to join our research team in an ambitious project at the intersection of Safe AI, Resource Constrained, and Multi-Modal Learning. The focus of this PhD is on developing novel state-of-the-art AI models for safety-critical applications in which resilience, autonomy, and intelligence are required. 

The overarching goal of this project is to develop efficient, trustworthy AI models that have a robust understanding of their environment based on various data sources. For example, the model should be able to integrate camera, radar, and other sensor modalities. Particular attention will be paid to transformer- and post-transformer architectures, and how to adapt them to learn efficiently on resource-constrained hardware, for instance with transfer learning and quantization techniques.

A key challenge of this project will be to design novel transformer-based architectures and adaptation techniques that are both efficient and reliable. The latter will require you to investigate and ultimately control the latent representations of knowledge in the model. This can be tackled from the viewpoint of various areas of machine learning, such as disentanglement of features, out-of-distribution awareness, robustness to adversarial attacks, and explainability, including the development of formal robustness guarantees that provide insights into the fundamental characteristics of the model that contribute to its safety and reliability in real-world applications. You will also assist in the integration and experimental evaluation of these techniques. Explainability will play an important role here, ensuring that models validate their explanations and provide insight into their latent representations, identifying what the model deems relevant for its predictions.

An auxiliary goal of this PhD is to make the employed models resource-efficient, ensuring they can run effectively on embedded accelerators (e.g., TPUs, NPUs, FPGAs) in constrained environments. The research will explore techniques such as model compression and quantization to optimize AI models for real-time operation on embedded platforms.

Key Research Areas:

• Multi-modal perception (e.g., radar, LiDAR, camera fusion) for robust real-world understanding.
• Transformer-based and post-transformer architectures optimized for embedded accelerators.
• Latent representation of knowledge for novel situation awareness and uncertainty estimation.
• Uncertainty-aware AI: Developing models that can identify when they "don’t know" and communicate uncertainty in decision-making.
• Resource-efficient AI techniques, including compression, quantization, and low-power optimization.

This research has strong implications for autonomous robotics, industrial automation, and safety-critical AI applications, where models must react to unexpected situations and make trustworthy decisions with limited computational resources.

This research will be performed under the supervision of professors Joaquin Vanschoren, Vlado Menkowski, Mykola Pechenizkiy, and Sibylle Hess, in collaboration with several key industrial players in advanced semiconductors.

Job Requirements

• A master’s degree (or an equivalent university degree) in Computer Science or a related field.
• A research-oriented attitude.
• Ability to work in an interdisciplinary team and interested in collaborating with industrial partners.
• Experience in programming and empirical analysis in Deep Learning (e.g. in Python, PyTorch). 
• Excellent problem-solving skills and ability to work independently and collaboratively.
• Motivated to develop your teaching skills and coach students.
• Fluent in spoken and written English (C1 level).

Conditions of Employment

• A meaningful job in a dynamic and ambitious university, in an interdisciplinary setting and within an international network. You will work on a beautiful, green campus within walking distance of the central train station. In addition, we offer you: 
• Full-time employment for four years, with an intermediate assessment after nine months. You will spend a minimum of 10% of your four-year employment on teaching tasks, with a maximum of 15% per year of your employment. 
• Salary and benefits (such as a pension scheme, paid pregnancy and maternity leave, partially paid parental leave) in accordance with the Collective Labour Agreement for Dutch Universities, scale P (min. € 2,901 max. € 3,707).  
• A year-end bonus of 8.3% and annual vacation pay of 8%. 
• High-quality training programs and other support to grow into a self-aware, autonomous scientific researcher. At TU/e we challenge you to take charge of your own learning process. 
• An excellent technical infrastructure, on-campus children's day care and sports facilities.  
• An allowance for commuting, working from home and internet costs. 
• A Staff Immigration Team and a tax compensation scheme (the 30% facility) for international candidates. 

Information

Do you recognize yourself in this profile and would you like to know more? Please contact the hiring manager Joaquin Vanschoren, Associate Professor, j.vanschoren@tue.nl.

Visit our website for more information about the application process or the conditions of employment. You can also contact Sibylle Hess, Assistant Professor, s.c.hess@tue.nl.

Are you inspired and would like to know more about working at TU/e? Please visit our career page.

Application

We invite you to submit a complete application by using the apply button. The application should include a:

• Cover letter in which you describe your motivation and qualifications for the position.
• Curriculum vitae, including a list of your publications and the contact information of three references.

We look forward to receiving your application and will screen it as soon as possible. The vacancy will remain open until the position is filled.