Rethinking Transformers Through Duality Principles
From 01-10-2024 to 30-09-2028
Description
The AI renaissance has been propelled by groundbreaking deep learning models, with the transformer architecture and its attention mechanism at the forefront. Notably, transformers are the foundation for famous models like ChatGPT and BERT and have witnessed lesser-known successes in computer vision, robotics, etc. Our goal is to develop a framework using generalized duality so as to understand and improve performance and efficiency of transformer architectures and the associated training procedures. This framework will improve current transformer architectures, apply to other deep learning models and even motivate entirely new architectures. We aim to unlock the full potential of transformers, making them more accessible to a wider range of users and applications, thereby democratizing the benefits of this technology, reducing the dependency on immense resources and data and fostering broader innovation and application across various domains of AI research.
Team
- Panagiotis Patrinos, Co-promoter
- Johan Suykens, Promoter
Financing
Funding: KU Leuven - Internal Funding KU Leuven
Program/Grant Type: C1 - KU Leuven Category 1 Funding
Events
2/09/2024:
PhD defense - Martijn Oldenhof
Machine Learning for Advanced Chemical Analysis and Structure Recognition in Drug Discovery
3/09/2024:
Meet the Jury Igor Tetko on Advanced Machine Learning in Drug Discovery
12/09/2024:
Multimodal analysis of cell-free DNA for sensitive cancer detection in low-coverage and low-sample settings
Seminar by Antoine Passemiers
News
STADIUS Alumni Herman Verrelst – new CEO of Biocartis
08 June 2017
Herman Verrelst, the founder of KU Leuven spin-off Cartagenia, who has been working in Silicon Valley, US for the last few years will be returning to Belgium to follow the steps of Rudi Pauwels as CEO of the Belgian diagnostic company, Biocartis.
Supporting healthcare policymaking via machine learning – batteries included!
29 May 2017
STADIUS takes the lead in the data analytics efforts in an ambitious European Project MIDAS.
Marc Claesen gives an interview about his PhD for the magazine of the Faculty of Engineering Sciences "Geniaal"
10 February 2017
Did you know that in Belgium approximately one third of type 2 diabetes patients are unaware of their condition?
Joos Vandewalle is nieuwe voorzitter KVAB
09 October 2016
Op 5 oktober 2016 heeft de Algemene Vergadering van de Academie KVAB Joos Vandewalle verkozen tot voorzitter van de KVAB.