Welcome 👋
I am a computational linguist and senior researcher at the Multilinguality and Language Technology (MLT) lab at the German Research Center for Artificial Intelligence, where I am currently leading mid-sized research projects both administratively and scientifically. I manage the MLT lab with over 20 employees and lead the research group on Efficient and Explainable NLP (E&E) at the MLT lab.
My research interests are mainly on transparent and robust language models and language technology.
- Transparency: Making the parameters and behavior of language models more explainable and understandable to both end users and researchers.
- Robustness: Improving language models in two dimensions:
- Data consumption (e.g., for low-resource settings or languages) by using structured data, new learning techniques, or other modalities.
- Size (e.g., for settings without powerful hardware).
In my work, I mostly use large language models trained with deep learning methods. I also have profound experience in data acquisition and crowdsourcing, as well as educational NLP.
Selected Publications 📰
Teaching 🏫
I am also teaching at Saarland University. Here is a list of my ongoing courses:
-
Seminar, Winter 2024: Recent Advances in Mechanistic Interpretability
A course examining the latest developments in understanding the mechanisms of AI models.
-
Software Project, Winter 2024: Recent Advances in Mechanistic Interpretability
A practical course on applying mechanistic interpretability methods in natural language processing.
- Seminar, Summer 2024: XPLN - Exploring Explainability in NLP
- Seminar, Summer 2024: Efficient and Robust Natural Language Processing
- Seminar, Summer 2023: XPLN - Exploring Explainability in NLP
- Software Project, Summer 2023: BERT and Friends – Pretrained LMs in Natural Language Understanding
- Seminar, Summer 2022: BERT and Friends - Pretrained LMs in Computational Semantics
Supervision 🎓
If you are interested in writing your MSc thesis with me and my team, please send me a mail with a short topic outline, a CV and a transcript of records. Please note that we get many such requests and cannot collaborate with everybody that is interested.
Ongoing
- Lucas Lage, MSc – Topic: Hallucinations in RAG Systems
- Julian Schlenker, MSc – Topic: Language Transfer with Monolingual Models
- Daniil Gurgurov, MSc – Topic: Knowledge-Based Adaptation of Multilingual LMs for Low-Resource Languages
- Katja Konermann, MSc – Topic: Massive Multilabel Classification
- Gregory Charles Shook, MSc – Topic: Interpretability of Language Adapters
- Tatiana Anikina - Topic: Efficient Adaptation of Language Models for Low-Resource Natural Language Understanding
- Cennet Oguz - Topic: Multimodal Processing of Procedural Knowledge
- Tanja Bäumel - Topic: Mechanistic Interpretability of LLMs
Finished
- Bangyao Tang, MSc, 2024, Reviewer – "Analysis of Calibrated Confident Text Classification"
- Akshai Joshi, MSc, 2024, Reviewer – "Self-Supervised Multimodal Representation Learning for Diagram Understanding"
- Konstantin Chernychew, MSc, 2024, Advisor – "Fast and Efficient Structured Pruning of LLMs with Gradient-Based Meta-Mask"
- Gokul Srinivasagan, MSc, 2023, Advisor – "Extreme model compression for large-scale transformer-based language models"
- Daria Fedorova, MSc, 2023, Advisor – "Cross-lingual and Cross-domain Knowledge Transfer for Sequence Labeling Tasks"
- Hannah Seitz, MSc, 2016, Advisor – "Investigating Instantiations of Script Structures in Narrative Texts"