Welcome 👋
I am a computational linguist and senior researcher at the Multilinguality and Language Technology lab at the German Research Center for Artificial Intelligence, where I am currently leading mid-sized research projects both administratively and scientifically. I manage the MLT lab with over 20 employees and lead one of the four research groups at the lab.
My research interests are mainly on transparent and robust language models and language technology.
- Transparency: Making the parameters and behavior of language models more explainable and understandable to both end users and researchers.
- Robustness: Improving language models in two dimensions:
- Data consumption (e.g., for low-resource settings or languages) by using structured data, new learning techniques, or other modalities.
- Size (e.g., for settings without powerful hardware).
In my work, I mostly use large language models trained with deep learning methods. I also have profound experience in data acquisition and crowdsourcing, as well as educational NLP.
Selected Publications 📰
Teaching 🏫
I am also teaching at Saarland University. Here is a list of my ongoing courses:
-
Seminar, Winter 2024: Recent Advances in Mechanistic Interpretability
A course examining the latest developments in understanding the mechanisms of AI models.
-
Software Project, Winter 2024: Recent Advances in Mechanistic Interpretability
A practical course on applying mechanistic interpretability methods in natural language processing.
- Seminar, Summer 2024: XPLN - Exploring Explainability in NLP
- Seminar, Summer 2024: Efficient and Robust Natural Language Processing
- Seminar, Summer 2023: XPLN - Exploring Explainability in NLP
- Software Project, Summer 2023: BERT and Friends – Pretrained LMs in Natural Language Understanding
- Seminar, Summer 2022: BERT and Friends - Pretrained LMs in Computational Semantics
Supervision 🎓
If you are interested in writing your MSc thesis with me and my team, please send me a mail with a short topic outline, a CV and a transcript of records. Please note that we get many such requests and cannot collaborate with everybody that is interested.
Ongoing
- Lucas Lage, MSc – Topic: Hallucinations in RAG Systems
- Julian Schlenker, MSc – Topic: Language Transfer with Monolingual Models
- Daniil Gurgurov, MSc – Topic: Knowledge-Based Adaptation of Multilingual LMs for Low-Resource Languages
- Katja Konermann, MSc – Topic: Massive Multilabel Classification
- Gregory Charles Shook, MSc – Topic: Interpretability of Language Adapters
- Tatiana Anikina - Topic: Efficient Adaptation of Language Models for Low-Resource Natural Language Understanding
- Cennet Oguz - Topic: Multimodal Processing of Procedural Knowledge
- Tanja Bäumel - Topic: Mechanistic Interpretability of LLMs
Finished
- Akshai Joshi, MSc, 2024, Reviewer – "Self-Supervised Multimodal Representation Learning for Diagram Understanding"
- Konstantin Chernychew, MSc, 2024, Advisor – "Fast and Efficient Structured Pruning of LLMs with Gradient-Based Meta-Mask"
- Gokul Srinivasagan, MSc, 2023, Advisor – "Extreme model compression for large-scale transformer-based language models"
- Daria Fedorova, MSc, 2023, Advisor – "Cross-lingual and Cross-domain Knowledge Transfer for Sequence Labeling Tasks"
- Hannah Seitz, MSc, 2016, Advisor – "Investigating Instantiations of Script Structures in Narrative Texts"