Welcome 👋
I am a senior researcher and deputy director at the Multilinguality and Language Technology (MLT) lab at the German Research Center for Artificial Intelligence, where I am currently leading mid-sized research projects. I lead the research group on Efficient and Explainable NLP (E&E) at the MLT lab.
My research broadly focuses on making language technology more accessible and fair. I focus on transparent and robust language models and language technologies. Specifically, I aim to make the parameters and behavior of language models more interpretable to both end users and researchers, drawing on methods from Explainable Artificial Intelligence and Mechanistic Interpretability. I also work on improving the robustness and efficiency of language models, particularly by enhancing their data efficiency using structured inputs, novel learning and adaptation techniques, and multimodal approaches. I especially focus on Low-Resource Languages. Additionally, I am interested in reducing model size to enable deployment in resource-constrained environments.
In side projects, I'm also exploring the factuality of model-generated outputs and I have worked on multimodal models. My work primarily involves large language models trained using deep learning techniques. I also have extensive experience in data acquisition and crowdsourcing, as well as in educational NLP.
Selected Publications 📰
For a full list of publications, including preprints, please check my Google Scholar Profile
Teaching 🏫
I am also teaching at Saarland University. Here is a list of my ongoing courses:
-
Seminar, Summer 2025: Efficient and Robust Natural Language Processing
A course on efficient NLP, with a focus on robust models for low-resource data settings and parameter efficient models.
-
Seminar, Winter 2024: Recent Advances in Mechanistic Interpretability
A course examining the latest developments in understanding the mechanisms of AI models.
-
Software Project, Winter 2024: Recent Advances in Mechanistic Interpretability
A practical course on applying mechanistic interpretability methods in natural language processing.
- Seminar, Summer 2024: XPLN - Exploring Explainability in NLP
- Seminar, Summer 2024: Efficient and Robust Natural Language Processing
- Seminar, Summer 2023: XPLN - Exploring Explainability in NLP
- Software Project, Summer 2023: BERT and Friends – Pretrained LMs in Natural Language Understanding
- Seminar, Summer 2022: BERT and Friends - Pretrained LMs in Computational Semantics
Supervision 🎓
I'm happy to collaborate with and advise the following PhD students:- Tatiana Anikina - Topic: Efficient Adaptation of Language Models for Low-Resource Natural Language Understanding
- Cennet Oguz - Topic: Multimodal Processing of Procedural Knowledge
- Tanja Bäumel - Topic: Mechanistic Interpretability of LLMs
- Yusser al Ghussin - Topic: Mechanistic Steering of LLMs in Multilingual and Multicultural Aspects
- Ivan Vykopal (at KInIT) - Topic: Multilingual Low-resource NLP
Ongoing MSc Theses
My team and I are happy to supervise the following MSc theses:- Lucas Lage, MSc – Topic: Hallucinations in RAG Systems
- Daniil Gurgurov, MSc – Topic: Knowledge-Based Adaptation of Multilingual LMs for Low-Resource Languages
- Katja Konermann, MSc – Topic: Massive Multilabel Classification
- Gregory Charles Shook, MSc – Topic: Interpretability of Language Adapters
- Mikhail Sonkin, MSc – Topic: Explainability of Word and Subword Order in Multilingual Language Models
If you are interested in writing your MSc thesis with my E&E group, please send me a mail with a short topic outline, a CV and a transcript of records. Please note that we get many such requests and cannot collaborate with everybody that is interested.
Finished MSc Theses
- Julian Schlenker, MSc, 2025, Advisor – "On the Efficacy of Language Adapters for Cross-Lingual Transfer in English-Centric LLMs"
- Bangyao Tang, MSc, 2024, Reviewer – "Analysis of Calibrated Confident Text Classification"
- Akshai Joshi, MSc, 2024, Reviewer – "Self-Supervised Multimodal Representation Learning for Diagram Understanding"
- Konstantin Chernychew, MSc, 2024, Advisor – "Fast and Efficient Structured Pruning of LLMs with Gradient-Based Meta-Mask"
- Gokul Srinivasagan, MSc, 2023, Advisor – "Extreme model compression for large-scale transformer-based language models"
- Daria Fedorova, MSc, 2023, Advisor – "Cross-lingual and Cross-domain Knowledge Transfer for Sequence Labeling Tasks"
- Hannah Seitz, MSc, 2016, Advisor – "Investigating Instantiations of Script Structures in Narrative Texts"