Simon Ostermann

Welcome 👋

I am a computational linguist and senior researcher at the Multilinguality and Language Technology (MLT) lab at the German Research Center for Artificial Intelligence, where I am currently leading mid-sized research projects both administratively and scientifically. I manage the MLT lab with over 20 employees and lead the research group on Efficient and Explainable NLP (E&E) at the MLT lab.

My research interests are mainly on transparent and robust language models and language technology.

• Transparency: Making the parameters and behavior of language models more explainable and understandable to both end users and researchers.
• Robustness: Improving language models in two dimensions:
  • Data consumption (e.g., for low-resource settings or languages) by using structured data, new learning techniques, or other modalities.
  • Size (e.g., for settings without powerful hardware).

In my work, I mostly use large language models trained with deep learning methods. I also have profound experience in data acquisition and crowdsourcing, as well as educational NLP.

Selected Publications 📰

Teaching 🏫

I am also teaching at Saarland University. Here is a list of my ongoing courses:

• Seminar, Summer 2025: Efficient and Robust Natural Language Processing Details

  A course on efficient NLP, with a focus on robust models for low-resource data settings and parameter efficient models.

  Course Website

• Seminar, Winter 2024: Recent Advances in Mechanistic Interpretability Details

  A course examining the latest developments in understanding the mechanisms of AI models.

  Course Website

• Software Project, Winter 2024: Recent Advances in Mechanistic Interpretability Details

  A practical course on applying mechanistic interpretability methods in natural language processing.

  Course Website

Finished Courses

Supervision 🎓

• Tatiana Anikina - Topic: Efficient Adaptation of Language Models for Low-Resource Natural Language Understanding
• Cennet Oguz - Topic: Multimodal Processing of Procedural Knowledge
• Tanja Bäumel - Topic: Mechanistic Interpretability of LLMs
• Yusser al Ghussin - Topic: Mechanistic Steering of LLMs in Multilingual and Multicultural Aspects
• Ivan Vykopal (at KInIT) - Topic: Multilingual Low-resource NLP

Ongoing MSc Theses

• Lucas Lage, MSc – Topic: Hallucinations in RAG Systems
• Daniil Gurgurov, MSc – Topic: Knowledge-Based Adaptation of Multilingual LMs for Low-Resource Languages
• Katja Konermann, MSc – Topic: Massive Multilabel Classification
• Gregory Charles Shook, MSc – Topic: Interpretability of Language Adapters
• Mikhail Sonkin, MSc – Topic: Explainability of Word and Subword Order in Multilingual Language Models

If you are interested in writing your MSc thesis with my E&E group, please send me a mail with a short topic outline, a CV and a transcript of records. Please note that we get many such requests and cannot collaborate with everybody that is interested.

Finished MSc Theses

• Julian Schlenker, MSc, 2025, Advisor – "On the Efficacy of Language Adapters for Cross-Lingual Transfer in English-Centric LLMs"
• Bangyao Tang, MSc, 2024, Reviewer – "Analysis of Calibrated Confident Text Classification"
• Akshai Joshi, MSc, 2024, Reviewer – "Self-Supervised Multimodal Representation Learning for Diagram Understanding"
• Konstantin Chernychew, MSc, 2024, Advisor – "Fast and Efficient Structured Pruning of LLMs with Gradient-Based Meta-Mask"
• Gokul Srinivasagan, MSc, 2023, Advisor – "Extreme model compression for large-scale transformer-based language models"
• Daria Fedorova, MSc, 2023, Advisor – "Cross-lingual and Cross-domain Knowledge Transfer for Sequence Labeling Tasks"
• Hannah Seitz, MSc, 2016, Advisor – "Investigating Instantiations of Script Structures in Narrative Texts"