Simon Ostermann

DFKI

Seminar: Efficient and Robust Natural Language Processing

Wed 14:15–15:45, Room -1.05

Dr. Simon Ostermann, Tatiana Anikina, Natalia Skachkova
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)
If you would like to participate, please drop an email to efficient-nlp-seminar@dfki.de
until April 6th (23:59).
In your email, please:

Prerequisites

This seminar is primarily targeted at Master students, but is also open to advanced Bachelor students. We expect you to have a curious mind and advanced familiarity with large language models. At the very least, we expect all students to have read (and understood :-)) the BERT paper and the Transformer paper.

Seminar Content

Nowadays, Large Language Models (LLMs) achieve impressive results on a variety of NLP tasks and languages. This increase in performance comes with a specific cost: Better performance is typically attributed to an increased number of parameters and training data. This means that better and larger models also need more computational resources, more time, more memory, and more energy, which makes training processes prohibitively expensive.

How can we solve the problem of ever larger and ever hungrier models? Efficient NLP is an umbrella term that includes a lot of different approaches to address these problems in terms of model design and data efficiency, by introducing more efficient fine-tuning, inference, or hardware utilization. In this seminar we will mostly focus on model and data-level efficiency and will explore various approaches including efficient prompting methods, adapters, transfer learning, data augmentation and active learning.

A preliminary list of papers to choose from can be found here.

Some words on grading: This seminar is meant to be as interactive as possible. Final grades will be based on students' presentations, term papers (optional), but also on participation and discussion in class.

The participants are expected to prepare for classes accordingly, by reading the relevant papers and also doing background reading, if necessary. Based on this preparation, the participants should be able to discuss the presented papers in depth and to understand relevant context during the discussion.

Back to Main Page