Prerequisites: This seminar is primarily targeted at Master students, but is also open to advanced Bachelor students. We expect you to have a curious mind and some familiarity with large language models. At the very least, we expect all students to have read (and understood :-)) the
BERT paper and the
Transformer paper.
Seminar Content
The rise of deep learning in AI has dramatically increased the performance of models across many sub-fields such as natural language processing or computer vision. In the last 5 years, large petrained language models (LLMs) and their variants (BERT, ChatGPT etc.) have changed the NLP landscape drastically. Such models got larger and larger over the last years, reaching increasingly impressive performance peeks, sometimes even surpassing humans.
A central issue with deep learning models with millions or billions of parameters is that the are essentially black boxes: From the model's parameters, it is not inherently clear why a model exhibits a certain behavior or makes certain classification decision. Understanding the inner workings of such large models is however extremely important, especially when AI takes on critical tasks e.g. in the medical or financial domain. Trustworthiness and fairness are important dimensions that such large models should adhere to, that are often not taken into account.
In this seminar we will try to shine a spotlight on the rapidly growing field of interpretable and explainable AI (XAI), that develops methods to peak into the black box that LLMs are. We will (1) introduce general methods used in XAI, then (2) turn to insights about BERT and its variants gained from applying these methods. In this context we will (3) also make use of our gained knowledge to discuss more critical views on the trend of ever-growing deep learning models.
Preliminary (incomplete) list of papers
Explainable AI
- Probing: What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties | paper
- Amnesic Probing: Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals | paper
- LIME: "Why Should I Trust You?": Explaining the Predictions of Any Classifier | paper
- Attention Interpretation:
- Is Attention Interpretable? | paper
- Attention is not Explanation | paper
- Attention is not not Explanation | paper
Interpreting BERT & Co.
- Prompt Learning: Personalized Prompt Learning for Explainable Recommendation | paper
- Syntax in BERT:
- Open Sesame: Getting Inside BERT’s Linguistic Knowledge | paper
- A Structural Probe for Finding Syntax in Word Representations | paper
- Semantics in BERT:
- What do you learn from context? Probing for sentence structure in contextualized word representations | paper
- What’s in a Name? Are BERT Named Entity Representations just as Good for any other Name? | paper
- World Knowledge in BERT:
- Do Neural Language Representations Learn Physical Commonsense? | paper
- BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA | paper
- ChatGPT:
- Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbook | paper
- ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text | paper
- ChatGPT: Jack of all trades, master of none | paper
Fairness and Ethics
- Measuring Fairness with Biased Rulers: A Comparative Study on Bias Metrics for Pre-trained Language Models | paper
- Bigger Data or Fairer Data? Augmenting BERT via Active Sampling for Educational Text Classification | paper
- Mitigating Language-Dependent Ethnic Bias in BERT | paper
Some words on grading: This seminar is meant to be as interactive as possible. Final grades will be based on students' presentations, term papers (optional), but also on participation and discussion in class.
The participants are expected to prepare for classes accordingly, by reading the relevant papers and also doing background reading, if necessary. Based on this preparation, the participants should be able to discuss the presented papers in depth and to understand relevant context during the discussion.