Deep Learning in NLP (SS 2023)


General Information

Instructors Dr. Younes Samih, Dr. Christian Wurm and David Arps
Practical sessions: Tuesday, 14:30 – 16:00 Raum: PC-Pool 23.21.00.93
Theoretical sessions: Thursday, 12:30 – 14:00 Raum: 24.21.05.61 (Z20)
Course web page: https://user.phil-fak.uni-duesseldorf.de/~samih/teaching/dl-nlp2023/
Office hours: by appointment
Language: German and English

Course Description

The aim of this course is to develop an understanding of the state-of-the-art techniques of neural networks and to apply them in practice, to natural language processing problems in particular. Thursday sessions will be typically dedicated to theory, Tuesday sessions – programming. During the practical sessions, we will use the PyTorch framework to implement our networks.

Script

The theoretical content can be found in the script (caution, frequent updates!).

Etherpad

for sharing code snippets etc.

Requirements

  • BN: Complete the theoretical and the programming homework exercises. The homeworks will be published on this web page as we go.
  • AP: Term paper based on a practical project: 4-5 pages for undergrad students, 7-10 pages for master students. Guidelines

Schedule

Time Week Content Homework Solutions
04.04.2023 01 Introduction and overview | Software installation
11.04.2023 02 Tensors | Vektoren Matrizen | Lecture Code Coding Ex_01 Solution_01
18.04.2023 03 Encoding and embedding | Intro Python Coding Ex_02 Solution_02
25.04.2023 04 Building neural modules Theory EX_01 Solution
02.05.2023 05 Linear regression | LR_Sklearn
09.05.2023 06 Intro | Gradient Descent (Colab Zip) Coding EX_03 (Colab Download) Solution
16.05.2023 07 POS Tagging Continued (1 2) Theory Ex_02
23.05.2023 08 Dev Sets Data Analysis Batches Tensors (1 2)
30.05.2023 09 Tensors continued  Coding Ex_04 Solution
06.06.2023 10 Contextualization Coding Ex_05  Solution
13.06.2023 11 LM from Scratch Coding Ex_06 Solution
20.06.2023 12 LM to POS tagger Theory Ex_03 bis 29.6.
27.06.2023 13 LM to POS tagger 2 project prep
04.07.2023 14 Tue: Huggingface + Transformers (Stanford Colab Tutorial) – Thu: final project discussion tba
11.07.2023 15 MLM Scoring