{"id":858,"date":"2023-03-26T20:25:19","date_gmt":"2023-03-26T19:25:19","guid":{"rendered":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/?page_id=858"},"modified":"2023-09-04T11:47:56","modified_gmt":"2023-09-04T10:47:56","slug":"dl-nlp2023","status":"publish","type":"page","link":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/teaching\/dl-nlp2023\/","title":{"rendered":"Deep Learning in NLP (SS 2023)"},"content":{"rendered":"<hr style=\"width: 50%; text-align: left; margin-left: 0;\" \/>\n<h2><strong>General Information<\/strong><\/h2>\n<table style=\"border: hidden;\" align=\"left\">\n<tbody>\n<tr style=\"border: hidden;\">\n<td style=\"border: hidden;\"><strong>Instructors<\/strong><\/td>\n<td style=\"border: hidden;\">Dr. Younes Samih, Dr. Christian Wurm and David Arps<\/td>\n<\/tr>\n<tr style=\"border: hidden;\">\n<td style=\"border: hidden;\"><strong>Practical sessions:<\/strong><\/td>\n<td style=\"border: hidden;\">Tuesday, 14:30 \u2013 16:00 Raum: <strong>PC-Pool 23.21.00.93<\/strong><\/td>\n<\/tr>\n<tr style=\"border: hidden;\">\n<td style=\"border: hidden;\"><strong>Theoretical sessions:<\/strong><\/td>\n<td style=\"border: hidden;\">Thursday, 12:30 &#8211; 14:00 Raum: <strong>24.21.05.61 (Z20)<\/strong><\/td>\n<\/tr>\n<tr style=\"border: hidden;\">\n<td style=\"border: hidden;\"><strong>Course web page:<\/strong><\/td>\n<td style=\"border: hidden;\"><a href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/teaching\/dl-nlp2023\/\">https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/teaching\/dl-nlp2023\/<\/a><\/td>\n<\/tr>\n<tr style=\"border: hidden;\">\n<td style=\"border: hidden;\"><strong>Office hours:<\/strong><\/td>\n<td style=\"border: hidden;\">by appointment<\/td>\n<\/tr>\n<tr style=\"border: hidden;\">\n<td style=\"border: hidden;\"><strong>Language:<\/strong><\/td>\n<td style=\"border: hidden;\">German and English<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<hr style=\"width: 50%; text-align: left; margin-left: 0;\" \/>\n<h2>Course Description<\/h2>\n<p>The aim of this course is to develop an understanding of the state-of-the-art techniques of neural networks and to apply them in practice, to natural language processing problems in particular. Thursday sessions will be typically dedicated to theory, Tuesday sessions \u2013 programming. During the practical sessions, we will use the <strong><a href=\"https:\/\/pytorch.org\/\">PyTorch<\/a><\/strong> framework to implement our networks.<\/p>\n<h4 id=\"script\">Script<\/h4>\n<p>The theoretical content can be found in\u00a0the <a href=\"http:\/\/user.phil.uni-duesseldorf.de\/~cwurm\/wp-content\/uploads\/2023\/05\/deep_architectures.pdf\"><strong>script<\/strong>\u00a0<\/a>(caution, frequent updates!).<\/p>\n<h4>Etherpad<\/h4>\n<p><a href=\"http:\/\/etherpad.phil-fak.uni-duesseldorf.de\/mPowPxNYrq\">for sharing code snippets etc.<\/a><\/p>\n<h4><\/h4>\n<h4 id=\"requirements\">Requirements<\/h4>\n<ul>\n<li><b>BN<\/b>: Complete the theoretical and the programming homework exercises. The homeworks will be published on this web page as we go.<\/li>\n<li><b>AP<\/b>: Term paper based on a practical project:\u00a0<b>4-5 pages<\/b>\u00a0for undergrad students,\u00a0<b>7-10 pages<\/b> for master students. <a href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-content\/uploads\/2023\/06\/guidelines.pdf\">Guidelines<\/a><\/li>\n<\/ul>\n<hr style=\"width: 50%; text-align: left; margin-left: 0;\" \/>\n<h2 id=\"schedule\">Schedule<\/h2>\n<div class=\"table-responsive\"><table id=\"someid\" style=\"width:100%; \"  class=\"easy-table easy-table-default \" border=\"0\">\n<thead>\r\n<tr><th ><strong>Time<\/strong><\/th>\n<th ><strong>Week<\/strong><\/th>\n<th ><strong>Content<\/strong><\/th>\n<th ><strong>Homework<\/strong><\/th>\n<th ><strong>Solutions<\/strong><\/th>\n<\/tr>\n<\/thead>\n<tbody>\r\n<tr><td ><strong>04.04.2023<\/strong><\/td>\n<td ><strong>01<\/strong><\/td>\n<td ><a href=\"https:\/\/cloud.phil.hhu.de\/index.php\/s\/9Bvs71EheAesJ17\" target=\"_blank\" rel=\"noopener\">Introduction and overview<\/a> | <a href=\"https:\/\/cloud.phil.hhu.de\/index.php\/s\/94XXWX1xaOo1QA5\">Software installation<\/a><\/td>\n<td > &#8212;<\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>11.04.2023<\/strong><\/td>\n<td ><strong>02<\/strong><\/td>\n<td ><a href=\"https:\/\/cloud.phil.hhu.de\/index.php\/s\/9hI8Bvm10xj46Wl\" target=\"_blank\" rel=\"noopener\">Tensors<\/a> | <a href=\"https:\/\/cloud.phil.hhu.de\/index.php\/s\/lk9qMNCmq2CqMMI\" target=\"_blank\" rel=\"noopener\">Vektoren Matrizen<\/a> | <a href=\"https:\/\/colab.research.google.com\/drive\/1t4KMqrvQ3wQ37N5naewpYQLpa27nOzjO?usp=sharing\" target=\"_blank\" rel=\"noopener\">Lecture Code<\/a><\/td>\n<td ><a href=\"https:\/\/colab.research.google.com\/drive\/1zO0xFNG4f52ttZCGandsS4I5IbBV8Drt?usp=sharing\" target=\"_blank\" rel=\"noopener\"> Coding Ex_01<\/a><\/td>\n<td ><a href=\"https:\/\/colab.research.google.com\/drive\/1hgzWTTJ39iJQRTFLfkpPQ04CdT_1rhO4\" target=\"_blank\" rel=\"noopener\">Solution_01<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>18.04.2023<\/strong><\/td>\n<td ><strong>03<\/strong><\/td>\n<td ><a href=\"https:\/\/colab.research.google.com\/drive\/1v03GaA0pGPawwm1wLpd8Qb2IgDs7wH1T?usp=sharing\" target=\"_blank\" rel=\"noopener\">Encoding and embedding<\/a> | <a href=\"https:\/\/colab.research.google.com\/drive\/154D4Avuxte-9_AIbTiDTqxWandBlRGh7?usp=sharing\" target=\"_blank\" rel=\"noopener\">Intro Python <\/td>\n<td ><\/a><a href=\"https:\/\/colab.research.google.com\/drive\/1PVU9R9_uQngZuPZUUnZjg585DgE-D329?usp=sharing\">Coding Ex_02<\/a><\/td>\n<td > <a href=\"https:\/\/colab.research.google.com\/drive\/1crbIeIlXea-ee6K3yVJTCwMgidCy12rc?usp=sharing\" target=\"_blank\" rel=\"noopener\">Solution_02<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>25.04.2023<\/strong><\/td>\n<td ><strong>04<\/strong><\/td>\n<td > <a href=\"https:\/\/colab.research.google.com\/drive\/1evnXXtDDTVIILrZLcDYxKortj5BSeCMS#scrollTo=G3ozJYf81AGK\">Building neural modules<\/a> <\/td>\n<td ><a href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-content\/uploads\/2023\/04\/haTheorie1.pdf\">Theory EX_01<\/a><\/td>\n<td > <a href=\"https:\/\/user.phil.hhu.de\/~cwurm\/wp-content\/uploads\/2023\/05\/HA1-sol.pdf\">Solution<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>02.05.2023<\/strong><\/td>\n<td ><strong>05<\/strong><\/td>\n<td > <a href=\"https:\/\/colab.research.google.com\/drive\/1J-Lf-XOhmi_EYPNnl5vLR5R5LstDjwKW?usp=sharing\" target=\"_blank\" rel=\"noopener\">Linear regression<\/a> | <a href=\"https:\/\/colab.research.google.com\/drive\/1Niu07kFjaULKUS6TdpM-ER7ptjxqOYuO?usp=sharing\" target=\"_blank\" rel=\"noopener\">LR_Sklearn<\/a> <\/td>\n<td >&#8212;<\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>09.05.2023<\/strong><\/td>\n<td ><strong>06<\/strong><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1G164zSKpMv4lbTRzQwnDcZ2AVY5_BNxe\/view\">Intro<\/a> | Gradient Descent (<a href=\"https:\/\/drive.google.com\/file\/d\/1HyyM8AIyZ1QLI4laSUf08zwKwSpTGe1_\/view\">Colab<\/a> <a href=\"https:\/\/drive.google.com\/file\/d\/14Zh8nD6cFovjMV_tnjEsTK3uUfMiOAnh\/view?usp=share_link\">Zip<\/a>) <\/td>\n<td >Coding EX_03 (<a href=\"https:\/\/drive.google.com\/file\/d\/1X_zjBVYwGytgRNRjS9_ABBTxbVyFW9if\/view?usp=sharing\">Colab<\/a> <a href=\"https:\/\/drive.google.com\/file\/d\/1X_zjBVYwGytgRNRjS9_ABBTxbVyFW9if\/view?usp=share_link\">Download<\/a>)<\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1RcHNIgEVfGwJjfU1zkQIGs3_X3qP0N0G\/view?usp=share_link\">Solution<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>16.05.2023<\/strong><\/td>\n<td ><strong>07<\/strong><\/td>\n<td > POS Tagging Continued (<a href=\"https:\/\/drive.google.com\/file\/d\/11YdseF8vUSy8-uVwiKmRiWwpPk5SYr9x\/view?usp=share_link\">1<\/a> <a href=\"https:\/\/drive.google.com\/file\/d\/1kZ-YEJJEOvraU8hZNrUUoTq8luEpkUz2\/view?usp=share_link\">2<\/a>) <\/td>\n<td ><a href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-content\/uploads\/2023\/05\/HA_Backpropagation.pdf\">Theory Ex_02<\/a><\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>23.05.2023<\/strong><\/td>\n<td ><strong>08<\/strong><\/td>\n<td > Dev Sets Data Analysis Batches Tensors (<a href=\"https:\/\/drive.google.com\/file\/d\/1cncYlDLup_zvWAnNqh0rSGYWObuQ2JCm\/view?usp=share_link\">1<\/a> <a href=\"https:\/\/drive.google.com\/file\/d\/1kfggyHPNe4-8cbUY6hDGoj3zb6p5XZIi\/view?usp=share_link\">2<\/a>) <\/td>\n<td > &#8212;<\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>30.05.2023<\/strong><\/td>\n<td ><strong>09<\/strong><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1XEXr_FV44laXaIQF8npjZ1y3WsjsRMtZ\/view?usp=sharing\">Tensors continued\u00a0<\/a><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1Q3PxJ4DWyVYatIURc7bCyMqk438zu9f_\/view?usp=sharing\">Coding Ex_04<\/a> <\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1r8GgX3aEf-qf7k2wkLdihQaVhCzYDdO5\/view?usp=sharing\">Solution<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>06.06.2023<\/strong><\/td>\n<td ><strong>10<\/strong><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1qU9Wt9ARkN7yZzFVY8j0OfoWHy6SNzgE\/view?usp=sharing\">Contextualization<\/a> <\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1PNAGuX_mes8RDkP9BmhHR0yOqhHOjBIg\/view?usp=sharing\">Coding Ex_05<\/a>\u00a0 <\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1bNtXPpuvE3rZTn73urZuuVMg3H0sllrN\/view?usp=sharing\">Solution<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>13.06.2023<\/strong><\/td>\n<td ><strong>11<\/strong><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1sFyOp3-H3WDNnz8o2fU_dSABvQGkwizL\/view?usp=sharing\">LM from Scratch<\/a><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1ONCBfumDHGYzlw0oosAB67ZP3JDfiG19\/view?usp=sharing\">Coding Ex_06<\/a><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/file\/d\/1UfNaH2_OZ1ikIgZk7gVi2ljV2ObMM6I3\/view?usp=sharing\">Solution<\/a><\/td>\n<\/tr>\n\r\n<tr><td ><strong>20.06.2023<\/strong><\/td>\n<td ><strong>12<\/td>\n<td ><\/strong> <a href=\"https:\/\/drive.google.com\/drive\/folders\/1EDkgMIxqY0SjuhT7L9qFCXC89OeUzfvO?usp=sharing\">LM to POS tagger<\/a><\/td>\n<td > <a href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-content\/uploads\/2023\/06\/ha3.pdf\">Theory Ex_03<\/a> bis 29.6.<\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>27.06.2023<\/strong><\/td>\n<td ><strong>13<\/strong><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/drive\/folders\/15OoM0oUW8ZPzQjWoFOyVUvRNK-trmQJ3?usp=sharing\">LM to POS tagger 2<\/a><\/td>\n<td > <a href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-content\/uploads\/2023\/06\/project_prep.pdf\">project prep<\/a><\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>04.07.2023<\/strong><\/td>\n<td ><strong>14<\/strong><\/td>\n<td > Tue: Huggingface + Transformers (<a href=\"https:\/\/colab.research.google.com\/drive\/1pxc-ehTtnVM72-NViET_D2ZqOlpOi2LH?usp=sharing\">Stanford Colab Tutorial<\/a>) &#8211; Thu: final project discussion<\/td>\n<td > tba<\/td>\n<td > &#8212;<\/td>\n<\/tr>\n\r\n<tr><td ><strong>11.07.2023<\/strong><\/td>\n<td ><strong>15<\/strong><\/td>\n<td > <a href=\"https:\/\/drive.google.com\/drive\/folders\/1zLW2s6fBOuQdZ4ma1BwOpxia93k-JDBI?usp=sharing\">MLM Scoring<\/a><\/td>\n<td > &#8212;<\/td>\n<td > &#8212;<\/td>\n<\/tr>\n<\/tbody><\/table><\/div>\n","protected":false},"excerpt":{"rendered":"<p>General Information Instructors Dr. Younes Samih, Dr. Christian Wurm and David Arps Practical sessions: Tuesday, 14:30 \u2013 16:00 Raum: PC-Pool 23.21.00.93 Theoretical sessions: Thursday, 12:30 &#8211; 14:00 Raum: 24.21.05.61 (Z20) Course web page: https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/teaching\/dl-nlp2023\/ Office hours: by appointment Language: German &hellip; <a class=\"more-link\" href=\"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/teaching\/dl-nlp2023\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":81,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"_links":{"self":[{"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/pages\/858"}],"collection":[{"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/comments?post=858"}],"version-history":[{"count":213,"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/pages\/858\/revisions"}],"predecessor-version":[{"id":1108,"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/pages\/858\/revisions\/1108"}],"up":[{"embeddable":true,"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/pages\/81"}],"wp:attachment":[{"href":"https:\/\/user.phil-fak.uni-duesseldorf.de\/~samih\/wp-json\/wp\/v2\/media?parent=858"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}