Course

LIN281/481: Statistical and Neural Computational Linguistics

Description

This course covers advanced topics in computational linguistics, with a focus on the deployment of statistical and neural methods for advancing linguistic theory as well as the use of linguistic theory for designing statistical and neural models. Topics include morphophonological grammar learning, syntactic grammar learning, semantic representation learning for words and phrases, syntactic parsing, semantic parsing, and natural language inference.

Logistics

Classroom513 Lattimore Hall
TimeMonday & Wednesday 12:30-1:45pm
InstructorAaron Steven White
Office 511A Lattimore Hall
Office hours Tuesday 9-11am
Email aaron.white@rochester.edu

Readings

There will be a substantial amount of reading per week. A tentative schedule is provided below. This tentative schedule will almost certainly change as the semester goes forward. Consult the table below for the most current list.

Project

Throughout the semester you will be developing a course project that will culminate in an ACL-style conference paper. This project consists of five main components: (i) six checkpoints; (ii) a prospectus; (iii) a mid-semester prospectus presentation; (iv) a final project presentation; and (v) the paper itself. A list of potential projects will be made available. You may opt to do something not included in this list, but I must approve it.

Checkpoints

Starting in the sixth week of class, you will be required to complete weekly checkpoints logging your progress on the project. Each checkpoint should at least include a summary of one paper (or more) that you read for the project. Later in the semester, as the project develops, the checkpoints should also include a description of the code you have written for the project (along with a link to that code on github) and/or results of running that code on some dataset. The point of these checkpoints is two-fold: (i) to ensure that you are making sufficient incremental progress on the project throughout the semester; and (ii) to provide you grist for the paper, so that by the time you go to produce a full draft, you have content to draw on.

Prospectus

A 500-word prospectus detailing the goals of your project will be due on Wednesday, March 20. This prospectus should specify the question your final paper aims to answer and summarize the literature you have read so far, including two papers you have not yet summarized in the checkpoints.

Mid-semester presentation

You will be required to give a 15-minute presentation detailing your prospectus to the class on either March 11 or March 13.

Final presentation

You will be required to give a 30-minute presentation detailing your completed project to the class on either April 29 or May 1.

Final paper

Your final paper should use the formatting instructions specified for submission to Transactions of the Association for Computational Linguistics (TACL). Students enrolled in LIN281 are required to write either exactly four pages or exactly eight pages (though you may include appendices). Students enrolled in LIN481 are required to write exactly ten pages (no more, no less). These numbers are not arbitrary. A standard short paper for an ACL venue is four pages with an optional fifth page for the “camera ready” version; a standard long paper is eight pages with an optional ninth for the “camera ready” version; and a standard submission to TACL is ten pages with more allowed for revisions.

Tools

I will assume competence in LaTeX and the conventional core of Python – i.e. built-in types, control flow, functions, classes, etc. In the first full week of class, I will cover basic numerical computing in Python using the scipy stack. As the class progresses, we will also use pytorch (for neural network development) and pyro (for probabilistic model development).

You may use any programming language that you like for developing your course project, though I would encourage you to use Python, since it is so prevalent in Computational Linguistics and Natural Language Processing.

All written work – including checkpoints, the prospectus, and the final paper must be produced in LaTeX. All work must furthermore be submitted through Overleaf, where templates will be hosted – i.e. I will provide templates for the checkpoints, the prospectus, and the final paper. Do not hand in handwritten work or email me electronic documents. You will not receive credit for that assignment unless you subsequently turn the assignment in through Overleaf (subject to late penalties).

Final grades

The grading breakdown is: project checkpoints (48%), project prospectus (12%), mid-term project presentation (10%), final project presentation (10%), project write-up (20%). (Percentages represent percent of total grade.)

Late work

Assignments should be submitted by 11:59pm the day they are due. An automatic 10% deduction will be applied after this time, and you must explicitly notify both Venkat and me by email when you turn the late assignment in or you will not receive any credit.

Starting from the end of class, assignments will lose 10% per day late according to the UTC time-stamp of submission that Overleaf reports. Late assignments may not be turned in for credit after a week unless explicit permission was sought before the due date.

Exceptions

Students will not be penalized because of important civic, ethnic, family or religious obligations, or university service. You will be have a chance, whenever feasible, to make up within a reasonable time any assignment that is missed for these reasons. Absences for these reasons will count as excused for the sake of the participation grade. But it is your job to inform me of any expected absences in advance, as soon as possible.

Honesty

All assignments and activities associated with this course must be performed in accordance with the University of Rochester’s Academic Honesty Policy. More information is available here.

Personal needs

Any student who needs special accommodations due to a disability should let me know privately, at the start of the semester.

Schedule

Date Topic Reading Due
Jan. 16 Cancelled -  
Jan. 21 MLK Day -  
Jan. 23 Introduction -  
Jan. 28 Clustering in Python I -  
Jan. 30 Clustering in Python II -  
Feb. 4 Mixture Models in Python -  
Feb. 6 Phonetic Categories I Feldman et al. 2013  
Feb. 11 Phonetic Categories II -  
Feb. 13 Phonetic Categories III -  
Feb. 18 Phonotactic Grammars I Hayes & Wilson 2008, Sec. 1-5  
Feb. 20 Phonotactic Grammars II Hayes & Wilson 2008, Sec. 6-9  
Feb. 25 Phonotactic Grammars III Futrell et al. 2017, Sec. 1-2  
Feb. 27 Phonotactic Grammars IV Futrell et al. 2017, Sec. 3-7 C1
Mar. 4 Morphophonological Grammars I Cotterell et al. 2015, Sec. 1-3  
Mar. 6 Morphophonological Grammars II Cotterell et al. 2015, Sec. 4-9 C2
Mar. 11 Spring Break -  
Mar. 13 Spring Break -  
Mar. 18 Project Presentations -  
Mar. 20 Linear Models and Multilayer Perceptrons Goldberg 2017, Ch. 2-5 P
Mar. 25 Word Embeddings Goldberg 2017, Ch. 6, 8, 10-11  
Mar. 27 Recurrent Neural Networks Goldberg 2017, Ch. 15-16 C3
Apr. 1 Morphophonological Grammars III Goldberg 2017, Ch. 17-17.2  
    Faruqui et al. 2016  
Apr. 3 Morphophonological Grammars IV - C4
Apr. 8 Syntactic Parsing I Linzen et al. 2016  
    Marvin and Linzen 2018  
    Gulordava et al. 2018  
    Wilcox et al. 2018  
Apr. 10 Syntactic Parsing II Goodkind and Bicknell 2018, C5
    van Schijndel and Linzen 2018, 2019  
Apr. 15 Natural Language Inference I Bowman et al. 2015  
Apr. 17 Natural Language Inference II White et al. 2017, Poliak et al. 2018 C6
Apr. 22 Decompositional Semantic Parsing I Rudinger et al. 2018a, b  
Apr. 24 Decompositional Semantic Parsing II Zhang et al. 2018  
Apr. 29 Project Presentations -  
May 1 Project Presentations -  

References