Schedule

• Course Introduction
• Logistics

• Word embeddings and vector semantics

• Jurafsky & Martin Chapter 6

• Word embeddings and vector semantics (cont.)
• Sparse representations
• Dense representations

• Jurafsky & Martin Chapter 6

• Distributed Representation of Words and Phrases and their Compositionality (Mikolov et al., 2013) [link]
• Efficient Estimation of Word Representations in Vector Space (Mikolov et al., 2013) [link]
• Word2vec Explained- deriving Mikolov et al.'s negative-sampling word-embedding method (Goldberg and Levy, 2014) [link]

HW 1 out

• Deriving the gradient of Word2vec
• Evaluation of word embeddings

• Jurafsky & Martin Chapter 6
• Distributed Representation of Words and Phrases and their Compositionality (Mikolov et al., 2013) [link]

• N-Gram Language Models
• Smoothing
• Evaluation of Language Models

• Jurafsky & Martin Chapter 7

• Neural network basics
• Autograd

• The Matrix Calculus You Need For Deep Learning (Terence Parr and Jeremy Howard) [link]
• Little book of deep learning (François Fleuret) - Ch 3, 4

• Auto Grad
• Building blocks of Deep Learning for Language Modeling
• CNNs

• Goldberg Chapter 9

• RNNs
• Task specific neural network architectures
• Training RNNs
• Machine translation
• Attention

• Understanding LSTM Networks (Christopher Olah) [link]
• Neural Machine Translation and Sequence-to-sequence Models- A Tutorial (Graham Neubig) [link]
• Learning to Align and Translate with Attention (Bahdanau et al., 2015) [link]
• Statistical Machine Translation (Koehn) [link]

• Neural Machine Translation and Sequence-to-sequence Models- A Tutorial (Graham Neubig) [link]

• Transformers

• Luong et al. (2015) Effective Approaches to Attention-based Neural Machine Translation [link]
• Attention is All You Need (Vaswani et al., 2017) [link]
• Illustrated Transformer [link]

Project teams due on 02/09

HW1 due 02/10

• Attention
• Transformers

• Neural Machine Translation and Sequence-to-sequence Models- A Tutorial (Graham Neubig) [link]
• Learning to Align and Translate with Attention (Bahdanau et al., 2015) [link]
• Luong et al. (2015) Effective Approaches to Attention-based Neural Machine Translation [link]
• Attention is All You Need (Vaswani et al., 2017) [link]
• Illustrated Transformer [link]

02/14
HW 2 out

• Tokenization
• Language modeling with Transformers
• Early language models
• Transfer Learning

• Illustrated Transformer [link]
• Attention is All You Need (Vaswani et al., 2017) [link]
• The Annotated Transformer (Harvard NLP) [link]
• GPT-2 (Radford et al., 2019) [link]

• Transfer Learning (contd.)
• Objective functions for pre-training
• Encoder-decoder pretrained models
• Architecture and pretraining objectives

• The Illustrated BERT, ELMo, and co. (Jay Alammar) [link]
• BERT- Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., 2018) [link]
• GPT-2 (Radford et al., 2019) [link]
• T5- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (Raffel et al., 2020) [link]
• BART- Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (Lewis et al., 2019) [link]
• What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? (Wang et al, 2022) [link]

• Decoding and generation
• Large language models and impact of scale
• In-context learning and prompting

• The Curious Case of Neural Text Degeneration (Holtzman et al., 2019) [link]
• How to generate text- using different decoding methods for language generation with Transformers [link]
• Scaling Laws for Neural Language Models (Kaplan et al., 2020) [link]
• Training Compute-Optimal Large Language Models (Hoffmann et al., 2022) [link]
• GPT3 paper - Language Models are Few-Shot Learners (Brown et al., 2020) [link]

• Post-training
• Supervised Finetuning
• Instruction Following

• Multitask Prompted Training Enables Zero-Shot Task Generalization (Sanh et al., 2021) [link]
• Scaling Instruction-Finetuned Language Models (Chung et al., 2022) [link]
• Are Emergent Abilities of Large Language Models a Mirage? (Sha et al., 2023) [link]
• Emergent Abilities of Large Language Models (Wei et al., 2022) [link]

HW 2 due 03/04

• From Assistants to Collaborators: Building Agents for Long-Form, Open-ended Collaboration

• Talk abstract: The ever-increasing capabilities of language models call for rethinking how we build agents: moving from assistants that passively help with simple, well-defined tasks, to collaborators that proactively support users in long-form, exploratory work — and continue learning alongside them. In this talk, I will present a systematic view of what it takes to build collaborator agents, organized around three questions. First, what is the right objective for building collaborator agents, and how should we evaluate whether collaboration succeeds? Second, how do we train agents to operate in settings where goals are open-ended and evolve over time? And third, what architectural improvements to language models are needed to make collaboration practical? I will share lessons learned in our recent projects in this direction, including training deep research agents for open-ended long-form generation with evolving rubrics and enabling interleaved latent and text chain of thought for more efficient long-form reasoning in language models.

Project proposals due 03/06

• Guest Lecture 2: TBD

• Post-training
• Reinforcement learning from Human Feedback
• Alignment

• Training language models to follow instructions with human feedback (Ouyang et al., 2022) [link]
• Fine-Tuning Language Models from Human Preferences (Ziegler et al., 2019) [link]
• Direct Preference Optimization- Your Language Model is Secretly a Reward Model (Rafailov et al., 2023) [link]
• RLAIF- Scaling Reinforcement Learning from Human Feedback with AI Feedback (Lee et al., 2023) [link]

• Post-training (contd...)

HW 3 out

• Retrieval Augmented Generation (RAG)

• Guest Lecture 4: TBD

• RAG continued, Intro to Agent-based systems

• Project presentations session 1

Final project presentations

• Project presentations session 2

Final project presentations, HW 3 due on 4/27,
Final project reports due on 4/30