Slot Session Readings Logistics
Wed 09/02/26
  • Course introduction and logistics
  • The frontier model landscape in 2026
Slides
Main readings:
  • GPT-4 Technical Report (2023) [link]
  • The Llama 3 Herd of Models (2024, skim) [link]
Fri 09/04/26
  • Assignment 0 overview
  • Foundations refresher
  • Transformers, pretraining objectives, tokenization, compute analysis
Slides
Main readings:
  • Attention Is All You Need (2017) [link]
  • Scaling Laws for Neural Language Models (2020) [link]
  • Monday classes meet on Friday because of Labor Day
  • Presentation teams and slots announced Fri 09/11
  • Assignment 0 due Sun 09/06
Mon 09/07/26
Labor Day - No class
Wed 09/09/26
  • Theme 1 begins: Frontier LLM architectures
  • The modern decoder stack
  • RMSNorm, SwiGLU, RoPE, MQA/GQA, multi-head latent attention, KV caching, FlashAttention
Slides
Main readings:
  • RoFormer - Enhanced Transformer with Rotary Position Embedding (2021) [link]
  • GQA - Training Generalized Multi-Query Transformer Models (2023) [link]
  • DeepSeek-V2 (2024, MLA sections) [link]
  • FlashAttention (2022) [link]
Mon 09/14/26
  • Sparsity and attention alternatives
  • Mixture-of-experts and routing, latent/low-rank MoE, state-space models, linear attention, hybrid stacks
  • Why frontier models converged on sparse hybrids; inference cost and low-precision training
Slides
Main readings:
  • Switch Transformers (2021) [link]
  • Mixtral of Experts (2024) [link]
  • Mamba - Linear-Time Sequence Modeling with Selective State Spaces (2023) [link]
Wed 09/16/26
  • Paper session 1: Frontier MoE at scale
Kimi K3 slides Nemotron 3 Ultra slides
Main readings:
  • Kimi K3 - Open Frontier Intelligence (2026) [link]
  • Nemotron 3 Ultra - MoE Hybrid Mamba-Transformer for Agentic Reasoning (2026) [link]
Optional readings:
  • DeepSeek-V3 Technical Report (2024) [link]
  • OLMoE - Open Mixture-of-Experts Language Models (2024) [link]
  • Kimi K2 - Open Agentic Intelligence (2025) [link]
  • Qwen3 Technical Report (2025) [link]
Mon 09/21/26
  • Paper session 2: Beyond softmax attention
Mamba 2 slides MiniMax-01 slides
Main readings:
  • Transformers are SSMs - Mamba-2 (2024) [link]
  • MiniMax-01 - Scaling Foundation Models with Lightning Attention (2025) [link]
Optional readings:
  • Jamba - A Hybrid Transformer-Mamba Language Model (2024) [link]
  • Gated Delta Networks (2024) [link]
  • Byte Latent Transformer - Patches Scale Better Than Tokens (2024) [link]
  • Large Language Diffusion Models - LLaDA (2025) [link]
  • Titans - Learning to Memorize at Test Time (2025) [link]
  • Final project teams due on Sat 09/26/26
Wed 09/23/26
  • Paper session 3: Long context and learned sparse attention
DeepSeek-V3.2 slides NSA slides
Main readings:
  • DeepSeek-V3.2 - Pushing the Frontier of Open LLMs, DSA sparse attention (2025) [link]
  • Native Sparse Attention (2025) [link]
Optional readings:
  • YaRN - Efficient Context Window Extension of LLMs (2023) [link]
  • Ring Attention with Blockwise Transformers (2023) [link]
  • RULER - What's the Real Context Size of Your Long-Context LMs? (2024) [link]
Mon 09/28/26
  • Paper session 4: Multimodal architectures - how images enter the LLM
Qwen2.5-VL slides Chameleon slides
Main readings:
  • Qwen2.5-VL Technical Report (2025) [link]
  • Chameleon - Mixed-Modal Early-Fusion Foundation Models (2024) [link]
Optional readings:
  • Background - CLIP (2021) [link], Flamingo (2022) [link], LLaVA (2023) [link]
  • Molmo and PixMo (2024) [link]
  • Transfusion - Predict the Next Token and Diffuse Images (2024) [link]
  • Kimi K2.5 - Visual Agentic Intelligence (2026) [link]
Wed 09/30/26
  • Theme 2 begins: Training and post-training
  • Pretraining at the frontier
  • Data curation, scaling laws in practice, annealing and midtraining, muP
Training landscape of frontier models
Main readings:
  • Training Compute-Optimal Large Language Models - Chinchilla (2022) [link]
  • The FineWeb Datasets (2024) [link]
  • The Llama 3 Herd of Models (2024, pretraining sections) [link]
Optional readings:
  • Tensor Programs V - muP (2022) [link]
  • Observational Scaling Laws (2024) [link]
Mon 10/05/26
  • Post-training, from RLHF to reasoning models
  • SFT, reward models, PPO, DPO and preference optimization, RLVR, long chain-of-thought
Slides
Main readings:
  • Training LMs to Follow Instructions - InstructGPT (2022) [link]
  • Direct Preference Optimization (2023) [link]
  • DeepSeek-R1 (2025) [link]
Optional readings:
  • Constitutional AI (2022) [link]
  • STaR - Self-Taught Reasoner (2022) [link]
  • SimPO - Simple Preference Optimization (2024) [link]
  • Tulu 3 - Pushing Frontiers in Open Post-Training (2024) [link]
  • Project proposal due on 10/05/26
Wed 10/07/26
  • A Geometric Perspective on Foundation Models
Slides
Guest lecturer:
Rex Ying, Yale University
Photo of Rex Ying 		Main readings:
  • TBD
Mon 10/12/26
  • Paper session 5: Reward design, LLM judges and agentic verifiers
Chasing the tail slides DCLM slides
Main readings:
  • Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training (2025, ICLR 2026) [link]
  • DataComp-LM: In search of the next generation of training sets for language models (2026) [link]
Optional readings:
  • The Verification Horizon: No Silver Bullet for Coding Agent Rewards (2026) [link]
  • Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains (2025, ICLR 2026) [link]
  • Online Rubrics Elicitation from Pairwise Comparisons (2025, ICML 2026) [link]
Wed 10/14/26
  • Paper session 6: Training methods, optimization and objectives
Muon slides RLP slides
Main readings:
  • Muon is Scalable for LLM Training (2025) [link]
  • RLP: Reinforcement as a Pretraining Objective (2025, ICLR 2026) [link]
Optional readings:
  • Fantastic Pretraining Optimizers and Where to Find Them (2025, ICLR 2026) [link]
  • Rethinking Language Model Scaling under Transferable Hypersphere Optimization (2026) [link]
Mon 10/19/26
  • Paper session 7: On-policy distillation
OPSD slides
Main readings:
  • Self-Distilled Reasoner - On-Policy Self-Distillation, OPSD (2026) [link]
  • GKD - On-Policy Distillation of Language Models (2023) [link]
Optional readings:
  • On-Policy Distillation (Thinking Machines Lab, 2025) [link]
  • MiniLLM - Knowledge Distillation of Large Language Models (2023) [link]
  • Nemotron 3 Ultra (2026, multi-teacher on-policy distillation sections) [link]
Wed 10/21/26
October recess - No class
  • Progress report 1 due
Mon 10/26/26
  • Learning to do RL, quickly
Guest lecturer:
Hamish Ivison, University of Washington / Meta FAIR
Photo of Hamish Ivison 		Main readings:
  • Async RLHF, https://arxiv.org/abs/2410.18252v3
  • Olmo 3, https://arxiv.org/abs/2512.13961
  • NGU, https://arxiv.org/abs/2609.13443
  • RLVE, https://arxiv.org/abs/2511.07317
Wed 10/28/26
  • Paper session 8: RL algorithms after GRPO
 Main readings:
  • Understanding R1-Zero-Like Training - Dr. GRPO (2025) [link]
  • Rethinking the Trust Region in LLM Reinforcement Learning - DPPO (2026) [link]
Optional readings:
  • GSPO - Group Sequence Policy Optimization (2025) [link]
  • The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models (2025) [link]
Mon 11/02/26
Midterm - in class, 75 minutes
  • Covers lectures and required papers through 10/26
Wed 11/04/26
  • Paper session 9: Test-time compute
 Main readings:
  • s1 - Simple Test-Time Scaling (2025) [link]
  • Scaling LLM Test-Time Compute Optimally (2024) [link]
Optional readings:
  • Let's Verify Step by Step (2023) [link]
Mon 11/09/26
  • Theme 3 begins: Agents and harnesses
  • Anatomy of an agent
  • Tool use, the agent loop, harness design, context management, memory, MCP
 Main readings:
  • ReAct - Synergizing Reasoning and Acting in Language Models (2022) [link]
  • Toolformer (2023) [link]
  • CodeAct - Executable Code Actions Elicit Better LLM Agents (2024) [link]
  • Building Effective Agents (Anthropic, 2024) [link]
Wed 11/11/26
  • Evaluating agents, and agent safety
  • Agentic benchmarks and their pitfalls, reward hacking, prompt injection
 Main readings:
  • SWE-bench - Can LMs Resolve Real-World GitHub Issues? (2023) [link]
  • tau-bench - Tool-Agent-User Interaction Benchmark (2024) [link]
  • Terminal-Bench - Benchmarking Agents on Hard, Realistic CLI Tasks (2026) [link]
  • Not What You've Signed Up For - Indirect Prompt Injection (2023) [link]
Optional readings:
  • GAIA - A Benchmark for General AI Assistants (2023) [link]
Mon 11/16/26
  • Paper session 10: Coding agents
 Main readings:
  • SWE-agent - Agent-Computer Interfaces Enable Automated SWE (2024) [link]
  • Agentless - Demystifying LLM-Based Software Engineering Agents (2024) [link]
Wed 11/18/26
  • Guest lecture on Mechanistic Interpretability, Nishant Subramani
Guest lecturer:
Nishant Subramani, CMU
Photo of Nishant Subramani 		Main readings:
  • TBD
11/21/26 - 11/29/26
Thanksgiving recess - No classes
Mon 11/30/26
  • Paper session 11: Computer and web use
 Main readings:
  • OSWorld - Benchmarking Multimodal Agents in Real Computer Environments (2024) [link]
  • UI-TARS - Pioneering Automated GUI Interaction with Native Agents (2025) [link]
  • SWE-RL - Advancing LLM Reasoning via RL on Open Software Evolution (2025) [link]
Optional readings:
  • WebArena (2023) [link]
  • Kimi K2.5 - Visual Agentic Intelligence, Agent Swarm sections (2026) [link]
  • Reflexion - Language Agents with Verbal Reinforcement Learning (2023) [link]
  • Why Do Multi-Agent LLM Systems Fail? (2025) [link]
  • MemGPT - Towards LLMs as Operating Systems (2023) [link]
  • Voyager - An Open-Ended Embodied Agent (2023) [link]
  • WebRL (2024) [link]
  • Progress report 2 due
Wed 12/02/26
  • Final project presentations (extended session, 3 hours)
  • Format will be announced
  • Final report due 12/12/26