Welcome to CPSC 477/577 – Natural Language Processing!

This year, we have updated the syllabus and we’ll focus heavily on Language Modeling and recent advances in the field. The curriculum spans both foundational concepts and cutting-edge developments in the field including Large Language Models (LLMs). The course begins with core neural network concepts in NLP, covering word embeddings, sequence modeling, and attention mechanisms. Students will gain a strong understanding of these building blocks while learning their practical implementations. Building on these foundations, we explore transformer architectures and their evolution, including landmark language models such as GPT. The course examines how these models enable sophisticated language understanding and generation through pre-training and transfer learning. The latter portion covers contemporary advances: LLMs, parameter-efficient fine-tuning, post-training, and efficiency techniques. We’ll analyze the capabilities and limitations of current systems while discussing emerging research directions.

Prerequisites:

Intro to ML or Intro to AI are required.

FAQ:

Q: Can I take this course?
You should have taken one of the above courses. If you have not, please consult the instructor before enrolling.
This course requires:
  • Strong programming skills in Python and basic prior exposure to libraries such as numpy, PyTorch, or TensorFlow
  • Familiarity with basic concepts in probability and statistics, and linear algebra
  • Prior exposure to foundational machine learning concepts through courses like CPSC 381/581 (Intro to Machine Learning) or CPSC 370/570 (Artificial Intelligence)
Q: I have equivalent experience but haven’t taken the prereq courses. Can I enroll?
Please contact the instructor with: Relevant coursework/grades and your programming experience (including ML projects) and math background
Q: I may miss several lectures. Can I still take the course?
Regular attendance is mandatory, and we do not guarantee lecture recordings. Missing more than a few classes significantly impacts learning outcomes and participation grade. If you anticipate scheduling conflicts, consider taking the course in a future semester.

Important Note: We strongly advise against taking this course if you do not meet the prerequisites.

Resources

  • Dan Jurafsky and James H. Martin. Speech and Language Processing (2024 pre-release)
  • Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing

We will also using papers from major conferences in the field including ACL, EMNLP, NAACL, ICLR, NeurIPS, etc.