Welcome to CPSC 477/577!

This course provides a deep dive into modern Natural Language Processing (NLP), focusing on deep learning approaches and their applications. The curriculum spans both foundational concepts and cutting-edge developments in the field including Large Language Models (LLMs). The course begins with core neural network concepts in NLP, covering word embeddings, sequence modeling, and attention mechanisms. Students will gain a strong understanding of these building blocks while learning their practical implementations. Building on these foundations, we explore transformer architectures and their evolution, including landmark models like BERT, GPT, and T5. The course examines how these models enable sophisticated language understanding and generation through pre-training and transfer learning. The latter portion covers contemporary advances: LLMs, parameter-efficient fine-tuning, and efficiency techniques. We’ll analyze the capabilities and limitations of current systems while discussing emerging research directions. Through lectures, hands-on assignments, and projects, students will gain both theoretical understanding and practical experience implementing modern NLP systems. The course emphasizes reproducible experimentation and real-world applications.

Prerequisites:

Intro to ML or Intro to AI are required.

FAQ: 

Q: Can I take this course?

*A: You should have taken one of the above courses. If you have not, please consult the instructor before enrolling.*

This course requires: 
    - Strong programming skills in Python and prior exposure to libraries such as numpy, PyTorch, or TensorFlow
    - Familiarity with probability and statistics, and linear algebra
    - Prior exposure to machine learning concepts through courses like CPSC 381/581 (Intro to Machine Learning) or CPSC 370/570 (Artificial Intelligence)

Q: I have equivalent experience but haven't taken the prereq courses. Can I enroll?

Contact the instructor with: Relevant coursework/grades and your programming experience (including ML projects) and math background

Q: I may miss several lectures. Can I still take the course?

Regular attendance is mandatory, and we do not guarantee lecture recordings. 
Missing more than a few classes significantly impacts learning outcomes and participation grade. 
If you anticipate scheduling conflicts, consider taking the course in a future semester.

Important Note: We strongly advise against taking this course if you do not meet the prerequisites.

Resources

  • Dan Jurafsky and James H. Martin. Speech and Language Processing (2024 pre-release)
  • Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing

We will also using papers from major conferences in the field including ACL, EMNLP, NAACL, ICLR, NeurIPS, etc.

  • Lectures: Tue/Thur 2:30PM - 3:45PM
  • Lecture Location: TBD
  • Office Hours Location: 17HH #326
  • Discussion: Ed Discussions