Fall 2024 - CMPT 983 G100

Special Topics in Artificial Intelligence (3)

Generative Models

Class Number: 7479

Delivery Method: In Person

Overview

  • Course Times + Location:

    Sep 4 – Dec 3, 2024: Wed, 1:30–2:20 p.m.
    Burnaby

    Sep 4 – Dec 3, 2024: Fri, 12:30–2:20 p.m.
    Burnaby

Description

COURSE DETAILS:

This course covers the fundamentals and applications of generative models, a branch of machine learning focused on learning unknown probability distributions from observed examples. Generative models are used to automatically generate complex data such as images, text and sound from limited user input, simulate alternative possible outcomes that are not observed in the real world, generate multiple possible predictions when the input cannot uniquely determine the output, quantify the amount of uncertainty in the model prediction and incorporate domain knowledge into otherwise uninformed domain-agnostic algorithms. Both classical approaches and modern techniques developed within the last 10 years will be covered, and their applications to different areas of artificial intelligence, such as computer vision, natural language processing and audio processing will be highlighted. The goal is to provide students with a comprehensive understanding of the latest techniques and bring them up to speed on the current scientific literature. By the end of the course, students will understand when generative models should be applied and how they can be applied in the context of their own research.

COURSE-LEVEL EDUCATIONAL GOALS:

Topics

  • Mathematical foundations, i.e., maximum entropy, maximum likelihood, variational inference, stochastic processes
  • Prescribed generative models, e.g.: latent variable models, variational autoencoders (VAEs)
  • Implicit generative models, e.g.: generative adversarial networks (GANs), implicit maximum likelihood estimation (IMLE)
  • Specially parameterized generative models, e.g.: autoregressive models, diffusion models

Grading

NOTES:

The course grade will be based on quizzes, scribing, participation and a final project.

REQUIREMENTS:

We will assume knowledge of the basics of machine learning (at the level of CMPT 726 or equivalent) and its mathematical foundations, namely probability, linear algebra, multivariate calculus and optimization. We recommend going over this resource for students who would like to brush up on mathematical foundations: http://gwthomas.github.io/docs/math4ml.pdf. Additionally, students should be familiar with programming in Python, using Python packages such as NumPy and PyTorch, and typesetting with LaTeX.

Materials

REQUIRED READING NOTES:

Your personalized Course Material list, including digital and physical textbooks, are available through the SFU Bookstore website by simply entering your Computing ID at: shop.sfu.ca/course-materials/my-personalized-course-materials.

Graduate Studies Notes:

Important dates and deadlines for graduate students are found here: http://www.sfu.ca/dean-gradstudies/current/important_dates/guidelines.html. The deadline to drop a course with a 100% refund is the end of week 2. The deadline to drop with no notation on your transcript is the end of week 3.

Registrar Notes:

ACADEMIC INTEGRITY: YOUR WORK, YOUR SUCCESS

SFU’s Academic Integrity website http://www.sfu.ca/students/academicintegrity.html is filled with information on what is meant by academic dishonesty, where you can find resources to help with your studies and the consequences of cheating. Check out the site for more information and videos that help explain the issues in plain English.

Each student is responsible for his or her conduct as it affects the university community. Academic dishonesty, in whatever form, is ultimately destructive of the values of the university. Furthermore, it is unfair and discouraging to the majority of students who pursue their studies honestly. Scholarly integrity is required of all members of the university. http://www.sfu.ca/policies/gazette/student/s10-01.html

RELIGIOUS ACCOMMODATION

Students with a faith background who may need accommodations during the term are encouraged to assess their needs as soon as possible and review the Multifaith religious accommodations website. The page outlines ways they begin working toward an accommodation and ensure solutions can be reached in a timely fashion.