Simon Fraser University
Engaging the World

Course Outlines

Spring 2022 - CMPT 983 G100

Special Topics in Artificial Intelligence (3)

Grounded Natural Language Understanding

Class Number: 5549

Delivery Method: In Person

Overview

  • Course Times + Location:

    Jan 10 – Apr 11, 2022: Mon, 10:30–11:50 a.m.
    Burnaby

    Jan 10 – Apr 11, 2022: Wed, 10:30–11:50 a.m.
    Burnaby

Description

COURSE DETAILS:

A seminar-oriented course covering topics at the intersection of language, vision, graphics, and robotics. The class focuses on the grounding of language to various representations and modalities. For topics covered in Spring 2021 (see https://angelxuanchang.github.io/cmpt983-sp2021/). The format of the class will be a mix of lectures and student-led presentations. Students are expected to have prior experience with deep learning concepts and framework (Pytorch, Tensorflow, etc), and should also have familiarity with one of the areas of natural language processing, vision, graphics or robotics. Each week, students will read papers in a particular area of language grounding, and discuss the contributions, limitations and interconnections between the papers. Students will also work on a research project during the course, culminating in a final presentation and written report. The course aims to provide practical experience in comprehending, analyzing and synthesizing research in grounded natural language understanding. Note: This course is NOT an introductory course to natural language processing. If you are interested in learning about natural language processing, CMPT 413/713 is offered in the fall.

Topics

  • Multimodal embeddings and contrastive learning
  • Pre-training for multimodal grounding
  • Grounding of language to machine interpretable programs (semantic parsing)
  • Visual grounding of language and tasks (captioning, VQA models, referring expressions)
  • Interpretation of language commands for embodied navigation and interaction
  • Interactive language learning through language games and dialogue
  • Grounded knowledge representations for mapping language to the 3D world
  • Generative models for content creation from text
  • Grounded language acquisition and understanding

Grading

NOTES:

Based on paper critiques, presentations, and class project.

Graduate Studies Notes:

Important dates and deadlines for graduate students are found here: http://www.sfu.ca/dean-gradstudies/current/important_dates/guidelines.html. The deadline to drop a course with a 100% refund is the end of week 2. The deadline to drop with no notation on your transcript is the end of week 3.

Registrar Notes:

ACADEMIC INTEGRITY: YOUR WORK, YOUR SUCCESS

SFU’s Academic Integrity web site http://www.sfu.ca/students/academicintegrity.html is filled with information on what is meant by academic dishonesty, where you can find resources to help with your studies and the consequences of cheating.  Check out the site for more information and videos that help explain the issues in plain English.

Each student is responsible for his or her conduct as it affects the University community.  Academic dishonesty, in whatever form, is ultimately destructive of the values of the University. Furthermore, it is unfair and discouraging to the majority of students who pursue their studies honestly. Scholarly integrity is required of all members of the University. http://www.sfu.ca/policies/gazette/student/s10-01.html

TEACHING AT SFU IN SPRING 2022

Teaching at SFU in spring 2022 will involve primarily in-person instruction, with safety plans in place.  Some courses will still be offered through remote methods, and if so, this will be clearly identified in the schedule of classes.  You will also know at enrollment whether remote course components will be “live” (synchronous) or at your own pace (asynchronous).

Enrolling in a course acknowledges that you are able to attend in whatever format is required.  You should not enroll in a course that is in-person if you are not able to return to campus, and should be aware that remote study may entail different modes of learning, interaction with your instructor, and ways of getting feedback on your work than may be the case for in-person classes.

Students with hidden or visible disabilities who may need class or exam accommodations, including in the context of remote learning, are advised to register with the SFU Centre for Accessible Learning (caladmin@sfu.ca or 778-782-3112) as early as possible in order to prepare for the spring 2022 term.