Overview

Description

COURSE DETAILS:

Prerequisites note:
There are no formal prerequisites for this class. However, you should have solid knowledge and skills in Machine Learning. Familiarity with Deep Learning is preferred.

This course is a graduate-level, seminar-oriented research course covering the topics of causal discovery, causal inference, and transfer learning. Causal discovery is the task of discovering the causal relationships between a set of variables, in the form of a Directed Acyclic Graph (or Structural Causal Model), from observational data alone, i.e. without access to experimental data. Causal inference is a related task, which assumes knowledge of the causal factors and estimates the strength of the effect of the causal variables (treatments) on an outcome variable. Typical applications of causal discovery and inference are for the evaluation of medical treatments or public policies. Transfer learning is the task of transferring a machine learning model from a source domain to a target domain and is commonly used if only a small or unlabelled dataset is available for the target domain. The general approach is to pre-train the machine learning model on the source domain and to fine-tune it on the target domain. Transfer learning plays an important role in the context of foundation models and large language models, which are pre-trained, usually in an unsupervised manner, on a huge source domain dataset and fine-tuned on the target dataset. There are strong connections between the three course topics, since causal models are expected to transfer better between domains than models based on correlations.

This course will teach you the general research methodology in the area of machine learning and more specifically introduce you to the forefront of research in the causal discovery and inference as well as transfer learning. We will begin with a series of lectures by the instructor, providing the foundations for the class topics, followed by tutorials on research communication in written and oral form. Students will then take turns presenting recent research papers, discussing the contributions, limitations and directions for future research. Students will also work on a research project during the course, leading up to final presentation and written report.

Topics

•   Structural causal models (SCMs)
•   Discrete and continuous algorithms for the discovery of SCMs
•   Machine learning methods for causal inference
•   Individual and average treatment effect estimation
•   Types of transfer learning: domain adaptation, domain generalization, etc.
•   Unsupervised pre-training methods
•   Supervised fine-tuning methods
•   Continual learning

Outline

• Tutorial introductions to causal discovery, causal inference, and transfer learning by the instructor
• Tutorials on paper writing and research presentations by the instructor
• Research paper presentations by the students
• Project presentations by the students
• Submission of project reports

Materials

REQUIRED READING NOTES: