Overview

Description

CALENDAR DESCRIPTION:

Machine Learning is the study of computer algorithms that improve automatically through experience. Provides students who conduct research in machine learning, or use it in their research, with a grounding in both the theoretical justification for, and practical application of, machine learning algorithms. Covers techniques in supervised and unsupervised learning, the graphical model formalism, and algorithms for combining models. Students who have taken CMPT 882 (Machine Learning) in 2007 or earlier may not take CMPT 726 for further credit.

COURSE DETAILS:

Machine Learning is the study of computer algorithms that improve automatically through experience. Machine learning algorithms play an important role in industrial applications and commercial data analysis. The goal of this course is to present students with both the theoretical justification for, and practical application of, machine learning algorithms. At the end of the course, students should:
* Be familiar with the strengths and weaknesses of important statistical models (e.g. Gaussians, neural networks, logistic regression)
* Understand the applications and use of general computational-statistical methods such as the EM algorithm and maximum likelihood estimation
* Know the main methods for solving the key problems that arise for these models, including:
   * Model selection: select a model based on data
   * Parameter estimation: use data to assign values to adaptive parameters
   * Inference: use a parametrized model to derive probabilistic predictions.

No previous knowledge of pattern recognition or machine learning concepts is assumed, but students are expected to have, or obtain, background knowledge in mathematics and statistics.

Topics

• Graphical models: directed graphs
• Temporal models and algorithms: hidden Markov Models, Kalman filtering
• Classification: nearest neighbour, linear models, decision trees, naive Bayes, neural nets
• Regression: linear regression, logistic regressin, regularization, neural nets
• Unsupervised learning: kmeans, Gaussian mixtures
• Latent variable models: Principal Components, Expectation-maximization
• Ensemble learning, Boosting
• Theory: Maximum Likelihood, Bias-Variance, time permitting: consistency
• Time permitting: learning to act, reinforcement learning

Grading

Materials

MATERIALS + SUPPLIES:

Reference Books

• Machine Learning, Tom Mitchell, McGraw Hill, 1997, 9780070428072
• Pattern Classification (2nd ed.), Richard O. Duda, Peter E. Hart, and David G. Stork, Wiley Interscience, 2000, 9780471056690
• All of Statistics, Larry Wasserman, Springer, 2009, 9780387402727
• Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer, 2006, 9780387310732, available on-line at http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop - Pattern Recognition And Machine Learning - Springer 2006.pdf

REQUIRED READING: