MATH 309
Continuous Optimization

Fall 2019


INSTRUCTOR: John Stockie
Office: K10518
Phone: 778-782-3553
E-mail: stockie  [at]  math.sfu.ca
Web: http://www.sfu.ca/~jstockie/math309/
CLASS TIMES: M      9:30-10:20   WMC 3210 
WF    9:30-10:20   C 9000 
MY OFFICE HOURS: Wednesdays    10:30-12:00
(If you can't see me during this time, then please email me to make an appointment)
TUTORIALS: Each of you is assigned to a tutorial section which is run by your TA:

Aven Bross (email: abross [at] sfu.ca)
Tuesday - 12:30-1:20   WMC 2830   (D101)
Tuesday - 1:30-2:20   WMC 2830   (D102)

I strongly encourage you to attend your scheduled tutorial session. Your TA is available to provide help with homework questions or explaining material from class. He will also review homework and midterm solutions if asked. When needed, your TA may provide additional illustrative examples or supplementary material to complement what is presented in lectures.
PREREQUISITES: MATH 240 or 232, and MATH 251. Recommended: MATH 308.
TEXTBOOK: The required textbook for this course is:
    Numerical Optimization, 2nd edition, by J. Nocedal and S.J. Wright (Springer, 2006)

This book has an electronic version that you can access directly through the SFU Library (just click on the link above). Another excellent book that I recommend for extra reading is:
     Nonlinear Optimization with Engineering Applications, by M. Bartholomew-Biggs (Springer, 2008)

 I have placed hard-copies of both books on reserve in the library and both can also be conveniently accessed as e-books through the "Library Reserves" tab in Canvas.
LECTURES:
Attendance in lectures is expected and strongly recommended. In the event that you miss a class, it is your responsibility to obtain the material from another student. Textbook readings or partial lecture notes will regularly be assigned before lectures, and it is absolutely vital that you read this material in advance so that you are better prepared to understand new concepts and to ask targeted questions about material that may not be clear to you.
HOMEWORK:
Homework will be assigned bi-weekly and you will have approximately two weeks to complete each assignment. Some homework questions will be assigned from the textbook, with the remaining problems taken from other sources. Homework is due on Fridays at 12:00 noon and should be deposited in the appropriate locked submission box outside the Applied Calculus Workshop (K9503, one floor below the Math Department). Any late assignment will receive a mark of zero, with no exceptions. To most easily deal with unforeseen illness or other problems, I will drop every student's lowest homework mark when calculating the final homework grade.
Some computer programming will be required on assignments. The language I will use in class is Matlab, which is available to all students in university computing labs and can also be installed for home use. Although you are expected to be able to write your own code from scratch, the length of the code required will actually be quite short. To help you to familiarize yourself with Matlab, I will provide many samples of Matlab code from examples in class and many assignment questions will involve modifying an existing Matlab code that is provided to you.
TESTS: There will be one midterm test held in class on Wednesday October 23. No makeup test will be given, so if you miss the midterm for a documented medical reason then the midterm portion of your grade will be transferred to the final exam. The final exam is scheduled for Wednesday December 4 at 12:00pm and will cover all material in the course.
ACADEMIC INTEGRITY:
Academic dishonesty has no place in a university and I have zero tolerance for it. All students must understand the meaning and consequences of cheating, plagiarism and other academic offences identified under the SFU Student Academic Integrity Policy. Cheating includes, but is not limited to:
  • Handing in assignment solutions copied from other sources such as solution manuals, other students' work, on-line sources, etc.
  • Using calculators or unauthorized reference materials during tests or examinations, unless they are explicitly allowed.
  • Looking at the work of other students during examinations.
In all of these cases, all students involved in the act will receive a mark of zero for the entire work in question. The Chair of the Mathematics Department will be notified and the incident will be noted in your official academic record. Further action may also be taken as outlined in the SFU Student Academic Integrity Procedures.
OUTLINE: (below is the official outline for Fall 2019, which may differ slightly from what is posted elsewhere on other SFU web pages)
This course covers elementary aspects of the theory, numerical algorithms, and practical applications of convex optimization, which refers to finding maxima/minima of a function of several real variables with or without (in)equality constraints. Examples will be drawn from applications in scheduling and assignment, image reconstruction, signal processing, economics, and engineering design. An essential component of this course will be understanding optimization algorithms (mostly implemented in Matlab) and using them to solve real problems.

Below is a rough outline of topics to be covered along with pointers to relevant sections of the textbook. Additional examples and material will be added as required from the text and other sources.

  1. Review of basic concepts:
    • Elements of multivariable calculus, analysis and topology (App. A)
    • Elements of linear algebra (App. A)
    • Introduction to Matlab programming

  2. Unconstrained optimization:
    • Necessary and sufficient conditions for an optimum (Chs. 1-2)
    • Line search methods (Ch. 3)
    • Conjugate gradient methods (Ch. 5)
    • Newton-type methods (Sec. 7.1)
    • Quasi-Newton methods (Chs. 6-7)

  3. Constrained optimization:
    • First- and second-order optimality (KKT) conditions (Ch. 12)
    • Quadratic programming (Ch. 16)
    • Penalty, barrier and augmented-Lagrangian methods (Ch. 17)
    • Sequential quadratic programming (Ch. 18)

MARKING SCHEME:
   Assignments (bi-weekly, best 5/6):   25%
Midterm test (Wed Oct 23):   25%
Final examination (Wed Dec 4):    50%
Students requesting any special accommodations for reasons of disability, religion, etc. MUST inform me during the first week of the semester.


Last modified: Mon Apr 20 2020