CAVEAT:
- This is a theory course.
- There will not be coding assignments.
- This course requires reading and writing mathematical proofs.

This is a course on "optimization for machine learning." Classic optimization theories focus on specific optimization problem templates, such as linear programs and semidefinite programs, and typically overlook the computational complexities of optimization algorithms. Modern machine learning applications, however, require solving a variety of optimization problems that do not obviously fit in the optimization problem templates, and require the computational complexity of an optimization algorithm to scale with respect to the data size and dimension. In this course, we adopt the "black-box approach" to optimization that aims to develop optimization algorithms for a class of optimization problems, and focus on "first-order optimization algorithms" that efficiently solve optimization problems defined on big and high-dimensional datasets.

The algorithms we plan to study in this course are: gradient descent, mirror descent, proximal gradient methods, and the Frank-Wolfe method.

====

[Course registration information]
This is a "type-3" course ( 第三類加簽 ). Please try your luck during the "online course add period".