課程資訊

Optimization Algorithms

110-1

CSIE5410

922 U4500

3.0

[Course registration information]

This is a "type-3" course ( 第三類加簽 ). Please try your luck during the "online course add period". If you are unlucky but want to take this course, please fill in the form on https://bit.ly/2Vzk2Iq and we can do "manual course add" ( 人工加簽 ) as long as the total number of students is below 50.

Lecture recordings will be provided during the whole semester. Subission of homework and project reports will be online.

====

CAVEAT:
- This is a theory course.
- There will not be coding assignments.
- This course requires reading and writing mathematical proofs.

This is a course on "optimization for machine learning." Classic optimization theories focus on specific optimization problem templates, such as linear programs and semidefinite programs, and typically overlook the computational complexities of optimization algorithms. Modern machine learning applications, however, require solving a variety of optimization problems that do not obviously fit in the optimization problem templates, and require the computational complexity of an optimization algorithm to scale with respect to the data size and dimension. In this course, we adopt the "black-box approach" to optimization that aims to develop optimization algorithms for a class of optimization problems, and focus on "first-order optimization algorithms" that efficiently solve optimization problems defined on big and high-dimensional datasets.

After taking this course, the students are expected to
1) understand *precisely* how and why standard first-order optimization algorithms work,
2) work out basic convergence analyses of optimization algorithms, and
2) be able to read research literature on optimization.

Familiarity with (multivariate) calculus, linear algebra, and probability theory and math maturity are required. Knowledge of convex analysis and machine learning can be helpful but is not necessary.

Office Hours

The order is alphabetical.

- A. Beck. First-Order Methods in Optimization. 2017.
- S. Bubeck. Convex Optimization: Algorithms and Complexity. 2015.
- Books and lecture notes by A. Nemirovski. (https://www2.isye.gatech.edu/~nemirovs/)
- Yu. Nesterov. Lectures on Convex Optimization. 2018.
- S. Shalev-Shwartz. Online Learning and Online Convex Optimization. 2011.

Lecture slides and notes by the instructor.

(僅供參考)

 No. 項目 百分比 說明 1. Homework 60% At least three homework assignments. 2. Final project 40% Survey of a theoretic topic in optimization and/or novel research results.

 課程進度
 週次 日期 單元主題 無資料