課程資訊
課程名稱
最佳化演算法
Optimization Algorithms 
開課學期
110-2 
授課對象
電機資訊學院  資訊工程學研究所  
授課教師
李彥寰 
課號
CSIE5410 
課程識別碼
922 U4500 
班次
 
學分
3.0 
全/半年
半年 
必/選修
選修 
上課時間
星期一8,9,10(15:30~18:20) 
上課地點
資105 
備註
總人數上限:30人 
 
課程簡介影片
 
核心能力關聯
核心能力與課程規劃關聯圖
課程大綱
為確保您我的權利,請尊重智慧財產權及不得非法影印
課程概述

CAVEAT:
- This is a theory course.
- There will not be coding assignments.
- This course requires reading and writing mathematical proofs.

This is a course on "optimization for machine learning." Classic optimization theories focus on specific optimization problem templates, such as linear programs and semidefinite programs, and typically overlook the computational complexities of optimization algorithms. Modern machine learning applications, however, require solving a variety of optimization problems that do not obviously fit in the optimization problem templates, and require the computational complexity of an optimization algorithm to scale with respect to the data size and dimension. In this course, we adopt the "black-box approach" to optimization that aims to develop optimization algorithms for a class of optimization problems, and focus on "first-order optimization algorithms" that efficiently solve optimization problems defined on big and high-dimensional datasets.

The algorithms we plan to study in this course are: gradient descent, mirror descent, proximal gradient methods, and the Frank-Wolfe method.

====

[Course registration information]
This is a "type-3" course ( 第三類加簽 ). Please try your luck during the "online course add period".  

課程目標
After taking this course, the students are expected to
1) understand *precisely* how and why standard first-order optimization algorithms work,
2) work out basic convergence analyses of optimization algorithms, and
3) be able to read research literature on optimization theory. 
課程要求
Familiarity with (multivariate) calculus, linear algebra, and probability theory and math maturity are required. Knowledge of convex analysis and machine learning can be helpful but is not necessary. 
預期每週課後學習時數
 
Office Hours
備註: TBD. 
參考書目
The order is alphabetical.

- A. Beck. First-Order Methods in Optimization. 2017.
- S. Bubeck. Convex Optimization: Algorithms and Complexity. 2015.
- Books and lecture notes by A. Nemirovski. (https://www2.isye.gatech.edu/~nemirovs/)
- Yu. Nesterov. Lectures on Convex Optimization. 2018.
- S. Shalev-Shwartz. Online Learning and Online Convex Optimization. 2011. 
指定閱讀
待補 
評量方式
(僅供參考)
 
No.
項目
百分比
說明
1. 
Final project 
40% 
Survey of a theoretic topic in optimization and/or novel research results. 
2. 
Homework 
60% 
At least three homework assignments. 
 
課程進度
週次
日期
單元主題