課程資訊
課程名稱
最佳化演算法
Optimization Algorithms 
開課學期
109-1 
授課對象
理學院  數學系  
授課教師
李彥寰 
課號
CSIE5410 
課程識別碼
922 U4500 
班次
 
學分
3.0 
全/半年
半年 
必/選修
選修 
上課時間
星期二7,8,9(14:20~17:20) 
上課地點
新504 
備註
總人數上限:40人 
 
課程簡介影片
 
核心能力關聯
核心能力與課程規劃關聯圖
課程大綱
為確保您我的權利,請尊重智慧財產權及不得非法影印
課程概述

Caveat: This is a theory course. ALL of the homework and exam problems ask you to work out mathematical proofs.

This is a course on optimization for machine learning. Therefore, we will focus on the black-box approach and first-order methods. We will NOT study classical topics such as linear programming and semi-definite programming.

This course consists of two parts. The first part introduces basic notions in convex analysis and standard first-order convex optimization algorithms. The second part focuses on solving online optimization and minimax problems. Below is a tentative list of topics.
- Basic convex analysis I: Convexity, strong convexity, Lipschitzness, smoothness, optimality condition, etc.
- Gradient descent.
- Mirror descent.
- Acceleration.
- Proximal methods.
- Smoothing.
- Frank-Wolfe method.
- Basic convex analysis II: Subgradient & subdifferential.
- Mirror descent revisited.
- Online mirror descent.
- Learning in games.
- Follow the regularized leader.
- Optimistic methods.  

課程目標
After taking this course, the students are expected to have the ability to:
.Characterize the pros and cons of an optimization algorithm.
.Choose an appropriate optimization algorithm for a given application scenario.
.Do basic complexity analyses of optimization algorithms.
.Read advanced literature in optimization and algorithm design. 
課程要求
Prerequisites: Multivariate calculus, linear algebra, and probability.

Knowledge in convex analysis, statistics, and/or machine learning are helpful but not necessary. 
預期每週課後學習時數
 
Office Hours
每週五 16:30~18:00
每週二 17:20~21:00 備註: 1. Yen-Huan's office hour: After class every Tuesday or by appointment. 2. TA's office hour: 16h30--18h00 every Friday @ Lab 407, CSIE-Der Tian Hall.  
參考書目
1. Yu. Nesterov. 2004. Introductory Lectures on Convex Optimization.
2. S. Bubeck. 2015. Convex Optimization: Algorithms and Complexity. (Available online: http://sbubeck.com/Bubeck15.pdf)
3. A. Ben-Tal and A. Nemirovski. 2015. Lectures on Modern Convex Optimization.(Available online: http://www2.isye.gatech.edu/~nemirovs/LMCO_LN.pdf)
4. N. Cesa-Bianchi and G. Lugosi. 2006. Prediction, Learning, and Games.
5. S. Bubeck. 2011. Introduction to Online Optimization. (Available online: http://sbubeck.com/BubeckLectureNotes.pdf)
6. S. Shalev-Shwartz. 2012. Online Learning and Online Convex Optimization.
7. E. Hazan. 2016. Introduction to Online Convex Optimization. (Available online: http://ocobook.cs.princeton.edu/) 
指定閱讀
待補 
評量方式
(僅供參考)
   
課程進度
週次
日期
單元主題