|
課程名稱 |
最佳化演算法
Optimization Algorithms |
|
開課學期 |
109-1 |
|
授課對象 |
理學院 應用數學科學研究所 |
|
授課教師 |
李彥寰 |
|
課號 |
CSIE5410 |
|
課程識別碼 |
922 U4500 |
|
班次 |
|
|
學分 |
3.0 |
|
全/半年 |
半年 |
|
必/選修 |
選修 |
|
上課時間 |
星期二7,8,9(14:20~17:20) |
|
上課地點 |
新504 |
|
備註 |
總人數上限：40人 |
|
|
|
|
課程簡介影片 |
|
|
核心能力關聯 |
核心能力與課程規劃關聯圖 |
|
課程大綱
|
|
為確保您我的權利,請尊重智慧財產權及不得非法影印
|
|
課程概述 |
Caveat: This is a theory course. ALL of the homework and exam problems ask you to work out mathematical proofs.
This is a course on optimization for machine learning. Therefore, we will focus on the black-box approach and first-order methods. We will NOT study classical topics such as linear programming and semi-definite programming.
This course consists of two parts. The first part introduces basic notions in convex analysis and standard first-order convex optimization algorithms. The second part focuses on solving online optimization and minimax problems. Below is a tentative list of topics.
- Basic convex analysis I: Convexity, strong convexity, Lipschitzness, smoothness, optimality condition, etc.
- Gradient descent.
- Mirror descent.
- Acceleration.
- Proximal methods.
- Smoothing.
- Frank-Wolfe method.
- Basic convex analysis II: Subgradient & subdifferential.
- Mirror descent revisited.
- Online mirror descent.
- Learning in games.
- Follow the regularized leader.
- Optimistic methods. |
|
課程目標 |
After taking this course, the students are expected to have the ability to:
．Characterize the pros and cons of an optimization algorithm.
．Choose an appropriate optimization algorithm for a given application scenario.
．Do basic complexity analyses of optimization algorithms.
．Read advanced literature in optimization and algorithm design. |
|
課程要求 |
Prerequisites: Multivariate calculus, linear algebra, and probability.
Knowledge in convex analysis, statistics, and/or machine learning are helpful but not necessary. |
|
預期每週課後學習時數 |
|
|
Office Hours |
每週五 16:30~18:00
每週二 17:20~21:00 備註： 1. Yen-Huan's office hour: After class every Tuesday or by appointment.
2. TA's office hour: 16h30--18h00 every Friday @ Lab 407, CSIE-Der Tian Hall. |
|
指定閱讀 |
待補 |
|
參考書目 |
1. Yu. Nesterov. 2004. Introductory Lectures on Convex Optimization.
2. S. Bubeck. 2015. Convex Optimization: Algorithms and Complexity. (Available online: http://sbubeck.com/Bubeck15.pdf)
3. A. Ben-Tal and A. Nemirovski. 2015. Lectures on Modern Convex Optimization.(Available online: http://www2.isye.gatech.edu/~nemirovs/LMCO_LN.pdf)
4. N. Cesa-Bianchi and G. Lugosi. 2006. Prediction, Learning, and Games.
5. S. Bubeck. 2011. Introduction to Online Optimization. (Available online: http://sbubeck.com/BubeckLectureNotes.pdf)
6. S. Shalev-Shwartz. 2012. Online Learning and Online Convex Optimization.
7. E. Hazan. 2016. Introduction to Online Convex Optimization. (Available online: http://ocobook.cs.princeton.edu/) |
|
評量方式
(僅供參考) |
|
|