Optimization has become a prevalent tool in modern algorithm design. In this course, we will study optimization algorithms from the perspective of machine learning theory. We will focus on the black-box approach; that is, we consider accessing information about the function to be optimized via an oracle, and derive non-asymptotic worst-case oracle complexity bounds with respect to a given class of objective functions.

This course consists of two parts. The first part introduces basic notions in convex analysis, and standard convex optimization algorithms. The second part focuses on online optimization algorithms, which allow one to do model-free sequential predictions, and can be computationally efficient when the data size is large.