Eric Moulines will be visiting the Heilbronn Institute of Mathematical Research as a Data Science Visitor from 27th January- 31st January. During his stay Eric will be delivering a series of lectures on Convex optimization for machine learning.
Tuesday 28th January 11:00- 12:00 G.09, The Fry Building
Thursday 30th January 13:00- 14:00, G.10, The Fry Building
Friday 31st January 10:00- 11:00, G.09, The Fry Building
Title: Convex optimization for machine learning
Abstract: The purpose of this course is to give an introduction to convex optimization and its applications in statistical learning.
In the first part of the course, I will recall the importance of convex optimisation in statistical learning. I will briefly introduce some useful results of convex analysis. I will then analyse gradient descent algorithms for strongly convex and then convex smooth functions. I will take this opportunity to establish some results on complexity lower bounds for such problems. I will show that the gradient descent algorithm is suboptimal and does not reach the optimal possible speed of convergence. I will the present a strategy to accelerate gradient descent algorithms in order to obtain optimal speeds.
In the second part of the course, I will focus on non smooth optimisation problems. I we will introduce the proximal operator of which I will establish some essential properties. I will then study the proximal gradient algorithms and their accelerated versions.
In a third part, I will look at stochastic versions of these algorithms.