Learning Theory of Stochastic Gradient Descent

Speaker:Dr. Yunwen Lei(University of Birmingham)

Date:November 5 Fri. 3:00pm -- 4:30pm

Venue: ZOOM ID: 158 874 395  Password:310027

Abstract:Stochastic gradient descent (SGD) has become the workhorse behind many machine learning problems. Optimization and sampling errors are two contradictory factors responsible for the statistical behavior of SGD. In this talk, we report our generalization analysis of SGD by considering simultaneously the optimization and sampling errors. We remove some restrictive assumptions in the literature and significantly improve the existing generalization bounds. Our results help to understand how to stop SGD early to get a best statistical behavior.