Distributed Statistical Learning via Refitting Bootstrap Samples

报告题目：Distributed Statistical Learning via Refitting Bootstrap Samples

报告人： 竺紫威（University of Michigan）

时间：2023年04月17日（星期一）下午13:30-3:00

地点：紫金港校区海纳苑2幢205教室

摘要：In this talk, I will introduce a one-shot distributed learning algorithm via refitting Bootstrap samples, which we refer to as ReBoot. Given the local models that are fit on multiple independent subsamples, ReBoot refits a new model on the union of the Bootstrap samples drawn from these local models. The whole procedure requires only one round of communication of model parameters. Theoretically, we analyze the statistical rate of ReBoot for generalized linear models (GLM) and noisy phase retrieval, which represent convex and non-convex problems respectively. In both cases, ReBoot provably achieves the full-sample statistical rate whenever the subsample size is not too small. In particular, we show that the systematic bias of ReBoot, i.e., the error that is independent of the number of subsamples, is O(n ^ -2) in GLM, where n is the subsample size. This rate is sharper than that of model parameter averaging and its variants, implying the higher tolerance of ReBoot with respect to data splits to maintain the full-sample rate. Simulation study demonstrates the statistical advantage of ReBoot over competing methods including averaging and CSL (Communication-efficient Surrogate Likelihood) with one round of gradient communication. Finally, we propose FedReBoot, an iterative version of ReBoot, to aggregate convolutional neural networks for image classification, which exhibits substantial superiority over FedAvg within early rounds of communication.

欢迎参加！

联系人: 张朋pengz@zju.edu.cn

报告人简介：竺紫威博士于 2019 年至 2022 年在密歇根大学安娜堡分校 (UMich) 担任统计学助理教授。在加入 UMich 之前，他于 2018 年至 2019 年在剑桥大学统计实验室担任助理研究员，与Richard J. Samworth 教授一起工作。师从范建青教授，在普林斯顿大学运筹与金融工程系 (ORFE)获得了博士学位。他的研究兴趣包括联邦/分布式统计学习、高维统计、鲁棒统计和缺失数据。