Soft-Sign Stochastic Gradient Descent Algorithm for Wireless Federated Learning

Seunghoon Lee; Chanho Park; Songnam Hong; Yonina C Eldar; Namyoon Lee

doi:10.1109/SPAWC51858.2021.9593212

Back

Conference paper

Soft-Sign Stochastic Gradient Descent Algorithm for Wireless Federated Learning

Seunghoon Lee, Chanho Park, Songnam Hong, Yonina C Eldar and Namyoon Lee

2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Vol.2021-, pp.241-245

IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 22 ( Lucca, Italy , 27/Sep/2021 - 30/Sep/2021)

15/Nov/2021

DOI: https://doi.org/10.1109/SPAWC51858.2021.9593212

Files and links (1)

url

doi.org/10.1109/SPAWC51858.2021.9593212View

Published (Version of record)

Abstract

Federated learning over wireless networks requires aggregating locally computed gradients at a server where the mobile devices send statistically distinct gradient information over heterogenous communication links. This paper proposes a Bayesian approach for wireless federated learning referred to as soft-sign stochastic gradient descent (soft-signSGD). The idea of soft-signSGD is to aggregate the one-bit quantized local gradients at the server by jointly exploiting i) the prior distributions of the local gradients, ii) the gradient quantizer function, and iii) channel distributions. This aggregation method is optimal in the sense of minimizing the mean-squared error (MSE) under a simplified Gaussian prior assumption on the local gradient. From simulations, we demonstrate that soft-signSGD considerably outperforms the conventional sign stochastic gradient descent algorithm when training and testing neural networks using the MNIST dataset and the CIFAR-10 dataset over heterogeneous wireless networks.

Neural networks

Signal processing algorithms

Stochastic processes

Training

Wireless networks

Wireless sensor networks

Signal Processing

Details

Title: Soft-Sign Stochastic Gradient Descent Algorithm for Wireless Federated Learning
Creators: Seunghoon Lee - POSTECH,Department of Electrical Engineering,Pohang,Korea,37673
Chanho Park - POSTECH,Department of Electrical Engineering,Pohang,Korea,37673
Songnam Hong - Hanyang University,Department of Electrical Engineering,Korea
Yonina C Eldar - 972WIS_INST___83
Namyoon Lee - POSTECH,Department of Electrical Engineering,Pohang,Korea,37673
Resource Type: Conference paper
Publication Details: 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Vol.2021-, pp.241-245; 15/Nov/2021
Number of pages: 5
Publisher: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Language: English
DOI: https://doi.org/10.1109/SPAWC51858.2021.9593212
Conference: IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 22 ( Lucca, Italy , 27/Sep/2021 - 30/Sep/2021)
Grant note: NA
Record Identifier: 993285122303596

Metrics

19 Record Views