Factors Influencing Cost and Performance of Federated and Centralized Machine Learning

Document Type

Conference Article

Publication Title

INDICON 2022 - 2022 IEEE 19th India Council International Conference

Abstract

With the growing concern of the customer data privacy, serving them with personalized engaging experiences through Machine Learning (ML) models built on centralized servers is becoming a challenge. Recent developments such as Federated Learning(FL), which is a privacy preserving ML scheme are gaining much attention. In FL model training happens with data federated across devices and not leaving them to sustain user privacy. This is ensured by making the untrained or partially trained models to reach directly the individual devices and getting locally trained "on-device"using the device owned data, and the server aggregating all the partially trained model learnings to update a global model. Although almost all the model learning schemes in the federated learning setup use gradient descent, there are certain characteristic differences brought about by the non-IID nature of the data availability, that affects the training in comparison to the centralized schemes. In this paper, we discuss the various factors that affect the federated learning training, because of the non-IID distributed nature of the data, as well as the inherent differences in the federating learning approach as against the centralized gradient descent techniques. We empirically demonstrate the effect of number of samples per device and the distribution of labels on federated learning. In addition to the privacy advantage we seek through federated learning, we also study if there is a cost advantage while using federated learning frameworks. The cost includes the cloud infrastructure cost for training and deployment, including communication costs, and download and upload cost(of models). We show that federated learning does have an advantage in cost when the model sizes to be trained are not reasonably large. All in all, we present the need for careful design of model for both performance and cost advantages of FL system adoption across services.

DOI

10.1109/INDICON56171.2022.10040166

Publication Date

1-1-2022

This document is currently not available here.

Share

COinS