In this post on Federated Learning Architectures and Their Applications, I will talk about various architectures for implementing Federated Learning and application scenarios of each of them. In my earlier post on Federated Learning, I have discussed the basic concepts and pros and cons of this new technique of building machine learning models. However, when we consider applying this technique in real-life, not all situations are same.
Hence, the architectural approach to implementing Federated Learning can vary. The most popular architectural approaches are Horizontal Federated Learning and Vertical Federated Learning. However, there are applications of Federated Transfer Learning and Blockchain Federated Learning that make these techniques significant.
Federated Learning Architectures
In this section, I will explain different architectural approaches for Federated Learning. Further, to make the point clear, the application scenario for each type of approach is also provided.
Horizontal Federated Learning
Basically, Horizontal Federated Learning (HFL) is about the common feature space or the common set of attributes. For instance, consider an example of an application to determine the employee satisfaction level. In such a case, the feature space or the set of attributes remain the same for each participant. Evidently, some of the attributes such as job security, flexibility, responsibility level, remuneration, and work-life balance are common in determining the job satisfaction level in every organization irrespective of the industry.
In fact, the Horizontal Federated Learning allows each participant to build the model locally and update only the model parameters. Later, the centralized server on receiving the updates from each participant creates the global model and sends this global model to all participants so that they can update their respective local models.
Vertical Federated Learning
In like manner, the Vertical Federated Learning also enables model training by taking the distributed approach. However, now the participants remain the same and the feature space varies.
For example, suppose we want to build a machine learning model to predict the health status of employees according to job satisfaction. In such a case, the profiles of the participants remain the same. However, the attributes indicating job satisfaction and health parameters are disjoint.
Federated Transfer Learning
Now that, we have a situation in which both the participants set as well as the feature set is disjoint, in such case, the Federated Transfer Learning can take place. The Federated Transfer Learning (FTL) is suitable in situations where there is a strict privacy requirement of the user data.
Take the case of a dataset of home buyers from a real estate company and another dataset of home insurance. Now, both types of datasets have non-overlapping features. Therefore, if the machine learning model is trained on one dataset and can be used to make predictions on the other domain, then Transfer Learning (TL) takes place.
Further, assume that most of the participants are also not overlapping. Additionally, each of the participants trains its local model on their personal device and sends only the trained gradient to the server in order to create the aggregated model. Therefore. in such a case, Federated Transfer Learning (FTL) takes place.
Summary
In this article on Federated Learning Architectures and Their Applications, I have discussed some popular architectures of Federated Learning. Also, the application scenarios and use cases appropriate to each architecture are also discussed here.