Taming backdoors in federated learning with FLAME

Federated learning is a collaborative approach that allows many participants to jointly train a machine learning model without having to share potentially sensitive data. Each participant trains a local machine learning model, which is then aggregated in either a centralized or decentralized fashion  into a global model or series of higher-level models with higher performance than any of the local models. These higher-level models are then distributed back to each local device. By sharing local models instead of data, federated learning mechanisms protect user data privacy and reduce communication overheads commonly associated with training machine learning models with large amounts of distributed data.

Despite obvious privacy and efficiency benefits, federated learning mechanisms are, by default, vulnerable to backdoor attacks. These attacks work by adversarially crafting local models in order to poison the global model such that it provides incorrect outputs for selected inputs. In the cyber security context, a backdoor attack might be used to poison detection models so that they are unable to correctly identify spam emails or malicious executables. Existing defenses against backdoor attacks are limited to countering only a very small number of specific attacks. They also, in many cases, significantly deteriorate the performance of the resulting model.

To address these deficiencies and to find solutions to the backdoor problem in federated learning, we collaborated with researchers from the Technical University of Darmstadt in Germany, the University of California San Diego and Google. The result of this collaboration is FLAME, a defense framework that computes and injects an optimal amount of noise into local models, just before they are aggregated into the global model, ensuring the elimination of backdoors while preserving global model performance. The evaluation of FLAME on several datasets from such application areas as image classification, word prediction, and IoT intrusion detection demonstrates that FLAME removes backdoors effectively and with a negligible impact on model performance. FLAME is thus a solution that adds security to the existing benefits of federated learning – namely performance, privacy protection, and communication efficiency.

The FLAME framework is presented in the publication: “FLAME: Taming Backdoors in Federated learning”, which has been accepted for presentation and publication at the 2022 USENIX Security Symposium to be held on August 10-12 2022. The USENIX Security Symposium is a leading academic conference on information system security established over 30 years ago. It is organized by the USENIX association, whose mission is to foster technical excellence and innovation and to support and disseminate research with a practical bias. The conference brings together researchers, practitioners, system administrators, system programmers, and others interested in the latest advances in the security and privacy of computer systems and networks. Our paper has already been published on the USENIX conference site.

The FLAME work contributes to and was partially supported by the EU Horizon 2020 project SPATIAL (https://spatial-h2020.eu, received funding from the Horizon 2020 programme under grant agreement No 101021808) and WithSecure™’s Project Blackfin. SPATIAL investigates how to enhance AI-powered solutions in terms of accountability, privacy and resilience in specific application scenarios from the cybersecurity, 5G services, IoT services, and Edge intelligence domains, where federated learning is a popular approach. WithSecure™’s Project Blackfin is a multi-year research effort with the goal of applying collective intelligence techniques, including federated learning, to the cyber security domain.