Decentralized Federated Learning Algorithm Under Adversary Eavesdropping

Lei Xu; Danya Xu; Xinlei Yi; Chao Deng; Tianyou Chai; Tao Yang

doi:10.1109/JAS.2024.125079

Volume 12 Issue 2

Feb. 2025

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2025 > 12(2): 448-456

L. Xu, D. Xu, X. Yi, C. Deng, T. Chai, and T. Yang, “Decentralized federated learning algorithm under adversary eavesdropping,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 2, pp. 448–456, Feb. 2025. doi: 10.1109/JAS.2024.125079

Citation:

L. Xu, D. Xu, X. Yi, C. Deng, T. Chai, and T. Yang, “Decentralized federated learning algorithm under adversary eavesdropping,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 2, pp. 448–456, Feb. 2025. doi: 10.1109/JAS.2024.125079

Citation:

PDF( 1374 KB)

Decentralized Federated Learning Algorithm Under Adversary Eavesdropping

doi: 10.1109/JAS.2024.125079

Funds: This work was supported by the National Key Research and Development Program of China (2022YFB3305904), the National Natural Science Foundation of China (62133003, 61991403, 61991400), the Open Project of State Key Laboratory of Synthetical Automation for Process Industries (SAPI-2024-KFKT-05, SAPI-2024-KFKT-08), China Academy of Engineering Institute of Land Cooperation Consulting Project (2023-DFZD-60-02, N2424004), the Fundamental Research Funds for the Central Universities and Shanghai Municipal Science and Technology Major Project (2021SHZDZX0100), and the Key Research and Development Program of Liaoning Province (2023JH26/10200011)

More Information

Author Bio:
Lei Xu received the B.S. and M.S. degrees in control theory and engineering from LiaoNing Petrochemical University, in 2017 and 2020, respectively. He is currently the first Ph.D. candidate in control science and engineering with the Department of Mechanical Engineering, University of Victoria, Canada, and the second Ph.D. candidate in control science and engineering with the State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University. His research focuses on distributed optimization, federated learning, event-triggered control, and Markovian jump systems

Danya Xu received the B. S. degree in computer science and technology from Beijing University of Posts and Telecommunications in 2018, and the M.S. degree in control science and engineering from Shandong University in 2021. She is currently a Ph.D. candidate in control science and engineering with the State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University. She has been a visiting Ph.D. student in the School of Chemistry and Chemical Engineering at the University of Surrey, UK, since 2024. Her research interests are intelligent fault diagnosis, federated learning, and transfer learning

Xinlei Yi received the B.S. and M.S. degrees in mathematics from China University of Geoscience and Fudan University, in 2011 and 2014, respectively, and the Ph.D. degree in electrical engineering from KTH Royal Institute of Technology, Sweden in 2020. He was a Postdoc with KTH Royal Institute of Technology from 2020 to 2022 and with the Laboratory for Information & Decision Systems, Massachusetts Institute of Technology, USA, from 2022 to 2024. He is a Tenure-Track Professor of Shanghai Institute of Intelligent Science and Technology, Tongji University. His current research interests include distributed online and optimization, online optimization, meta-learning and graph neural networks. He was selected as one of the four finalists for the 2021 European Systems & Control Ph.D. Thesis Award

Chao Deng (Senior Member, IEEE) received the Ph.D. degree in control engineering from Northeastern University in 2018. From May 2018 to May 2021, he was a Research Fellow with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. He is currently a Faculty Member with the Institute of Advanced Technology, Nanjing University of Posts and Telecommunications. His research interests include distributed secondary control of microgrids and cyber-physical systems

Tianyou Chai (Life Fellow, IEEE) received the Ph.D. degree in control theory and engineering from Northeastern University in 1985, where he became a Professor in 1988. He is the Founder and Director of the Center of Automation, which became a National Engineering and Technology Research Center and a State Key Laboratory. He has served as Director of Department of Information Science of National Natural Science Foundation of China from 2010 to 2018. His current research interests include modeling, control, optimization and integrated automation and intelligence of complex industrial processes. He has published 356 peer reviewed international journal papers. His paper titled Hybrid intelligent control for optimal operation of shaft furnace roasting process was selected as one of three best papers for the Control Engineering Practice Paper Prize for 2011–2013. He has developed control technologies with applications to various industrial processes. For his contributions, he has won 5 prestigious awards of National Natural Science, National Science and Technology Progress and National Technological Innovation, the 2007 Industry Award for Excellence in Transitional Control Research from IEEE Multiple-Conference on Systems and Control, and the 2017 Wook Hyun Kwon Education Award from Asian Control Association. He is a Member of Chinese Academy of Engineering, IFAC Fellow and IEEE Fellow

Tao Yang (Senior Member, IEEE) received the Ph.D. degree in electrical engineering from Washington State University, USA in 2012. Between August 2012 and August 2014, he was an ACCESS Postdoctoral Researcher with the ACCESS Linnaeus Centre, Royal Institute of Technology, Sweden. He then joined the Pacific Northwest National Laboratory as a Postdoc, and was promoted to Scientist/Engineer II in 2015. He was an Assistant Professor at the Department of Electrical Engineering, University of North Texas, USA, from 2016–2019. He is a Professor at the State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University. His research interests include industrial artificial intelligence, integrated optimization and control, distributed control and optimization with applications to process industries, cyber physical systems, networked control systems, and multi-agent systems. He is an Associate Editor for IEEE Transactions on Control of Network Systems, IEEE Transactions on Control Systems Technology, and IEEE Transactions on Neural Networks and Learning System. He received Ralph E. Powe Junior Faculty Enhancement Award and Best Student Paper award (as an Advisor) of several international conference
Corresponding author: Tao Yang, e-mail: yangtao@mail.neu.edu.cn
Received Date: 2024-10-17
Accepted Date: 2024-11-27

Available Online: 2025-01-20

Abstract

Abstract

In this paper, we study the decentralized federated learning problem, which involves the collaborative training of a global model among multiple devices while ensuring data privacy. In classical federated learning, the communication channel between the devices poses a potential risk of compromising private information. To reduce the risk of adversary eavesdropping in the communication channel, we propose TRADE (transmit difference weight) concept. This concept replaces the decentralized federated learning algorithm’s transmitted weight parameters with differential weight parameters, enhancing the privacy data against eavesdropping. Subsequently, by integrating the TRADE concept with the primal-dual stochastic gradient descent (SGD) algorithm, we propose a decentralized TRADE primal-dual SGD algorithm. We demonstrate that our proposed algorithm’s convergence properties are the same as those of the primal-dual SGD algorithm while providing enhanced privacy protection. We validate the algorithm’s performance on fault diagnosis task using the Case Western Reserve University dataset, and image classification tasks using the CIFAR-10 and CIFAR-100 datasets, revealing model accuracy comparable to centralized federated learning. Additionally, the experiments confirm the algorithm’s privacy protection capability.
- Adversary eavesdropping,
- decentralized federated learning,
- privacy protection,
- stochastic gradient descent

FullText(HTML)

References(38)

References

[1]	M. Qi, Z. Wang, Q.-L. Han, J. Zhang, S. Chen, and Y. Xiang, “Privacy protection for blockchain-based healthcare IoT systems: A survey,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 8, pp. 1757–1776, 2022.
[2]	J. Zhang, L. Pan, Q.-L. Han, C. Chen, S. Wen, and Y. Xiang, “Deep learning based attack detection for cyber-physical system cybersecurity: A survey,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 377–391, 2021.
[3]	Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu, “Federated learning,” Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 13, no. 3, pp. 1–207, 2019. doi: 10.1007/978-3-031-01585-4
[4]	Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu, and B. He, “A survey on federated learning systems: Vision, hype and reality for data privacy and protection,” IEEE Trans. Knowledge and Data Engi- neering, vol. 35, no. 4, pp. 3347–3366, 2021.
[5]	B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. 20th Int. Conf. Artificial Intelligence and Statistics, 2017, pp. 1273–1282.
[6]	J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv: 1610.05492, 2016.
[7]	A. G. Roy, S. Siddiqui, S. Pölsterl, N. Navab, and C. Wachinger, “Braintorrent: A peer-to-peer environment for decentralized federated learning,” arXiv preprint arXiv: 1905.06731, 2019.
[8]	Y. Yuan, J. Liu, D. Jin, et al., “DeceFL: A principled fully decentralized federated learning framework,” National Science Open, vol. 2, no. 1, p. 20220043, 2023. doi: 10.1360/nso/20220043
[9]	Y. Wang and A. Nedić, “Decentralized gradient methods with time-varying uncoordinated stepsizes: Convergence analysis and privacy design,” IEEE Trans. Autom. Control, vol. 69, no. 8, pp. 5352–5367, 2023.
[10]	O. Goldreich, Foundations of Cryptography: Volume 2, Basic Applications. Cambridge, UK: Cambridge University Press, 2009.
[11]	W. Yang, Z. Zheng, G. Chen, Y. Tang, and X. Wang, “Security analysis of a distributed networked system under eavesdropping attacks,” IEEE Trans. Circuits and Systems II: Express Briefs, vol. 67, no. 7, pp. 1254–1258, 2019.
[12]	Y. Aono, T. Hayashi, L. Wang, and S. Moriai, “Privacy-preserving deep learning via additively homomorphic encryption,” IEEE Trans. Infor. Forensics and Security, vol. 13, no. 5, pp. 1333–1345, 2017.
[13]	C. Fontaine and F. Galand, “A survey of homomorphic encryption for nonspecialists,” EURASIP J. Infor. Security, vol. 2007, pp. 1–10, 2007.
[14]	C. Dwork, “Differential privacy: A survey of results,” in Proc. Int. Conf. Theory and Applications of Models of Computation, 2008, pp. 1–19.
[15]	D. Maity and V. S. Mai, “Ensuring system-level protection against eavesdropping adversaries in distributed dynamical systems,” arXiv preprint arXiv: 2409.09539, 2024.
[16]	A. Krizhevsky, “Learning multiple layers of features from tiny images,” University of Toronto, Toronto, Canada, Tech. Rep., 2009.
[17]	S. Warnat-Herresthal, H. Schultze, K. L. Shastry, et al., “Swarm learning for decentralized and confidential clinical machine learning,” Nature, vol. 594, no. 7862, pp. 265–270, 2021. doi: 10.1038/s41586-021-03583-3
[18]	H. Kim, J. Park, M. Bennis, and S.-L. Kim, “Blockchained on-device federated learning,” IEEE Communi. Lett., vol. 24, no. 6, pp. 1279–1283, 2019.
[19]	M. Blot, D. Picard, M. Cord, and N. Thome, “Gossip training for deep learning,” arXiv preprint arXiv: 1611.09726, 2016.
[20]	J. Daily, A. Vishnu, C. Siegel, T. Warfel, and V. Amatya, “Gossipgrad: Scalable deep learning using gossip communication based asynchronous gradient descent,” arXiv preprint arXiv: 1803.05880, 2018.
[21]	R. L. Rivest, L. Adleman, and M. L. Dertouzos, “On data banks and privacy homomorphisms,” Foundations of Secure Computation, vol. 4, no. 11, pp. 169–180, 1978.
[22]	C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Proc. Theory of Cryptography Conf., 2006, pp. 265–284.
[23]	P. Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” in Proc. Int. Conf. Theory and Applications of Cryptographic Techniques, 1999, pp. 223–238.
[24]	T. ElGamal, “A public key cryptosystem and a signature scheme based on discrete logarithms,” IEEE Trans.n Inform. Theory, vol. 31, no. 4, pp. 469–472, 1985. doi: 10.1109/TIT.1985.1057074
[25]	C. Gentry, “Fully homomorphic encryption using ideal lattices,” in Proc. 41th Annual ACM Symp. Theory Computing, 2009, pp. 169–178.
[26]	K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in Proc. ACM SIGSAC Conf. Computer and Communi. Security, 2017, pp. 1175–1191.
[27]	C. Zhang, S. Li, J. Xia, W. Wang, F. Yan, and Y. Liu, “BatchCrypt: Efficient homomorphic encryption for Cross-Silo federated learning,” in Proc. USENIX Annu. Technical Conf., 2020, pp. 493–506.
[28]	F. D. McSherry, “Privacy integrated queries: An extensible platform for privacy-preserving data analysis,” in Proc. ACM SIGMOD Int. Conf. Management of Data, 2009, pp. 19–30.
[29]	B. Bebensee, “Local differential privacy: A tutorial,” arXiv preprint arXiv: 1907.11908, 2019.
[30]	F. McSherry and K. Talwar, “Mechanism design via differential privacy,” in Proc. 48th Annual IEEE Symp. Foundations of Computer Science, 2007, pp. 94–103.
[31]	X. Yi, S. Zhang, T. Yang, T. Chai, and K. H. Johansson, “A primal-dual SGD algorithm for distributed nonconvex optimization,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 5, pp. 812–833, 2022. doi: 10.1109/JAS.2022.105554
[32]	X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-IID data,” in Proc. Int. Conf. Learning Representations, 2019, pp. 1–26.
[33]	X. Liang, S. Shen, J. Liu, Z. Pan, E. Chen, and Y. Cheng, “Variance reduced local SGD with lower communication complexity,” arXiv preprint arXiv: 1912.12844, 2019.
[34]	F. Haddadpour and M. Mahdavi, “On the convergence of local descent methods in federated learning,” arXiv preprint arXiv: 1910.14425, 2019.
[35]	D. Maity and P. Tsiotras, “Multiagent consensus subject to communication and privacy constraints,” IEEE Trans. Control Network Systems, vol. 9, no. 2, pp. 943–955, 2021.
[36]	A. Tsiamis, K. Gatsis, and G. J. Pappas, “State-secrecy codes for networked linear systems,” IEEE Trans. Autom. Control, vol. 65, no. 5, pp. 2001–2015, 2019.
[37]	W. A. Smith and R. B. Randall, “Rolling element bearing diagnostics using the case western reserve university data: A benchmark study,” Mechanical Systems and Signal Processing, vol. 64, pp. 100–131, 2015.
[38]	D. Misra, “Mish: A self regularized non-monotonic activation function,” arXiv preprint arXiv: 1908.08681, 2019.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(8) / Tables(2)

Get Citation

PDF

XML

Article Metrics

Article views (50) PDF downloads(17)

Decentralized Federated Learning Algorithm Under Adversary Eavesdropping

doi: 10.1109/JAS.2024.125079

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content