WPEC 2024 Talk 3a3: Optimizing ML MPC from System & Theoretical Perspectives

Abstract. In this talk, we will delve into advancements in optimizing n-party Multi-Party Computation (MPC) protocols for machine learning (ML), focusing on system and theoretical innovations presented in two key papers. Firstly, we explore MPC-Pipe, an efficient pipeline scheme designed to enhance the performance of n-party MPC protocols. Traditional MPC implementations suffer from significant performance bottlenecks due to the sequential implementation of communication and computation phases. MPC-Pipe introduces three innovative pipeline schemes, thereby improving GPU utilization and reducing idle times. This approach demonstrates substantial performance gains, with throughput improvements of up to 50% and latency reductions of up to 16% for deep neural networks and transformer models in various network settings. Additionally, we present CompactTag, a scheme that significantly reduces the overhead of actively secure n-party MPC by compacting the tags associated with input data. CompactTag leverages a novel tagging mechanism that ensures data integrity and authenticity while minimizing the computation costs typically associated with traditional tagging methods. This innovation is particularly effective in large-scale ML training scenarios, where reducing overhead can lead to substantial performance improvements. Optimizing MPC for machine learning from both system and theoretical perspectives is essential for advancing privacy-preserving technologies. The innovations presented in MPC-Pipe and CompactTag offer practical solutions to overcome existing bottlenecks, enhancing the performance of MPC protocols in ML applications.

[Slides]