PNNL

Abstract

The proliferation of the Machine-Learning-As-A-Service (MLaaS) market has brought to light a number of clients’ data privacy and security concerns. One promising solution is private inference (PI) techniques using cryptographic primitives. These techniques often come with high computation and communication overhead associated with the non-linear operator such as ReLU. Several approaches have been developed in reducing the number of ReLU operations, however, they either require a heuristic threshold selection or introduce significant accuracy drop. This work presents AutoReP, a gradient-based framework for non-linear operators reduction that aims to mitigate these concerns from a systematic perspective. AutoReP automates the process of discrete selection of ReLU and polynomial functions on neurons to accelerate PI applications. We also introduce distribution-aware polynomial approximation (DaPa) to accurately approximate ReLUs under given distribution, preserving model expressivity. Our experimental results demonstrate significant accuracy improvements of 6.12% (94.31%, 12.9K ReLU budget, CIFAR-10), 8.39% (74.92%, 12.9K ReLU budget, CIFAR-100), and 9.45% (63.69%, 55K ReLU budget, Tiny-ImageNet) over current state-of-the-art methods, e.g., SNL. Morever, AutoReP is applied to EfficientNet-B2 on ImageNet dataset, and achieved 75.55% accuracy with 176.1 × ReLU budget reduction.