PNNL

Abstract

Graph learning on dynamical systems has recently surfaced as an emerging research domain. By leveraging a novel electronic Dynamical System (DS), various graph learning challenges have been effectively tackled through a rapid, spontaneous natural annealing process. This method has attracted increasing attention due to its orders-of-magnitude improvements in speed and energy efficiency compared to traditional Graph Neural Network (GNN) approaches for inference tasks. However, (1) the current DS hardware only supports inference, missing its native solution for training; while relying on conventional hardware is even more expensive than GNNs. (2) The current DS architecture only allows linear interactions among its nodes, limiting training accuracy. In this work, we present a Dynamical-System Training-Processing Unit (DS-TPU) developed through algorithm-architecture co-design to tackle the two major challenges: (1) An on-device lifelong learning mechanism that leverages feedback electric current as the loss function in response to the observed training data, allowing electron-speed refinement on the present model parameters. (2) A nonlinear DS node interaction mechanism constructed from Chebyshev polynomials to significantly improve the compatibility between the DS hardware and the embedded relation of graph data. Extensive evaluations using six real-world graph learning applications demonstrate that for accuracy, DS-TPU achieves 10.8% MAE reduction over the best results of five widely used GNNs. In terms of training performance, the 5-Watt DS-TPU architecture achieves on-average 810x speedup over the offline training for DS on an Nvidia A100 GPU, and 640x over GNN training on the same GPU. In terms of inference performance, DS-TPU achieves 2548x over the A100 GPU and 115x over the best state-of-the-art GNN accelerators.