Today MemComputing released a whitepaper highlighting the advantages of the company’s new training approach compared to traditional deep learning methods.
The paper addresses the inherent limitations associated with today’s most popular gradient-based methods, such as Adaptive Moment Estimation (ADAM) and Stochastic Gradient Descent (SGD), which incorporate backpropagation. MemComputing’s approach instead aims towards a more global and parallelized optimization algorithm, achievable through its entirely new computing architecture. Each data point contributes toward the network update concurrently and in parallel with the others, resulting in faster and more robust learning.
MemComputing represents an alternative to gradient descent-based methods,” said Fabio Traversa, CTO, and co-inventor of MemComputing. “Our approach allows for the trainable parameters of the network to be optimized together in a truly parallelized fashion, allowing for the training of deeper networks without the need of most hotfixes used for Deep Neural Network training.”
As a case study, MemComputing reported performance on the detection of Exotic High-Energy Particles compared to ADAM and SGD. Traversa adds, “MemComputing training was 5 times faster than traditional methods with higher accuracy.”
Although MemComputing has been focusing on solving combinatorial optimization problems, John Beane, CEO of MemComputing commented, “This opens a whole new door for our technology; when you look at the Artificial Intelligence market, we have been using the same methods to train our neural networks for over 3 decades. Even ignoring our superior performance, the introduction of an alternative method is significant in and of itself.”
