Exponential machines and tensor trains


Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called Tensor Train (TT). The Tensor Train format regularizes the model and lets you control the number of underlying parameters. To train the model, we develop a stochastic version of Riemannian optimization, which allows us to fit tensors with \(2^{30}\) entries. We show that the model achieves state-of-the-art performance on synthetic data with high-order interactions.


26/05/2016 A TT-eigenvalue solver that finally works
12/05/2016 Exponential machines and tensor trains
06/04/2016 Convergence analysis of a projected fixed-point iteration
30/03/2016 Compress-and-eliminate solver for sparse matrices
01/12/2015 New paper in SIMAX


We are located at the 2-nd floor of the new "Technopark-3” building in Skolkovo (few kilometers outside Moscow Ring Road). The building is accessible from Skolkovo Road (Сколковское шоссе) and Minskoe Highway (Минское шоссе).