Exponential machines and tensor trains

12/05/2016

Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called Tensor Train (TT). The Tensor Train format regularizes the model and lets you control the number of underlying parameters. To train the model, we develop a stochastic version of Riemannian optimization, which allows us to fit tensors with \(2^{30}\) entries. We show that the model achieves state-of-the-art performance on synthetic data with high-order interactions.

News

26/05/2016 A TT-eigenvalue solver that finally works
12/05/2016 Exponential machines and tensor trains
06/04/2016 Convergence analysis of a projected fixed-point iteration
30/03/2016 Compress-and-eliminate solver for sparse matrices
01/12/2015 New paper in SIMAX

Contact

We are located at the 2-nd floor of the new "Technopark-3” building in Skolkovo (few kilometers outside Moscow Ring Road). The building is accessible from Skolkovo Road (Сколковское шоссе) and Minskoe Highway (Минское шоссе).

email: