Making Linear Autoencoders Work for Large Scale Recommendation Systems

Vojtech Vancura

Aug 29, 2022

Linear autoencoders for collaborative filtering in recommender systems are simple and surprisingly accurate as we explained in our blogpost on how linear methods work. The critical disadvantage of methods like EASE is that they are not applicable to real-world problems, where the number of items to be recommended is high. In this blogpost, we will explain how we made linear methods applicable even to the largest customers of Recombee with tens of millions of recommendable items.

We introduce ELSA, a scalable linear model that is basically a shallow autoencoder. What is interesting is that ELSA is not only more scalable, but it actually outperforms EASE and other linear methods in a number of tasks. We will first explain how ELSA is different and how we managed to make it scalable.

The problem of linear autoencoders (EASE, SLIM) is that the Item to Item weight matrix grows significantly with the number of items and using a sparse representation does not help. It makes these methods practically applicable to problems with tens of thousands of items. However many real-world recommendation problems are orders of magnitude bigger than that. In ELSA, we were able to replace this matrix by a trick allowing us to use a smaller matrix (items x d) instead, where d is a small constant (e.g. 512) you can choose. This trick makes both memory and time complexity of ELSA grow linearly with the number of items.

As you can see from our results on benchmarking data, the offline performance of ELSA is similar to EASE and it is beneficial to use ELSA when you have more than 20k recommendable items. What is surprising is that ELSA is consistently better than EASE or MF in online tests, probably thanks to a more robust training procedure.

We will make ELSA source codes available for everyone. Also, our paper was accepted for publication at the RecSyS 2022 conference in Seattle, so you can meet us there in case you are interested in our research.