# SVD++

**SVD++** refers to a matrix factorization model which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users' preference.

## Contents

## Model Formalization

The SVD++ model is formally described as following equation:

<math>r_{ui} = \mu + b_u + b_i + \left(p_u + \frac{1}{\sqrt{|N(u)|}}\sum_{j\in N(u)} y_j \right)^T q_i,</math>

where <math>N(u)</math> is the set of implicit information( the set of items user u rated ).

## General Formalization for User Feedback Information

A more general form of utilizing implicit/explicit information as user factor can be described in following equation

<math>r_{ui} = \mu + b_u + b_i + \left(p_u + \sum_{i\in Ufeed(u)} \alpha_i y_i \right)^T q_i.</math>

Here <math>Ufeed(u)</math> is the set of user feedback information( e.g: the web pages the user clicked, the music on users' favorite list,
the movies user watched, any kinds of information that can be used to describe the user). <math> \alpha_i </math> is a *feature weight* associates
with the user feedback information. With the most two common choices: (1) <math>\frac{1}{\sqrt{|N(u)|}}</math> for implicit feedback, (2) <math>\frac{r_{uj} - b_u}{\sqrt{|R(u)|}}</math> for explicit feedback.

## Learning

SVD++ can be trained using ALS.

It is slow to train a SVD++-style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used. [1] describes efficient training with user feedback information in section 4

## Literature

- Yehuda Koren: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008, http://portal.acm.org/citation.cfm?id=1401890.1401944

## Implementations

- The GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
- SVDFeature is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and its extensions.
- LibFM can also be used to implement SVD++
- wooflix is a (not very fast) Python implementation of SVD++
- MyMediaLite: SVD++ source code on GitHub; see also [2]