RecSysWiki - User contributions [en]

SVDFeature

2012-12-12T02:09:44Z

Tqchen:

'''SVDFeature''' is a toolkit designed to solve the [[feature-based matrix factorization]] efficiently.
Unlike traditional engineering approaches for collaborative filtering which requires writing specific code for each algorithm, SVDFeature develop
new models just by defining new features. The feature-based setting allows us to include many kinds of information into the model, making the model informative. Using the toolkit, we can easily incorporate information such as temporal dynamics, neighborhood relationship, and hierarchical information into the model. Besides [[rating prediction]], it is also capable of doing pairwise ranking tasks for [[item prediction]].

Using the toolkit, we built the best single model reported in track 1 [[KDD Cup 2011]], and champion of track1 [[KDD Cup 2012]]
SVDFeature is distributed under the [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0].

== Features ==
* Big data handling: The toolkit buffers the training data on disk thus memory cost is invariant to training data size. For track 1 of [[KDD Cup 2011]], SVDFeature trains a very complex model using less than 2G memory.
* Strong description ability: Many variants of [[matrix factorization]] can be described in feature-based matrix factorization. One can try new approaches by generating corresponding features, and no modification of code is required.

== When to use SVDFeature ==
* SVDFeature is not a toolkit implementing many specific matrix factorization models. Maybe it is not the best choice for users who are looking for a ready-to-use implementation of a specific algorithm. Some other toolkits (e.g [[LensKit]], [[Mahout]], or [[MyMediaLite]]) may be a better choice.
* SVDFeature is a generic toolkit for developing new algorithms by defining features. If you want to research new algorithms for [[context-aware recommendation]] or compose some existing models together (such as [[SVD++]], [[neighborhood-based models]]), you may want to use SVDFeature, since you only need to write a script for feature generation, and the new model can be learned using SVDFeature.

== External links ==
* Project Homepage: http://svdfeature.apexlab.org
* Project at mloss.org: http://mloss.org/software/view/333/

[[Category: CPlusPlus]]
[[Category: Software]]

SVDFeature

2012-12-12T02:08:41Z

Tqchen: /* External links */

'''SVDFeature''' is a toolkit designed to solve the [[feature-based matrix factorization]] efficiently.
Unlike traditional engineering approaches for collaborative filtering which requires writing specific code for each algorithm, SVDFeature develop
new models just by defining new features. The feature-based setting allows us to include many kinds of information into the model, making the model informative. Using the toolkit, we can easily incorporate information such as temporal dynamics, neighborhood relationship, and hierarchical information into the model. Besides [[rating prediction]], it is also capable of doing pairwise ranking tasks for [[item prediction]].

Using the toolkit, we built the best single model reported in track 1 [[KDD Cup 2011]].
SVDFeature is distributed under the [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0].

== Features ==
* Large-scale data handling: The toolkit buffers the training data on disk thus memory cost is invariant to training data size. For track 1 of [[KDD Cup 2011]], SVDFeature trains a very complex model using less than 2G memory.
* Strong description ability: Many variants of [[matrix factorization]] can be described in feature-based matrix factorization. One can try new approaches by generating corresponding features, and no modification of code is required.

== When to use SVDFeature ==
* SVDFeature is not a toolkit implementing many specific matrix factorization models. Maybe it is not the best choice for users who are looking for a ready-to-use implementation of a specific algorithm. Some other toolkits (e.g [[LensKit]], [[Mahout]], or [[MyMediaLite]]) may be a better choice.
* SVDFeature is a generic toolkit for developing new algorithms by defining features. If you want to research new algorithms for [[context-aware recommendation]] or compose some existing models together (such as [[SVD++]], [[neighborhood-based models]]), you may want to use SVDFeature, since you only need to write a script for feature generation, and the new model can be learned using SVDFeature.

== Usage Examples ==
* The toolkit provide a demo folder with example scripts.
* A non-trivial experiment on KDDCup 2011 track1 dataset using SVDFeature: http://apex.sjtu.edu.cn/apex_wiki/kddtrack1
* A non-trivial experiment on KDDCup 2011 track2 dataset using SVDFeature: http://apex.sjtu.edu.cn/apex_wiki/kddtrack2

== External links ==
* Project Homepage: http://svdfeature.apexlab.org
* Project at mloss.org: http://mloss.org/software/view/333/

[[Category: CPlusPlus]]
[[Category: Software]]

SVDFeature

2011-12-05T08:48:54Z

Tqchen: /* Usage Examples */

'''SVDFeature''' is a toolkit designed to solve the [[feature-based matrix factorization]] efficiently.
Unlike traditional engineering approaches for collaborative filtering which requires writing specific code for each algorithm, SVDFeature develop
new models just by defining new features. The feature-based setting allows us to include many kinds of information into the model, making the model informative. Using the toolkit, we can easily incorporate information such as temporal dynamics, neighborhood relationship, and hierarchical information into the model. Besides [[rating prediction]], it is also capable of doing pairwise ranking tasks for [[item prediction]].

Using the toolkit, we built the best single model reported in track 1 [[KDD Cup 2011]].
SVDFeature is distributed under the [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0].

== Features ==
* Large-scale data handling: The toolkit buffers the training data on disk thus memory cost is invariant to training data size. For track 1 of [[KDD Cup 2011]], SVDFeature trains a very complex model using less than 2G memory.
* Strong description ability: Many variants of [[matrix factorization]] can be described in feature-based matrix factorization. One can try new approaches by generating corresponding features, and no modification of code is required.

== When to use SVDFeature ==
* SVDFeature is not a toolkit implementing many specific matrix factorization models. Maybe it is not the best choice for users who are looking for a ready-to-use implementation of a specific algorithm. Some other toolkits (e.g [[LensKit]], [[Mahout]], or [[MyMediaLite]]) may be a better choice.
* SVDFeature is a generic toolkit for developing new algorithms by defining features. If you want to research new algorithms for [[context-aware recommendation]] or compose some existing models together (such as [[SVD++]], [[neighborhood-based models]]), you may want to use SVDFeature, since you only need to write a script for feature generation, and the new model can be learned using SVDFeature.

== Usage Examples ==
* The toolkit provide a demo folder with example scripts.
* A non-trivial experiment on KDDCup 2011 track1 dataset using SVDFeature: http://apex.sjtu.edu.cn/apex_wiki/kddtrack1
* A non-trivial experiment on KDDCup 2011 track2 dataset using SVDFeature: http://apex.sjtu.edu.cn/apex_wiki/kddtrack2

== External links ==
* Project Homepage: http://apex.sjtu.edu.cn/apex_wiki/svdfeature
* Project at mloss.org: http://mloss.org/software/view/333/
[[Category: Software]]

Singular value decomposition

2011-11-12T07:37:29Z

Tqchen:

'''SVD''' refers to singular value decomposition in linear algebra. However, in the field of collaborative filtering, '''SVD''' often means [[Matrix factorization]].

== SVD in Math ==
The traditional '''SVD''' can also be used as collaborative filtering algorithm. It's also named Latent Semantic Indexing(LSI) in IR.

* Wikipedia page about SVD: http://en.wikipedia.org/wiki/Singular_value_decomposition
* Wikipedia page about LSI: http://en.wikipedia.org/wiki/Latent_semantic_indexing

== SVD as Matrix Factorization ==
Please refer to [[Matrix factorization]]

== See Also ==
* [[SVD++]] is a generalization of matrix factorization to make use of implicit feedback.

[[Category:Method]]

Singular value decomposition

2011-11-12T02:02:15Z

Tqchen:

Singular value decomposition

2011-11-12T02:00:54Z

Tqchen: /* Matrix Factorization */

Singular value decomposition

2011-11-12T02:00:26Z

Tqchen: /* SVD in Math */

'''SVD''' refers to singular value decomposition in linear algebra. However, in the field of collaborative filtering, '''SVD''' often means [[Matrix factorization]].

== SVD in Math ==
The traditional '''SVD''' can also be used as collaborative filtering algorithm. It's also named Latent Semantic Indexing(LSI) in IR.

* Wikipedia page about SVD: http://en.wikipedia.org/wiki/Singular_value_decomposition
* Wikipedia page about LSI: http://en.wikipedia.org/wiki/Latent_semantic_indexing

== Matrix Factorization ==
Please refer to [[Matrix factorization]]

Singular value decomposition

2011-11-12T01:59:12Z

Tqchen: /* SVD in Math */

'''SVD''' refers to singular value decomposition in linear algebra. However, in the field of collaborative filtering, '''SVD''' often means [[Matrix factorization]].

== SVD in Math ==
The traditional '''SVD''' can also be used as collaborative filtering algorithm. It's also named Latent Semantic Indexing in IR.

* [http://en.wikipedia.org/wiki/Singular_value_decomposition]
* [http://en.wikipedia.org/wiki/Latent_semantic_indexing]

== Matrix Factorization ==
Please refer to [[Matrix factorization]]

Singular value decomposition

2011-11-12T01:58:45Z

Tqchen: Created page with "'''SVD''' refers to singular value decomposition in linear algebra. However, in the field of collaborative filtering, '''SVD''' often means Matrix factorization. == SVD in M..."

'''SVD''' refers to singular value decomposition in linear algebra. However, in the field of collaborative filtering, '''SVD''' often means [[Matrix factorization]].

== SVD in Math ==
The traditional '''SVD''' can also be used as collaborative filtering algorithm. It's also named Latent Semantic Indexing in IR.

* [ http://en.wikipedia.org/wiki/Singular_value_decomposition ]
* [ http://en.wikipedia.org/wiki/Latent_semantic_indexing ]
== Matrix Factorization ==
Please refer to [[Matrix factorization]]

Implicit feedback

2011-11-12T01:47:47Z

Tqchen: /* Use Implicit Feedback */

'''Implicit feedback''' is [[user]] activity that can be used to indirectly infer user [[preference]]s, e.g. clicks, page views, purchase actions. Sometimes only positive feedback is known, e.g. the products customers have bought, but not the ones they have decided against.
== Prediction from Implicit Feedback ==
Prediction from implicit feedback(implicit feedback ranking)
refers to the task that ranks a list of item so that the items user prefer( more likely to click,view etc. ) will be in higher order.

== Using Implicit Feedback ==
Implicit feedback information can be used to enhance collaborative filtering algorithms. The most famous example is [[SVD++]]

== See also ==
* The [[MyMediaLite]] software supports [[item prediction]] from implicit feedback.
* The [[SVDFeature]] software supports prediction from implicit feedback and using implicit feedback to improve recommendation.

== External links ==
* [[Wikipedia: Implicit data collection]]

Implicit feedback

2011-11-12T01:46:49Z

Tqchen:

'''Implicit feedback''' is [[user]] activity that can be used to indirectly infer user [[preference]]s, e.g. clicks, page views, purchase actions. Sometimes only positive feedback is known, e.g. the products customers have bought, but not the ones they have decided against.
== Prediction from Implicit Feedback ==
Prediction from implicit feedback(implicit feedback ranking)
refers to the task that ranks a list of item so that the items user prefer( more likely to click,view etc. ) will be in higher order.

== Use Implicit Feedback ==
Implicit feedback information can be used to enhance collaborative filtering algorithms. The most famous example is [[SVD++]]

== See also ==
* The [[MyMediaLite]] software supports [[item prediction]] from implicit feedback.
* The [[SVDFeature]] software supports prediction from implicit feedback and using implicit feedback to improve recommendation.

== External links ==
* [[Wikipedia: Implicit data collection]]

KDD Cup 2011

2011-11-02T09:04:02Z

Tqchen: /* See also */

The '''KDD Cup 2011''' is a two-track recommender system competition, involving both [[rating prediction]] and [[item prediction|prediction of highly rated items]].

== See also ==
* [[SVDFeature]]: A toolkit developed by InnerPeace Team as part of solution to track1, can also be used for track2 task.
* The developers of the [[MyMediaLite]] software participated in track 2.

== External links ==
* http://www.kdd.org/kdd2011/kddcup.shtml
* [http://kddcup.yahoo.com/workshop.php Workshop program]
* [http://tech.groups.yahoo.com/group/kddcup2011/ Yahoo! Group about the KDD Cup 2011]
* Blog posts mentioning/discussing the KDD Cup 2011:
** [http://musicmachinery.com/2011/02/22/is-the-kdd-cup-really-music-recommendation/ Is the KDD Cup really music recommendation?] by [[Paul Lamere]]
** [http://urbanmining.wordpress.com/2011/08/30/on-data-mining-competitions/ On Data Mining Competitions] by [[Neal Lathia]]
** [http://zenoga.tumblr.com/post/9704751024/data-mining-competitions-they-are-very-very-useful Data Mining Competitions: They Are Very, Very Useful] by [[Zeno Gantner]]

[[Category: Competition]]
[[Category: Dataset]]
[[Category: Music recommendation]]
[[Category: Workshop]]

KDD Cup 2011

2011-11-02T09:03:20Z

Tqchen: /* See also */

The '''KDD Cup 2011''' is a two-track recommender system competition, involving both [[rating prediction]] and [[item prediction|prediction of highly rated items]].

== See also ==
* [[SVDFeature]]: A toolkit developed by InnerPeace Team as part of solution to track1, can also work for track2.
* The developers of the [[MyMediaLite]] software participated in track 2.

== External links ==
* http://www.kdd.org/kdd2011/kddcup.shtml
* [http://kddcup.yahoo.com/workshop.php Workshop program]
* [http://tech.groups.yahoo.com/group/kddcup2011/ Yahoo! Group about the KDD Cup 2011]
* Blog posts mentioning/discussing the KDD Cup 2011:
** [http://musicmachinery.com/2011/02/22/is-the-kdd-cup-really-music-recommendation/ Is the KDD Cup really music recommendation?] by [[Paul Lamere]]
** [http://urbanmining.wordpress.com/2011/08/30/on-data-mining-competitions/ On Data Mining Competitions] by [[Neal Lathia]]
** [http://zenoga.tumblr.com/post/9704751024/data-mining-competitions-they-are-very-very-useful Data Mining Competitions: They Are Very, Very Useful] by [[Zeno Gantner]]

[[Category: Competition]]
[[Category: Dataset]]
[[Category: Music recommendation]]
[[Category: Workshop]]

SVDFeature

2011-11-02T09:01:41Z

Tqchen: /* Usage Examples */

'''SVDFeature''' is a toolkit designed to solve the [[feature-based matrix factorization]] efficiently.
Unlike traditional engineering approaches for collaborative filtering which requires writing specific code for each algorithm, SVDFeature develop
new models just by defining new features. The feature-based setting allows us to include many kinds of information into the model, making the model informative. Using the toolkit, we can easily incorporate information such as temporal dynamics, neighborhood relationship, and hierarchical information into the model. Besides [[rating prediction]], it is also capable of doing pairwise ranking tasks for [[item prediction]].

Using the toolkit, we built the best single model reported in track 1 [[KDD Cup 2011]].
SVDFeature is distributed under the [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0].

== Features ==
* Large-scale data handling: The toolkit buffers the training data on disk thus memory cost is invariant to training data size. For track 1 of [[KDD Cup 2011]], SVDFeature trains a very complex model using less than 2G memory.
* Strong description ability: Many variants of [[matrix factorization]] can be described in feature-based matrix factorization. One can try new approaches by generating corresponding features, and no modification of code is required.

== When to use SVDFeature ==
* SVDFeature is not a toolkit implementing many specific matrix factorization models. Maybe it is not the best choice for users who are looking for a ready-to-use implementation of a specific algorithm. Some other toolkits (e.g [[LensKit]], [[Mahout]], or [[MyMediaLite]]) may be a better choice.
* SVDFeature is a generic toolkit for developing new algorithms by defining features. If you want to research new algorithms for [[context-aware recommendation]] or compose some existing models together (such as [[SVD++]], [[neighborhood-based models]]), you may want to use SVDFeature, since you only need to write a script for feature generation, and the new model can be learned using SVDFeature.

== Usage Examples ==
* The toolkit provide a demo folder with example scripts.
* A non-trivial experiment on KDDCup'11 track2 dataset using SVDFeature, getting 3.1x error rate: http://apex.sjtu.edu.cn/apex_wiki/kddtrack2

== External links ==
* Project Homepage: http://apex.sjtu.edu.cn/apex_wiki/svdfeature
* Project at mloss.org: http://mloss.org/software/view/333/
[[Category: Software]]

SVDFeature

2011-11-02T08:59:56Z

Tqchen:

'''SVDFeature''' is a toolkit designed to solve the [[feature-based matrix factorization]] efficiently.
Unlike traditional engineering approaches for collaborative filtering which requires writing specific code for each algorithm, SVDFeature develop
new models just by defining new features. The feature-based setting allows us to include many kinds of information into the model, making the model informative. Using the toolkit, we can easily incorporate information such as temporal dynamics, neighborhood relationship, and hierarchical information into the model. Besides [[rating prediction]], it is also capable of doing pairwise ranking tasks for [[item prediction]].

Using the toolkit, we built the best single model reported in track 1 [[KDD Cup 2011]].
SVDFeature is distributed under the [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0].

== Features ==
* Large-scale data handling: The toolkit buffers the training data on disk thus memory cost is invariant to training data size. For track 1 of [[KDD Cup 2011]], SVDFeature trains a very complex model using less than 2G memory.
* Strong description ability: Many variants of [[matrix factorization]] can be described in feature-based matrix factorization. One can try new approaches by generating corresponding features, and no modification of code is required.

== When to use SVDFeature ==
* SVDFeature is not a toolkit implementing many specific matrix factorization models. Maybe it is not the best choice for users who are looking for a ready-to-use implementation of a specific algorithm. Some other toolkits (e.g [[LensKit]], [[Mahout]], or [[MyMediaLite]]) may be a better choice.
* SVDFeature is a generic toolkit for developing new algorithms by defining features. If you want to research new algorithms for [[context-aware recommendation]] or compose some existing models together (such as [[SVD++]], [[neighborhood-based models]]), you may want to use SVDFeature, since you only need to write a script for feature generation, and the new model can be learned using SVDFeature.

== Usage Examples ==
* The toolkit provide a demo folder with example scripts.
* A non-trivial experiment on KDDCup'11 track2 dataset using SVDFeature, getting 3.1x error: http://apex.sjtu.edu.cn/apex_wiki/kddtrack2

== External links ==
* Project Homepage: http://apex.sjtu.edu.cn/apex_wiki/svdfeature
* Project at mloss.org: http://mloss.org/software/view/333/
[[Category: Software]]

SVD++

2011-10-04T06:16:24Z

Tqchen:

'''SVD++''' refers to the matrix factorization algorithm which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users'
preference.

== Model Formalization ==
The SVD++ model is formally described as following equation:

<math>r_{ui} = \mu + b_u + b_i + \left(p_u + \frac{1}{\sqrt{|N(u)|}}\sum_{j\in N(u)} y_j \right)^T q_i</math>

Where <math>N(u)</math> is the set of implicit information( the set of items user u rated ).

== General Formalization for User Feedback Information ==
A more general form of utilizing implicit/explicit information as user factor can be described in following equation

<math>r_{ui} = \mu + b_u + b_i + \left(p_u + \sum_{i\in Ufeed(u)} \alpha_i y_i \right)^T q_i</math>

Here <math>Ufeed(u)</math> is the set of user feedback information( e.g: the web pages the user clicked, the music on users' favorite list,
the movies user watched, any kinds of information that can be used to describe the user). <math> \alpha_i </math> is a ''feature weight'' associates
with the user feedback information. With the most two common choices: (1) <math>\frac{1}{\sqrt{|N(u)|}}</math> for implicit feedback, (2) <math>\frac{r_{uj} - b_u}{\sqrt{|R(u)|}}</math> for explicit feedback.

== Model Learning ==
* SVD++ can be trained using ALS.
* It's slow to train a SVD++ style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used.

== Efficient SGD Training for SVD++ ==
http://arxiv.org/abs/1109.2271 describes efficient training with user feedback information in section 4

== Literature ==
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008, http://portal.acm.org/citation.cfm?id=1401890.1401944

== Implementations ==
* GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
* [[SVDFeature]] is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and it's extensions.

[[Category:Method]]

Summary of Matrix Factorization Tricks

2011-10-01T03:17:52Z

Tqchen:

This page tries to list the tricks and components used in matrix factorization for collaborative filtering.
There are many kinds of variants for matrix factorization, and in general they can be divided into three kinds:
* Different variant predictor, how to use existing information to do prediction
* Different variant loss functions, whether to use square-loss, log-loss or max-margin loss
* To do rate prediction or rank prediction
== Predictor ==
Predictor refers to the way we give the prediction given input information. Basic predictor for matrix factorization is given by

<math>y_{ui} = \mu + b_u+b_i+p_u^T q_i</math>

List of predictor variants(try to add more):
* [[SVD++]]
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008, http://portal.acm.org/citation.cfm?id=1401890.1401944
* [[Yehuda Koren]]: Collaborative Filtering with Temporal Dynamics, KDD 2009, http://research.yahoo.com/files/kdd-fp074-koren.pdf
* [[Feature-based matrix factorization]]

== Loss Function ==
Loss function specifies how we train our model. It's more or less independent with the predictor.

List of loss functions:
* Square-loss: most commonly used in collaborative filtering task for rate prediction
* Logistic log-likelihood loss: used for sigmoid matrix factorization, sometimes performs better than square-loss
* Hinge-loss( smoothed hinge loss ): used for maximum margin matrix factorization. http://portal.acm.org/citation.cfm?id=1102441

== Pairwise Rank Model==
It's not hard to convert a rate prediction predictor to pairwise rank model. We only need to follow two steps: (1) choose a predictor <math>y</math> (2) choose a loss function for binary classification( either logistic loss or hinge-loss ) (3) train a classification predictor for pairwise order prediction, using predictor <math>y_{ui}-y_{uj}</math> when we compare <math>i</math> to <math>j</math> for user <math>u</math>. This idea is also referred as [[Bayesian Personalized Ranking]].

[[Category:Method]]

Summary of Matrix Factorization Tricks

2011-10-01T03:16:31Z

Tqchen: try to summarize the tricks for MF

Feature-based matrix factorization

2011-10-01T02:44:04Z

Tqchen: /* Model Formalization */

'''Feature-based matrix factorization''' is an abstract [[matrix factorization]] model that uses features to describe the global bias and user/item factors.
The the model allows development of new model simply by feature defining. We can incorporate information such as temporal information, neighborhood information,
taxonomy information into feature-based matrix factorization to make the model ''informative''.
If we have a solver for feature-based matrix factorization, we only need to design context-aware or informative collaborative filtering(or ranking) models by feature-defining,
without engineering efforts for writing codes for each new model.
== Model Formalization ==
The feature-based matrix factorization model can be formalized as follows:

<math>y = \mu+\left(\sum_j \gamma_j b^g_j +\sum_j \alpha_j b^u_j + \sum_j \beta_j b^i_j\right) +\left(\sum_j \alpha_j p_j\right)^T\left( \sum_j \beta_j q_j\right)</math>

We call <math>\gamma </math> global feature, <math>\alpha </math> user feature and <math>\beta </math> item feature. Defining different kinds of features will result in different variants of models. We can add global bias effects to global feature, use user feature to describe user preference and use item feature to describe aspects about the item

== Example Information that can be Incorporated into the Model ==
* Neighborhood information( global feature )
* Global item temporal bias( global feature )
* Item taxonomy information( item feature )
* User temporal factor( user feature )
* User implicit/explicit feedback( user feature )

== Related Models ==
* [[Factorization Machine]]: feature-based matrix factorization can be viewed as a restricted case of factorization machine to distinguish different types of features, allowing us to overcome some shortcomings of FM(e.g: including neighborhood information, exact implementation of rank,svd++ ). We call the model feature-based matrix factorization instead of restricted FM because the idea descends more naturally from matrix factorization using features.

== Implementation ==
*[[SVDFeature]] is an efficient and scalable implementation of feature-based matrix factorization.

== References ==
* [[User:Tqchen | Tianqi Chen]], Zhao Zheng, Qiuxia Lu and Yong Yu: Feature-based Matrix Factorization, http://arxiv.org/abs/1109.2271

[[Category: Method]]

Feature-based matrix factorization

2011-10-01T02:39:02Z

Tqchen: /* Example Information that can be Incorporated into the Model */

'''Feature-based matrix factorization''' is an abstract [[matrix factorization]] model that uses features to describe the global bias and user/item factors.
The the model allows development of new model simply by feature defining. We can incorporate information such as temporal information, neighborhood information,
taxonomy information into feature-based matrix factorization to make the model ''informative''.
If we have a solver for feature-based matrix factorization, we only need to design context-aware or informative collaborative filtering(or ranking) models by feature-defining,
without engineering efforts for writing codes for each new model.
== Model Formalization ==
The feature-based matrix factorization model can be formalized as follows:

<math>y = \mu+\left(\sum_j \gamma_j b^g_j +\sum_j \alpha_j b^u_j + \sum_j \beta_j b^i_j\right) +\left(\sum_j \alpha_j p_j\right)^T\left( \sum_j \beta_j q_j\right)</math>

We call <math>\gamma </math> global feature, <math>\alpha </math> user feature and <math>\beta </math> item feature. Defining different kinds of features will result in different variants of models.

== Example Information that can be Incorporated into the Model ==
* Neighborhood information( global feature )
* Global item temporal bias( global feature )
* Item taxonomy information( item feature )
* User temporal factor( user feature )
* User implicit/explicit feedback( user feature )

== Related Models ==
* [[Factorization Machine]]: feature-based matrix factorization can be viewed as a restricted case of factorization machine to distinguish different types of features, allowing us to overcome some shortcomings of FM(e.g: including neighborhood information, exact implementation of rank,svd++ ). We call the model feature-based matrix factorization instead of restricted FM because the idea descends more naturally from matrix factorization using features.

== Implementation ==
*[[SVDFeature]] is an efficient and scalable implementation of feature-based matrix factorization.

== References ==
* [[User:Tqchen | Tianqi Chen]], Zhao Zheng, Qiuxia Lu and Yong Yu: Feature-based Matrix Factorization, http://arxiv.org/abs/1109.2271

[[Category: Method]]

Feature-based matrix factorization

2011-10-01T01:41:55Z

Tqchen:

'''Feature-based matrix factorization''' is an abstract [[matrix factorization]] model that uses features to describe the global bias and user/item factors.
The the model allows development of new model simply by feature defining. We can incorporate information such as temporal information, neighborhood information,
taxonomy information into feature-based matrix factorization to make the model ''informative''.
If we have a solver for feature-based matrix factorization, we only need to design context-aware or informative collaborative filtering(or ranking) models by feature-defining,
without engineering efforts for writing codes for each new model.
== Model Formalization ==
The feature-based matrix factorization model can be formalized as follows:

<math>y = \mu+\left(\sum_j \gamma_j b^g_j +\sum_j \alpha_j b^u_j + \sum_j \beta_j b^i_j\right) +\left(\sum_j \alpha_j p_j\right)^T\left( \sum_j \beta_j q_j\right)</math>

We call <math>\gamma </math> global feature, <math>\alpha </math> user feature and <math>\beta </math> item feature. Defining different kinds of features will result in different variants of models.

== Example Information that can be Incorporated into the Model ==
* Neighborhood information( global feature )
* Global item temporal bias( global feature )
* Item taxonomy information( item feature )
* User temporal factor( user feature )
* User implicit/explicit feedback( user feature )

== Related Models ==
* [[Factorization Machine]]: feature-based matrix factorization can be viewed as a restricted case of factorization machine to distinguish different types of features, allowing us to overcome some shortcomings of FM(e.g: including neighborhood information, exact implementation of rank,svd++ ). We call the model feature-based matrix factorization instead of restricted FM because the idea descends more naturally from matrix factorization using features.

== Implementation ==
*[[SVDFeature]] is an efficient and scalable implementation of feature-based matrix factorization.

== References ==
* [[User:Tqchen | Tianqi Chen]], Zhao Zheng, Qiuxia Lu and Yong Yu: Feature-based Matrix Factorization, http://arxiv.org/abs/1109.2271

[[Category: Method]]

SVD++

2011-10-01T01:27:13Z

Tqchen:

'''SVD++''' refers to the matrix factorization algorithm which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users'
preference.

== Model Formalization ==
The SVD++ model is formally described as following equation:

<math>r_{ui} = \mu + b_u + b_i + \left(p_u + \frac{1}{\sqrt{|N(u)|}}\sum_{j\in N(u)} y_j \right)^T q_i</math>

Where <math>N(u)</math> is the set of implicit information( the set of items user u rated ).

== General Formalization for User Feedback Information ==
A more general form of utilizing implicit/explicit information as user factor can be described in following equation

<math>r_{ui} = \mu + b_u + b_i + \left(p_u + \sum_{i\in Ufeed(u)} \alpha_i y_i \right)^T q_i</math>

Here <math>Ufeed(u)</math> is the set of user feedback information( e.g: the web pages the user clicked, the music on users' favorite list,
the movies user watched, any kinds of information that can be used to describe the user). <math> \alpha_i </math> is a ''feature weight'' associates
with the user feedback information. With the most two common choices: (1) <math>\frac{1}{\sqrt{|N(u)|}}</math> for implicit feedback, (2) <math>\frac{r_{uj} - b_u}{\sqrt{|R(u)|}}</math> for explicit feedback.

== Model Learning ==
* SVD++ can be trained using ALS.
* It's slow to train a SVD++ style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used.

== Efficient SGD Training for SVD++ ==
http://arxiv.org/abs/1109.2271 describes efficient training with user feedback information in section 4

== Literature ==
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008,http://portal.acm.org/citation.cfm?id=1401890.1401944

== Implementations ==
* GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
* [[SVDFeature]] is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and it's extensions.

[[Category:Method]]

SVD++

2011-10-01T01:20:17Z

Tqchen: /* Model Formalization */

SVD++

2011-10-01T01:19:34Z

Tqchen: /* Model Formalization */

Feature-based matrix factorization

2011-10-01T01:13:41Z

Tqchen: /* Related Models */

Feature-based matrix factorization

2011-10-01T01:10:30Z

Tqchen: /* Related Models */

Feature-based matrix factorization

2011-10-01T01:08:48Z

Tqchen: /* Related Models */

Feature-based matrix factorization

2011-10-01T01:08:21Z

Tqchen: /* Related Models */

Tianqi Chen

2011-09-24T02:25:05Z

Tqchen: Redirected page to User:Tqchen

#REDIRECT [[User:Tqchen]]

SVDPlusPlus

2011-09-24T02:24:07Z

Tqchen: Redirected page to SVD++

#REDIRECT [[SVD++]]

SVD++

2011-09-24T02:21:53Z

Tqchen:

SVD++ refers to the matrix factorization algorithm which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users'
preference.

== Model Formalization ==
currently seems that Latex formula is not supported, wait for another solution.

== Model Learning ==
* SVD++ can be trained using ALS.
* It's slow to train a SVD++ style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used.

== Efficient SGD Training for SVD++ ==
please refer to http://arxiv.org/abs/1109.2271. Describe efficient SVD++ training in section 4

== Literature ==
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008,http://portal.acm.org/citation.cfm?id=1401890.1401944

== Implementations ==
* GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
* [[SVDFeature]] is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and it's extensions.

[[Category:Method]]

Main Page

2011-09-24T02:20:56Z

Tqchen:

{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}
<div id="mainpage"></div>
__NOTOC__

{|style="width:100%;margin-top:+.9em;background-color:#fcfcfc;border:1px solid #ccc"
|style="width:56%;color:#000"|
{|style="width:100%;border:solid 0px;background:none"
|-
|style="width:100%;text-align:center;white-space:nowrap;color:#000" |
<div style="font-size:162%;border:none;margin: 0;padding:.1em;color:#000">Welcome to the Recommender Systems Wiki (RecSysWiki) </div>
<div style="top:+0.2em;font-size: 95%">''to facilitate the sharing of information on all aspects of [[Recommender System|Recommender Systems]]''</div>
<div id="articlecount" style="width:100%;text-align:center;font-size:85%;">{{NUMBEROFPAGES}} pages and {{NUMBEROFARTICLES}} articles in the RecSysWiki as of {{CURRENTDAYNAME}}, {{CURRENTMONTHNAME}} {{CURRENTDAY}}, {{CURRENTYEAR}}</div>
<div style="width:100%;text-align:center;font-size:85%;">started on February 10th, 2011</div>
|}
|}

This Wiki is intended as a space for everything related to the topic of [[Recommender System|Recommender Systems]].

Please note that creation and editing of pages has been disabled for unregistered users due to excessive spamming.

Registration is open to anyone who wishes to contribute.

Currently there are very few pages in the wiki but we're hoping content will be added over time.


{|style="border-spacing:8px;margin:0px -8px"
|class="MainPageBG" style="width:55%;border:1px solid #cef2e0;background-color:#f5fffa;vertical-align:top;color:#000"|
{|width="100%" cellpadding="2" cellspacing="5" style="vertical-align:top;background-color:#f5fffa"
! <h2 style="margin:0;background-color:#cef2e0;font-size:120%;font-weight:bold;border:1px solid #a3bfb1;text-align:left;color:#000;padding:0.2em 0.4em;">Categories in RecSysWiki</h2>
|-
|style="color:#000"|
* [[:Category:Books|Books]]
* [[:Category:Class|Classes]]
* [[:Category:Company|Companies]]
* [[:Category:Competition|Competitions]]
* [[:Category:Conference|Conferences]]
* [[:Category:Datasets|Datasets]]
* [[:Category:Educational Recommender Systems|Educational Recommender Systems]]
* [[:Category:Evaluation|Evaluation]]
* [[:Category:Evaluation measure|Evaluation Measures]]
* [[:Category:Job offer|Job Offers]]
* [[:Category:Literature|Literature]]
* [[:Category:Method|Methods]]
* [[:Category:Movie recommendation|Movie Recommendation]]
* [[:Category:Music recommendation|Music Recommendation]]
* [[:Category:Papers|Papers]]
* [[:Category:Research Groups|Research Groups]]
* [[:Category:Software|Software]]
* [[:Category:Task|Task]]
* [[:Category:User-centric evaluation|User-centric evaluation]]
* [[:Category:Workshop|Workshops]]

|-
|}
|class="MainPageBG" style="width:65%;border:1px solid #cedff2;background-color:#f5faff;vertical-align:top"|
{| width="100%" cellpadding="2" cellspacing="5" style="vertical-align:top;background-color:#f5faff"
!
<h2 style="margin:0;background-color:#cedff2;font-size:120%;font-weight:bold;border:1px solid #a3b0bf;text-align:left;color:#000;padding:0.2em 0.4em;">Information on RecSysWiki</h2>
|-
|style="color:#000"|
* [[:RecSysWiki:Current events| Current Events]]
* [[Special:Statistics|RecSysWiki statistics]]
* [[Special:AllPages|All pages]]
* [[Special:Categories|All categories]]
* [[Special:WantedPages|Wanted pages]]
* [[To Do List]] - ''please help by contributing''

<h2 style="margin:0;background-color:#cedff2;font-size:120%;font-weight:bold;border:1px solid #a3b0bf;text-align:left;color:#000;padding:0.2em 0.4em;">Recent additions and updates</h2>
|-
|style="color:#000"|
* [[ACM Recommender Systems 2011]] 
* [[ACM Recommender Systems 2012]] 
* [[Context-aware recommendation]] 
* [[Discounted Cumulative Gain]] 
* [[Gravity]] 
* [[Hulu]] 
* [[KDD 2011]] 
* [[KDD Cup 2011]] 
* [[Matrix factorization]] 
* [[MyMediaLite]] 
* [[SVDFeature]] 
* [[SVD++]] 
* [[Tag recommendation]] 
|-
!
|-
|}

|-
|}

[[Category:RecSys Wiki Information]]
{{#TwitterFBLike:}}

SVDFeature

2011-09-24T02:05:22Z

Tqchen:

'''SVDFeature''' is a toolkit designed to solve the [[feature-based matrix factorization]] efficiently.
Unlike traditional engineering approaches for collaborative filtering which requires writing specific code for each algorithm, SVDFeature develop
new models just by defining new features. The feature-based setting allows us to include many kinds of information into the model, making the model informative. Using the toolkit, we can easily incorporate information such as temporal dynamics, neighborhood relationship, and hierarchical information into the model. Besides [[rating prediction]], it is also capable of doing pairwise ranking tasks for [[item prediction]].

Using the toolkit, we built the best single model reported in track 1 [[KDD_Cup_2011|KDDCup'11]].
SVDFeature is distributed under apache-2.0.

== Features of Toolkit ==
* Large-scale data handling: The toolkit buffer the training data in disk thus memory cost is invariant to training data size. For track1 of [[KDD_Cup_2011|KDDCup'11]], SVDFeature train a very complex model using less than 2G memory.
* Strong description ability: Many variants of matrix factorization can be described in feature-based matrix factorization. One can try new approaches by generating corresponding features, and no modification of code is required.

== When to use SVDFeature ==
* SVDFeature is not a toolkit implementing many specific matrix factorization models. Maybe it's not the best choice for users who's looking for a ready-to-useimplementation for a specific algorithm. Some other softwares(e.g [[MyMediaLite]]) may be a better choice.
* SVDFeature is like a generic toolkit for developing new algorithms by defining features. If you want to try to research new algorithms for contextual aware recommendation or composite some existing models together( such as SVD++, neighborhood ), you may want to use SVDFeature, since you only need to write script for feature generation, and new model can be learned using SVDFeature.

== External links ==
* Project Homepage: http://apex.sjtu.edu.cn/apex_wiki/svdfeature
* Project at mloss.org: http://mloss.org/software/view/333/
[[Category: Software]]

SVDFeature

2011-09-24T01:28:07Z

Tqchen:

SVDFeature

2011-09-24T01:23:09Z

Tqchen:

SVDFeature

2011-09-24T01:22:11Z

Tqchen:

SVDFeature

2011-09-24T01:21:14Z

Tqchen:

Feature-based matrix factorization

2011-09-24T01:12:00Z

Tqchen:

Feature-based matrix factorization is an abstract matrix factorization model that use features to describe the global bias and user/item factors.
The the model allows development of new model simply by feature defining. We can incorporate information such as temporal information, neighborhood information,
taxonomy information into feature-based matrix factorization to make the model ''informative''.
If we have a solver for feature-based matrix factorization, we only need to design context-aware or informative collaborative filtering(or ranking) models by feature-defining,
without engineering efforts for writing codes for each new model.

= Related Models =
* [[Factorization Machine]]: feature-based matrix factorization can be viewed as a restricted case of factorization machine to distinguish different types of features.

= Implementation =
*[[SVDFeature]] is an efficient and scalable implementation of feature-based matrix factorization.

= References =
* [[User:Tqchen | Tianqi Chen]], Zhao Zheng, Qiuxia Lu and Yong Yu: Feature-based Matrix Factorization, http://arxiv.org/abs/1109.2271
[[Category:Method]]

Feature-based matrix factorization

2011-09-24T01:11:36Z

Tqchen:

Feature-based matrix factorization is an abstract matrix factorization model that use features to describe the global bias and user/item factors.
The the model allows development of new model simply by feature defining. We can incorporate information such as temporal information, neighborhood information,
taxonomy information into feature-based matrix factorization to make the model ''informative''.
If we have a solver for feature-based matrix factorization, we only need to design context-aware or informative collaborative filtering(or ranking) models by feature-defining,
without engineering efforts for writing codes for each new model.

= Implementation =
*[[SVDFeature]] is an efficient and scalable implementation of feature-based matrix factorization.

= Related Models =
* [[Factorization Machine]]: feature-based matrix factorization can be viewed as a restricted case of factorization machine to distinguish different types of features.

= References =
* [[User:Tqchen | Tianqi Chen]], Zhao Zheng, Qiuxia Lu and Yong Yu: Feature-based Matrix Factorization, http://arxiv.org/abs/1109.2271
[[Category:Method]]

Feature-based matrix factorization

2011-09-24T01:11:12Z

Tqchen: Created page with "Feature-based matrix factorization is an abstract matrix factorization model that use features to describe the global bias and user/item factors. The the model allows development..."

Feature-based matrix factorization is an abstract matrix factorization model that use features to describe the global bias and user/item factors.
The the model allows development of new model simply by feature defining. We can incorporate information such as temporal information, neighborhood information,
taxonomy information into feature-based matrix factorization to make the model ''informative''.
If we have a solver for feature-based matrix factorization, we only need to design context-aware or informative collaborative filtering(or ranking) models by feature-defining,
without engineering efforts for writing codes for each new model.

= Implementation =
*[[SVDFeature]] is an efficient and scalable implementation of feature-based matrix factorization.

= Related Models =
* [[Factorization Machine]]: feature-based matrix factorization can be viewed as a restricted case of factorization machine to distinguish different types of features.

= References =
* [[User:Tqchen | Tianqi Chen]], Zhao Zheng, Qiuxia Lu and Yong Yu: Feature-based Matrix Factorization, http://arxiv.org/abs/1109.2271

User talk:Zeno Gantner

2011-09-24T00:51:54Z

Tqchen:

Hi Zeno:
It seems that math formula in Latex is not supported in this wiki. Is it possible to enable the supported so that we can write description for details?
--[[User:Tqchen|Tianqi Chen]]

User talk:Zeno Gantner

2011-09-24T00:51:39Z

Tqchen: Created page with "Hi Zeno: It seems that math formula in Latex is not supported in this wiki. Is it possible to enable the supported so that we can write description for details? --[[User:Tqchen..."

Hi Zeno:
It seems that math formula in Latex is not supported in this wiki. Is it possible to enable the supported so that we can write description for details?
--[[User:Tqchen|Tianqi Chen]]

SVD++

2011-09-24T00:45:02Z

Tqchen: /* Efficient SGD Training for SVD++ */

SVD++ refers to the matrix factorization algorithm which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users'
preference.

== Model Formalization ==
currently seems that Latex formula is not supported, wait for another solution.

== Model Learning ==
* SVD++ can be trained using ALS.
* It's a bit unwise to train a SVD++ style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used.

== Efficient SGD Training for SVD++ ==
please refer to http://arxiv.org/abs/1109.2271. Describe efficient SVD++ training in section 4

== Literature ==
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008,http://portal.acm.org/citation.cfm?id=1401890.1401944

== Implementations ==
* GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
* [[SVDFeature]] is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and it's extensions.

[[Category:Method]]

SVDPlusPlus

2011-09-24T00:43:40Z

Tqchen: Created page with "Please refer to SVD++"

Please refer to [[SVD++]]

SVD++

2011-09-24T00:42:33Z

Tqchen:

SVD++ refers to the matrix factorization algorithm which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users'
preference.

== Model Formalization ==
currently seems that Latex formula is not supported, wait for another solution.

== Model Learning ==
* SVD++ can be trained using ALS.
* It's a bit unwise to train a SVD++ style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used.

== Efficient SGD Training for SVD++ ==
please refer to http://arxiv.org/abs/1109.2271

== Literature ==
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008,http://portal.acm.org/citation.cfm?id=1401890.1401944

== Implementations ==
* GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
* [[SVDFeature]] is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and it's extensions.

[[Category:Method]]

SVD++

2011-09-24T00:41:09Z

Tqchen:

SVD++ refers to the matrix factorization algorithm which makes use of implicit feedback information.
In general, implicit feedback can refer to any kinds of users' history information that can help indicate users'
preference.

== Model Formalization ==
currently seems that Latex formula is not supported, wait for another solution.

== Model Learning ==
* SVD++ can be trained using ALS.
* It's a bit unwise to train a SVD++ style model using stochastic gradient descent due to the size of user feedback information, however, an efficient SGD training algorithm can be used.

== Efficient SGD Training for SVD++ ==
please refer to http://arxiv.org/abs/1109.2271

== Literature ==
* [[Yehuda Koren]]: Factorization meets the neighborhood: a multifaceted collaborative filtering model, KDD 2008,http://portal.acm.org/citation.cfm?id=1401890.1401944

== Implementations ==
* GraphLab Collaborative Filtering Library has implemented SVD++ for multicore: http://graphlab.org/pmf.html
* [[SVDFeature]] is a toolkit designed for feature-based matrix factorization, can be used to implement SVD++ and it's extensions.

SVD++

2011-09-24T00:15:18Z

Tqchen: Created page with "SVD++ refers to the matrix factorization algorithm which makes use of implicit feedback information. <math>r = (p_u+\frac{1}{\sqrt{|N(u)|}}\sum_{i\in N(u)} y_i) q_i + </math>"

SVD++ refers to the matrix factorization algorithm which makes use of implicit feedback information.

<math>r = (p_u+\frac{1}{\sqrt{|N(u)|}}\sum_{i\in N(u)} y_i) q_i + </math>

User:Tqchen

2011-09-23T16:36:42Z

Tqchen:

Tianqi Chen is from Shanghai Jiao Tong University, China.
* [http://apex.sjtu.edu.cn/apex_wiki/tqchen homepage]
I a team member of InnerPeace, we get the 3rd place in track 1 [[KDD_Cup_2011|KDDCup'11]].

== Project ==
[[SVDFeature]]: A toolkit for developing new models by defining features:)

User:Tqchen

2011-09-23T16:31:15Z

Tqchen:

Tianqi Chen is from Shanghai Jiao Tong University, China.
* [http://apex.sjtu.edu.cn/apex_wiki/tqchen homepage]

== Project ==
[[SVDFeature]]: A toolkit for developing new models by defining features:)

User:Tqchen

2011-09-23T16:30:34Z

Tqchen: /* Project */

Tianqi Chen is from Shanghai Jiao Tong University, China.
* [http://apex.sjtu.edu.cn/apex_wiki/tqchen homepage]

==Project==
[[SVDFeature]]: A toolkit for developing new models by defining features:)