Area Under the ROC Curve

The area under the ROC curve (AUC) is a measure for ranking quality.

In recommender systems, we are often interested in how well method can rank a given set of items. The best possible value is 1, and any non-random ranking that makes sense would have an AUC > 0.5.

An intuitive explanation:

The AUC specifies the probability that, when we draw two examples at random, their predicted pairwise ranking is correct.

(adapted from [1], which we found via [2])

AUC does not give a higher weight to items higher up in the ranking. Some measures that put more weight on higher-ranking items are normalized discounted cumulative gain (NDCG) and mean average precision (MAP).

External links

• Wikipedia: Receiver operating characteristic
• C# implementation of AUC evaluation for binary responses (part of the MyMediaLite library)