Could someone give me an example of how each one works and how their error measures differ? Thanks!
0
votes
1 Answers
0
votes
Normalised discounted cumulative gain (NDCG) measures the performance of a recommendation system based on the graded relevance of the recommended entities. It varies from 0.0 to 1.0, with 1.0 representing the ideal ranking of the entities. This metric is commonly used in information retrieval and to evaluate the performance of web search engines.
Root Mean Squared Error (RMSE) is perhaps the most popular metric used in evaluating accuracy of predicted ratings. The system generates predicted ratings Rˆui for a test set T of user-item pairs (u,i) for which the actual ratings Rui are known. Typically, Rui are known because they are hidden in an offline experiment, or because they were obtained through a user study or online experiment. It varies from 0.0 to 1.0 with lower values being signalling less error (therefore "better").
You are more likely to see academic papers include RMSE, it is not too often you see NDCG, although it does have it's place, especially when you are examining recommender systems with a strong Information Retrieval component. Root Mean Squared Error is not application-specific as such, and so tends to be included in most academic literature when evaluating the predictive accuracy of Recommender Systems.
This information is easily accessible via Google so please spend a bit of time in the future to do some research before posting a question like this. There are lots of worked examples and the formula are detailed clearly on each respective Wiki page NDCG: http://en.wikipedia.org/wiki/Discounted_cumulative_gain#Normalized_DCG and RMSE: http://en.wikipedia.org/wiki/Root-mean-square_deviation.
