An Analysis of Probabilistic Methods for Top-N Recommendation in Collaborative Filtering
Nicola Barbieri 1,2 and Giuseppe Manco 11ICAR-CNR via Bucci 41c, 87036 Rende (CS) - Italy manco@https://www.wendangku.net/doc/bf8351510.html,r.it
2DEIS - University of Calabria, via Bucci 41c, 87036 Rende (CS) - Italy nbarbieri@deis.unical.it
Analysis of Probabilistic Methods for Top-N Recommendations Context
Prediction accuracy vs recommendation accuracy
The recommendation problem has been traditionally interpreted as a missing value
prediction problem, in which, given an active user, the system is asked to predict her
preference for a set of items.
?Since a user is more prone to access items for which she will likely provide a positive
feedback, a recommendation list can be be hence built by drawing upon the
(predicted) highly-rated items
?Standard approach: minimize statistical error metrics, such as the Root Mean
Squared Error (RMSE).
?The common belief is that small improvements in RMSE would re?ect an increase of the
accuracy of the recommendation lists ?However, recent works have shown that there is no monotonic relation between error
metrics and accuracy metrics:
lower RMSE does not imply higher recommendation accuracy
Evaluating Recommendation Accuracy
Precision/Recall
-Let L u denote the recommendation list provided to a user u during a generic session
-Let T u denote the test-set entries for the user (u)
-
N is the size of the recommendation list Probabilistic Modeling of Preference Data Latent factors modeling : the state of the hidden variable associated to each preference observation ?u, i ? models the underlying reason why u has chosen/rated i .
Relevant items Unrelevant items User satisfaction -For each user u and each item i in -Generate a candidate list -Add i to and sort the list according to the scoring function -Record the position of item i in the sorted list:-if i belongs to the top-N items, we have a hit -otherwise, we have a miss Exploiting Probabilities for Item Ranking The underlying probabilistic framework provides high ?exibility in the choice of the item ranking function:
?Predicted Preference ?Item Selection ?Item Selection and Relevance Experimental Evaluation We study the effects of the ranking function on the accuracy of the recommendation list, by employing a MonteCarlo 5-folds validation on Movielens1M data.The results we report are obtained by varying the length of the recommendation list in the range [1,20] and the dimension of the random candidate list is ?xed to 1000.Summary of Results
Probabilistic models, equipped with the proper ranking function, exhibit competitive advantages over state-of-the-art
RS in terms of recommendation accuracy.
Whereas the predicted preference item ranking provides poor accuracy results, strategies based on item selection
guarantee signi?cant improvements.
Item selection component plays the most important role in recommendation ranking. Better results can be achieved
by considering also a rating prediction component.–For each user u and for each positively-rated item i ∈T r u :?Generate the candidate list C u by randomly drawing from I R ?(I T (u )∪{i });?add i to C u and sort the list according to the scoring function;?Record the position of the item i in the ordered list:–if i belongs to the top-k items,we have a hit –otherwise,we have a miss Practically speaking,we ask the RS to rank an initial random sample which also contains i .If i is actually recommended,we have an hit,otherwise the RS has failed in detecting an item of high interest for the considered user.Recall and precision can hence be tuned accordingly:US Recall (N )=#hits |T r u |(1)US Precision (N )=#hits N ·|T r u |=US Recall (N )N (2)Notice that the above de?nition of precision does not penalize false positives:the recommendation is considered successful if it matches at least an item of interest.However,neither the amount of non-relevant“spurious”items,nor the position of the relevant item within the top-N is taken into account.3Collaborative Filtering in a Probabilistic Framework Probabilistic approaches assume that each preference observation is randomly drawn from the joint distribution of the random variables which model users,items and preference values (if available).Typically,the random generation pro-cess follows a bag of words assumption and preference observations are assumed to be generated independently.A key di ?erence between probabilistic and deter-ministic models relies in the inference phase:while the latter approaches try to minimize directly the error made by the model,probabilistic approaches do not focus on a particular error metric;parameters are determined by maximizing the likelihood of the data,typically employing an Expectation Maximization pro-cedure.In addition,background knowledge can be explicitly modeled by means prior probabilities,thus allowing a direct control on over?tting within the infer-ence procedure [?].By modeling prior knowledge,they implicitly solve the need for regularization which a ?ects traditional gradient-descent based latent factors approaches.Further advantages of probabilistic models can be found in their easy inter-pretability:they can often be represented by using a graphical model,which summarizes the intuition behind the model by underlying causal dependencies between users,items and hidden factors.Also,they provide an uni?ed frame-work for combining collaborative and content features [?,?,?],to produce more accurate recommendations even in the case of new users/items.Moreover,as-suming that an explicit preference value is available,probabilistic models can user u and for each positively-rated item i ∈T r u :erate the candidate list C u by randomly drawing from I R ?(I T (u )∪i to C u and sort the list according to the scoring function;rd the position of the item i in the ordered list:i belongs to the top-k items,we have a hit herwise,we have a miss peaking,we ask the RS to rank an initial random sample which also f i is actually recommended,we have an hit,otherwise the RS has ecting an item of high interest for the considered user.Recall and n hence be tuned accordingly:US Recall (N )=#hits |T r u |(1)US Precision (N )=#hits N ·|T r u |=US Recall (N )N (2)at the above de?nition of precision does not penalize false positives:endation is considered successful if it matches at least an item of wever,neither the amount of non-relevant“spurious”items,nor the he relevant item within the top-N is taken into account.borative Filtering in a Probabilistic Framework approaches assume that each preference observation is randomly the joint distribution of the random variables which model users,eference values (if available).Typically,the random generation pro-a bag of words assumption and preference observations are assumed ted independently.A key di ?erence between probabilistic and deter-dels relies in the inference phase:while the latter approaches try to ectly the error made by the model,probabilistic approaches do not articular error metric;parameters are determined by maximizing the the data,typically employing an Expectation Maximization pro-ddition,background knowledge can be explicitly modeled by means ilities,thus allowing a direct control on over?tting within the infer-ure [?].By modeling prior knowledge,they implicitly solve the need ation which a ?ects traditional gradient-descent based latent factors advantages of probabilistic models can be found in their easy inter-they can often be represented by using a graphical model,which the intuition behind the model by underlying causal dependencies rs,items and hidden factors.Also,they provide an uni?ed frame-mbining collaborative and content features [?,?,?],to produce more ommendations even in the case of new users/items.Moreover,as-an explicit preference value is available,probabilistic models can –For each user u and for each positively-rated item i ∈T r u :?Generate the candidate list C u by randomly drawing from I R ?(I T (u )∪{i });?add i to C u and sort the list according to the scoring function;?Record the position of the item i in the ordered list:–if i belongs to the top-k items,we have a hit –otherwise,we have a miss Practically speaking,we ask the RS to rank an initial random sample which also contains i .If i is actually recommended,we have an hit,otherwise the RS has failed in detecting an item of high interest for the considered user.Recall and precision can hence be tuned accordingly:US Recall (N )=#hits |T r u |(1)US Precision (N )=#hits N ·|T r u |=US Recall (N )N (2)Notice that the above de?nition of precision does not penalize false positives:the recommendation is considered successful if it matches at least an item of interest.However,neither the amount of non-relevant“spurious”items,nor the position of the relevant item within the top-N is taken into account.3Collaborative Filtering in a Probabilistic Framework Probabilistic approaches assume that each preference observation is randomly drawn from the joint distribution of the random variables which model users,items and preference values (if available).Typically,the random generation pro-cess follows a bag of words assumption and preference observations are assumed to be generated independently.A key di ?erence between probabilistic and deter-ministic models relies in the inference phase:while the latter approaches try to minimize directly the error made by the model,probabilistic approaches do not focus on a particular error metric;parameters are determined by maximizing the likelihood of the data,typically employing an Expectation Maximization pro-cedure.In addition,background knowledge can be explicitly modeled by means prior probabilities,thus allowing a direct control on over?tting within the infer-ence procedure [?].By modeling prior knowledge,they implicitly solve the need for regularization which a ?ects traditional gradient-descent based latent factors approaches.Further advantages of probabilistic models can be found in their easy inter-pretability:they can often be represented by using a graphical model,which summarizes the intuition behind the model by underlying causal dependencies between users,items and hidden factors.Also,they provide an uni?ed frame-work for combining collaborative and content features [?,?,?],to produce more accurate recommendations even in the case of new users/items.Moreover,as-
suming that an explicit preference value is available,probabilistic models can For each user u and for each positively-rated item i ∈T r u :?Generate the candidate list C u by randomly drawing from I R ?(I T (u )∪{i });?add i to C u and sort the list according to the scoring function;?Record the position of the item i in the ordered list:–if i belongs to the top-k items,we have a hit –otherwise,we have a miss tically speaking,we ask the RS to rank an initial random sample which also ains i .If i is actually recommended,we have an hit,otherwise the RS has d in detecting an item of high interest for the considered user.Recall and sion can hence be tuned accordingly:US Recall (N )=#hits |T r u |(1)US Precision (N )=#hits N ·|T r u |=US Recall (N )N (2)otice that the above de?nition of precision does not penalize false positives:ecommendation is considered successful if it matches at least an item of est.However,neither the amount of non-relevant“spurious”items,nor the ion of the relevant item within the top-N is taken into account.Collaborative Filtering in a Probabilistic Framework abilistic approaches assume that each preference observation is randomly n from the joint distribution of the random variables which model users,s and preference values (if available).Typically,the random generation pro-ollows a bag of words assumption and preference observations are assumed generated independently.A key di ?erence between probabilistic and deter-stic models relies in the inference phase:while the latter approaches try to mize directly the error made by the model,probabilistic approaches do not on a particular error metric;parameters are determined by maximizing the hood of the data,typically employing an Expectation Maximization pro-re.In addition,background knowledge can be explicitly modeled by means probabilities,thus allowing a direct control on over?tting within the infer-procedure [?].By modeling prior knowledge,they implicitly solve the need egularization which a ?ects traditional gradient-descent based latent factors oaches.urther advantages of probabilistic models can be found in their easy inter-bility:they can often be represented by using a graphical model,which marizes the intuition behind the model by underlying causal dependencies een users,items and hidden factors.Also,they provide an uni?ed frame-for combining collaborative and content features [?,?,?],to produce more rate recommendations even in the case of new users/items.Moreover,as-ng that an explicit preference value is available,probabilistic models can