NUS崖, S|G|R2018 National University Lab for Media Search Adversarial Personalized Ranking for Recommendation Xiangnan He, Zhankui He, Xiaoyu du, Tat-Seng Chua School of Computing National University of Singapore
Adversarial Personalized Ranking for Recommendation Xiangnan He, Zhankui He, Xiaoyu Du, Tat-Seng Chua School of Computing National University of Singapore 1 SIGIR 2018
DeNUS Motivation ational University The core of ir tasks is ranking Search Given a query, ranking documents Recommendation Given a user, ranking items a personalized ranking task Ranking is usually supported by the underlying scoring model Linear. Probabilistic, Neural network models etc Model parameters are learned by optimizing learning- to-rank loss Question: is the learned model robust in ranking? Will small change on inputs/parameters lead to big change on the ranking result? This concerns model generalization ability
Motivation • The core of IR tasks is ranking. • Search: Given a query, ranking documents • Recommendation: Given a user, ranking items – A personalized ranking task • Ranking is usually supported by the underlying scoring model. – Linear, Probabilistic, Neural network models etc. – Model parameters are learned by optimizing learning-to-rank loss • Question: is the learned model robust in ranking? – Will small change on inputs/parameters lead to big change on the ranking result? – This concerns model generalization ability. 2
Adversarial Examples on DeNUS ational University Classification ( Goodfellow et al, ICLR' 15 Recent efforts on adversarial machine learning show many well-trained classitiers sutter from adversarial examples: +.007× panda nematode” gibbon 57.7 confidence 8. 2% confidence 99.3 confidence This implies weak generalization ability of the classifier Question: do such adversarial examples also exist for IR ranking methods?
Adversarial Examples on Classification (Goodfellow et al, ICLR’15) • Recent efforts on adversarial machine learning show many well-trained classifiers suffer from adversarial examples: – This implies weak generalization ability of the classifier • Question: do such adversarial examples also exist for IR ranking methods? 3
Adversarial Examples on DeNUS ational University Personalized Ranking We train visually-aware BPR(He et al, AAAl16 on a user image interaction dataset for visualization VBPR is a pairwise learning-to-rank method Effect of adversarial examples on personalized ranking Top-4 image ranking 360 5.5 +0.007 of a sampled user. o0a0o05 3.50 +0.007x before vs. after adversarial noise: 49 +0.007 翻 34 测+000 4.13 ariginal Images Perturbed Ir Small adversarial noises on images ( noise level e= 0.007)leads to big change on ranking
Adversarial Examples on Personalized Ranking • We train Visually-aware BPR (He et al, AAAI’16) on a userimage interaction dataset for visualization. – VBPR is a pairwise learning-to-rank method • Effect of adversarial examples on personalized ranking: 4 Small adversarial noises on images (noise level ϵ = 0.007)leads to big change on ranking. Ranking scores (before) Ranking scores (after) Top-4 image ranking of a sampled user. before vs. after adversarial noise:
Quantitative Analysis on DeNUS ational University Adversarial Attacks We train matrix factorization(Mf)with BPR loss MF is a widely used model in recommendation BPR is a standard pairwise loss for personalized ranking We add noises on model parameters of mf Random noise vs. Adversarial noise Performance change w.r.t. different noise levels E(i. e, L, norm Conclusion: 0.16 MF-BPR is robust to 0.06 0.1 random noise but not 0.08 for adversarial noise! ● Adversarial Noise 004 Adversarial Noise Random Noise 0 040.60.8 (a) Testing NDCG vs.∈ (c) Testing NDCG vs.∈
Quantitative Analysis on Adversarial Attacks • We train matrix factorization (MF) with BPR loss – MF is a widely used model in recommendation – BPR is a standard pairwise loss for personalized ranking • We add noises on model parameters of MF – Random noise vs. Adversarial noise – Performance change w.r.t. different noise levels ε (i.e., L2 norm): 5 Conclusion: MF-BPR is robust to random noise, but not for adversarial noise!