near model ING SHEN SSE TONGJIUNIVERSITY SEP.2016
Linear Model Y I NG SH EN SSE, TO NG JI UNI VERSITY SEP. 2 0 1 6
The basic form of the linear model Given a samplex=(i,x2,,x d)' with d attributes The linear model tries to a learn a prediction function using a linear combination of all attributes, i e f(r)=Wix,+W2x2+.+Waxd tb The vector form of the function is f(=w'x+ b Where w=(W1,W2,…,wd Once w and d have been learned from samples, f will be determined For example 好瓜=02*x色泽+0.5*x根蒂+0.3*x敲声+ 2/3/2021 PATTERN RECOGNITION
The basic form of the linear model Given a sample 𝒙 = 𝑥1 , 𝑥2 , … , 𝑥𝑑 𝑇with d attributes The linear model tries to a learn a prediction function using a linear combination of all attributes, i.e. 𝑓 𝒙 = 𝑤1𝑥1 + 𝑤2𝑥2 + ⋯ + 𝑤𝑑𝑥𝑑 + 𝑏 The vector form of the function is 𝑓 𝒙 = 𝒘𝑇𝒙 + 𝑏 where 𝒘 = 𝑤1 , 𝑤2 , … , 𝑤𝑑 𝑇 Once w and d have been learned from samples, f will be determined. For example 2/3/2021 PATTERN RECOGNITION 2 𝑓好瓜 = 0.2 ∗ 𝑥色泽 + 0.5 ∗ 𝑥根蒂 + 0.3 ∗ 𝑥敲声 + 1
near regression Given a dataset=(1, yi,(x2,y2),,(mmbx (i1, xi2,u, Mid), the task of a linear regression is to learn a linear model which can predict a value for a new samplex' that close to its true value y When d=1, i=xi 400}0 Hours Spent Studying 1014 4 1222 Math SAT 390580650730410530600790350400590 Sc core 2/3/2021 PATTERN RECOGNITION
Linear regression Given a dataset 𝐷 = 𝒙1 , 𝑦1 , 𝒙2 , 𝑦2 , … , 𝒙𝑚, 𝑦𝑚 ; 𝒙𝑖 = 𝑥𝑖1, 𝑥𝑖2, … , 𝑥𝑖𝑑 𝑇 , the task of a linear regression is to learn a linear model which can predict a value for a new sample x' that close to its true value y'. When 𝑑 = 1, 𝒙𝑖 = 𝑥𝑖 2/3/2021 PATTERN RECOGNITION 3 Hours Spent Studying 4 9 10 14 4 7 12 22 1 3 8 Math SAT Score 390 580 650 730 410 530 600 790 350 400 590
near regression We will learn a linear regression model f(i=wxi+b, such that f(xi=y How do we determine w and 6? >600 2/3/2021 PATTERN RECOGNITION
Linear regression We will learn a linear regression model 𝑓 𝑥𝑖 = 𝑤𝑥𝑖 + 𝑏, 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝑓(𝑥𝑖 ) ≅ 𝑦𝑖 How do we determine w and b? 2/3/2021 PATTERN RECOGNITION 4
near regression Mean squared error(Mse) is a commonly used performance measure m MSE We want to minimize MSe between f(xi)and y (w*, b*)=arg min)((xi-yi (w,b) m = arg min>(i-wxi-b (w,b) 1 2/3/2021 PATTERN RECOGNITION
Linear regression Mean squared error (MSE) is a commonly used performance measure: We want to minimize MSE between f(xi ) and yi : 2/3/2021 PATTERN RECOGNITION 5 𝑀𝑆𝐸 = 1 𝑚 𝑖=1 𝑚 𝑦𝑖 ′ − 𝑦𝑖 2 𝑤 ∗ , 𝑏 ∗ = arg min (𝑤,𝑏) 𝑖=1 𝑚 𝑓 𝑥𝑖 − 𝑦𝑖 2 = arg min (𝑤,𝑏) 𝑖=1 𝑚 𝑦𝑖 − 𝑤𝑥𝑖 − 𝑏 2