Intro. Edge features Region features Corner features tracking by correlation Harris corner detector procedures[1] Harris matrix(structure tensor): A(, y) ° Harris algorithm allx. in N al(x,y) Scan through all X ,y in the image f(al(x, y )Yal(x, y )yi(a(x, yy For each pixel (x,y find the neighborhood Nx, y) which is a (21 21 window Inside N(x, y a 21 21 window, Find The whole image Harris matriⅸx(A) 21pixeIs Find Eigen 2 values of A(x, y=n 21 Pixel(x,y) p Sort all 2min, discard pixel with N= Neighborhood sm入 (21x21)window Discard pixels with large %max- 7min Remaining features are corner points features vo. a 16
Intro. | Edge features | Region features | Corner features | tracking by correlation Harris corner detector procedures[1] • Harris Algorithm – Scan through all x,y in the image – For each pixel (x,y) find the neighborhood N(x,y) which is a (21x21) window. • Inside N(x,y) a 21x21 window, Find Harris Matrix (A) • Find Eigen 2 values of A(x,y)=max, min – Sort all min , discard pixel with small min. – Discard pixels with large max- min. . – Remaining features are corner points features v0.a 16 Pixel(x,y) N= Neighborhood (21x21) window 21pixels 21 pixels = all x,y in N , 2 all , in , all , in N , all , in N , 2 ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) Harris matrix (structure tensor): ( , ) x y x y N x y x y x y x y x y y I x y y I x y x I x y y I x y x I x y x I x y A x y The whole image
Intro. Edge features Region features Corner features tracking by correlation mplementation of the harris agorithm for feature extraction(step1) %scan through the pixel (X=1,y=1) ‰ For each pixel position《x,y), obtain its neighborhood image %e.g. size of window n=21x21 %e.g. x_ max=1024,y_max=1024 XX_max For (x=1, X<=x max; x++)%scan thru the image y=y_ max) i For ly=1, y<=y_ max; y++) 1024X1024 i %for each window centered at x, y %Extract neighborhood X-D, y-D Window Neighbor- obtain a 21x21 N(x,y %D=10 N hood A(x, y) s(x,) window centered at x,y) Image X+D, V+D e.g D=10 features vo. a 17
Intro. | Edge features | Region features | Corner features | tracking by correlation Implementation of the Harris algorithm for feature extraction (step1) • % scan through the pixel • %For each pixel position (x,y), • % obtain its neighborhood image, • % e.g. size of window N =21x21 • %e.g. x_max=1024, y_max=1024 • For (x=1,x<=x_max;x++) %scan thru the image • { For (y=1,y<=y_max;y++) • { %for each window centered at x,y • %Extract neighborhood • obtain a 21x21 N(x,y) %D=10 • window centered at (x,y) • } • } features v0.a 17 (x=1,y=1) (x=x_max, y=y_max) 1024x1024 (x,y) x-D,y-D x+D,y+D e.g. D=10 Neighborhood A(x,y) image Window N
Intro. Edge features Region features Corner features tracking by correlation The harris algorithm for feature extraction(step2 %For each window centered at xy Nxy %in the 21x21 neighborhood image N(x, y) The 21x21 window For(x=1, x<=21 X++%inside window N centered at x, y is o For y=1, y<=21 y++%inside window N I find N(x,y) al(x, y)/O %convolve Ix, y with x edge mask al(x, y/ay %convolve l(x, y) with y edge mask (x=1, y=1)Window N(x,y) then find a 2 2 matrix A(x, y for N/x, y z(x,y) or use the image different, W is the gradient processing window ∑(x,y]-1x+1y)egW=2×2 window X=21,y=21 ∑([xy-;y+1) I in y features MO a
Intro. | Edge features | Region features | Corner features | tracking by correlation The Harris algorithm for feature extraction (step2) • %For each window centered at x,y : N(x,y) • %in the 21x21 neighborhood image N(x,y) • For (x=1, x<=21 x++) %inside window N • { For (y=1, y<=21 y++) %inside window N • { find • I(x,y)/x %convolve I(x,y) with x_edge mask • I(x,y) /y %convolve I(x,y) with y_edge mask • } } • % then find a 2x2 matrix A(x,y) for N(x,y) • features v0.a 18 • The 21x21 window centered at x,y is • N(x,y) (x,y) (x=1,y=1) x=21,y=21 Window N (x,y) ( ) ( ) = − + = − + = all I in W all I in W I x y I x y y I I x y I x y W x I W _ _ _ _ _ _ , , 1 , 1, , e.g. 2 2 window or use the image different, is the gradient processing window
Intro. I Edge teatures I Region teatures I Corner teatures tracking by correation Example: Look at the data of lx2 a point with a corner feature 5I |x2=(O(x,y)/×)^2; Example Ⅳy2=(o(xy)/y)^2 A neighborhood Ixy=(o(x,y)/×)*(o|(x,y)/y) windown with a corner feature We see that variations in x, y directions are high A(x, y), a 2x2 matrix, x23 ly2 representing the intensity variations in the neighborhood image N)] is obtained from these Example data Aneighborhood window w without corner feature
Intro. | Edge features | Region features | Corner features | tracking by correlation Example: Look at the data of a point with a corner feature • Ix2= (I(x,y)/x)^2; • Iy2= (I(x,y)/y)^2 • Ixy= (I(x,y)/x)*(I(x,y)/y) • We see that variations in x,y directions are high • A(x,y), a 2x2 matrix, representing the intensity variations in the neighborhood image {N)} is obtained from these data features v0.a 19 Example: A neighborhood window N with a corner feature Example: A neighborhood window W without corner feature Ix2 Iy2 Ix2 Iy2 Ixy Ixy
Intro. Edge features Region features Corner features tracking by correlation Reason Bad: Amin is too small: the image change is too flat, amin represents smallest intensity variation in the patch Bad: ,max > 7min the image has horizontal or vertical edges Good:λmaX≈ min and amin is big enough features vo. a
Intro. | Edge features | Region features | Corner features | tracking by correlation Reason • Bad: min is too small: the image change is too flat, min represents smallest intensity variation in the patch • Bad: max >> min: the image has horizontal or vertical edges • Good: max min and min is big enough features v0.a 20