Intro. Edge features Region features Corner features tracking by correlation Harris Interest corner detector for feature tracking [1] S1 2D S2 For two images taken at t1 and T1+At,,we want x+u, y+v) to find how a patch of image(2DX 2D )is moved Center of search window S1 (at T1 is moved from (x,y to S2 (at T1+At )with center at(x+u, y+v) Image feature search criterion: Square pixe/ difference of sl and S2 is minimum Problem definition and notations pixel value at (x,y=1(x, y) nage gradient= VI(x, y) aI(x, y)aI(x, y) If the position of a square window N(size 2D. 2D) has a pixel shift of (u, v), the intensity change the square pixel difference)of the windows is E(u, v) Referencehttp:/cmp.felk.cvut.cz/Cmp/courses/dzo/resources/lectureharrisurbanpdf features vo. a
Intro. | Edge features | Region features | Corner features | tracking by correlation Harris Interest corner detector for feature tracking [1] (the square pixel difference) of the windowsis E (u, v) has a pixelshift of ( , ), the intensity change If the position of a square window N (size 2 2 ) ( , ) , ( , ) image gradient ( , ) pixel value at Problem definition and notations: u v D D y I x y x I x y I x y (x,y) I(x,y) • = = = features v0.a 11 • For two images taken at T1 and T1+t, , we want to find how a patch of image (2Dx2D) is moved. • Center of search window S1 (at T1) is moved from (x,y) to S2 (at T1+t ) with center at (x+u,y+v) • Image feature search criterion: Square pixel difference of S1 and S2 is minimum. (x,y) 2D (x+u,y+v) Reference : http://cmp.felk.cvut.cz/cmp/courses/dzo/resources/lecture_harris_urban.pdf S1 S2
Intro. Edge features Region features Corner features tracking by correlation Taylor series: f(x)=f(xo)+f(xo(x-xo)+higher order aI(x,y) al(x, y) Harris interest So, I(x+u,y+v)al(x,y)+ v+ Operator Since E(u1)=∑[(x+u,y+)-1(xy) Basic -D<x<D, -D<y<D) 2D ∑ a(x +2 aI(x,y)a(x, y) aI(x, y) s1 +1 (-D<x<D,-D<y<D) 2D S2 (X以 aI(x,y) (,y)a(x,y) X(+u, y+y) aI(x, y)al(x,y al(x,y) al(x,y) aI(x, y)aI(x,y aI(x,y)Yal(x, y) The tree top has moved ax ∑ we want to find uv to minimize e A=Harris Matrix( structure tensor) features vo. a 12
Intro. | Edge features | Region features | Corner features | tracking by correlation Harris Interest Operator Basic = = = + + = + + − + + + + + = + − + − − − − − − vu u v A vu y I x y y I x y x I x y y I x y x I x y x I x y u v vu y I x y y I x y x I x y y I x y x I x y x I x y u v y I x y v y I x y x I x y uv x I x y u E u v I x u y v I x y Since v y I x y u x I x y I x u y v I x y f x f x f x x x higher order D x D D y D D x D D y D D x D D y D 2 2 2 2 ( , ) 2 2 ( , ) 2 2 2 2 2 ( , ) 0 0 0 ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , ) 2 ( , ) ( , ) ( , ) ( , ) .. ( , ) ( , ) So, ( , ) ( , ) Taylor series: ( ) ( ) '( )( ) _ ... features v0.a 12 • (x,y) 2D (x+u,y+v) S1 S2 2D The tree top has moved, we want to find u,v to minimize E A=Harris Matrix (or structure tensor)
Intro. Edge features Region features Corner features tracking by correlation Continue E()=[y] If n(a scalar) is the eigen value of A, where A is a 2x2 matrix hence E(u, v)=u val OI E(2)={ Discussion and conclusion: So Elu, v(image change or square pixel difference) depends on two independent factors n and ( u, v Our target is to find a minimum Eu, v ), but e(u, =0 is a trivia solution because an all-white window can match another all white windows anywhere If we use elu, v as an indicator in the feature correspondence search algorithm testing different u, v), so E must be large enough to make the search effective Since u, v depends on the search algorithm but n depends on the image patch you select, so pick a large n will benefit the search features vo. a There are 2ns (amin imax for Harris Matrix aa 2x 2 matrix) So amin should not be too small
Intro. | Edge features | Region features | Corner features | tracking by correlation Continue = = = = vu E u v u v or vu E u v u v vu vu A A A vu E u v u v A ( , ) ,hence ( , ) , If (a scalar) is the eigen value of ,where is a 2x2 matrix. ( , ) 2 2 features v0.a 13 • Discussion and conclusion: • So E(u,v) (image change or square pixel difference) depends on two independent factors: and (u,v) • Our target is to find a minimum E (u,v), but E(u,v) =0 is a trivial solution because an all-white window can match another all white windows anywhere. If we use E(u,v) as an indicator in the feature correspondence search algorithm (testing different u,v) , so E must be large enough to make the search effective. • Since (u,v) depends on the search algorithm, but depends on the image patch you select, so pick a large will benefit the search. • There are 2 s (min, max) for Harris Matrix A (a 2x2 matrix) • So min should not be too small
Intro. Edge features Region features Corner features tracking by correlation Rules for finding the suitable feature patch window Two Eigen values (may, 2min )exist for Harris Matrix a nmin must be big enough T Amin is a good criterion for corner features See appendix for eigen values tutorial features vo. a
Intro. | Edge features | Region features | Corner features | tracking by correlation Rules for finding the suitable feature patch window • Two Eigen values (max , min ) exist for Harris Matrix A • min must be big enough • max min is a good criterion for corner features. • See appendix for Eigen values tutorial features v0.a 14
Intro. Edge features Region features Corner features tracking by correlation Our notations Our algorithm deals with the whole image Within the whole image we search for windows(N)to determine if window n is a corner feature or not Within n we use windows w to find image gradients For a window n, we calculate harris matrix a to determine if window n is a corner or not features vo. a
Intro. | Edge features | Region features | Corner features | tracking by correlation Our notations • Our algorithm deals with the whole image • Within the whole image we search for windows (N) to determine if window N is a corner feature or not. • Within N, – we use windows W to find image gradients. – For a window N, we calculate Harris Matrix A to determine if window N is a corner or not. features v0.a 15