Intro. I Essential Mat. FUndamental Mat> Epipolar Geom. I Corresp. I Reconst From E(Essential matrix to F(Fundamental matrix We can combine(essential matrix camera intrinsic parameters to become the Fundamental matrix. It is more compact So we can use two un-calibrated cameras (unknown focal length image centers etc. to make a stereo system For essential matrix you need to know the intrinsic parameters of the two cameras For the fundamental matrix F, it encapsulates all intrinsic parameters. a 21
Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. From E (Essential matrix) to F (Fundamental matrix) • We can combine (Essential matrix + Camera intrinsic parameters) to become The Fundamental matrix. It is more compact. • So we can use two un-calibrated cameras (unknown focal length , image centers etc.) to make a stereo system • For Essential matrix you need to know the intrinsic parameters of the two cameras • For the Fundamental matrix F , it encapsulates all intrinsic parameters Stereo v0.a 21
Intro. I Essential Mat. FUndamental Mat> Epipolar Geom. I Corresp. I Reconst Exercise 2: Fundamental Matrix F(a 3x3 matrix).Recall X*E*Ⅹ,=0 x2y2f]*E区xy1分=0 Assume both camera intrinsic parameters (Mint )are known sau sl S1v2=Min2)y2 hence y2=(Min2)s2 f2 and similar for the left -camera s,u S ence Note in linear algebra: (ab)=b a If f=M int 2 ERM Prove thatu2 v2 1]*F*u, v 1=0--(ii) ? Ans Stereo vO. a
Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Stereo v0.a 22 ( ) ( ) ( ) ? : __ Prove that 1 * * 1 0 ( ) If * * Note in Linear algebra : ,hence and similar for the left- camera ,hence Assume both camera intrinsic parameters( ) are known * * 0 ( ) X * *X 0 ( ) Exercise 2 :Fundamental Matrix (a 3x3 matrix).Recall 2 2 1 1 1 int 2 int1 1 1 1 1 1 1 int1 1 1 1 1 1 1 int1 1 1 1 1 1 2 2 2 2 2 1 int 2 2 2 2 2 2 2 int 2 2 2 2 2 2 int 2 2 2 1 1 1 1 T 2 Ans u v F u v iii F M E M ab b a s s v s u M fy x fy x M s s v s u s s v s u M fy x fy x M s s v s u M x y f E x y f ii E i F T T T T T T = − − − = = = = = = = − − − = − − − − − − − − − − − − •
Intro. Essential Mat. Fundamental Mat. Epipolar Geom. Corresp. Reconst Answer 2 Fromx, y,f*E*x y f=0--(ii) since y2=(Min2 52v2, he ence Advantages of using F f2 S2 u, v are just pixel su sul y2 S,l Sv coordinates f2 Mint,2 are encapsulated lul inside f y=(in )s, b put these in(i) S Note: (x1, y1), (x2, y2 are S2 s,u real image point positions v2(Mm**(Mm-1S=0 fuf2 are focal lengths S2 s,u [SulS,V1,S,, [S,u2, S,V2,S,, s,(M ERM -1 sP1|=0 or homogeneous image S points Since F=(Mint 2)*E*Mint )choose s1=52=1 u, v2 1*F*u, v I=0--(done Stereo vO. a
Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Answer 2 • Advantages of using F • u,v are just pixel coordinates • Mint1,2 are encapsulated inside F • Note: (x1,y1),(x2,y2) are real image point positions • f1 ,f2 are focal lengths • [s1u1 ,s1v1 ,s1 ], [s2u2 ,s2v2 ,s2 ], or homogeneous image points Stereo v0.a 23 ( ) ( ) ( ) ( ) ( ) ( ) ( ) (( ) ) 1* * 1 0 ( ) Since * * ,choose 1 * * * 0 * * 0 ,put these in ( ) since ,hence From * * 0 ( ) 2 2 1 1 1 2 1 int 2 int1 1 1 1 1 1 1 2 2 2 int 2 int1 1 1 1 1 1 1 int 2 int1 2 2 2 2 2 1 1 1 1 1 1 int1 1 1 1 int 2 2 2 2 2 2 2 2 2 2 2 1 int 2 2 2 2 2 2 2 2 2 1 int 2 2 2 2 2 2 2 1 1 1 u v F u v done F M E M s s s s v s u su sv s M E M s s v s u M E M s s v s u ii s s v s u M fy x M s s v s u s s v s u M fy x s s v s u M fy x x y f E x y f ii T T T T T T T T T T = − − − = = = = = = = = = = − − − − − − − − − − − − − −
Intro. I Essential Mat. FUndamental Mat> Epipolar Geom. I Corresp. I Reconst Exercise 3: Eight (n>=8) point algorithm for finding e or f n= number of corresponding point teatures left i-th image point=lu,(i)v1le1], ight i-th image point=[uli)vali) 1 fu fr fi F=f f22 f F l241+2°12+2/13+21f21+2f2+n2/23+l1f31+v2+f3=0 n>=8, Reshape F3x3, make it become a F9rvector F'=u f2 fi3 f2 f2f23 fa f2 f l2(1)1(1)u2(1)v1(1)a2(1)v2(1)4(1)v2(1)(1)v2(l)u(1)v1() AF u2(n)u, (n)u(n),(n)u,(n) v2(n)u,(n) v2(n),(n) v2(n)u,(n)v,(n) Write the relation of these terms in a formula, F, u2(3),v3),u2(3),v23) Answer Stereo vO. a 24
Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Exercise 3: Eight (n>=8) point algorithm for finding E or F n= number of corresponding point features left i-th image point=[u1 (i) v1 (i) 1]T , right i-th image point=[u2 (i) v2 (i) 1]T • ?___ : Write the relation of these terms in a formula, F', (3), (3), (3), (3). ' 0 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1 : : : : : : : : : (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) 1 ' ' 8, Reshape , make it become a vector 0 1 * * 1 0 ( ) 2 1 2 2 2 1 2 1 2 2 1 2 1 2 1 1 2 1 2 1 2 2 1 2 1 2 1 1 1 1 1 2 1 3 2 1 2 2 2 3 3 1 3 2 3 3 3 3 9 1 2 1 1 1 2 1 1 2 2 1 3 2 1 2 1 2 1 2 2 2 2 3 1 3 1 1 3 2 3 3 2 2 1 1 3 1 3 2 3 3 2 1 2 2 2 3 1 1 1 2 1 3 Answer u v u v F u n u n u n v n u n v n u n v n v n v n u n v n u u u v u v u v v v u v A F F f f f f f f f f f n F F ' u u f u v f u f v u f v v f v f u f v f f u v F u v iv f f f f f f f f f F T x x T = = = = + + + + + + + + = = − − − − − − − − − = Stereo v0.a 24
Intro. I Essential Mat. FUndamental Mat> Epipolar Geom. I Corresp. I Reconst Eight Point algorithm (for n28 to solve the Fundamental matrix F equation,(p91[1A],265[1B]) Find correspondences n28 left-right image feature points Normalize image points [u, vl, this makes the algorithm more stable and accurate Normalization means for each image See appendix for how Lu, v]All images points should have zero mean normalization works Average distance of [u, v] around the center [0, 0] is 21/2 apply to each image independently See Ref: Richard l. Hartley june 1997). In Defense of the Eight-Point Algorithm". IEEE Transaction on Pattern Recognition and Machine lnte∥ ligence19(6):580-593.do:10.1109/34601246 Rank of a is 8, at least 8 points are needed From A(size is nx 9), left side [x, y, 1], right side =[x,y' 1]image features, use the method of svd to find f f size is 9x 1) AF=0, and [U, S, V=SVD(A, SVD=Singular Value Decomposition Estimated f=last column of v from f find f. Stereo vO. a 25
Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Eight Point algorithm (for n8) to solve the Fundamental matrix F equation , (p91[1A], 265[1B]) • Find correspondences n≥8 left-right image feature points • Normalize image points [u,v], this makes the algorithm more stable and accurate. Normalization means for each image – [u,v]All images points should have zero mean – Average distance of [u,v] around the center [0,0] is 21/2 – Apply to each image independently. – See Ref: Richard I. Hartley (June 1997). "In Defense of the Eight-Point Algorithm". IEEE Transaction on Pattern Recognition and Machine Intelligence 19 (6): 580–593. doi:10.1109/34.601246. • Rank of A is 8, at least 8 points are needed • From A(size is nx9) , left side [x,y,1] , right side =[x’,y’,1] image features, use the method of SVD to find f (f size is 9x1). • AF’=0, and [U,S,V]=SVD(A), SVD=Singular Value Decomposition • Estimated F’=last column of V, from F’ find F. Stereo v0.a 25 See appendix for how normalization works