Image processing and Computer Vision Chapter 11: Bundle adjustment Structure reconstruction sfm from N-frames Pose estimation vo.a
Image Processing and Computer Vision Chapter 11: Bundle adjustment Structure reconstruction SFM from N-frames Pose estimation V0.a 1
Reconstruction from n-frames Factorization(inear, fast, not too accurate) Bundle adjustment (slower but more accurate) can use factorization results as the first guess Non linear iterative methods are more accurate han linear method require first guess (e.g. from factorization) Many different implementations but the concept is the same Pose estimation vo.a
Reconstruction from N-frames • Factorization (linear, fast, not too accurate) • Bundle adjustment (slower but more accurate), can use factorization results as the first guess. – Non linear iterative methods are more accurate than linear method, require first guess (e.g. From factorization). – Many different implementations, but the concept is the same. Pose estimation V0.a 2
Problem definition There are n features in the 3d object We takei pictures of the object at different VIews Input · Image sequence I2…lr Each image has n image feature points Output (structure=model, and motion=pose) 3-d coordinates of all 3-d model points x,x.X N Camera pose for each image taken [r()t(t)t=l, .T Pose estimation vo.a
Problem definition • There are N features in the 3D object . • We take pictures of the object at different views. • Input : • Image sequence I1 ,I2 ,…I . • Each image has n image feature points • Output (structure=model, and motion=pose) • 3-D coordinates of all 3-D model points X1 ,X2 ,..,XN . • Camera pose for each image taken [R(t),T(t)] t=1,… Pose estimation V0.a 3
Example: Bundle adjustment 3D reconstruction (see also http://www.cse.cuhk.edu.hk/khwong/demo/index.html) grand canyon demo lask · robot http://www.youtube.com/watch?v=2klfrillOjc http://www.youtube.com/watch?v=xgcnv--wf2k http://www.youtube.com/watch?v=onx4cyyyyri http://www.youtube.com/watch?v=4h1pn2dis6g Pose estimation vo.a
Example: Bundle adjustment 3D reconstruction (see also http://www.cse.cuhk.edu.hk/khwong/demo/index.html) • Grand Canyon Demo • Flask • Robot Pose estimation V0.a 4 http://www.youtube.com/watch?v=2KLFRILlOjc http://www.youtube.com/watch?v=4h1pN2DIs6g http://www.youtube.com/watch?v=ONx4cyYYyrI http://www.youtube.com/watch?v=xgCnV--wf2k
The iterative Sfm alternating bundle adjustment method Break down the system into two phases SFM1: find pose phase SFM2: find model phase Initialize first guess of model The first guess is a flat model perpendicular to the image and is Zinit away e.g. Zinit =0.5 meters or any reasonable guess Iterative while( Err is not small SFM1: find pose phase SFM2: find model phase Measurement error(Err)or(model and pose stabilized) Pose estimation vo.a
The iterative SFM alternating bundle adjustment method • Break down the system into two phases: --SFM1: find pose phase --SFM2: find model phase • Initialize first guess of model – The first guess is a flat model perpendicular to the image and is Zinit away (e.g. Zinit = 0.5 meters or any reasonable guess) • Iterative while ( Err is not small ) • { – SFM1: find pose phase – SFM2: find model phase – Measurement error(Err) or(model and pose stabilized) • } Pose estimation V0.a 5