Intro.MotivationPoseest.Newton's method| Iterative method Motivation of studying pose estimation In order to understand bundle adjustment for 3-d model structure reconstruction from -frames, we need to understand pose estimation first Pose estimation problem definition there are n features in a known 3D object For pose estimation We take / pictures of the object at different views.(Assume object stationary, camera moving at time t=12…,F Input We know there are n 3-d model points of the object: X,Y, Z)12. N and Image sequence I,I2,Ir: Each image has N 2-D image feature points Output (pose=rotation and translation the camera) 0,1=(R Tt1, 2=(R=,.,0 =[ R, Tt=r poses are found Pose estimation vo.a 6
Intro. | Motivation | Pose est.| Newton’s method | Iterative method Motivation of studying pose estimation • In order to understand bundle adjustment for 3-D model structure reconstruction from -frames, we need to understand pose estimation first. • Pose estimation problem definition: There are N features in a known 3D object . • For pose estimation: We take pictures of the object at different views. (Assume object stationary, camera moving at time t=12,.., . • Input : • We know there are N 3-D model points of the object : (X, Y, Z)1,2,..,N, and • Image sequence I1 ,I2 ,…I. Each image has N 2-D image feature points • Output (pose=rotation and translation the camera) • t=1={R,T} t=1 , t=2={R,T} t=2 , …., t=={R,T} t= poses are found Pose estimation V0.a 6
Intro.MotivationPoseest.Newton's method| Iterative method Example: Bundle adjustment 3d reconstruction (see also http://www.cse.cuhk.edu.hk/khwong/demo/index.html) Grand Canyon Demo flask Robot http://www.youtube.com/watch?v=2klfriliojc http://www.youtube.com/watch?v=xgcnv--wf2k http://www.youtube.com/watch?v=onx4cyyyyri http://www.youtube.com/watch?v=4h1pn2dis6g Pose estimation vo.a 7
Intro. | Motivation | Pose est.| Newton’s method | Iterative method Example: Bundle adjustment 3D reconstruction (see also http://www.cse.cuhk.edu.hk/khwong/demo/index.html) • Grand Canyon Demo • Flask • Robot Pose estimation V0.a 7 http://www.youtube.com/watch?v=2KLFRILlOjc http://www.youtube.com/watch?v=4h1pN2DIs6g http://www.youtube.com/watch?v=ONx4cyYYyrI http://www.youtube.com/watch?v=xgCnV--wf2k
Intro. MOtivation) Pose est. I Newton's method I Iterative method The iterative SFM alternating(2-stepbundle adjustment Break down the system into two phases SM1: find pose estimation phase --SFM2: find model estimation phase Initialize first guess of model The first guess is a flat model perpendicular to the image and is Zinit away ( e. g. Zinit =0. 5 meters or any reasonable guess) Iterative while( err is not small SFM1: find pose phase SFM2 find model phase Measurement error(Error(model and pose stabilized Pose estimation vo.a 8
Intro. | Motivation | Pose est.| Newton’s method | Iterative method The iterative SFM alternating (2-step)bundle adjustment • Break down the system into two phases: --SFM1: find pose estimation phase --SFM2: find model estimation phase • Initialize first guess of model – The first guess is a flat model perpendicular to the image and is Zinit away (e.g. Zinit = 0.5 meters or any reasonable guess) • Iterative while ( Err is not small ) • { – SFM1: find pose phase – SFM2: find model phase – Measurement error(Err) or(model and pose stabilized) • } Pose estimation V0.a 8
Intro.MotivationPoseest.Newton's method| Iterative method Define SFM1: the pose estimation algorithm (assume the model is found or given, or guessed Use Klt (Feature Detector interface( or lkdemo. c)in opencv, or http://www.cs.ubcca/mlowe/keypoints/toobtainfeaturesin[u,v]t There are t=1, 2 ,, I image frames So there are 0 1 (R, Ts1, 0 =2=(R,+2,.,0 =R, T]=r poses Given: focal length f and one model M=XY,Zl with j=1,., Features Initialize first guess of model The first guess is a flat model perpendicular to the image and is Zinit away(e.g. Zinit =0.5 meters or any reasonable guess For(=1; t<=/; t++//runs Itimes here i or every time frame t, use all N features, run SFM1 once; so SFM1SFM1: find pose: to find 8) After the above is run e=1 R, Tt=, 0+2=R,TIts,.,0.=R, Tt r poses are found Pose estimation vo.a
Intro. | Motivation | Pose est.| Newton’s method | Iterative method Define SFM1: the pose estimation algorithm (assume the model is found or given, or guessed) • Use KLT (FeatureDetector interface (or lkdemo.c) in opencv, or http://www.cs.ubc.ca/~lowe/keypoints/) to obtain features in [u,v] T • There are t=1,2,…,image frames, • So there are t=1={R,T} t=1 , t=2={R,T} t=2 , …., t=={R,T} t= poses. • Given: focal length f and one model Mi=[X,Y,Z]I ,with i=1,..,N features • Initialize first guess of model – The first guess is a flat model perpendicular to the image and is Zinit away (e.g. Zinit = 0.5 meters or any reasonable guess) – For (t=1; t<=; t++) // runs times here – { (for every time frame t, use all N features, run SFM1 once); – so SFM1 {SFM1: find pose : to find t } – } – After the above is run – t=1={R,T} t=1 , t=2={R,T} t=2 , …., t=={R,T} t= poses are found Pose estimation V0.a 9
Intro. I Motivation I Pose est. I Newton's method I Iterative method Define SFM2 the structure estimation algorithm (assume poses are found or given o be discussed in the next chapter: Bundle adjustment Similar to pose estimation In pose estimation model is known pose is unknown Here(model finding by the iterative method Assume pose is known model is unknown The ideas of the algorithms are similar Pose estimation vo.a
Intro. | Motivation | Pose est.| Newton’s method | Iterative method Define SFM2: the structure estimation algorithm (assume poses are found or given) To be discussed in the next chapter : Bundle adjustment • Similar to pose estimation. – In pose estimation: model is known, pose is unknown. – Here (Model finding by the iterative method) Assume pose is known, model is unknown. – The ideas of the algorithms are similar. Pose estimation V0.a 10