Stanford OnlineStanford AA274A Principles of Robotic Autonomy | Autumn 2019 | Stereo Vision
Stanford Online
Stereo vision setup 0:05
Stereo vision tries to recover 3D structure from two images, because one image alone leaves depth ambiguous. The main task is matching points between the two views. Epipolar constraints help by limiting the search to a single line in the other image, instead of the whole frame.
Epipolar geometry basics 2:01
Two camera centers, O and O', define a baseline. The point where that baseline meets each image plane is an epipole, and the line through an epipole and a matched image point is an epipolar line. The plane through O, O', and a 3D point is the epipolar plane, and its intersection with the image plane gives the line you search along.
Fundamental matrix 10:31
The epipolar constraint can be written as p'ᵀFp = 0, where F is the fundamental matrix. From F you can get an epipolar line in one image from a point in the other, and the reverse direction uses Fᵀ. F is 3 by 3, singular, and can be estimated from point matches, with the eight-point algorithm as a simple method.
Enforcing rank 21:01
The eight-point method still leaves scale unclear, so the solution is normalized by fixing the norm of the learned F to one. That gives you a least-squares estimate, then SVD is used to force F to be rank two, which makes it singular and able to send epipoles to zero as required.
Rectifying and matching 24:31
The matrix F depends on the camera setup at that moment, so it changes if you move or rotate the cameras. To find matching points, you often start by clicking or otherwise giving initial correspondences, then use either region-based matching on image patches or feature-based matching with methods like SIFT, SURF, and ORB. With rectified images, matching is easier because you search along one row, and triangulation from disparity gives depth. Wide baselines improve distant detail, but too much separation hides points; short baselines keep points visible, but depth becomes noisy.
Motion and structure 41:00
When the camera pose is unknown, the problem becomes structure from motion. You try to recover both the 3D points in the scene and the camera matrices from multiple views. The scale of the scene cannot be recovered absolutely, because the same images can come from a large object far away or a small object close by, so the result is only defined up to similarity, including rotation, translation, and scale.
Bundle adjustment and odometry 45:00
Stereo uses pairs of images to recover 3D points, then moves on to the next pair. Bundle adjustment is the more common non-linear way to solve structure from motion. It minimizes reprojection error, the gap between a projected 3D point and the point actually seen in the image. Visual odometry repeats this process to estimate a robot's motion from camera input alone.
Mars rover example 47:02
Visual odometry can also produce a 3D map, but its main goal is position and motion. Mars rovers use it to locate themselves on terrain that can look like a feature-poor desert. When rocks or other landmarks appear, they provide enough detail to track pixel movement from one frame to the next.
Next vision topics 49:31
The discussion ends by turning away from 3D reconstruction. The next material will cover image processing tasks such as blurring images and detecting edges.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
