Stanford OnlineStanford AA274A Principles of Robotic Autonomy | Autumn 2019 | Camera Models & Calibration
Stanford Online
Camera projection goals 0:05
The lesson starts by linking a real-world point to its image and then turning that relation around so you can rebuild the scene from images. It breaks the job into three steps: project the point from the camera frame to the image plane, convert that position to pixel coordinates, and relate the camera frame to a global world frame.
Pixels and homogeneous form 4:01
The image-plane coordinates are then shifted to the camera’s pixel grid with a translation, a scale by pixels per unit, and rounding. Because projection uses division by depth, it is nonlinear, so the points are lifted into homogeneous coordinates. In that form, the camera matrix K makes the projection a simple matrix times vector, and the final division by the third component brings you back to ordinary coordinates.
World to camera frame 27:02
The coordinate change starts with the case where the world and camera origins match. A point can then be written in either frame, and multiplying by the unit axes pulls out the matching coordinates. This gives a matrix of dot products, called the rotation matrix, that captures how one frame is turned relative to the other. It is an orthogonal matrix, with determinant one.
Rotation plus translation 36:01
When the two origins do not overlap, you first add the translation vector from the camera origin to the world origin, then apply the same rotation. In homogeneous coordinates, that becomes one matrix with R and T together, which makes the full world-to-camera transform compact. That same form is often used in reverse to estimate camera motion from image sequences, a process tied to visual odometry. The pieces then feed into the full projection equation with the camera matrix K, which carries the intrinsic settings such as focal length and the shift between reference frames. Because K, R, and T are usually unknown, camera calibration tries to recover them so you can reconstruct 3D from 2D images.
Calibration setup 45:00
Calibration starts with a known pattern, often a checkerboard. You place a world frame on that pattern, then collect pairs of 3D world points and their 2D pixel locations. Those pairs are easy to match because the pattern fixes exactly which point in the world maps to which pixel.
Solve then decompose 47:31
The first goal is to estimate a single projection matrix M from the point pairs. Each correspondence gives two linear equations, so with n points you get 2n constraints. Since M has 11 degrees of freedom up to scale, you need at least six correspondences, and more is better because real measurements contain error. You stack the equations into one matrix problem, solve for the nonzero vector that minimizes the residual, and pick the smallest eigenvector or singular vector. Once M is found, you split it into K, R, and T with an RQ decomposition to recover the camera’s intrinsic and extrinsic parts. Lens distortion is left out here, but it can matter, especially for fisheye or barrel distortion.
Depth from images 1:10:01
A single image cannot recover exact depth. Any point on the ray through the pinhole can produce the same image point, so the scene scale stays ambiguous unless you add more information.
Ways to recover structure 1:12:00
Four routes come up here. You can use landmarks or semantic understanding, use focus and the thin lens equation, use stereo vision with two cameras, or use structure from motion with multiple images over time.
Stereo triangulation 1:14:30
With two calibrated cameras, matching points from each image give two rays in space. Their intersection gives the 3D point, or as close as noise allows. In practice you minimize reprojection error, the gap between observed and predicted image points, and you need distinctive features to match the same point across both views.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
