Camera resectioning

In this context, we use ${\displaystyle [u\ v\ 1]^{T}}$ to represent a 2D point position in pixel coordinates and ${\displaystyle [x_{w}\ y_{w}\ z_{w}\ 1]^{T}}$ is used to represent a 3D point position in world coordinates. In both cases, they are represented in homogeneous coordinates (i.e. they have an additional last component, which is initially, by convention, a 1), which is the most common notation in robotics and rigid body transforms.

Referring to the pinhole camera model, a camera matrix ${\displaystyle M}$ is used to denote a projective mapping from world coordinates to pixel coordinates.

${\displaystyle z_{c}{\begin{bmatrix}u\\v\\1\end{bmatrix}}=K\,{\begin{bmatrix}R&T\end{bmatrix}}{\begin{bmatrix}x_{w}\\y_{w}\\z_{w}\\1\end{bmatrix}}=M{\begin{bmatrix}x_{w}\\y_{w}\\z_{w}\\1\end{bmatrix}}}$

where ${\displaystyle M=K\,{\begin{bmatrix}R&T\end{bmatrix}}}$. ${\displaystyle u,v}$ by convention are the x and y coordinates of the pixel in the camera, ${\displaystyle K}$ is the intrinsic matrix as described below, and ${\displaystyle R\,T}$ form the extrinsic matrix as described below. ${\displaystyle x_{w},y_{w},z_{w}}$ are the coordinates of the source of the light ray which hits the camera sensor in world coordinates, relative to the origin of the world. By dividing the matrix product by ${\displaystyle z_{c}}$, the z-coordinate of the camera relative to the world origin, the theoretical value for the pixel coordinates can be found.

${\displaystyle K={\begin{bmatrix}\alpha _{x}&\gamma &u_{0}&0\\0&\alpha _{y}&v_{0}&0\\0&0&1&0\end{bmatrix}}}$

The ${\displaystyle K}$ contains 5 intrinsic parameters of the specific camera model. These parameters encompass focal length, image sensor format, and camera principal point. The parameters ${\displaystyle \alpha _{x}=f\cdot m_{x}}$ and ${\displaystyle \alpha _{y}=f\cdot m_{y}}$ represent focal length in terms of pixels, where ${\displaystyle m_{x}}$ and ${\displaystyle m_{y}}$ are the inverses of the width and height of a pixel on the projection plane and ${\displaystyle f}$ is the focal length in terms of distance. [1] ${\displaystyle \gamma }$ represents the skew coefficient between the x and the y axis, and is often 0. ${\displaystyle u_{0}}$ and ${\displaystyle v_{0}}$ represent the principal point, which would be ideally in the center of the image.

Nonlinear intrinsic parameters such as lens distortion are also important although they cannot be included in the linear camera model described by the intrinsic parameter matrix. Many modern camera calibration algorithms estimate these intrinsic parameters as well in the form of non-linear optimisation techniques. This is done in the form of optimising the camera and distortion parameters in the form of what is generally known as bundle adjustment.

${\displaystyle {}{\begin{bmatrix}R_{3\times 3}&T_{3\times 1}\\0_{1\times 3}&1\end{bmatrix}}_{4\times 4}}$

${\displaystyle R,T}$ are the extrinsic parameters which denote the coordinate system transformations from 3D world coordinates to 3D camera coordinates. Equivalently, the extrinsic parameters define the position of the camera center and the camera's heading in world coordinates. ${\displaystyle T}$ is the position of the origin of the world coordinate system expressed in coordinates of the camera-centered coordinate system. ${\displaystyle T}$ is often mistakenly considered the position of the camera. The position, ${\displaystyle C}$, of the camera expressed in world coordinates is ${\displaystyle C=-R^{-1}T=-R^{T}T}$ (since ${\displaystyle R}$ is a rotation matrix).

Camera calibration is often used as an early stage in computer vision.

When a camera is used, light from the environment is focused on an image plane and captured. This process reduces the dimensions of the data taken in by the camera from three to two (light from a 3D scene is stored on a 2D image). Each pixel on the image plane therefore corresponds to a shaft of light from the original scene.

Zhang model [2][3] is a camera calibration method that uses traditional calibration techniques (known calibration points) and self-calibration techniques (correspondence between the calibration points when they are in different positions). To perform a full calibration by the Zhang method at least three different images of the calibration target/gauge are required, either by moving the gauge or the camera itself. If some of the intrinsic parameters are given as data (orthogonality of the image or optical center coordinates) the number of images required can be reduced to two.

In a first step, an approximation of the estimated projection matrix ${\displaystyle H}$ between the calibration target and the image plane is determined using DLT method.[4] Subsequently, applying self-calibration techniques to obtained the image of the absolute conic matrix [Link]. The main contribution of Zhang method is how to extract a constrained instrinsic ${\displaystyle K}$ and ${\displaystyle n}$ numbers of ${\displaystyle R}$ and ${\displaystyle T}$ calibration parameters from ${\displaystyle n}$ pose of the calibration target.

Assume we have a homography ${\displaystyle {\textbf {H}}}$ that maps points ${\displaystyle x_{\pi }}$ on a "probe plane" ${\displaystyle \pi }$ to points ${\displaystyle x}$ on the image.

The circular points ${\displaystyle I,J={\begin{bmatrix}1&\pm j&0\end{bmatrix}}^{\mathrm {T} }}$ lie on both our probe plane ${\displaystyle \pi }$ and on the absolute conic ${\displaystyle \Omega _{\infty }}$. Lying on ${\displaystyle \Omega _{\infty }}$ of course means they are also projected onto the image of the absolute conic (IAC) ${\displaystyle \omega }$, thus ${\displaystyle x_{1}^{T}\omega x_{1}=0}$ and ${\displaystyle x_{2}^{T}\omega x_{2}=0}$. The circular points project as

${\displaystyle {\begin{aligned}x_{1}&={\textbf {H}}I={\begin{bmatrix}h_{1}&h_{2}&h_{3}\end{bmatrix}}{\begin{bmatrix}1\\j\\0\end{bmatrix}}=h_{1}+jh_{2}\\x_{2}&={\textbf {H}}J={\begin{bmatrix}h_{1}&h_{2}&h_{3}\end{bmatrix}}{\begin{bmatrix}1\\-j\\0\end{bmatrix}}=h_{1}-jh_{2}\end{aligned}}}$.

We can actually ignore ${\displaystyle x_{2}}$ while substituting our new expression for ${\displaystyle x_{1}}$ as follows:

${\displaystyle {\begin{aligned}x_{1}^{T}\omega x_{1}&=\left(h_{1}+jh_{2}\right)^{T}\omega \left(h_{1}+jh_{2}\right)\\&=\left(h_{1}^{T}+jh_{2}^{T}\right)\omega \left(h_{1}+jh_{2}\right)\\&=h_{1}^{T}\omega h_{1}+j\left(h_{2}^{T}\omega h_{2}\right)\\&=0\end{aligned}}}$

It is a 2-stage algorithm, calculating the pose (3D Orientation, and x-axis and y-axis translation) in first stage. In second stage it computes the focal length, distortion coefficients and the z-axis translation.[5]

Selby's camera calibration method[6] addresses the auto-calibration of X-ray camera systems. X-ray camera systems, consisting of the X-ray generating tube and a solid state detector can be modelled as pinhole camera systems, comprising 9 intrinsic and extrinsic camera parameters. Intensity based registration based on an arbitrary X-ray image and a reference model (as a tomographic dataset) can then be used to determine the relative camera parameters without the need of a special calibration body or any ground-truth data.