Method to compute the projection matrix of a Stereo Vision system

Last Revision Date: 7/3/2015

This article details the calculations required to compute the projection matrix of a Stereo Vision system with respect to the Faugeras method.

NOTE: This article refers to Olivier Faugeras, a well-known author in the field of computer vision. We cannot guarantee the accuracy of our interpretation of Faugeras' work. This article refers to the 1996 (2nd printing) version of Faugeras' book, "Three-Dimensional Computer Vision", from The MIT Press.

The "general" P matrix seems to be given on page 57 eq 3.31. In our stereo vision system:

f = focal length
ku = kv = 1 because our pixels are square
theta = 90 because our axes are orthogonal.

So P is very simple and is like it is given at the bottom of page 43 dq 3.12 but with ku = kv = 1:

    [ -f   0   u0  0 ] 
P = [ 0   -f   v0  0 ] 
    [ 0    0   1   0 ]

For example, at 1024x768 resolution, the projection matrix might be:

[ -802.68   0        -607.60    0 ] 
[ 0        -802.68   -383.56    0 ] 
[ 0         0           1       0 ]

Note: u0 = -center column and v0 = -center row. In doing this, we make some assumptions that are explained below.

With the triclopsRCDFloatToXYZ() function, we apply the following equations:

X/Z = u/f 
Y/Z = v/f 
B = baseline
f = focal length

In these equations, the origin of (u,v) is the principal point, and the u axis is parallel with the X axis and in the XZ plane, similarly for the v axis.

In this geometry, positive u == positive X and also for v and Y. We align these axes with the row/column of the image. So the X axis in our world system points to the right of the camera, the Y axis points down and the Z axis points forward. In this arrangement, increasing column == increasing u == increasing X and again, also for row, v and Y.

To calculate u,v you use:

u = c - c0
v = r - r0

Where c0, r0 are the "image center" as returned from triclopsGetImageCenter()

(u,v) are in the same units (pixels) as (r,c) but (r,c) has its origin at the upper left corner of the image and (u,v) has its origin at the principal point (or the "image center). Using our simple projective equations above:

u = Xf/Z, v = Yf/Z - converting to row
col c = Xf/Z + c0, r = Yf/Z + r0

Applying the P matrix from Faugeras we get:

[u]   [ -f  0  u0  0] [x]   [ -fx + u0z ]   [ -fx/z + u0 ]
[v] = [ 0  -f  v0  0] [y] = [ -fy + v0z ] = [ -fy/z + v0 ]
[s]   [ 0   0  1   0] [z]   [     z     ]   [      1     ]

The result is similar but not identical. If we interpret Faugeras' u,v for our row,column, then the following needs to happen to make the systems match.

1. Set u0 = -c0, v0 = -r0. Now Faugeras gives

u = -fx/z - c0
v = -fy/z - r0

2. Reverse the u / c and v / r axes i.e. u = -c, v = -r, which gives:

c = fx/z + c0 
r = fx/z + r0

These are the same as the equations we use, and why we interpret u0 = -c0 and v0 = -r0.

NOTE: Please make sure to understand which directions your axes are pointing when you convert from the Triclops library to your projective space representation.