Face Detection

After applying motion detection and heuristics, we have a fairly good indication of where there may be a human face present. However, we still need to determine if the object we have detected is a face. To do this, we scan a region close to the location detected in the image at three sizes ( $40\times40$, $60\times60$, $80\times80$). The scanning is illustrated in figure 4 (nine frames at each size, for a total of twenty-seven frames). For each frame we apply the following algorithm:

1.

Resize frame to $20\times20$ pixels

2.

Preprocess frame:

(a)

subtract best-fit linear function

(b)

histogram equalize

3.

Project frame onto vectors which define a "face space"

4.

Reconstruct the frame from the projection

5.

Compute reconstruction error

From the reconstruction error we can decide if the frame contains a human face or not. A face is deemed present at the frame with the smallest reconstruction error. This error also has to be smaller than a threshold. This threshold is set manually from experience.

The preprocessing step is adapted from [7] and [5]. First we subtract a best fit linear function to correct for extreme lighting conditions. This basically removes some shades and normalizes the intensity values to a small degree. Then we histogram equalize the subframe to further normalize the intensity values and enhance features.

The "face space" is computed by a principal components analysis of the ORL face database² (figure 5), which consist of 400 human face images $\boldsymbol{\Gamma}_{1}, \boldsymbol{\Gamma}_{2},\ldots, \boldsymbol{\Gamma}_{400}$. This technique of representing face images using principal components was originally suggested by [6], and further developed by [8]. The average face is defined by

$$\boldsymbol{\Psi} = \frac{1}{400}\sum_{i=1}^{400}\boldsymbol{\Gamma}_{i}$$

The average face is then subtracted from each image and vectorized:

$$\boldsymbol{\Phi}_{i} = (\boldsymbol{\Gamma}_{i} - \boldsymbol{\Psi})^{\nu}$$

Let $D = [\boldsymbol{\Phi}_{1}\boldsymbol{\Phi}_{2}\ldots\boldsymbol{\Phi}_{400}]$, and C = DD^t. We want to find the eigenvectors u_i of C. Since the matrix C is of size $10384 \times 10384$ (the images are of size $92 \times 112 = 10384$ pixels)³ and we have a maximum of 400 independent vectors, we apply the following technique:

Let L = D^tD and v_i be the eigenvectors of L. We can now find u_i from the following equation (a linear combination of the face images) [8]:

$$\boldsymbol{u}_{i} = \sum_{k=1}^{400}\boldsymbol{v}_{ik}\boldsymbol{\Phi}_{i}, \,\,\,\, i=1,\ldots,400$$

u_i are called the principal components of D, and when converted back to matrices we can view these vectors as the eigenfaces of the ORL face database. Some of these eigenfaces are shown in figure 6 (ordered according to eigenvalue).

The preprocessed subframe $\boldsymbol{\Phi}$ (mean subtracted) is projected into face space by

$$\omega_{k} = \boldsymbol{u}_{k}^{t}\boldsymbol{\Phi}, \,\,\,\, k=1,\ldots,20$$

We only use the first twenty principal components to define face space. It is reconstructed by

$$\boldsymbol{\Phi}_{r} = \sum_{k=1}^{20}\omega_{k}\boldsymbol{u}_{k}$$

The reconstruction error $\epsilon =\, \parallel\!\!\boldsymbol{\Phi} - \boldsymbol{\Phi}_{r}\!\!\parallel^{2}$is a measure of how typical a face this is, and is applicable for face/non-face classification.

  

Figure 4: The region located from motion detection and heuristics is scanned (nine frames, eight shown here plus the center location).

  

Figure 5: The Olivetti Research Laboratory (ORL) face database.

  

Figure 6: Some examples of eigenfaces (the number indicates the principal component number, ordered according to eigenvalues).