|
1.
2.
Introduction
3.
In many current face recognition applications, such as
video surveillance security and content annotation in web environment,
low-resolution faces are commonly encountered and negatively impact on
reliable recognition performance. Especially, the recognition accuracy of
current intensity-based FR systems can be dropped off significantly if
the resolution of facial images is smaller than a certain level (e.g.,
less than 20 x 20 pixels). To cope with low-resolution faces, this
research proposes a novel color FR method, which is not only simple in
implementation, but also guarantees extended applicability to most of the
FR applications. Experimental results show that color features decrease
recognition error rate by at least an order of magnitude over
intensity-driven features when low-resolution faces (25 x 25
pixels or less) are applied to three FR methods.
4. The proposed
color face recognition framework
We formulate the base-line color FR framework that can
make efficient use of facial color features to overcome low-resolution
faces. RGB color face images are first converted into another different
color space (e.g, YCbCr color space). Let  be a color
face image generated in the color space conversion process. Then, let  be a  spectral
component vector of  (in the form
of column vector by lexicographic ordering of pixel elements of
two-dimensional spectral images), where  and  denotes  dimensional real space. Then, face vector is defined as
augmentation (or combination) of each spectral component such that  where   and  represents
the transpose operator of matrix. Note that each should be
normalized to zero mean and unit variance prior to their augmentation.
Face vector  can be
generalized in that, for  the face
vector could be defined by gray-scale only while, for  it could be
defined by spectral component configuration like YCbCr
or YQCr by column order from YCbCr and
YIQ color spaces.
Most subspace
FR methods are separately divided into the training and testing stages.
Given a set  of  color face
images,  should be
first rescaled into the prototype template size to be used for creation
of a corresponding face vector  With a formed training set  of face vector
samples, feature subspace is trained and constructed. Rationale behind
the feature subspace construction is to find a projection matrix  by
optimizing criteria to get lower dimensional feature representation  where each
column vector  is a basis
vector spanning the feature subspace,  and  It should
be noted that  For the testing
phase, let  be a
gallery (or target) set consisting of  prototype
enrolled face vectors of known individuals, where  In
addition, let  be an
unknown face vector to be identified or verified, denoted as probe (or
query), where  To perform
FR tasks on probe,  and are projected onto the feature subspace to get
corresponding feature representations such that
|
 and 
|
(1)
|
where  and  A nearest
neighbor classifier is then applied to determine identity of by finding
the smallest distance between  and  in the
feature subspace as following:
|
 and 
|
(2)
|
where  returns a
class label of face vector and  denotes
distance metric. In order to exploit why role of color is getting
significant as face resolution is decreased within our base-line color FR
framework, theoretical analysis will be given in the following sections.
5. Experiments
In the practical FR
systems, there are two possible FR approaches to perform FR tasks over
lower resolution probe images. The first method is to prepare multiple
training sets of multi-resolution facial images, and then construct
multiple feature subspaces, each of which is charged with a particular
face resolution of probe. Alternative method is that lower resolution
probe is reconstructed to be matched with prototype resolution of
training and gallery facial images by adopting resolution enhancement or
interpolation techniques. The second method would be appropriate in
typical surveillance FR applications in which high-quality training and
gallery images are usually employed, but probe images transmitted from
surveillance cameras (e.g., CCTV) are often low-resolution. To demonstrate
the effect of color on low-resolution faces in both FR scenarios, two
sets of experiments have been carried out in our experimentation, namely
Experiment 1 and 2. The Experiment 1 is to assess the impact of color on
recognition performance with varying face resolution of probe given
multi-resolution trained feature subspaces. On the other hand, the
Experiment 2 is to conduct the same assessment when a single-resolution
feature subspace trained with high-resolution facial images is only
available to the actual testing operation.
Fig. 1 shows examples of
facial images containing face resolution variations used in our
experiments.
Fig. 1. The examples of facial
images from color FERET according to six different face resolutions.
Low-resolution observation below original 112 x 112 pixels is
interpolated using nearest neighbor interpolation.
5.1 Experiment 1
Fig. 2 shows CMC curves for
the identification rate (or BstCRR) comparisons between the gray-scale
and color features with respect to six different face resolutions in PCA,
FLDA, and Bayesian FR methods. As can be seen in CMC curves obtained from
gray-scale R feature (in the left side of Fig. 5), the differences in
BstCRR between face resolutions of 112 x 112, 86 x 86, and
44 x 44 pixels are relatively marginal in all three FR
methods. However, BstCRRs obtained from low-resolution of 25 x 25
pixels and below are tended to be significantly deteriorated in all three
FR methods. For example, for PCA, FLDA, and Bayesian methods, rank one
BstCRRs (identification rate of top response being correct) decline from
77.20%, 83.69%, and 82.46% to 56.03%, 37.29%, and 62.32%, respectively,
as face resolution is reduced from 112 x 112 to
15 x 15 pixels.
Fig. 2. Identification rate
(or BstCRR) comparison between gray-scale and color features with respect
to six different face resolutions of each pair of training, gallery, and
probe facial images in three FR methods. The graphs on the left side were
resulted from gray-scale feature ¡®R¡¯, while those on the right side were
generated from color feature ¡®RQCr¡¯ for each face resolution.
(a) PCA. (b) FLDA. (c) Bayesian.
5.2 Experiment 2
In Experiment
2, face resolution of training images was fixed as 112 x 112 pixels,
while the resolution of probe was varied as six different resolutions
depicted in Fig. 4. Since the high-quality gallery images are usually
pre-registered in FR systems before testing probes, we assume that the
resolution of gallery is the same as the training facial images, i.e.,
112 x 112 pixels. In Experiment 2, R from RGB color space was used as
gray-scale feature. Due to the best performance from Experiment 1, RQCr
was adopted as a color feature.
Fig. 3 shows CMC curves
with respect to six different probe resolutions in the both cases of
gray-scale (in the left side) and color features (in the right side) in
PCA, FLDA, and Bayesian. To obtain a low-dimensional feature representation
for a lower face resolution probe, the probe has been up-sampled to have
the same resolution of training faces by using a cubic interpolation
technique in Fig. 3. From Fig. 3, in case of a gray-scale feature, we can
see considerable identification rate degradation in all three FR methods,
considering low-resolution (25 x 25 pixels and below) probes compared to relatively
high-resolution counterparts (above 44 x 44
pixels). Especially, similar to the results from Experiment 1,
identification rate resulting from FLDA is significantly deteriorated at
low-resolution probes. The margins of rank one identification rate
between 112 x 112 and each 25 x 25, 20 x 20, and 15 x 15 pixels gray-scale probes in FLDA are 25.66%, 43.77%,
and 62.41%, respectively. In case of color feature, BstCRR improvement is
made at all probe face resolutions in all three FR algorithms. As
expected, face color information greatly improves identification
performance obtained from low-resolution probes (25 x 25
pixels and below), compared to gray-scale feature. In PCA, by
incorporating color feature, the BstCRR margins between grayscale probe of the 112 x 112
resolution and color probe of
the 25 x 25, 20 x 20, and 15 x 15 resolutions are reduced to 3.33%, 4.77%, and 8.02%,
respectively. In FLDA, these differences are decreased to 6.65%, 7.28%,
and 11.60% at 25 x 25, 20 x 20, and 15 x 15 resolutions, respectively. In addition, in Bayesian,
1.47%, 2.61%, and 5.64% performance margins decrease are achieved with
the above three different probe resolutions thanks to color feature.
Fig.3. Identification rate
comparison between gray-scale and color features with respect to six
different face resolutions of probe images. The graphs on the left side
were resulted from ¡®R¡¯ as a gray-scale feature from RGB color space,
while those on the right side were generated from ¡®RQCr¡¯ as a
color feature for each face resolution. Note that a single feature
subspace trained with face images having resolution of 112 x 112 pixels was
given to test probe images with varying face resolutions. (a) PCA. (b)
FLDA. (c) Bayesian.
6.
Discussion
Traditionally, low-resolution FR modules have been
extensively used in video surveillance-like application. Recently, FR
applications in web environment are getting increasing attention due to
the popularity of online social networks (e.g., Myspace and Facebook) and
their high commercialization potentials [4-7]. Under web-based FR
paradigm, many devices such as cellular phone cameras and web cameras often
produce low-resolution or low quality face images which, however, can be
used for recognition purposes [4-5]. As shown in our experimentation,
color-based FR outperforms gray-scale based FR over all face resolutions.
Particularly, thanks to color information, both identification and
verification rates obtained using low-resolution 25 x 25 or 20
x 20 templates are comparable to rates obtained using much
larger gray-scale images such as 86 x 86
pixels. Moreover, as shown in Fig. 1, the face DB, used in our
experimentation, contain images obtained under varying illumination
conditions. Hence, the robustness of color in low-resolution FR appears
to be stable with respect to the variation in illumination, at least, in
our experimentation. These results
demonstrate that facial color can be reliably and effectively utilized in
real-world FR systems of practical interest, such as video surveillance
and promising web applications, which frequently have to
handle the low-resolution face images taken under uncontrolled
illumination conditions.
|
