Adaptive face modelling for reconstructing 3D face
Published in IET Computer Vision Received on7th March2013 Revised on27th December2013 Accepted on6th January2014 doi:
10.1049/iet-cvi.2013.0220
ISSN1751-9632 Adaptive face modelling for reconstructing3D face shapes from single2D images
Ashraf Maghari1,Ibrahim Venkat1,Iman Yi Liao2,Bahari Belaton1
1School of Computer Sciences,Universiti Sains Malaysia,Pinang,Malaysia
2School of Computer Science,University of Nottingham Malaysia Campus,Semenyih,Selangor,Malaysia
E-mail:myashraf2@https://www.wendangku.net/doc/787560170.html,
Abstract:Example-based statistical face models using principle component analysis(PCA)have been widely deployed for three-dimensional(3D)face reconstruction and face recognition.The two common factors that are generally concerned with such models are the size of the training dataset and the selection of different examples in the training set.The representational power(RP)of an example-based model is its capability to depict a new3D face for a given2D face image.The RP of the model can be increased by correspondingly increasing the number of training samples.In this contribution,a novel approach is proposed to increase the RP of the3D face reconstruction model by deforming a set of examples in the training dataset.A PCA-based3D face model is adapted for each new near frontal input face image to reconstruct the3D face shape.Further an extended Tikhonov regularisation method has been employed to reconstruct3D face shapes from a set of facial points.The results justify that the proposed adaptive PCA-based model considerably improves the RP of the standard PCA-based model and outperforms it with a95%con?dence level.
1Introduction
The problem of three-dimensional(3D)facial modelling remains as a partially solved problem in the?eld of computer vision.The deployment of3D faces in image processing applications has received substantial attention during the recent decades.Three-dimensional face modelling has potential applications in various?elds,such as virtual reality simulations,face recognition[1,2]and plastic surgery[3].For example,in biometric identi?cation, it has been shown that face recognition rate could be signi?cantly improved by incorporating3D face shapes with2D face images[4].
The objective of3D facial reconstruction systems is to recover the3D shape of individuals from their2D pictures or video sequences.However,accurate reconstruction of a person’s3D face model from his/her2D face images still remains as a challenge.Even with the aid of the most sophisticated technology,3D facial models are not directly obtained from images but by laser-scanning of people’s faces[5].This leads to the following limitations:
1.These scanners are usually expensive and targeted to work in tightly controlled environments.
https://www.wendangku.net/doc/787560170.html,ser-scan-based reconstruction could not be applied in certain scenarios;for example,a person’s face had been damaged during an accident and his/her face needs to be reconstructed in order to assist plastic surgery.Here,we cannot apply laser-scan but we can reconstruct the face using computational techniques with the aid of available photos of the person taken prior to the accident.
A3D face can be reconstructed from a single image,or from multiple images.Some of the typical approaches based on3D face reconstruction from multiple images include video-based and silhouette-based methods. Video-based techniques are used for3D face reconstruction from images captured from different viewpoints.There are a number of techniques based on video frames such as reconstruction using generic model morphing[6],linear basis functions that do not require a generic model[7]and shape-from-motion(SFM)[8,9]where motion information of feature points from multiple video frames are extracted to obtain3D face reconstruction.
This paper focuses on the problem of reconstructing3D face shapes from single2D image.This technique does not require setting up of multiple cameras to capture the objects simultaneously,and thereby it is not limited to working in controlled environments.
Shape-from-shading(SFS)[10–12]is one of the several approaches aim in the reconstruction of3D faces from a single image.This approach utilises the idea that the depth information is proportional to the intensity of a face image being acquired through a given/chosen re?ectance model. Based on this idea,SFS estimates the illumination direction in the2D face image to produce the3D shape of the surface. There are also conventional learning-based methods,such as neural networks[13,14]and typical statistical learning-based methods such as hidden Markov model (HMM)[15],Markov random?eld(MRF)[16]and analysis by synthesis using3D morphable model(3DMM)[17]. Recently,Kemelmacher-Shlizerman and Basri[18]have proposed an approach that combines shading information
https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454 doi:10.1049/iet-cvi.2013.0220
441 &The Institution of Engineering and Technology2014
with generic shape information derived from a single reference model by utilising the global similarity of faces. This method uses only a single reference model of a different person’s face to reconstruct the3D face shape.It does not need a learning stage to build a model for representing input faces.The involved?tting process requires boundary conditions and parameters to be adjusted during the reconstruction process.Such a3D reference model that ignores shape similarities pertaining to the input image may result in inaccurate3D shape estimation.
The advancement of3D scanning technology has led to the creation of more accurate3D face exemplar models[19]. Example-based modelling allows more realistic face reconstruction than other methods[20,21].However,the quality of face reconstruction using such models are affected by the chosen examples.For example, Kemelmacher-Shlizerman and Basri[18],and Jose et al.
[22]have emphasised that learning a generic3D face model requires large amounts of3D faces.Furthermore,analytical results in[23]show that in many cases the representational power(RP)of the model may vary if the model is trained with a different sample although the same sample size is retained.
The PCA-based model proposed by Blanz and Vetter with relatively small sample size(100faces)used for face recognition has obtained reasonable results[24].Castelan and van Horebeek[25]apply partial least square regression technique to predict3D face shape from single image by describing the relationship between the intensity images and 3D shape.However,the method was sensitive to pose and illumination changes[25].
Our?ndings in[23]indicate that shapes are more amendable to PCA-based modelling than textures,as textures,unlike shape features,are subject to vast variation [26].Therefore,the model we intend to propose here is based on modelling of shapes.In fact,the reconstruction of 3D faces from2D images using a shape-based model is relatively simple.A popular method is a regularisation-based reconstruction where a few feature points are selected as the observations for reconstruction [26].Alternatively,the regularisation that uses the3DMM to reconstruct a3D face shape from facial2D points has also been presented in[27,28].On the other hand,the face shapes reconstructed using regularisation are bound by the RP of the PCA-based model[23].
A short version of the adaptive model has been presented in our previous paper[29].In this paper,we have extended this idea,with extensive experiments,quantitative and qualitative analysis,to address the problem of increasing the RP of the PCA-model to improve its capability in depicting a new3D face from a given input face image.A3D face shape modelling scheme is proposed to handle the vital model adaptation part of the PCA-based model.Model adaptation here infers that the model can be tuned with the guidance of a given input image by deforming3D faces in the training data set.There are other methods that intend to create synthetic views in training sets for face recognition. These synthetic views include different pose[30]and different expression[31].However,this work is different and novel in the context of deforming3D faces using the given input face for the problem of3D face reconstruction.
A different representation of faces such as spherical harmonics[32–34]may have a different effect on the reconstruction accuracy.Further,we study the effect of deforming faces in the training set with respect to the quality of reconstruction of3D faces.
Before the training phase,exemplar face shapes are deformed as governed by the input face.In order to learn the deformation trend from the input face,information from a set of facial landmarks has been acquired.Further,we use the thin plate spline(TPS)technique[35,36]for transforming the exemplar faces into deformed faces based on those facial landmarks(see Section3.1.2).
An extended Tikhonov regularisation method,which was presented in[26],is also employed in this paper to test the adaptive model by reconstructing3D faces from a set of feature points.The regularisation mechanism intends to?nd a tradeoff between?tting3D shape to the given2D facial landmarks,thereby producing plausible solution in terms of prior knowledge[28].The de facto standard USF human ID 3D database[17]has been used to experiment the proposed adaptive PCA model.For a given2D face,the objective is to reconstruct its corresponding3D model using the proposed algorithm with a good degree of accuracy.For3D face shape reconstruction from real2D images,2D images from CMU-PIE database[37]have been used to qualitatively evaluate the proposed scheme.
This paper is organised as follows:Section1provides a brief introduction and throws some light on related studies about the problem of3D face reconstruction.Section2 demonstrates the RP of the PCA-based model.Section3 describes the methodology of the proposed adaptive3D face shape modelling approach.Section4is devoted to the formulation of the proposed reconstruction techniques,and Section5deals with the experimental evaluation and associated discussions.Finally,Section6concludes the research.
2Representational power of PCA-based model
The RP of the PCA-based model has been de?ned in[23]as its capability to depict a new3D face shape for a given input face image.The capability of the model under study can be measured by evaluating the quality of reconstruction of the output3D face yielded by the model(with respect to its ground-truth).To compare two given3D surfaces,we can compute the Euclidean distance between them as follows
Ed w=
n
i=1
(s i?s r i)2
n
(1)
where Ed w is the weighted Euclidean distance,s is the testing face shape,s r is the reconstructed face shape and n is the number of vertices of the face shape.
As an example,Fig.1demonstrates the signi?cance of RP in terms of evaluating the quality of reconstruction of a given face shape.A typical test face has been projected onto various PCA subspaces(PCA10,PCA30and so on by gradually increasing the size of the training set)and its corresponding 3D face shape is reconstructed.As one can see in Fig.1, the3D representation of reconstructed face shapes gets more realistic and more closer to the ground truth when Ed w decreases.This demonstration hypothesises that the PCA model that represents a new3D face with less Ed w possesses more RP.
In the following section,we propose a novel method that is capable of improving the RP of the PCA model for the same training data set by reducing the weighted Euclidean distance Ed w of the reconstructed face shapes.
https://www.wendangku.net/doc/787560170.html,
442
&The Institution of Engineering and Technology2014IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220
3Adaptive 3D face shape modelling:proposed method
The bird ’s eye view of the proposed scheme is shown in Fig.2.It shows how a given input face gets reconstructed via a deformation synthesis mechanism.Basically,the proposed adaptive PCA-based model intends to enhance the RP of the standard PCA-based model by adding deformed faces into the training examples.The following subsections demonstrate the steps involved in building an adaptive PCA model for a given face.3.1
Deforming 3D exemplar faces
To learn the deformation mapping between the input image and neutral 3D models in the database,initially a set of facial landmarks are extracted.Since the facial geometry and aspect ratios between the input face image and the 3D models are different,the following procedure is proposed to transfer the learned deformation into a 3D neutral model in the database for synthesis.
1.Register the input face with the 3D face shape using prede ?ned facial landmarks.
2.Deform 3D face shapes using TPS [35,36]:
(a)Establish a mapping between the landmark set of the input face and the 3D face shape.
(b)Apply the mapping established in step (a)to the remaining vertices in the 3D face shape to translate them to the new positions.
Fig.3demonstrates the deformation procedure applied to 3D faces using a set of 20chosen feature points.In the following two subsections,we explain the deformation procedure in detail.
3.1.1Facial feature points registration:Similar to Farkas [38]as referred by Knothe et al.[39],?rst we choose landmarks over facial regions such as eyes,nose,mouth and chin for the purpose of face alignment and shape deformation.The landmarks of the input image are ?rst aligned to the corresponding points of a reference 3D face shape (3D average face shape)so that the selected landmarks are in the same coordinate system with that of the training 3D face shapes.The input feature points are aligned using a standard algorithm called ‘Procrustes analysis ’.Procrustes determines a linear transformation (translation,re ?ection,orthogonal rotation and scaling)of the points in Shape A (input object)to best conform them to the points in Shape B (reference object).The concept of Procrustes analysis is similar to iterative closest point [40],which has the following four main steps:
1.Select control points from the ?rst object.
2.Determine the correspondence points in the second object.
3.Calculate the optimal transformation between these two point sets based on their correspondence relationships.
4.Transform the points.Repeat step 2,until convergence is achieved.
3.1.2Deforming 3D examples using the TPS technique:TPS is a commonly used basis function
for
Fig.1Typical test face being projected onto various PCA subspaces (PCA10,PCA30,…)
PCA10indicates that the training set has 10examples,PCA30indicates that the training set has 30examples and so on.The lesser the ED w ,the better the RP of the
model
Fig.2Proposed scheme of the adaptive 3D face shape reconstruction model
https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454doi:10.1049/iet-cvi.2013.0220
443
&
The Institution of Engineering and Technology 2014
representing coordinate mappings from rigid to non-rigid deformations.It is used for estimating a deformation function between two surfaces [31].
Let g 0and g 1be two 2D/3D shapes,and X =x 1,x 2,…,x m and Y =y 1,y 2,…,y m where X ,g 0and Y ,g 1,and m is the number of corresponding points.Then,we can de ?ne a warping function F that intend to warp point set X onto Y using
F (x j )=y j ,
j =1,2,...,m
(2)
The interpolation deformation model [41]is given in terms of
the warping function F (x )using
F (x )=x A +KW
(3)
where x ∈g 0,A is an af ?ne transformation,W is a ?xed m -dimensional column vector of non-af ?ne warping parameters constrained to X T W =0and K is an m -dimensional row vector.K i is the Green ’s function σ(|x ?x i |)which can be written as
K i (x )=s (|x ?x 1|),s (|x ?x 2|),...,s (|x ?x m |)
T
(4)
The Green ’s function σ(|x ?x i |)can be calculated by determining the Euclidean distances between vectors x and x i as
x ?x i =
x 2+ x i 2?2x ·x i
(5)
In our model,the sets X and Y correspond to a prede ?ned number of landmarks on 3D neutral scan and the input face image (2D/3D face),respectively.
A TPS can minimise the following energy function
E l =1m m i =1|y i
?F (x i )|+l J ,
i =1,...,m (6)
where y i represents the i th 2D/3D point (landmark)of the input face (base points)and x i =(x i 1,…,x id )is the i th point given in d -dimensions (in our case d could be 2D or 3D,x i represents the set of points used to warp an image),m is the total number of corresponding points,J is a smoothness
penalty function in d -dimensions and λis the smoothness parameter [36].
For the case when λis zero,the data points are interpolated without any smoothing.On the other hand,when λtends to in ?nity,we get the problem of ?nding a plane which is the least square ?t of the data.For the interpolating case,TPS provides a linear system of equations given by
W A =K T X ?1Y (7)
where K ij =σ(|x i ?x j |).For the approximating case,(6)can be written in the following matrix form
XA +(K +m l I )W =Y ,
X T W =0
(8)
which actually performs the standard QR decomposition.Obviously,a QR decomposition of the matrix X produces an orthogonal matrix Q and an upper triangular matrix R such that X =QR [36].The following two examples demonstrate the registration process using the TPS technique.Illustrative Example 1:Registration 3D faces to a test 3D face:Fig.4shows the prede ?ned landmark set on the pair of face scans and their corresponding deformed 3D faces generated based on the warping function F in (2).The deformation procedure works as follows:For 3D –3D deformation/registration,?rst 20prede ?ned vertices on the source mesh S are selected.Then 20corresponding vertices on the target mesh T are found.TPS is then used to map the sample source vertices to their corresponding targets.A new mesh T ′is generated based on the TPS map.TPS basically de ?nes a global warping pertaining to a given space and is therefore used to warp the entire source mesh onto the target mesh.
Illustrative Example 2:Registration of 3D faces to a given 2D face:Fig.5shows six deformed 3D faces for two original faces registered with reference to three typical 2D images.For 3D –2D deformation/registration,the source image is in 2D form.We further use the TPS technique in order to establish a correspondence between the 3D face mesh T and the 2D face mesh of the given image based on the 202D selected facial landmarks.A new 3D mesh T ′is generated based on the correspondence
mappings.
Fig.3Typical illustration of the TPS-based facial deformation process
https://www.wendangku.net/doc/787560170.html,
444
&The Institution of Engineering and Technology 2014
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220
3.2Adaptive PCA model construction
The characteristic shape properties of the 3D face shapes are derived from a dataset of 3D scans.The 3D shapes are aligned with one another in such a way that a 3D –3D correspondence for all vertices is obtained [17].In addition,each registered/deformed 3D face model captures only some characteristics from its corresponding input face.This leads to a 3DMM that is a linear combination of 3D face shapes,some of
which are obtained as a result of a deformation transfer mechanism from one input face to neutral 3D example face shapes.The number of p vertices corresponding to each face can be vectorised as
s i =x i 1,y i 1,z i 1,...,x ip ,y ip ,z ip T
,i =1,...,m (9)where s i is of dimension n =p ×3,p is the number of
vertices
Fig.4Typical 3D –3D registration scheme based on the proposed deformation model
The top row shows three typical test faces (Faces 81,89and 85).The leftmost column (column 1)shows original two 3D face images (Faces 1and 2).The corresponding deformed face images are shown in columns 2,3and 4(right
most)
Fig.5Typical 3D –2D registration scheme based on the proposed deformation model
The top row shows three 2D face images (Faces 15,68and 18)from CMU-PIE database.The leftmost column (column 1)shows original two 3D face images (Faces 1and 2).The corresponding deformed face images are shown in columns 2,3and 4(rightmost)https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454doi:10.1049/iet-cvi.2013.0220
445
&
The Institution of Engineering and Technology 2014
and m is the sum of the number of original training face shapes and the number of deformed face shapes.
The dimensions of the shape vectors are very large compared to the sample size,while the number of vertices, p,is equal to75972,and the sample size m comprises about100face shapes.If PCA is applied to the data,the covariance matrix will be n×n,which is very huge. However,the same eigenvectors and eigenvalues can be derived from a smaller m×m matrix,while the covariance matrix C[1]can be written as
C=1
m
XX T=1
m
m
i=1
x T
i
x
i
[R m×m(10)
where X=[x1,x2,…,x m]T∈R m×m is the data matrix with each vector,x i=s i?s0is centred around the mean s0,and i =1,…,m.PCA is then applied on the covariance matrix C. As a result of the analysis,a new shape vector,s rec,can be expressed as
s rec =s0+E a=s0+
m
i=1
a i e i(11)
where E=[e1,e2,…,e m]is the matrix of scaled basis vectors, andαi is the coef?cient of the scaled basis vector e i.Since E is an orthogonal matrix,the PCA coef?cientsαof the vector x=s?s0∈R n can be derived from(11)as
a=E T x(12)
After building the adaptive PCA model,3D faces can be reconstructed from a limited number of feature points using regularisation.In the following section,we will demonstrate the reconstruction procedure.
4Three-dimensional face shape reconstruction:fitting stage
The goal of a robust?tting algorithm is to reduce the chance of?tting noise and increase the accuracy in predicting new data.Noise in our case may occur due to intricacies in choosing the input feature points,which depends on the acquisition systems or the uncertainties imposed by the used alignment methods.Fitting the shape model to a given2D image is formulated as an optimisation problem to solve the linear system in(11), whereas the goal of this inverse problem is to?nd the shape coef?cientsαrapidly and ef?ciently,given a shape vector x=s i?s0and the orthogonal matrix E.However, there are three major dif?culties with3D face shape reconstruction,namely
1.The number of available feature coordinates is much fewer than the coordinates of the reconstructed face shape.
2.The relationship in(11)is usually not linear.
3.The inverse problem is ill-posed and ill-conditioned.
These dif?culties could cause the solution to be unstable and very sensitive to noise in the input data.Therefore, regularisation can be used in order to enforce the result to be plausible with the aid of acquiring some prior knowledge from the3D object domain[39].Tikhonov regularisation is a sophisticated method used to acquire such prior information.In the proposed adaptive model, prior knowledge of3D face shape properties are learned from the training set,some of which are deformed with the guidance of the input face image.
The following subsection will brief down the reconstruction procedures incurred in the regularisation algorithms.
4.1Three-dimensional face shape reconstruction from2D face image
The feature points have been aligned with the reference3D model using Procrustes analysis(Section3.1.1),which is the usual preliminary step before the reconstruction stage. Fig.6demonstrates the alignment procedure used.The aligned feature points have been used to compute the3D shape coef?cients of the eigenvectors using Tikhonov regularisation[42].Similar to[23],the coef?cient a of a new3D face shape can be estimated using
Tikhonov Fig.6Typical input face being aligned using the Procrustes analysis-based transformation
https://www.wendangku.net/doc/787560170.html,
446
&The Institution of Engineering and Technology2014
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220
regularisation as
a =
E T f E f +l L
?1
?1
E T f
S f ?S f 0
(13)
where Λis a diagonal m ×m matrix with eigenvalues,λbeing
the weighting factor,S f be the set of selected points on the 2D face image,S f 0be the corresponding points on S 0(the average 3D face shape)and E f be the corresponding columns on E (the matrix of row eigenvectors).Then,the whole 3D face shape can be obtained by applying αto (11).Jiang et al.[26]have used the same regularisation equation in an iterative procedure in order to bring about a stable solution.In this work,however,the shape coef ?cients are calculated directly using (13).
The following section demonstrates how many principle components (PCs)are used in the PCA representation.4.2
Selection of principle components
In this work,we are dealing with small sample size modelling.Under normal circumstances,that is,with large sample (e.g.number of faces)size of training data (where the sample size is comparable or larger than the data dimension (e.g.number of vertices in the face vector)),removing the PCs with small eigenvalues will be better than using all PC components since the latter one will introduce instability and model-over ?tting.However,when dealing with small sample size problem (i.e.the number of training data is much less than the data dimension),a PC component,although corresponding to small eigenvalues (even 0),however,if effectively representing a true dimension of the high dimensional data,will contribute to better modelling of the data.To assess the effect of PC number on representing new 3D faces,we use a sample size of 80training faces and the remaining 20faces for testing.The number of PC was varied from 1to 80as a sequence of 1,2,…,80.Fig.7shows the average representation error of 20testing faces for different numbers of PCs.The ?gure shows that as the number of PCs increases the accuracy of representation increases.This indicates that all PCs may be needed to be used for reconstruction.Therefore,in this work we used all PCA components for standard and adaptive PCA-based models.Further studies are suggested to test the effect of PCs on the representation accuracy with small sample sizes.
5.Experiments and discussion
In this section we intend to report the experimental evaluation aspects of the proposed adaptive PCA-based model in
comparison to the standard PCA-based model.We have systematically categorised our experimental study in terms of the following three phases:
1.Representation of testing 3D face shapes using both models.
2.Reconstruction of testing 3D face shapes from feature points.
3.3D face shapes reconstruction from real 2D images.
In the ?rst two phases,the adaptive PCA-based model is evaluated quantitatively and qualitatively in comparison to the standard PCA-based model.By quantitative evaluation,the RP of the represented/reconstructed 3D face shape has been computed for all testing faces.
In the ?rst phase,the full testing face shape vectors have been projected on both PCA models (adaptive and standard PCA),then the average representation error for all test faces has been computed.However,in the second phase,the complete 3D face shape vector has been reconstructed from limited number of feature points using Tikhonov regularisation.
For the ?rst two phases,the t -test has been applied to statistically compare the two models.The α-value of the t -test (level of signi ?cance)has been chosen to be 0.05which means that the two models have been compared at a 95%con ?dence level.On the other hand,the represented/reconstructed 3D face shapes have been visualised to clarify the qualitative features of the represented/reconstructed face shapes in order to complement the ?ndings in the quantitative results.
The 100examples of the USF database (Section 5.1)were randomly divided into two sets;80examples have been used for training and deformation purposes and the remaining 20for testing the adaptive https://www.wendangku.net/doc/787560170.html,ing testing faces from the USF database enables numerical evaluation of the adaptive model through the comparison of the represented/reconstructed 3D face shapes with its ground truth.
In the third phase,the model is qualitatively evaluated pertaining to the visualisation aspect of the reconstructed faces from their 2D face images which were selected from the CMU-PIE database [37].
To assess the impact of deformed faces on the 3D shape representation accuracy,we varied the number of deformed faces in the training sets from 2to 70as a sequence of 2,5,10,20,30,…,and max of 70.The experiment was conducted for three different training sets,40,60and 80.PCA40,PCA60and PCA80are trained with 40,60and 80faces,respectively.This variety of training set sizes is to show the effect of various training set sizes on the reconstruction accuracy in case of adding deformed examples to the original training sets.
We have used MATLAB to implement the models used in this study and experiments were conducted on a CPU E5620@2.40GH with an Intel(R)Xeon(R)processor.5.1
USF Human ID database
The adaptive and the standard PCA models have been experimented using the USF Human ID 3D Face database [17].This database includes shape and texture information of 1003D faces with neutral expression acquired using the Cyberware TM laser scanner.We have divided the data set into a training set of 80faces and a testing set comprising the remaining faces.Each face model in the database has
75
Fig.7Average representation error of 20testing faces using different numbers of PCs
https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454doi:10.1049/iet-cvi.2013.0220
447
&
The Institution of Engineering and Technology 2014
972vertices.The faces have been aligned with each other using the procedures explained by Blanz and Vetter[17]. The3D shapes in the database are aligned with each other in such a way that the3D–3D correspondence for all vertices are obtained[21].For example,if the tip of the nose is assigned an index,for example,k=587,then the tip of the nose for all3D face shapes will have the same index as k= 587,even though the x,y and z coordinates that represent the position of the tip on the nose on the face will change. So,the indices of the vertices that can be selected from a reference face can be used to directly select the correspondences of the feature point from the other face shapes.
This study emphasises on the usage of shape vectors for training and testing the models for the reason stated in Section1.
5.2Representational power(RP)of the adaptive PCA
The RP of new3D face shapes has been analysed to compare the adaptive PCA with the standard PCA.Initially,we would experiment how the number of deformed examples affects the representation accuracy.Then,a comparison between the adaptive PCA and the standard PCA model would be made for an example of20deformed faces.
5.2.1Effect of deformed faces on the adaptive PCA: Fig.8shows the average representation error of20testing faces for the three different training sets which vary on size. The?gure shows that by increasing the number of deformed examples in the training set leads to the increase in the RP of the model.
5.2.2Adaptive PCA against standard PCA in terms of RP:The RP of the PCA-based model was evaluated by comparing it with the standard PCA-based model.For this purpose,20face shapes were deformed and added to the original faces.Subsequently,this resulted in the following two comparison examples:
1.Each of the adaptive PCA and the standard PCA was trained with the same number of training face shapes.
2.The adaptive PCA was trained with more face shapes than the standard PCA.
In the?rst example,the adaptive PCA was trained with80 faces,including60original and20deformed face shapes (adaptive PCA80).However,the standard PCA was trained with only80original faces shapes(standard PCA80).Fig.9 shows the representation errors for20testing faces exhibited by the adaptive PCA and the standard PCA which were trained with the same number of face shapes.
In the second example,the adaptive PCA was trained with 100face shapes,including80original and20deformed face shapes.Whereas the standard PCA remained with same sample size,that is,80original face shapes.Fig.10shows the representation errors for20testing faces exhibited by the standard and the adaptive PCA with different sample sizes.
From Figs.9and10,it has been concluded that two factors affect the RP of the PCA-based model.These factors are the deformation of training shapes and the increase in the number of training shapes.That is,as in the?rst example,the representation errors of the PCA model decreased when20 original face shapes were replaced with20deformed face shapes in the training sample.Similarly,in the second example,when the20deformed face shapes were added to the80original face shapes,thus composing a training sample of100face shapes,the representation errors decreased more noticeably compared to the?rst example. Moreover,t-test was conducted in order to verify that signi?cant differences exist between the adaptive PCA and the standard PCA using20testing face shapes.The average results of the20testing face shapes for the said two examples as shown in Table1indicate that the adaptive PCA-based model outperforms the standard PCA-based model with a95%con?dence level.This con?rms that
the Fig.8Average representation error of20testing faces on
PCA-based models which vary on size
PCA40indicates that the training set has40original
examples
Fig.9Comparison between the standard PCA(sample size80)
and the adaptive PCA(sample size60+20)in terms of RP;the
two models have the same sample
sizes
Fig.10Comparison between the standard PCA(sample size80)
and the adaptive PCA(sample size80+20)in terms of RP https://www.wendangku.net/doc/787560170.html,
448
&The Institution of Engineering and Technology2014
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220
proposed method (adaptive PCA-based model)yields better RP than that of the standard PCA-based model.Thus,it is concluded that adaptive PCA-based model which includes deformed face shapes has better RP.However,the adaptive PCA is more effective when it has deformed faces as well as larger training sample size as demonstrated in Example 2above.
Three typical representations of testing face shapes represented using both models are visualised in Fig.11.The 3D representation of face shapes shown in Fig.11demonstrate that sharp features of the facial components (e.g.nose,lips)are well retained by the proposed adaptive PCA-based model when compared to the standard PCA-based model.This visual comparison aids one to clearly justify the qualitative features of the proposed model with a larger training sample size caused by adding the deformed face shapes to the training sample.
5.3Reconstruction of 3D testing faces from limited number of feature points
Here,we intend to compute the accuracy of 3D face reconstruction using the adaptive PCA against the standard PCA.Additionally,we would also like to determine the optimal number of deformed faces which are required to be
added to the training set with respect to the reconstruction accuracy.
5.3.1Determining the optimal number of deformed faces:We have selected feature points in sets of f =25,58and 78from the reference 3D face.These feature points were formed using salient points such as nose-tip and eyes corners.First,the 3D feature points were manually selected from the mean face.Then,the correspondences of the feature point from the testing face shapes have been directly selected using their indices.The xy coordinates of the feature points selected from the testing face shape have been used to calculate α,the 3D shape coef ?cients.
Fig.12shows the average reconstruction error of 20training faces based on the optimal λvalue for different sets of feature points using three different training sets which vary in size.The optimal λhas been determined by computing the minimum Euclidean distance between the reconstructed shape and the ground truth.Then,λ,which meets the minimum distance,is the optimal one.
Apparently,from the trends in the 2D plot in Fig.12,although additional deformed examples increase the RP of the prior PCA model (Fig.12)and improve the reconstruction accuracy of the 3D test faces (Fig.12,for the number of deformed faces N d =2,5and 10),yet an excessive addition of deformed examples (N d >10)
Table 1Comparative results between the representation/reconstruction errors of standard PCA-based model and the adaptive
PCA-based model Accuracy measures
Standard model Adaptive model Significance level
Mean
Std.dev.Mean Std.dev.(P -value)1:representational accuracy (different sample sizes –example1) 2.59×10?3 4.07×10?4 2.15×10?3 3.02×10?4 1.07×10?82:representational accuracy (same sample sizes –example2) 2.59×10?3 4.08×10?4 2.35×10?3 3.53×10?4 4.96×10?63:reconstruction accuracy 6.13×10?3
1.51×10?3
4.90×10?3
1.30×10?3
6.55×10?5
t -Test against adaptive
model
Fig.11Visual comparison of represented 3D face shapes using the standard PCA-based model and the adaptive PCA-based model
https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454doi:10.1049/iet-cvi.2013.0220
449
&
The Institution of Engineering and Technology 2014
increases the freedom degree of the model and,thus,may affect negatively the reconstruction accuracy.Note that Fig.12shows the average reconstruction errors of 20testing 3D faces which have been reconstructed using PCA models trained with different sample sizes.
These results suggest that the optimal range of N d would be 2to 10.Since this range is not so big,the reconstruction process will not be much time-consuming.That is why the consistency of selection of different faces was tested with sets of faces within the boundary 2–10face shapes.In order to examine if this selection of different sets of faces affects the reconstruction accuracy,examples of sets containing two and ?ve faces were separately selected for this purpose.Tables 2and 3provide the average reconstruction results of four experiments (in columns)on various face feature points (in rows).As can be seen in Table 2,when the number of deforming faces are two,the consistency of the average reconstruction error is not identical.For example,for Face 1-2on 25points,the average reconstruction error is 0.0049,whereas for Face 3-4on 25points,the average reconstruction error is 0.0057.However,when the deformed faces are ?ve,the consistency of the model gets increased (see Table 3).This is because the difference between the average reconstruction results on different faces is small and this fact is being witnessed by the results tabulated in Table 3.As shown in Tables 2and 3,the total number of training faces got increased to 85faces (80original and 5deformed faces).This number of training faces is potentially suitable for building a 3D face reconstruction model where only a limited number of feature points are required for 3D face reconstruction.As a conclusion,three crucial factors may be taken into consideration to build any adaptive model for reconstruction of 3D faces from limited feature points:https://www.wendangku.net/doc/787560170.html,putational time of the proposed PCA-model.2.The degree of freedom of the PCA-model.
3.The number of available feature points used for reconstruction.
5.3.2Adaptive PCA against standard PCA in terms of reconstruction accuracy:According to the ?ndings on the optimal number of deformed faces (Section 5.3.1),the selection of any ?ve training faces to be deformed would be appropriate to reduce the reconstruction errors.This is because the selection of ?ve faces ensures a combination of consistency,less time consumption and less reconstruction errors.Therefore,?ve deformed shapes have been added to PCA80(training set =80original face shapes +5deformed faces).The results in Fig.13show that the reconstruction errors yielded by reconstruction of 20testing face shapes from 25feature points using
the
Fig.12Average reconstruction error of 20testing faces reconstructed from different sets of feature points
Table 2Average reconstruction error of 20faces by deforming two faces
Experiments Standard PCA
Face1-2Face3-4Face5-6Face17-18Average 25points 0.00610.00490.00570.00510.00530.005358points 0.00560.00470.00550.00490.00510.005178points
0.0055
0.0047
0.0054
0.0048
0.0051
0.0050
Face1-2means face numbers 1and 2of the USF database;Face3-4means face numbers 3and 4and so on
Table 3Average reconstruction error of 20faces by deforming five faces
Experiments Standard PCA
Face1-5Face6-10Face11-15Face41-45Average 25points 0.00610.00520.00520.00500.00490.005158points 0.00560.00510.00510.00490.00480.005078points
0.0055
0.0051
0.0051
0.0049
0.0048
0.0050
Face1-5means face numbers 1,2,3,4and 5in the USF database;Face6-10means face numbers 6,7,8,9and 10and so on https://www.wendangku.net/doc/787560170.html,
450
&The Institution of Engineering and Technology 2014
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220
adaptive-PCA-based model is minimised when compared to the standard PCA-based model.
Moreover,the average results of the 20testing face shapes shown in Table 1indicates that the reconstruction errors of the adaptive PCA-based model are signi ?cantly lower than those observed for the standard PCA model at a 95%con ?dence level.Fig.14helps to visualise three typical reconstructed testing faces from 25feature points using both the models.
These results enable us to conclude that the inclusion of similar 3D faces in the training set strengthens the prior knowledge about the input face shape and results in a more accurate reconstruction of the 3D face shape.
5.4Three-dimensional face shape reconstruction from 2D image
The CMU-PIE database [37]has been used for testing the visual effects of the proposed model.We intend to
reconstruct 3D models for the 2D images of CMU-PIE database.In this work,the input 2D images are in near frontal pose with most of them containing neutral and smiling expression.The feature points were manually selected for reconstruction.The 3D shape coef ?cients of the eigenvectors were calculated using (13).Then,the coef ?cients are used to reconstruct the 3D face shape using (11).For each 2D face image,the adaptive PCA-based model was trained with 85face shapes,including 80original and 5deformed face shapes.The comparisons between some typical 2D face images and their 3D reconstructions using standard PCA-based model and the adaptive PCA-based model are illustrated in Fig.15.From the results in Fig.15c ,one could notice some visual improvements in the reconstructed 3D face shapes.For example,in the rightmost face image of Fig.15c ,the reconstructed 3D face shape has retained some expression of the input image (Fig.15a )such as the facial smile,chin features and lip expressions.
This means that the capability of the model to depict a new 3D face can be improved when 3D exemplar training faces are deformed with the guidance of the input 2D image.However,in addition to the number of feature points used for 3D face shape reconstruction [43],the accuracy of reconstruction can be affected by the following factors:
1.Number of feature points used for deformation modelling.
2.Number of deformed faces that are being added to the training set.
Interestingly,the proposed model is capable of reconstructing 3D faces from 2D face images by retaining facial expressions although the training samples contain only neutral expression.By this way we do not impose that the training samples should contain a variety of
expressions
Fig.13Comparison between the standard PCA and the adaptive PCA in terms of ‘reconstruction from limited number facial points =25
’
Fig.14Visual comparison between the standard PCA and the adaptive PCA;reconstruction has been performed using 25chosen feature points
https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454doi:10.1049/iet-cvi.2013.0220
451
&
The Institution of Engineering and Technology 2014
as imposed by certain recent approaches(e.g.Shu-Fan and Shang-Hong[44]).Moreover,the original2D texture has been registered with the reference texture and warped on the reconstructed3D shapes.Fig.16shows the texture-mapping results.We can see that,interestingly,the proposed model is capable of reconstructing3D face shapes from2D face images without losing the smile expression.5.5Comparison of the proposed adaptive
PCA-based model with other state-of-the-art methods
A comparison of the proposed method on par with other state-of-the-art approaches is synthesised in Table4.Our implementation of the algorithm(including deforming?
ve
Fig.15Visual comparison
a Typical input2D images
b3D reconstruction using standard PCA-based model
c3D reconstruction using adaptive PCA-based
model
Fig.16Reconstructed3D faces with texture
a Typical input2D images
b3D reconstruction using adaptive PCA-based model
Table4Comparison of the proposed method with the other state-of-the-art methods
Input face Feature pts.Efficiency Texture recovery proposed method near frontal,
smile
more than25manually39.7s interpolation and
warping
stochastic Newton optimisation method(SNO)
[17,24]
arbitrary view all facial points 4.5min find optimal texture
coeff.
linear shape and texture fitting(LiST)[45]arbitrary view all facial points54s find optimal texture
coeff.
shape alignment and interpolation method
correction(SAMIC)[26]
frontal view87automatic10s directly mapped using a single reference face shape(SRFS)[18]5different poses all facial points29s image overlay constructing a low-dimensional expression
deformation manifold(CLDEDM)[44]
frontal with
expression
84predefined feature pts.
(same as[46])
5–7min texture coeff.
estimation https://www.wendangku.net/doc/787560170.html,
452
&The Institution of Engineering and Technology2014
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220
faces,rebuilding the PCA model and reconstructing the complete face shape vector)takes approximately39.7s. Notably,as shown in Table4,the proposed method is able to outperform SNO[24]and CLDEDM[44]in terms of ef?ciency,and comparably comparative with others.
Note that the information in Table4is gathered from the survey paper[21]and the recently published papers[18,44]. 6Conclusion and future work
In this contribution,a novel approach for the problem of3D face reconstruction from single2D face images has been proposed.We have intuitively demonstrated how RP aids to compare the qualitative aspect of the proposed model.This study has investigated how a3D deformable PCA-based model can be adapted for a given input2D face image by deforming3D faces in the training data set so as to gain signi?cant RP.Then,the adapted PCA-based model is used to reconstruct a3D face shape from the given input2D image using a number of feature points.Our experimental results demonstrate that the proposed deformation model scheme increases the RP of the standard PCA-based model for any given input face image.The proposed model outperforms the standard PCA-based model with a95% con?dence level.Further,it could reconstruct3D faces from limited number of feature points.However,deforming3D face shapes using TPS tends to increase the computational cost of the proposed scheme compared with the standard PCA-based model.We would consider to work on the following directions as part of our future work:
1.Other deformation techniques would be considered to improve the deformation synthesis of the proposed adaptive model.
2.Addressing the effect of different representation of faces, such as spherical harmonics[32–34],could be explored.
3.Similar to other approaches[27,31],the proposed method uses manual facial landmark selection.An automatic facial landmark selection method using approaches such as active shape model[47]could be attempted.
7Acknowledgments
The?rst author would like to thank the Ministry of Higher Education(MOHE),Malaysia for supporting him with the MIS scholarship.This research is also partly supported by an RU Grant(1001/PKOMP/817055)offered by Universiti Sains,Malaysia.
8References
1Elyan,E.,Ugail,H.:‘Reconstruction of3D human facial images using partial differential equations’,https://www.wendangku.net/doc/787560170.html,put.,2007,2,(8),pp.1–8
2Avilaq, A.J.,Rezaie, A.H.:‘Accuracy enhancement of3D face reconstruction using undecimated wavelet transform’,J.Am.Sci., 2013,9,(2s),pp.28–34
3Bottino,A.,De Simone,M.,Laurentini,A.,Sforza,C.:‘A new3-d tool for planning plastic surgery’,IEEE Trans.Biomed.Eng.,2012,59,(12), pp.3439–3449
4Hu,Y.,Jiang, D.,Yan,S.,Zhang,L.,et al.:‘Automatic3D reconstruction for face recognition’.Proc.Sixth Int.Conf.Automatic Face and Gesture Recognition(FGR2004),Seoul,Korea,May2004, pp.843–848
5Zhang,Z.,Hu,Y.,Yu,T.,Huang,T.:‘Minimum variance estimation of 3D face shape from multi-view’.Seventh Int.Conf.Automatic Face and Gesture Recognition(FGR2006),Southampton,UK,April2006, pp.547–552
6Liu,Z.,Zhang,Z.,Jacobs,C.,Cohen,M.:‘Rapid modeling of animated faces from video’,https://www.wendangku.net/doc/787560170.html,put.Anim.,2001,12,(4),pp.227–240
7Bregler,C.,Hertzmann,A.,Biermann,H.:‘Recovering non-rigid3D shape from image streams’https://www.wendangku.net/doc/787560170.html,puter Vision and Pattern Recognition(CVPR2000),Hilton Head,SC,USA,June2000, pp.690–696
8Amin,S.H.,Gillies,D.:‘Analysis of3D face reconstruction’.14th Int.
Conf.Image Analysis and Processing(ICIAP2007),September2007, Modena,Italy,pp.413–418
9Chowdhury,A.,Chellappa,R.,Krishnamurthy,S.,Vo,T.:‘3D face reconstruction from video using a generic model’.Proc.Int.Conf.
Multimedia and Expo-Volume I(ICME2002),Lausanne,Switzerland, August2002,pp.449–452
10Atick,J.,Grif?n,P.,Redlich,A.:‘Statistical approach to shape from shading:reconstruction of three-dimensional face surfaces from single two-dimensional images’,Neural Comput.,1996,8,(6),pp.1321–1340 11Zhang,R.,Tsai,P.-S.,Cryer,J.E.,Shah,M.:‘Shape-from-shading:a survey’,IEEE Trans.Pattern Anal.Mach.Intell.,1999,21,(8), pp.690–706
12Smith,W.A.,Hancock,E.R.:‘Recovering facial shape using a statistical model of surface normal direction’,IEEE Trans.Pattern Anal.Mach.
Intell.,2006,28,(12),pp.1914–1930
13Nandy,D.,Ben-Arie,J.:‘Shape from recognition:a novel approach for 3D face shape recovery’,IEEE Trans.Image Process.,2001,10,(2), pp.206–217
14Lin,C.,Cheng,W.,Liang,S.:‘Neural-network-based adaptive hybrid re?ectance model for3D surface reconstruction’,IEEE Trans.Neural Netw.,2005,16,(6),pp.1601–1615
15Nagai,T.,Naruse,T.,Ikehara,M.,Kurematsu,A.:‘Hmm-based surface reconstruction from single images’.Proc.Int.Conf.Image Processing–Volume2(ICIP2002),Rochester,New York,USA,September2002, pp.II–561
16Saxena,A.,Sun,M.,Ng,A.:‘Make3d:learning3D scene structure from
a single still image’,IEEE Trans.Pattern Anal.Mach.Intell.,2009,31,
(5),pp.824–840
17Blanz,V.,Vetter,T.:‘A morphable model for the synthesis of3D faces’.
Proc.26th Annual https://www.wendangku.net/doc/787560170.html,puter Graphics and Interactive Techniques (SIGGRAPH1999),Los Angeles,CA,USA,August1999,pp.187–194 18Kemelmacher-Shlizerman,I.,Basri,R.:‘3D face reconstruction from a single image using a single reference face shape’,IEEE Trans.Pattern Anal.Mach.Intell.,2011,33,(2),pp.394–405
19Luximon,Y.,Ball,R.,Justice,L.:‘The3D Chinese head and face modeling’,Comput.Aid.Des.,2012,44,(1),pp.40–47
20Widanagamaachchi,W.,Dharmaratne,A.:‘3D face reconstruction from 2D images’.Digital image computing:techniques and applications (DICTA2008),Canberra,ACT,Australia,December2008,pp.365–371 21Levine,M.D.,Yu,Y.:‘State-of-the-art of3D facial reconstruction methods for face recognition based on a single2D training image per person’,Pattern Recognit.Lett.,2009,30,(10),pp.908–913
22Gonzalez-Mora,J.,De la Torre,F.,Guil,N.,Zapata,E.L.:‘Learning a generic3D face model from2D image databases using incremental structure-from-motion’,Image https://www.wendangku.net/doc/787560170.html,put.,2010,28,(7), pp.1117–1129
23Maghari,A.,Liao,I.,Belaton,B.:‘Quantitative analysis on PCA-based statistical3D face shape modeling’,Proc.Int.Symposium of Computational Modeling of Objects Presented in Images III: Fundamentals,Methods and Application(CompIMAGE2012),2012, pp.13–18
24Blanz,V.,Vetter,T.:‘Face recognition based on?tting a3D morphable model’,IEEE Trans.Pattern Anal.Mach.Intell.,2003,25,(9), pp.1063–1074
25Castelan,M.,Van Horebeek,J.:‘Relating intensities with three dimensional facial shape using partial least squares’,IET Comput.
Vis.,2009,3,(2),pp.60–73
26Jiang,D.,Hu,Y.,Yan,S.,Zhang,L.,Zhang,H.,Gao,W.:‘Ef?cient3D reconstruction for face recognition’,Pattern Recognit.,2005,38,(6), pp.787–798
27Blanz,V.,Vetter,T.:‘Reconstructing the complete3D shape of faces from partial information(rekonstruktion der dreidimensionalen form vongesichtern aus partieller information)’,IT-Inf.Technol.(vormals it +ti),2002,44,(6/2002),p.295
28Blanz,V.,Mehl,A.,Vetter,T.,Seidel,H.:‘A statistical method for robust3D surface reconstruction from sparse data’.Proc.Second Int.
Symposium on3D Data Processing,Visualization and Transmission (3DPVT2004),Thessaloniki,Greece,September2004,pp.293–300 29Maghari,A.Y.,Venkat,I.,Liao,I.Y.,Belaton,B.:‘Reconstructing3D face shapes from single2D images using an adaptive deformation model’.Proc.Third Int.Visual Informatics Conf.(IVIC2013), Selangor,Malaysia,November2013,pp.300–310
30Lu,X.,Jain,A.K.,Colbry,D.:‘Matching2.5D face scans to3D models’,IEEE Trans.Pattern Anal.Mach.Intell.,2006,28,(1), pp.31–43
https://www.wendangku.net/doc/787560170.html,
IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454 doi:10.1049/iet-cvi.2013.0220
453 &The Institution of Engineering and Technology2014
31Lu,X.,Jain,A.K.:‘Deformation modeling for robust3D face matching’, IEEE Trans.Pattern Anal.Mach.Intell.,2008,30,(8),pp.1346–1357 32Vranic,D.V.,Saupe,D.:‘Description of3D-shape using a complex function on the sphere’.Proc.Int.Conf.Multimedia and Expo-Volume I(ICME2002),Lausanne,Switzerland,August2002,pp.177–180
33Saupe,D.,Vranic,D.V.:‘3d model retrieval with spherical harmonics and moments’.Proc.23rd DAGM—Symposium on Pattern Recognition,Munich,Germany,September2001,pp.392–397
34Vranic,D.V.,Saupe,D.:‘3D model retrieval’.Proc.Spring Conf.
Computer Graphics and Its Applications(SCCG2000),Budmerice, Slovakia,Citeseer,May2000,pp.89–93
35Donato,G.,Belongie,S.:‘Approximate thin plate spline mappings’.
Proc.Seventh European https://www.wendangku.net/doc/787560170.html,puter Vision—Part III(ECCV 2002),Copenhagen,Denmark,May2002,pp.21–31
36Wahba,G.:‘Spline models for observational data’(Society for Industrial and Applied Mathematics(SIAM),1990,1st edn.)
37Sim,T.,Baker,S.,Bsat,M.:‘The CMU pose,illumination,and expression database’,IEEE Trans.Pattern Anal.Mach.Intell.,2003, 25,(12),pp.1615–1618
38Farkas,L.G.:‘Anthropometry of the head and face’(Raven Press,1994, 2nd edn.)
39Knothe,R.,Romdhani,S.,Vetter,T.:‘Combining PCA and LFA for surface reconstruction from a sparse set of control points’.Seventh Int.Conf.Automatic Face and Gesture Recognition(FGR2006), Southampton,UK,April2006,pp.637–644
40Besl,P.,McKay,H.:‘A method for registration of3D shapes’,IEEE Trans.Pattern Anal.Mach.Intell.,1992,14,(2),pp.239–25641Brown,B.,Rusinkiewicz,S.:‘Non-rigid range-scan alignment using thin-plate splines’.Proc.Second Int.Symp.3D Data Processing, Visualization and Transmission(3DPVT2004),Thessaloniki,Greece, September2004,pp.759–765
42Maghari, A.Y.,Venkat,I.,Liao,I.Y.,Belaton, B.:‘PCA-based reconstruction of3d face shapes using Tikhonov regularization’, Int.J.Adv.Soft Comput.Appl.,2013,5,(2),pp.1–15
43Maghari,A.Y.,Liao,I.Y.,Belaton,B.:‘Effect of facial feature points selection on3D face shape reconstruction using regularization’.Proc.
19th Int.Conf.Neural Information Processing—Volume Part V (ICONIP2012),Doha,Qatar,November2012,pp.516–524
44Wang,S.-F.,Lai,S.-H.:‘Reconstructing3D face model with associated expression deformation from a single face image via constructing a low-dimensional expression deformation manifold’,IEEE Trans.Pattern Anal.Mach.Intell.,2011,33,(10), pp.2115–2121
45Romdhani,S.,Blanz,V.,Vetter,T.:‘Face identi?cation by?tting a3D morphable model using linear shape and texture error functions’.
European https://www.wendangku.net/doc/787560170.html,puter Vision—Part IV(ECCV2002), Copenhagen,Denmark,May2002,pp.3–19
46Yin,L.,Wei,X.,Sun,Y.,Wang,J.,Rosato,M.J.:‘A3D facial expression database for facial behavior research’.Seventh Int.Conf.
Automatic Face and Gesture Recognition(FGR2006),Southampton, UK,April2006,pp.211–216
47Milborrow,S.,Nicolls,F.:‘Locating facial features with an extended active shape model’,Proc.10th European Conference on Computer Vision(ECCV2008),2008,pp.504–513
https://www.wendangku.net/doc/787560170.html,
454
&The Institution of Engineering and Technology2014IET Comput.Vis.,2014,Vol.8,Iss.5,pp.441–454
doi:10.1049/iet-cvi.2013.0220