transferfor
realtimeexpression Facialreenactment
renderingof
theentireface
seriesas a
theparametricfacemodel andanintermediaterepresentation tolaterenableprotorealisticre
statisticalprior
model
parametrictune
mesh the
facetemplate whitestuff
mgeoca.rs s embeds theverticeswiers ofageneric components
in alower sub
dimensional space in
reduction dimensionality principal
a setofdimensionsencodestheidentitywithweightsa
a setofdimensionsencodestheexpressionwithweights s
Eepemocontaintheoasisrectorofthelinearsubspaces
eigenvectors
matricesFia
columnsofthe
positionsv
vertex evis verses dosc ccis
are
whileuandc are byalinear
defined of vectors thenormalsn i canbederivedasthecrossproductofthepartialderive
combination basis
oftheshapew v t a cuis parametrization
e
Illuminationmode
Assumption lightingisdistant andsurfacesinthescenearepredominantlyLambertian
parametricmodel
fitting
earsDinputbyusingananalysisthroughsynthesisapproan
wewantto fitourmodeltoourperfectly
thebestparameterpwhich
find thesyntheticmodeetoourinput
aligns
energyFormulation
colorconsistencymetric
metric earsimagessynthesizedusing themodelarephotoconsistentwith
the
c onsistency
Geometric giveninputcolor
images
the
match observations
reconstructedgeometryshould
bythe
captured stream
depth
input we the
minimize dicerence
the
b etween rarsimageandthe
input view
rendered for
pixelper
every
I
minimises sumofpointtopointdistances
the featuresimilaritymetric
foranpixelsinthevisiblesetv facialfeaturesinthesynthesizedviewstobeclosetothe
epointer apointspill we enforce
E features
detected
eachdetectedfeature fj cuj.glis a 2Dlocationintheimagedome
thatcorresponds to aconsistentsovertexis inourgeometricface
t
solver
core
y
peg
Gpubased GmissNewtonsolver
wealreadyhaveanapproximate
weassumethat Pixs
solution initialguess
minimizestheresultaroundpts
weseekforaparameterincrementopthat the
with iterations
GaussNewtonproceeds
FCPk Dp a rCP J Pt Dp Il
k
g p g p gp g p pp iterationtheupdate d pistil
ateach
foundbyarranging
canbe
su
me
s istilIcs
D
sitJr D
realtimeexpression Facialreenactment
renderingof
theentireface
seriesas a
theparametricfacemodel andanintermediaterepresentation tolaterenableprotorealisticre
statisticalprior
model
parametrictune
mesh the
facetemplate whitestuff
mgeoca.rs s embeds theverticeswiers ofageneric components
in alower sub
dimensional space in
reduction dimensionality principal
a setofdimensionsencodestheidentitywithweightsa
a setofdimensionsencodestheexpressionwithweights s
Eepemocontaintheoasisrectorofthelinearsubspaces
eigenvectors
matricesFia
columnsofthe
positionsv
vertex evis verses dosc ccis
are
whileuandc are byalinear
defined of vectors thenormalsn i canbederivedasthecrossproductofthepartialderive
combination basis
oftheshapew v t a cuis parametrization
e
Illuminationmode
Assumption lightingisdistant andsurfacesinthescenearepredominantlyLambertian
parametricmodel
fitting
earsDinputbyusingananalysisthroughsynthesisapproan
wewantto fitourmodeltoourperfectly
thebestparameterpwhich
find thesyntheticmodeetoourinput
aligns
energyFormulation
colorconsistencymetric
metric earsimagessynthesizedusing themodelarephotoconsistentwith
the
c onsistency
Geometric giveninputcolor
images
the
match observations
reconstructedgeometryshould
bythe
captured stream
depth
input we the
minimize dicerence
the
b etween rarsimageandthe
input view
rendered for
pixelper
every
I
minimises sumofpointtopointdistances
the featuresimilaritymetric
foranpixelsinthevisiblesetv facialfeaturesinthesynthesizedviewstobeclosetothe
epointer apointspill we enforce
E features
detected
eachdetectedfeature fj cuj.glis a 2Dlocationintheimagedome
thatcorresponds to aconsistentsovertexis inourgeometricface
t
solver
core
y
peg
Gpubased GmissNewtonsolver
wealreadyhaveanapproximate
weassumethat Pixs
solution initialguess
minimizestheresultaroundpts
weseekforaparameterincrementopthat the
with iterations
GaussNewtonproceeds
FCPk Dp a rCP J Pt Dp Il
k
g p g p gp g p pp iterationtheupdate d pistil
ateach
foundbyarranging
canbe
su
me
s istilIcs
D
sitJr D
