face replacement, how to?

General discussion about 3D DCC and other topics
Post Reply
forton
Posts: 210
Joined: 25 Nov 2011, 17:57

face replacement, how to?

Post by forton » 20 Feb 2014, 13:16

I've got a question here asking how difficult it is to do full face replacement (only face, no hair).
The goal is mapping a filmed face to another moving person.
The quality should be good enough for film resolution, many shots, lots of camera movement.


Is there any specialized software to do this?
Any good tutorials?

Cheers, Wim

User avatar
Rork
Posts: 1359
Joined: 09 Jul 2009, 08:59
Location: Close to The Hague, Netherlands
Contact:

Re: face replacement, how to?

Post by Rork » 21 Feb 2014, 11:26

First thing that comes to mind is to do facial tracking with PFTrack or alike, and remap the footage back on.

With all the moving around and quality requirements, this probably will take some time to get right.

Regarding tutorials, I can recall something that covered this.
But not sure if it was for PFTrack, SynthEyes or something completely different though.

Some Googling/Binging might pop up something helpful.

rob
SI UI tutorials: Toolbar http://goo.gl/iYOL0l | Custom Layout http://goo.gl/6iP5xQ | RenderManager View http://goo.gl/b4ZkjQ
So long, and thanks for all the Fish!!

forton
Posts: 210
Joined: 25 Nov 2011, 17:57

Re: face replacement, how to?

Post by forton » 21 Feb 2014, 12:49

Thanks Rob,

Googling gave me one interesting result (besides allot of after effects tutorials that are ok but I think not accurate and not "3D" enough):
http://phys.org/news/2011-12-harvard-gr ... video.html
Its a few years old and I wonder if this tech has made it into some commercial software.
I'll search for Pftrack, I have syntheyes and I did try some facial tracking with it some time ago, but it was lots of steps.

Cheers, Wim

User avatar
Rork
Posts: 1359
Joined: 09 Jul 2009, 08:59
Location: Close to The Hague, Netherlands
Contact:

Re: face replacement, how to?

Post by Rork » 21 Feb 2014, 13:03

Hi,

Yes, all this can take quite some time to get right.

With PFTrack you can track the face and generate a animated geo model. With that exported, it's the first steps in reprojecting back the footage.
It's also interesting to look into Nuke for doing the 3d projections and such.

Keep us informed, stuff like this always makes interest threads ;)

did a quick google search on pftrack face tracking, second one from the top (pftrack - daft punk helmet) is probably a good start.

cheers!
SI UI tutorials: Toolbar http://goo.gl/iYOL0l | Custom Layout http://goo.gl/6iP5xQ | RenderManager View http://goo.gl/b4ZkjQ
So long, and thanks for all the Fish!!

forton
Posts: 210
Joined: 25 Nov 2011, 17:57

Re: face replacement, how to?

Post by forton » 21 Feb 2014, 14:03

If we do some tests and-or if we have the job i'll certainly do!

Chris_TC
Posts: 411
Joined: 22 Mar 2010, 17:43

Re: face replacement, how to?

Post by Chris_TC » 25 Feb 2014, 21:00

You will need tracking software and compositing software. For best results I'd also add 3d software (even though most compositing apps have 3d features as well).

Quick steps:
Step 1: You track the face that needs replacement and export the track to your 3d application.
Step 2: In your 3d application, you model a 3d face mockup (or use a 3d scan of the replacement face) and make it follow the motion track you imported.
Step 3: You project the replacement footage onto the 3d face and render the result.
Step 4: Using compositing software, you overlay the new face, color correct it and blend it into the background plate.

Post Reply

Who is online

Users browsing this forum: No registered users and 31 guests