3D face reconstruction algorithms

Question

Project: 3D face reconstruction

Input: 2D frontal face Image: 3D face Reconstruction and modeling of expressions Platform: Matlab or Opencv cpp.

I learned after studying that the 3D Morphable Models (3DMM) algorithm is a good starting point for my project. But to implement the algorithm, I do not have a Basel Face model (3D face Database). However, I downloaded GavabDB from http://gavab.escet.urjc.es/recursos_en.html . Can I develop 3DMM with GavabDB for three-dimensional face reconstruction from a frontal image? After reading the dataset description doc, I noticed that Gavab does not provide 3D scan texture data; Are texture data required?

Does the quality of the output depend on the three-dimensional database used for modeling?

Answer 1

The original morphing model , as well as the Basel Face Model, which belongs to the same main researcher, contain texture as well as geometry.

It is perfectly possible to calculate the geometric part of the Morphable model from three-dimensional geometry data without texture. This gives you a model that captures the statistically most significant changes in form.

However, comparing a model of pure geometry with a two-dimensional image of a face is more complicated than comparing a model with a texture. This essentially requires you ...

• identify (or annotate) the locations of some landmarks in a 2D face image
• optimize 3D pose and shape settings that bring the corresponding landmarks in the 3D Morphable model closer to your annotated landmarks after projecting with an unknown camera.

This is an interesting problem, but not trivial. For step 2, I would recommend starting with the intended spelling projection.

As for your question about the quality of the issue: if your database has a sufficiently large number of 3D models, individual scans do not have to be of especially high quality. Noise will not be displayed in the main components of the model that you are actually using. However, holes in the scan are a problem.

Finally, let me shamelessly point you to the paper that I wrote several years ago. It does not solve your problem, but contains a section on installing a 3D-simulated model (geometry only) to a two-dimensional face silhouette extracted from a photograph.