How to move face points in opencv / face distortion

Question

I need to model facial expressions in a facial image (say, an open mouth). To do this, I first identified facial features and found the corners of the lips. Now I need to warp the image by moving the dots down.

In the above image, I need to move points (2) and (3) a certain distance to the left and right, respectively. And paragraph (18) and (19) littele a bit down. So I get the expression like an open mouth.

My questions:

1) Is it right to continue to simulate a facial expression?

2) If correctly, how to move points and create a new image in opencv?

Answer 1

A recent survey and course of techniques that people have used in this area are here: http://old.siggraph.org/publications/2006cn/course30.pdf

TL: DR. There is no “right” way to do this in any absolute sense. You must define your goal in such a way as to calculate it. Then find out what additional (preliminary) information you need to get, in addition to the image data itself. Collapsing with “texture warp” or other interpolation schemes before you decide what you need to do is a waste of time.

You mention “an expression like an open mouth,” and I interpret it as meaning that you want to create an image similar to what a real face would look like if the subject were photographed with an open mouth. The selected markers clearly do not provide enough information about this - in particular, they do not express any concept of “mouth”. In fact, this concept is not found anywhere in the image. So, strictly speaking, your task is insoluble unless you add more information to it.

I suggest you take a look at the paper above and reconsider your problem again.

Answer 2

So here we are again and again.

I saw a lot of people using delauny triangulation from these points, and then texture warping or distortion in opengl or even opencv.

https://github.com/MasteringOpenCV/code/tree/master/Chapter7_HeadPoseEstimation

looks pretty connected, part of MasteringOpencv is on google.books