Synthesis of Hereditary Patterns of Facial Expressions

Introduction

Analysis of kinship from facial images or videos is an important problem. Prior machine learning and computer vision studies approach kinship analysis as a verification or recognition task. In this project, first time in the literature, we propose a kinship synthesis framework, which generates smile videos of (probable) children from the smile videos of parents. While the appearance of a child's smile is learned using a convolutional encoder-decoder network, another neural network models the dynamics of the corresponding smile. The smile video of the estimated child is synthesized by the combined use of appearance and dynamics models. The method is described in the following paper:
  • I. Onal Ertugrul and H. Dibeklioglu. What will Your Future Child Look Like? Modeling and Synthesis of Hereditary Patterns of Facial Dynamics. IEEE International Conference on Automatic Face and Gesture Recognition (FG), 2017, accepted.

Samples of Input/Output Videos

Sample 1 (Father → Daughter/Son)

Original Input (Father) Synthesized Daughter Synthesized Son

Sample 2 (Father → Daughter/Son)

Original Input (Father) Synthesized Daughter Synthesized Son

Sample 3 (Mother → Daughter/Son)

Original Input (Mother) Synthesized Daughter Synthesized Son

Sample 4 (Mother → Daughter/Son)

Original Input (Mother) Synthesized Daughter Synthesized Son

Legal Notice: Samples of input videos are property of the UvA-NEMO Smile Database, and may not be used/shared without permission of the owners. Synthesized samples may only be used/shared by citing this work.