Cascading CNNs for facial action unit detection

Abstract

The contractions of facial muscles are what shape the expressions produced by the human face. The Facial Action Coding System (FACS) stands as the predominant standard in describing all visual alterations in the face, defining them through Action Units (AU) that articulate the movements occurring in the facial muscles. In this paper, an end-to-end pipeline, CCNN2, is proposed as a deep pre-processing step to detect AUs by processing the features extracted from hidden CNN layers, without exploiting any landmark information in a recursive manner. Trials conducted on three spontaneous datasets (MMI, DISFA, BP4D) along with one in-the-wild dataset (EmotioNet) demonstrate that this method surpasses the results of state-of-the-art approaches in three of the datasets, and even more, its two-module structure increases the overall F1 score in detection in every experiment. The method being proposed is also adaptable to a diverse range of classification applications.