Deep Transfer Learning for Masked Face Reconstruction and Hybrid DCNN-ELM Framework for Recognition

Abstract

Facial reconstruction has always been a pivotal aspect of medical and forensic science. The increasing use of face masks in recent years has posed new challenges, making traditional facial recognition techniques less effective. To address this, our research explored innovative methods for reconstructing faces from images obscured by masks. We focused on post mask face reconstruction and facial recognition using cutting-edge techniques. We assess the effectiveness of three key unmasking algorithms: edgeconnect (EC), gated convolution (GC), and hierarchical variational vector quantized autoencoders (HVQVAE). Using two synthetic face datasets, MaskedFace-CelebA and MaskedFace-CelebAHQ, we rigorously evaluate the quality of the reconstructed faces based on metrics such as the PSNR, SSIM, UIQI, and NCORR. Among these, the Gated Convolution algorithm stands out as the superior choice in terms of image quality. For facial recognition, we employ a novel hybrid framework that combines a deep convolutional neural network and an extreme learning machine (DCNN-ELM). We tested five classifiers (Vgg16, Vgg19, ResNet50, ResNet101, and ResNet152) in combination with ELM and a support vector machine (SVM). Our comprehensive ablation study revealed that ResNet152 combined with ELM achieved the best performance, with a facial recognition accuracy of 60.9%, suggesting that the reconstructed faces were of high quality. Our paper presents a novel approach to image classification utilizing five classifiers within the DCNN+ELM hybrid framework and provides a complete ablation study of these classifiers. This research underscores the importance of face reconstruction in the current field and its potential to enhance facial recognition techniques.