The use of generative networks to create an eyewear personalization system
Dr. Richard Plesh
Clarkson University
Objective: Demonstrate how facial generation networks can be retasked using Targeted Subspace Modeling, in this case for the task of an eyewear personalization system.
Abstract: The increased use of digital avatars and online commerce has created a need for a tool that can allow personalization of eyewear virtually. In this talk I present GlassesGAN, a novel image editing framework for custom design of glasses, that sets a new standard in terms of output-image quality, edit realism, and continuous multi-style edit capability. To facilitate the editing process, GlassesGAN proposes a Targeted Subspace Modelling (TSM) procedure that, based on a novel mechanism for (synthetic) appearance discovery in the latent space of a pre-trained GAN generator, constructs an eyeglasses-specific (latent) subspace that the editing framework can utilize. Additionally, GlassesGAN introduces an appearance-constrained subspace initialization (SI) technique that centers the latent representation of the given input image in the well-defined part of the constructed subspace to improve the reliability of the learned edits. GlassesGAN is tested on two (diverse) high-resolution datasets (CelebA-HQ and SiblingsDB-HQf) and compared to three state-of-the-art baselines, i.e., InterfaceGAN, GANSpace, and MaskGAN. The reported results show that GlassesGAN convincingly outperforms all competing techniques, while offering functionality (e.g., fine-grained multi-style editing) not available with any of the competitors.
Bio: Richard is an artificial intelligence scientist at the Identity and Data Sciences Lab (IDSL) at the Maryland Test Facility. He has over six years of experience in developing and applying advanced machine learning techniques to biometric recognition and security problems. He received his PhD from Clarkson University in 2023 in computer and electrical engineering during which he published multiple papers on novel methods for generating synthetic fingerprints, reducing bias in facial recognition, detecting presentation attacks, and evaluating iris recognition. As a Fulbright Fellow, he worked as a researcher at the University of Ljubljana, Slovenia, exploring the potential of Generative Adversarial Networks (GANs) for image editing and personalization.
Wednesday, Feb 7th, 2024, 4 pm via ZOOM
ZOOM Link: https://clarkson.zoom.us/j/95106031702?pwd=NnRNNU5yU2k3WVRTSWdjSTlrUnlaUT09