Clarkson Computer Science PhD student and NSF Graduate Research Fellowship awardee Nikolas Lamb presented his research on using techniques from artificial intelligence (AI), particularly computer vision and deep learning, to repair damaged objects at the Eurographics Symposium on Geometry Processing (SGP), the highest ranking venue in geometry processing techniques on July 5. Nikolas is advised on his research by Drs. Natasha Banerjee and Sean Banerjee, both Associate Professors in the Department of Computer Science. Results of Nikolas’s research from the venue proceedings will appear as published work in the 2022 Computer Graphics Forum, the leading journal for in-depth technical articles on computer graphics. Nikolas’s work is the first from Clarkson to be presented at SGP and appear in print at the Computer Graphics Forum journal.
Nikolas’s work provides users with a novel approach, called MendNet—an Object Mending Deep Neural Network—that automatically synthesizes additively manufactured repair parts to 3D models of damaged objects. Nikolas’s approach for automated 3D repair synthesis is the first of its kind. Prior to Nikolas’s research, if a user’s valuable heirloom broke, with broken-off parts damaged beyond repair, restoring the broken object was a significant challenge, as the user would need to painstakingly 3D model the complex geometry of the broken part. This is something that most users are unlikely to do, and it is no surprise that a large number of damaged objects end up being thrown out, increasing environmental waste and greatly impacting sustainability.
Nikolas’s research plays a key role in advancing Clarkson’s commitment toward sustainability, by using AI to automate the repair process, incentivizing end users to choose ‘repair’ over ‘replace’. Users can now fix broken items, e.g., ceramic objects such as precious dinnerware with minimal effort. Nikolas’s automated repair algorithm allows users to scan in their broken item and can automatically synthesize the repair part and send the part to a 3D printer. Nikolas’s work takes advantage of the widespread ubiquity of 3D printers and the emergence of 3D printers for materials such as ceramics and wood in the consumer marketplace. By tying AI, computer vision, and deep learning to the manufacturing process, Nikolas’s work greatly transforms the landscape of advanced manufacturing, bringing rapid manufacturing within the hands of the average user.
Nikolas’s work has broader impact in advancing knowledge in domains such as archaeology, anthropology, and paleontology, by providing a user-friendly approach to repair cultural heritage artifacts, damaged fossil specimens, and fragmented remains, reducing the busy work for researchers and enabling them to focus attention toward addressing research questions of domain interest. The work also has an impact in automating repair in dentistry and medicine.
Nikolas is a member of the Terascale All-sensing Research Studio (TARS) at Clarkson University. TARS supports the research of 15 graduate students and nearly 20 undergraduate students every semester. TARS has one of the largest high-performance computing facilities at Clarkson, with 275,000+ CUDA cores and 4,800+ Tensor cores spread over 50+ GPUs, and 1 petabyte of (nearly full!) storage. TARS houses the Gazebo, a massively dense multi-viewpoint multi-modal markerless motion capture facility for imaging multi-person interactions containing 192 226FPS high-speed cameras, 16 Microsoft Azure Kinect RGB-D sensors, 12 Sierra Olympic Viento-G thermal cameras, and 16 surface electromyography (sEMG) sensors, and the Cube, a single- and two-person 3D imaging facility containing 4 high-speed cameras, 4 RGB-D sensors, and 5 thermal cameras. TARS performs research on using deep learning to glean understanding on natural multi-person interactions from massive datasets, in order to enable next-generation technologies, e.g., intelligent agents and robots, to seamlessly integrate into future human environments.
The team thanks the Office of Information Technology for providing access to the ACRES GPU node with 4 V100s containing 20,480 CUDA cores and 2,560 Tensor cores.
Nikolas and the TARS team are looking for your broken items that you want to throw out, for expanding the research. Please drop them a line at lambne@clarkson.edu if you have any damaged objects that you want to get rid of.
