MSC THESIS DEFENSE
DEPARTMENT OF COMPUTER SCIENCE
CLARKSON UNIVERSITY
THURSDAY DECEMBER 5 2024. 13:00. VIRTUAL
WILLIAM DUNKLIN
Department of Computer Science
Clarkson University
Loss of Plasticity Beyond Deep Learners
Kolmogorov-Arnold Networks (KANs) represent a novel approach to neural network design, utilizing learnable
spline-based activations on network edges rather than nodes. This architecture promises significant advantages
in interpretability and adaptability, particularly in continual learning contexts. However, the original claims
regarding the KAN architecture’s resistance to catastrophic forgetting and its suitability for continual learning
have not been rigorously tested.
This thesis investigates KANs through a series of targeted experiments aimed at evaluating their resistance to
the two prescient challenges in continual learning: catastrophic forgetting and plasticity loss. Initial replication of the original KAN experiments with a [1, 1]-KAN confirmed its effectiveness in retaining sequential knowledge. However, extending these tests to a multilayer [1, 1, 1]-KAN revealed critical limitations. The addition of network depth led to significant noise accumulation, ultimately undermining the network’s stability and
adaptability.
To explore the phenomenon of plasticity loss in KANs, a novel experimental framework was designed using a
moving Gaussian setup to simulate non-stationary data. The results indicate that while KANs are promisingly
interpretable for continual learning applications, they are susceptible to plasticity loss under drifting targets.
These findings provide new insights into the behavior of spline-based activations and highlight the need for
refined experimental designs and mitigation strategies to address noise accumulation and stability challenges
in KANs.
This work establishes KANs as a compelling architecture for studying plasticity loss and presents avenues for
future research to enhance their performance in dynamic learning environments.
Committee members: Christopher Lynch, Charles Thorpe, and Christino Tamon (advisor).
BIOGRAPHICAL SKETCH:
Will Dunklin is a Master’s candidate in Computer Science at Clarkson University, where he also completed his Bachelor’s degree. For the past three years, he has worked as an R&D Engineer at Kitware, where he is currently developing digital pathology tools in collaboration with researchers at Emory University and the University of Florida. His research interests largely span alternative machine learning approaches, particularly focusing on negative results in machine learning, such as the loss of plasticity phenomenon.
Zoom link: https://clarkson.zoom.us/j/91760513594.
