Enterprise AI Analysis

Selective Forgetting in Machine Learning and Beyond: A Survey

This survey investigates the multifaceted nature of selective forgetting in machine learning, drawing insights from neuroscientific research that posits forgetting as an adaptive function rather than a defect, enhancing the learning process and preventing overfitting. It redefines forgetting as an adaptive mechanism for enhancing model performance, adaptability, and data privacy compliance. The paper introduces a novel taxonomy categorizing forgetting mechanisms by content, recoverability, and extent, and explores active vs. passive approaches, including recent advancements in LLM unlearning. It also addresses challenges like model training, evaluation, verification, and ethical considerations, proposing future research directions for inter-disciplinary collaboration and robust AI systems.

Human brain is a complex system, where forgetting serves as a dynamic nature that allows us to avoid cognitive overload, update information to adapt to changing environments [82], and can potentially enhance our learning capabilities [28]. In the context of the human brain, overfitting arises when we simply memorise specific examples rather than generalise patterns from them [101]. This narrow focus can cause inflexibility in humans' thinking and problem-solving abilities, as well as lead to erroneous predictions or assumptions when confronted with unfamiliar situations. The advantages of forgetting have been investigated in various research fields, including education, philosophy, ecology, and linguistics, where forgetting has been found to contribute significantly to the enhancement of humans' decision-making, creativity, and diversity from multiple perspectives.