Avoiding Catastrophe: Active Dendrites Enable Multi Task Learning in Dynamic Environments ( Review)

, multitasklearning, biology, neuralnetworks Catastrophic forgetting is a big problem in mutlitask and continual learning. Gradients of different objectives tend to conflict, and new tasks tend to override past knowledge. In biological neural networks, each neuron carries a complex network of dendrites that mitigate such forgetting by recognizing the context of an input signal. This paper introduces Active Dendrites, which carries over the principle of contextsensitive gating by dendrites into the deep learning world. Various experiments show the benefit in combatting catastrophic forgetting, while preserving sparsity and limited parameter counts. OUTLINE: 0:00 Introduction 1:20 Paper Overview 3:15 Catastrophic forgetting in continuous and multitask learning 9:30 Dendrites in biological neurons 16:55 Sparse representations in biology 18:35 Active dendrites in deep learning 34:15 Experiments on multitask learning 39:00 Experiments in continual learning and adaptive prototyping 49:20 Anal