Transfer learning

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem. For example, knowledge gained while learning to recognize cars could apply when trying to recognize trucks. This area of research bears some relation to the long history of psychological literature on transfer of learning, although practical ties between the two fields are limited. From the practical standpoint, reusing or transferring information from previously learned tasks for the learning of new tasks has the potential to significantly improve the sample efficiency of a reinforcement learning agent.

History

In 1976, Stevo Bozinovski and Ante Fulgosi published a paper explicitly addressing transfer learning in neural networks training. The paper gives a mathematical and geometrical model of transfer learning. In 1981, a report was given on the application of transfer learning in training a neural network on a dataset of images representing letters of computer terminals. Both positive and negative transfer learning was experimentally demonstrated.

In 1993, Lorien Pratt published a paper on transfer in machine learning, formulating the discriminability-based transfer (DBT) algorithm.

In 1997, Pratt and Sebastian Thrun guest edited a special issue of Machine Learning devoted to transfer learning, and by 1998, the field had advanced to include multi-task learning, along with a more formal analysis of its theoretical foundations. Learning to Learn, edited by Thrun and Pratt, is a 1998 review of the subject.

Transfer learning has also been applied in cognitive science, with Pratt also guest editing an issue of Connection Science on reuse of neural networks through transfer in 1996.

Andrew Ng said in his NIPS 2016 tutorial that TL will be the next driver of ML commercial success after supervised learning to highlight the importance of TL.

Definition

The definition of transfer learning is given in terms of domains and tasks. A domain ${\mathcal {D}}$ consists of: a feature space ${\mathcal {X}}$ and a marginal probability distribution $P(X)$, where $X=\{x_{1},...,x_{n}\}\in {\mathcal {X}}$. Given a specific domain, ${\mathcal {D}}=\{{\mathcal {X}},P(X)\}$, a task consists of two components: a label space ${\mathcal {Y}}$ and an objective predictive function $f:{\mathcal {X}}\rightarrow {\mathcal {Y}}$. The function $f$ is used to predict the corresponding label $f(x)$ of a new instance $x$. This task, denoted by ${\mathcal {T}}=\{{\mathcal {Y}},f(x)\}$, is learned from the training data consisting of pairs $\{x_{i},y_{i}\}$, where $x_{i}\in X$ and $y_{i}\in {\mathcal {Y}}$.

Given a source domain ${\mathcal {D}}_{S}$ and learning task ${\mathcal {T}}_{S}$, a target domain ${\mathcal {D}}_{T}$and learning task ${\mathcal {T}}_{T}$, where ${\mathcal {D}}_{S}\neq {\mathcal {D}}_{T}$, or ${\mathcal {T}}_{S}\neq {\mathcal {T}}_{T}$, transfer learning aims to help improve the learning of the target predictive function $f_{T}(\cdot )$ in ${\mathcal {D}}_{T}$ using the knowledge in ${\mathcal {D}}_{S}$ and ${\mathcal {T}}_{S}$.

Applications

Algorithms are available for transfer learning in Markov logic networks and Bayesian networks. Transfer learning has also been applied to cancer subtype discovery, building utilization, general game playing, text classification, digit recognition, medical imaging and spam filtering.

In 2020 it was discovered that, due to their similar physical natures, transfer learning is possible between electromyographic (EMG) signals from the muscles and classifying the behaviors of electroencephalographic (EEG) brainwaves, from the gesture recognition domain to the mental state recognition domain. It was also noted that this relationship worked vice versa, showing that EEG can likewise be used to classify EMG. The experiments noted that the accuracy of neural networks and convolutional neural networks were improved through transfer learning at both the first epoch (prior to any learning, ie. compared to standard random weight distribution) and the asymptote (the end of the learning process). That is, algorithms are improved by exposure to another domain. Moreover, the end-user of a pre-trained model can change the structure of fully-connected layers to achieve superior performance.

In the domain of machine learning on code, it has been shown that transfer learning is useful for automatically repairing security vulnerabilities.

Softwares

Several compilations of transfer learning and domain adaptation algorithms have been implemented over the past decades:

• ADAPT (Python)
• TLlib (Python)
• Domain-Adaptation-Toolbox (Matlab)

See also

• Crossover (genetic algorithm)
• Domain adaptation
• General game playing
• Multi-task learning
• Multitask optimization
• Zero-shot learning

Sources

• Thrun, Sebastian; Pratt, Lorien (6 December 2012). Learning to Learn. Springer Science & Business Media. ISBN 978-1-4615-5529-2.