What Is Transfer Learning? Techniques, Uses and Examples

• 1.2 million images across 1,000 categories: the size of ImageNet, still the standard vision pretraining set, per a 2019 study on ImageNet-trained classification layers.
• 175 billion parameters: GPT-3 in 2020, ten times larger than any prior non-sparse language model, performing new tasks with no weight updates at all (OpenAI, 2020).
• 67% and 72%: phone error rate reductions on Swedish and Turkish from cross-lingual speech transfer, the most recent published measurements, from a 2020 study on unsupervised cross-lingual representation learning.
• 1.58x: compute efficiency gained by tuning one base learning rate at small scale and transferring it across budgets, per 2026 research on hypersphere optimization.

What is transfer learning? The core definition

Transfer learning is a machine learning method that reuses what a model learned on one task (the source) to learn a different but related task (the target) with less data and less compute than training from scratch. The formal version comes from Sinno Jialin Pan and Qiang Yang’s 2010 survey in IEEE Transactions on Knowledge and Data Engineering, which is still the definition most of the field cites.

The goal is to “improve the learning of the target predictive function in the target domain using the knowledge in the source domain and source task.” (Pan and Yang, 2010)

That sentence hides a bigger break with tradition than it sounds. Classical machine learning assumes your training data and your real-world data come from the same distribution, the independent and identically distributed (i.i.d.) assumption. Transfer learning drops it. A model trained on photographs of everyday objects can help read chest X-rays precisely because nobody expects the two datasets to match. A 2019 survey on transfer learning by Zhuang and colleagues makes this relaxation of the i.i.d. assumption the central reason the approach generalizes across related data.

Why anyone bothers. Three practical pressures push teams toward transfer instead of from-scratch training:

• Labeled data is scarce where it matters most. A hospital might have a few hundred annotated scans. Web-scale image collections have millions of labeled examples.
• Training from nothing costs compute and calendar time that most projects don’t have.
• The features a large network learns on general data turn out to be reusable. Edges, textures and word patterns look the same whether the downstream task is retail or radiology.

The idea predates the deep learning boom. In 2007, Rajat Raina, Andrew Ng and colleagues at Stanford published self-taught learning, which used unlabeled data via sparse coding to build a representation that improved supervised classification. The unlabeled data didn’t even need to share the target task’s classes. That is the pretrain-then-fine-tune recipe in embryo, thirteen years before it became the default way to build almost any model.

If you run an engineering team in healthcare, finance or logistics, the operational takeaway is blunt. You almost never start from zero anymore. The question is which pretrained knowledge to borrow, and how much of it to keep.

Transfer learning techniques: the main approaches explained

Transfer learning techniques fall into four practical groups: frozen feature extraction, partial or full fine-tuning, parameter-efficient fine-tuning (adapters and LoRA), and in-context transfer through prompting with no weight updates at all. Which one you pick comes down to how many parameters you’re willing to train, how much target data you have, and what GPU budget you can defend.

The evidence for why any of this works comes from a 2014 experiment by Jason Yosinski, Jeff Clune, Yoshua Bengio and Hod Lipson on how transferable features in deep neural networks really are. Using ImageNet-trained networks, they showed that early layers learn general features (Gabor-like filters, color blobs) that move well between tasks, while later layers grow task-specific. Their sharper finding: initializing from transferred features and then fine-tuning beat random initialization even when the source task was fairly distant, and the advantage survived full fine-tuning.

Frozen feature extraction. Keep every pretrained layer fixed and train only a new output head on your data. It needs the least data and the least compute, and it works best when the target looks like the source.

Partial and full fine-tuning. Unfreeze some layers (usually the later ones) or all of them, and keep training on target data. The gap between theory and practice here is the layer question. Yosinski’s results say later layers are the task-specific ones, but which cutoff is right changes from dataset to dataset, and nobody has a formula for it.

Parameter-efficient fine-tuning (PEFT). As pretrained models grew, updating every weight stopped being affordable. A 2019 paper on parameter-efficient transfer for NLP introduced adapter modules that add a small number of trainable parameters per task while the base model stays frozen. In 2026 the dominant PEFT method is LoRA (Low-Rank Adaptation), which injects trainable low-rank update matrices into transformer layers. Research from 2025 and 2026 is now probing its limits, including work on breaking the low-rank bottleneck with nonlinear mappings and on guided LoRA initialization using Fisher-subspace analysis. For enterprise LLM fine-tuning, LoRA is the default starting point.

In-context transfer. OpenAI’s 2020 GPT-3 paper showed a 175-billion-parameter model performing new tasks from demonstrations in the prompt, with no gradient updates. Knowledge transfers through text instead of weights.

Hugging Face’s documentation frames its checkpoint library exactly this way: reuse a pretrained model, adapt only the layers a new task needs, and cut both training time and GPU spend. I’d argue most production teams should treat that ordering as a ladder. Start frozen. Climb only when the validation numbers demand it.

How transfer learning is categorized: inductive, transductive and unsupervised

Transfer learning is categorized by label availability into three settings, inductive, transductive and unsupervised, following Pan and Yang’s 2010 taxonomy, with a second, independent axis describing whether instances, features or model parameters are the thing being transferred.

The label-based split works like this:

• Inductive transfer learning: the target domain has labels, whether or not the source does. Multi-task learning sits here, where one model shares output layers across several tasks trained together.
• Transductive transfer learning: the source is labeled and the target is unlabeled. This is home to classic domain adaptation, where the task stays the same (object recognition on both sides, say) but the data distributions differ. A 2021 review of transfer learning and domain adaptation for NLP on electronic health records treats clinical text this way: same extraction task, different hospitals, different writing styles.
• Unsupervised transfer learning: neither side has labels.

The second axis is about mechanism. Instance-based transfer reweights individual source examples so the useful ones count more. Feature-based transfer learns a shared representation in which source and target look alike. Parameter-based transfer copies model weights across.

That last one won. A 2018 survey on deep transfer learning describes parameter-based transfer as the dominant mode in deep networks, carried out as a two-phase pipeline: pretrain on abundant source data, then fine-tune some or all of the weights on scarce target data. Instance and feature reweighting still matter in domain adaptation work, and they still show up in the mitigation literature, but when an engineer says “transfer learning” in 2026 they nearly always mean moving parameters.

One thing the tidy taxonomy hides. Real projects straddle categories. A team fine-tuning a language model on a labeled support-ticket set is doing inductive, parameter-based transfer, yet the pretrained checkpoint they start from was itself built by unsupervised learning on unlabeled text. The categories describe stages of a pipeline more often than whole systems.

How a transfer model is built: the pretrain-then-fine-tune pipeline

A transfer model is built in two phases: a long, general pretraining run on abundant source data, then a short, task-specific fine-tuning run on the target data with a new output layer attached. The first phase is usually someone else’s work. The second is yours.

Phase one: pretraining. What the source model learns depends on the modality:

• Vision: ImageNet, roughly 1.2 million images across 1,000 categories, is still the standard pretraining set for convolutional architectures such as ResNet, VGG and GoogLeNet.
• Language: BERT pretrains with masked language modeling plus next-sentence prediction, a task-agnostic phase that runs far longer than any downstream fine-tune.
• Self-supervised: contrastive methods (MoCo, SimCLR, BYOL) build transferable representations from unlabeled images by learning which augmented views of an image belong together. No manual labels at all.

Phase two: fine-tuning. The steps, in the order they happen on a real project:

1. Pull a checkpoint whose source domain sits close to your target. Hugging Face’s guide to fine-tuning a pretrained model is the standard entry point for transformer checkpoints, and torchvision plays the same role for CNNs.
2. Remove the source output layer and attach a new head sized to your classes. For BERT-style text classification, a 2019 study by Sun and colleagues showed a single added output layer is enough.
3. Decide what stays frozen. Head only first. Then unfreeze the later layers. Then everything, if the validation numbers keep asking for it.
4. Train on the target data while watching validation loss instead of training loss, because a pretrained network can memorize a small dataset in a few epochs.

Learning rates are where people get burned. The freshly initialized head needs to move a lot. The pretrained layers need to move very little, or the first few hundred steps erase the features you borrowed the model for. A 2024 comparison of fine-tuning strategies in medical image classification found that fine-tuning every layer the same way is often worse than the alternatives. Separate learning rates per layer group are the usual fix, and on our own model fine-tuning work that one change turns more disappointing first runs into usable ones than any other.

Transfer learning examples across industries

Transfer learning examples in production span medical imaging, genomics, low-resource speech recognition, cybersecurity, reinforcement learning and, as of 2026, video understanding built on image-language foundation models. The pattern is the same everywhere: labeled target data is scarce, and a model pretrained elsewhere fills the gap. The measured results vary far more than the pattern does.

Medical imaging. ImageNet-pretrained networks are the default starting point for scan classification, and the caveats matter. Google Research’s 2021 study of supervised transfer at scale for medical imaging found the benefit appears only when the transfer is done at sufficient scale. A 2022 follow-up identified feature reuse as the mechanism doing the work. A December 2024 study of dataset transferability found that which source-target pairs transfer well is problem-dependent, and a 2025 survey of machine learning researchers’ intuitions about medical-image transfer confirmed the field still has no rule for which features to reuse.

Genomics and bioinformatics. Models move across species, cell types and datasets. Published examples include cancer survival prediction from gene-expression data (2020), cancer drug sensitivity prediction (2019), and a 2024 method that improved protein stability change prediction by pretraining on larger datasets before adapting to a small, high-quality target set.

Speech and low-resource languages. The numbers here are the most striking, and the most recent published measurements date from 2020. Cross-lingual representation transfer cut phone error rate by roughly 67% on Swedish and 72% on Turkish against non-transfer baselines in that 2020 work on unsupervised cross-lingual representation learning for speech. A separate 2020 study that added speech translation as an auxiliary task reported up to a 24.6% word-error-rate reduction. A 2021 paper on semi-supervised language expansion for end-to-end speech recognition calls high-to-low-resource transfer an efficient way to improve ASR, which reads as understatement next to those figures.

Cybersecurity. Rare and zero-day attack detection is the main use, since nobody has thousands of labeled examples of an attack that has happened twice. A 2025 Scientific Reports paper on federated transfer learning for rare attack class detection and a 2024 review of transfer learning for security both name intrusion detection and malware classification as the field’s predominant application. Both also warn that evolving attack patterns limit straightforward transfer.

Reinforcement learning. Taylor and Stone’s 2009 JMLR survey set the frame: reuse rewards, demonstrations, policies or representations across tasks. The 2023 IEEE survey on transfer in deep reinforcement learning sorts current methods into reward shaping, learning from demonstration, policy reuse, inter-task mapping and representation transfer.

Image-to-video. A January 2026 survey of image-to-video transfer built on image-language foundation models splits current techniques into frozen-feature and adapted-feature approaches. The source has shifted from a single ImageNet CNN to a multimodal foundation model stretched into a modality it never saw in training.

Foundation models and scaling laws changed what transfer means

Foundation models changed transfer learning by making weight updates optional: since OpenAI’s 2020 GPT-3 paper, a large enough pretrained model can take on a new task from a few examples in its prompt, and scaling-law research since 2021 has begun to measure how much knowledge that pretraining actually transfers.

GPT-3’s headline result in 2020 was a 175-billion-parameter model, ten times larger than any prior non-sparse language model, performing new tasks from in-context demonstrations “without any gradient updates or fine-tuning.” Every earlier technique moved knowledge by changing weights. Prompting moves it through text.

Stanford researchers named the new category in their 2021 report on the opportunities and risks of foundation models, and flagged the cost of building everything on one shared base:

Such models’ effectiveness across many tasks “incentivizes homogenization,” so that defects in the foundation model propagate to every model adapted from it downstream. (Stanford CRFM, 2021)

For an enterprise, that is the trade. One base model serves fifty use cases, and one flaw in it shows up in fifty places.

What the scaling laws say. Three findings, one per research generation:

• 2021, Scaling Laws for Transfer (OpenAI-affiliated): in the low-data regime, the effective amount of data transferred from pretraining follows a power law in both parameter count and fine-tuning dataset size. Bigger pretrained models transfer more, and predictably so.
• 2025, Overtrained Language Models Are Harder to Fine-Tune: pretraining a model past a certain point makes it harder to adapt afterward, which breaks the assumption that more pretraining is always better.
• 2026, hyperparameter transfer under hypersphere optimization: tuning a single base learning rate at small scale transfers across compute budgets, yielding roughly 1.58x compute efficiency over a strong baseline.

The 2025 result is the one I’d flag for anyone budgeting a fine-tuning program. It means the most heavily pretrained checkpoint on the leaderboard can be a worse starting point for your data than a smaller, less saturated one. Where the threshold sits for a given target is still an open question.

When transfer learning fails: negative transfer and domain shift

Transfer learning fails when the knowledge carried over from the source task lowers performance on the target instead of raising it, a condition the field calls negative transfer, and the usual cause is a domain shift too large for the borrowed features to fit. A 2020 survey on negative transfer defines it as source-domain knowledge that “undesirably reduces the learning performance in the target domain.”

Nobody warns you about this in the tutorials. The pretrained checkpoint loads, training loss drops, and the validation numbers land below what a small model trained from scratch would have produced. The transfer looked free. It cost you accuracy.

Three conditions raise the risk, per that 2020 survey and the medical imaging work published since:

• A source domain far larger than the target and full of data irrelevant to it. Scale helps only when the extra data is related.
• A wide distribution gap between source and target. Natural photos to satellite radar is a bigger jump than natural photos to dermatology, and the features transfer accordingly.
• Uniform fine-tuning. The 2024 comparison of fine-tuning strategies for medical image classification found that updating every layer identically is often worse than treating layer groups differently.

Mitigations exist, and most of them work by deciding what to keep rather than moving everything across. Instance weighting and importance sampling downweight source examples that don’t resemble the target, the standard tool in domain adaptation. Spectral filtering suppresses pretrained components identified as untransferable during fine-tuning. Gradient surgery reduces conflicting gradients when several tasks share one model. An October 2025 paper on mitigating negative transfer by reducing environmental disagreement extends that list, and the December 2024 work on dataset transferability in medical image classification treats the source-target pairing itself as something to test before you commit a GPU budget to it.

Four signs you should skip transfer for a given project:

1. Your target domain shares almost nothing with any available pretrained source.
2. You already hold enough labeled target data that a from-scratch model performs well.
3. The target distribution changes faster than you can re-fine-tune, the problem the 2024 security review flags for evolving attack patterns.
4. A from-scratch baseline beats the transferred model on held-out validation.

That fourth check is the only reliable detector of negative transfer. Run it every time.

Transfer learning vs fine-tuning vs domain adaptation vs multi-task learning

Transfer learning is the umbrella term, and fine-tuning, domain adaptation and multi-task learning are three specific ways of doing it that differ in what is labeled, whether the task changes, and when training happens. People swap the words freely in meetings. The differences decide which method you actually need.

Two distinctions do most of the work. The first is timing: fine-tuning happens after pretraining, while multi-task learning trains everything in one run with shared output layers. The second is task identity. Domain adaptation keeps the task fixed and only the data moves, which is why a 2021 review of transfer learning and domain adaptation for NLP on electronic health records treats moving a clinical extraction model between hospitals as adaptation rather than a new task.

A quick test for the right label. If the output you want is the same and only the inputs look different, say domain adaptation. If you are changing weights on a downloaded checkpoint, say fine-tuning. If several objectives train side by side, say multi-task learning. If you are unsure, transfer learning is always correct, just less precise.

Frequently asked questions about transfer learning

What is transfer learning in simple terms?

Transfer learning means starting from a model that already learned something related, instead of starting from random weights. A network trained on millions of everyday photos already knows edges, textures and shapes, so teaching it to read X-rays takes far fewer scans than teaching a blank one. Pan and Yang formalized the idea in 2010, and it is now the default way most production models get built.

What is the difference between transfer learning and fine-tuning?

Transfer learning is the goal and fine-tuning is one method of reaching it. Fine-tuning specifically means continuing to train some or all of a pretrained model’s weights on your target data. Frozen feature extraction, LoRA adapters and prompting a foundation model are also transfer learning, and none of them are full fine-tuning.

Is ChatGPT an example of transfer learning?

Yes, in the broad sense. OpenAI’s 2020 GPT-3 paper showed a 175-billion-parameter model performing new tasks from examples in the prompt “without any gradient updates or fine-tuning,” which is knowledge transfer through text rather than weight changes. Assistants built on large pretrained language models inherit that property, and Stanford’s 2021 foundation model report treats every downstream adaptation of such a base as transfer.

What are the advantages and disadvantages of transfer learning?

The advantages are less labeled data, less compute, and better accuracy than random initialization, a result Yosinski and colleagues measured on ImageNet networks in 2014. The disadvantages are negative transfer when source and target differ too much, the homogenization risk Stanford flagged in 2021 (one flaw in the base model surfaces in every adapted model), and the 2025 finding that overtrained language models are harder to fine-tune. Whether the trade pays off depends on how close your target sits to the source.

When should you not use transfer learning?

Skip it when no pretrained source resembles your target, when you already have enough labeled data for a from-scratch model to perform well, or when your data drifts faster than you can retrain. The deciding test is empirical: a from-scratch baseline on held-out validation tells you whether transfer helped. Teams unsure which case they’re in often benefit from an AI integration audit before the first training run, since the answer usually lives in the data rather than the model.

Is transfer learning supervised or unsupervised?

It can be either, and most modern pipelines are both. Pan and Yang’s 2010 taxonomy splits it into inductive (target labeled), transductive (source labeled, target unlabeled) and unsupervised (neither labeled) settings. A typical 2026 project pretrains without labels, through masked language modeling or contrastive methods like SimCLR, then fine-tunes with labels on the target task.

How to start using transfer learning on your own data

You start using transfer learning on your own data by picking a pretrained checkpoint close to your domain, freezing it, training only a new head, and unfreezing more of the network only when validation results demand it. The whole process fits in a checklist:

1. Choose the nearest source. A Hugging Face or torchvision checkpoint trained on data that resembles yours beats a bigger model trained on data that doesn’t.
2. Freeze everything and train the head. Hundreds of labeled examples are often enough to get a first number.
3. Unfreeze progressively, later layers first, with a smaller learning rate for pretrained weights than for the new head.
4. Use LoRA when the base model is large. Full fine-tuning of a modern language model rarely pays for itself.
5. Train a from-scratch baseline on the same split. If the transferred model doesn’t beat it on held-out data, you have negative transfer, and no amount of tuning fixes a bad source.

The fifth step gets skipped most. Skip it and you never learn whether the pretrained model helped or quietly hurt.

Budget one afternoon for the first frozen run. Most teams find out within that afternoon whether the source they picked is worth climbing the ladder with, and that answer decides the next month.