[CS330] 02. Multi-Task Learning & Transfer learning Basics

What is "Task"?

More formally, a task can be described as this format: $\mathcal{T}_i  \equiv \{p_i(\textbf{x}), p_i (\textbf{y|x}), \mathcal{L}_i\}$, based on data generating distribution.

• Multi-task learning: Learn  $\mathcal{T}_1, \mathcal{T}_2, \cdots, \mathcal{T}_T$ at once
  • Transfer learning: Solve target task  $\mathcal{T}_b$ after solving source task $\mathcal{T}_a$ by transferring knowledge learned from $\mathcal{T}_a$ ($\mathcal{T}_a$ include multiple tasks itself.)
    • Key assumption: Cannot access data $\mathcal{D}_a$ during transfer. 
    • Fine-tuning is a valid solution to transfer learning $\phi \leftarrow \theta -\alpha \nabla_\theta \mathcal{L}(\theta, \mathcal{D}^{tr})$
• The meta-learning: Given data on previous tasks, learn a new task more quickly and proficiently

Vanilla Multi-Task Learning 

We have to define the model, objective, and its optimization way.

1. We use task descriptor $z_i$: $f_\theta (\textrm{y|x}) \rightarrow f_\theta (\textrm{y|x, z_i})$
   • How does the Multi-task architecture should be: Concatenation-based conditioning, Additive conditioning, Multi-head architecture, and Multiplicative conditioning
2. Objective $\min_\theta \sum_{i=1}^T \mathcal{L}_i(\theta, \mathcal{D}_i)$
   • We can weight tasks differently: $\min_\theta \sum_{i=1}^T w_i \mathcal{L}_i (\theta, \mathcal{D}_i)$
     • The weighting can be a. heuristics, b. task uncertainty c. monotonic improvment towards Pareto optimal solutions or d. optimize for the worst-case task loss.
3. How should the objective be optimized: By mini-batch Backpropagation with optimizer
4. Challenges: Negative transfer, Overfitting, Lot of tasks, etc.

Multi-task learning → Transfer learning

• What are some problems/applications where transfer learning might make sense?
  • When $\mathcal{D}_a$ is very large (Don't want to retain & retrain on $\mathcal{D}_a$)
  • When you don't care about solving $\mathcal{T}_a$ and $\mathcal{T}_b$ simultaneously
  • More answers about. Personalization

 Transfer learning  → Fine-tuning

Transfer-learning: If we are solving target task $\mathcal{T}_b$ after solving source task $\mathcal{T}_a$ by transferring knowledge learned from $\mathcal{T}_a$

We can use fine-tuning by using $\theta$, which is the parameters pre-trained on $\mathcal{D}_a$. We have a training data for new task $\mathcal{T}_b$. We can fine-tune for many gradient steps by this formula:

$\phi \leftarrow - \alpha \nabla_\theta \mathcal{L}(\theta, \mathcal{D}_{b}^{tr})$

However, fine-tuning does not work well with very small target task datasets. This is where meta-learning can help.