Their point of view of why multi-task learning and meta-learning are important
- Robots can teach us things about intelligence.
- Faced with the real world
- Must generalize across tasks, objects, environments, etc
- Need some common sense understanding to do well
- Supervision can't be taken for granted
- Specialists vs. Generalists
- Specialist: Learn one task in one environment, starting from scratch using detailed supervision -> Impractical to learn from scratch for each disease, each robot, each person, each language, each task
- Humans are generalists
- We want
- a more general-purpose AI system
- when we don't have a large dataset
- need to quickly learn something new
Task?
Different tasks can vary based on: different objects, different people, different objectives, etc.
Not just different "tasks". There are many tasks with shared structure!
- The law of physics underly real data
- People are all organisms with intentions
- The rules of English underly English language data
- Languages all develop for similar purposes
More formally, a task can be described as this format: $\mathcal{T}_i \equiv \{p_i(\textbf{x}), p_i (\textbf{y|x}), \mathcal{L}_i\}$, based on data generating distribution
Informal Problem Definitions
- The multi-task learning problem: Learn all of the tasks more quickly/proficiently than learning them independently.
- Simply aggregating the data across tasks & learning a single model is one approach to multi-task learning
- Transfer learning is a valid solution to multi-task learning (but not vice versa)
- Exploit the fact that we know that data is coming from different tasks.
- The meta-learning problem: Given data on previous tasks, learn a new task more quickly/proficiently