by François Chollet
Skill is the output of intelligence, not its substance. A measure should control for priors and experience, and reward skill-acquisition efficiency over a scope of tasks.
tags: intelligence, generalization, ARC, priors, benchmarks, AGI
Looked at in one way, everyone knows what intelligence is; looked at in another way, no one does.
A reader's collection of passages from Chollet's paper, ordered as in the PDF. Full text: arXiv:1911.01547 · 62 pages. Toggle a term below to trace it across the quotes.
The thesis in one sentence.
To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans.
The second half of the abstract, doing the actual work.
Solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power.
Why dictionary definitions and the Turing Test won't do the job.
Common-sense dictionary definitions of intelligence may be useful to make sure we are talking about the same concepts, but they are not useful for our purpose, as they are not actionable, explanatory, or measurable. Similarly, the Turing Test and its many variants are not useful as a driver of progress (and have in fact served as a red herring), since such tests completely opt out of objectively defining and measuring intelligence, and instead outsource the task to unreliable human judges who themselves do not have clear definitions or evaluation protocols.
On the state of the field.
It is a testimony to the immaturity of our field that the question of what we mean when we talk about intelligence still doesn't have a satisfying answer. What's worse, very little attention has been devoted to rigorously defining it or benchmarking our progress towards it.
The cost of not defining terms.
The absence of widely-accepted explicit definitions has been substituted with implicit definitions and biases that stretch back decades. Though invisible, these biases are still structuring many research efforts today, as illustrated by our field's ongoing fascination with outperforming humans at board games or video games.
§I.2 - two visions of intelligence the field keeps oscillating between.
Two characterizations, which are nearly universally - but often separately - found in definitions of intelligence: one with an emphasis on task-specific skill ("achieving goals"), and one focused on generality and adaptation ("in a wide range of environments"). In this view, an intelligent agent would achieve high skill across many different tasks. Implicitly here, the tasks may not necessarily be known in advance: to truly achieve generality, the agent would have to be able to learn to handle new tasks (skill acquisition).
§I.2.1 - the Minsky-style vision, which produced the field's early wins.
This vision of the mind as a wide collection of vertical, relatively static programs that collectively implement "intelligence", was most prominently endorsed by influential AI pioneer Marvin Minsky. This view gave rise to definitions of intelligence and evaluation protocols for intelligence that are focused on task-specific performance. This is perhaps best illustrated by Minsky's 1968 definition of AI: "AI is the science of making machines capable of performing tasks that would require intelligence if done by humans".
§II.1.1 - what's wrong with the "moonshot" games.
Although OpenAI's DotA2-playing AI "Five" was trained on 45,000 years of play and was able to beat top human players, it has proven very brittle, as non-champion human players were able to find strategies to reliably beat it in a matter of days after the AI was made available for the public to play against. In addition, Five did not even generalize to DotA2 in the first place: it could only play a restricted version of the game, with 16 characters instead of over 100.
The hashtable example - a system with almost no intelligence can "solve" any finite task.
Consider a hashtable that uses a locality-sensitive hash function (e.g. nearest neighbor) to map new inputs to previously seen inputs. Such a system implements a learning algorithm capable of local generalization, the extent of which is fixed (independent of the amount of data seen). This system, despite only featuring trace amounts of generalization power, is already sufficient to "solve" any task for which unlimited training data can be generated, such as any video game. All that one has to do is obtain a dense sampling of the space of situations that needs to be covered, and associate each situation with an appropriate action vector.
The if/else chatbot - the paper's most memorable image.
When a human engineer implements a chatbot by specifying answers for each possible query via if/else statements, we do not assume this chatbot to be intelligent, and we do not expect it to generalize beyond the engineer's specifications. Likewise, if an engineer looks at a specific IQ test task, comes up with a solution, and writes down this solution in program form, we do not expect the program to generalize to new tasks, and we do not believe that the program displays intelligence - the only intelligence at work here is the engineer's. The program merely encodes the crystallized output of the engineer's thought process - it is this process, not its output, that implements intelligence.
The indictment of skill-focused evaluation.
In summary, the hallmark of broad abilities (including general intelligence) is the power to adapt to change, acquire skills, and solve previously unseen problems - not skill itself, which is merely the crystallized output of the process of intelligence. Testing for skill at a task that is known in advance to system developers can be gamed without displaying intelligence, in two ways: 1) unlimited prior knowledge, 2) unlimited training data. To actually assess broad abilities, and thus make progress toward flexible AI and eventually general AI, it is imperative that we control for priors, experience, and generalization difficulty in our evaluation methods, in a rigorous and quantitative way.
Crank priors or experience and skill climbs - but intelligence barely moves. Only the third slider, the system's own generalization power, earns it.
§II.1.2 - No Free Lunch applied to human intelligence itself.
The No Free Lunch theorem teaches us that any two optimization algorithms (including human intelligence) are equivalent when their performance is averaged across every possible problem, i.e. algorithms should be tailored to their target problem in order to achieve better-than-random performance. Thus we may ask: is the human g factor universal? Would it generalize to every possible task in the universe?
The anthropocentric move, stated baldly.
If there is such a thing as universal intelligence, and if human intelligence is an implementation of it, then this algorithm of universal intelligence should be the end goal of our field, and reverse-engineering the human brain could be the shortest path to reach it. It would make our field close-ended: a riddle to be solved. If, on the other hand, human intelligence is a broad but ad-hoc cognitive ability that generalizes to human-relevant tasks but not much else, this implies that AI is an open-ended, fundamentally anthropocentric pursuit, tied to a specific scope of applicability.
The cross-domain analogy that clarifies the spectrum idea.
Like intelligence, fitness is not easily reducible to any single factor (such as a person's age or muscle mass), rather, it seems to emerge from a constellation of interdependent factors. If we sought to rigorously measure physical fitness in humans, we would come up with a set of diverse tests such as running a 100m, running a marathon, swimming, doing sit-ups, doing basketball throws, etc., not unlike IQ test suites. But would this mean that human morphology and motor affordances are "general" in an absolute sense, and that a very fit person could handle any physical task at all? Certainly not; we are not adapted for the large majority of environments that can be found in the universe.
§II.1.3 - the two bad models of the mind, and why both are wrong.
The human mind is not merely a collection of special-purpose programs hard-coded by evolution; it is capable of a remarkable degree of generality and open-endedness. Simultaneously, the mind is not a single, general-purpose "blank slate" system capable of learning anything from experience. Our cognition is specialized, shaped by evolution in specific ways; we are born with priors about ourselves, about the world, and about how to learn, which determine what categories of skills we can acquire and what categories of problems we can solve.
Why priors enable generalization rather than limit it.
These priors are not a limitation to our generalization capabilities; to the contrary, they are their source, the reason why humans are capable of acquiring certain categories of skills with remarkable efficiency. The central message of the No Free Lunch theorem is that to learn from data, one must make assumptions about it - the nature and structure of the innate assumptions made by the human mind are precisely what confers to it its powerful learning abilities.
The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty.
The informal intuition beneath the formalism.
If you consider two systems that start from a similar set of knowledge priors, and that go through a similar amount of experience (e.g. practice time) with respect to a set of tasks not known in advance, the system with higher intelligence is the one that ends up with greater skills - the one that has turned its priors and experience into skill more efficiently. This definition of intelligence encompasses meta-learning priors, memory, and fluid intelligence. It is distinct from skill itself: skill is merely the output of the process of intelligence.
The honest disclaimer about the definition itself.
Many possible definitions of intelligence may be valid, across many different contexts, and we do not purport that the definition above and the formalism below represent the "one true" definition. Nor is our definition meant to achieve broad consensus. Rather, the purpose of our definition is to be actionable, to serve as a useful perspective shift for research on broad cognitive abilities, and to function as a quantitative foundation for new general intelligence benchmarks. As per George Box's aphorism, "all models are wrong, but some are useful": our only aim here is to provide a useful North Star towards flexible and general AI.
The consequence: a spectrum, not a threshold.
"General intelligence" is not a binary property which a system either possesses or lacks. It is a spectrum, tied to 1) a scope of application, which may be more or less broad, 2) the degree of efficiency with which the system translates its priors and experience into new skills over the scope considered, 3) the degree of generalization difficulty represented by different points in the scope considered.
The corollary that retires "AGI" in the absolute sense.
It is conceptually unsound to set "artificial general intelligence" in an absolute sense (i.e. "universal intelligence") as a goal. To set out to build broad abilities of any kind, one must start from a target scope, and one must seek to achieve a well-defined intelligence threshold within this scope: AI is a deeply contextual and open-ended endeavour, not a single one-time riddle to be solved.
§III.1.2 - the four priors a fair test is allowed to assume.
Objectness and elementary physics: humans assume that their environment should be parsed into "objects" characterized by principles of cohesion (objects move as continuous, connected, bounded wholes), persistence (objects do not suddenly cease to exist and do not suddenly materialize), and contact (objects do not act at a distance and cannot interpenetrate).
Agentness and goal-directedness: humans assume that, while some objects in their environment are inanimate, some other objects are "agents", possessing intentions of their own, acting so as to achieve goals (e.g. if we witness an object A following another moving object B, we may infer that A is pursuing B and that B is fleeing A), and showing efficiency in their goal-directed actions.
Natural numbers and elementary arithmetic: humans possess innate, abstract number representations for small numbers, which can be applied to entities observed through any sensory modality. These number representations may be added or subtracted, and may be compared to each other, or sorted.
Elementary geometry and topology: this core knowledge system captures notions of distance, orientation, in/out relationships for objects in our environment and for ourselves. It underlies humans' innate facility for orienting themselves with respect to their surroundings and navigating 2D and 3D environments.
Why Core Knowledge, and not a larger set of priors.
Core Knowledge has been written into our DNA by natural evolution. Natural evolution is an extremely low-bandwidth, highly selective mechanism for transferring information from the surrounding environment to an organism's genetic code. It can only transfer information associated with evolutionary pressures, and it can only write about aspects of the environment that are stable over sufficiently long timescales. As such, it would not be reasonable to expect humans to possess vast amounts of human-specific prior knowledge; core knowledge is evolutionarily ancient and largely shared across many species.
The proposal for a fair test.
We propose that an actionable test of general intelligence that would be fair for both humans and machines should only feature tasks that assume the four core knowledge systems listed above, and should not involve any acquired knowledge outside of these priors. We also argue that general AI systems should hard-code as fundamental priors these core knowledge principles.
Try one. ARC gives you a single training pair; you infer the rule and produce the output. Click cells to cycle colors.
One training example per task, one test. No statistics, no dataset. This is the shape of every ARC task.
Full PDF: arxiv.org/abs/1911.01547 · 62 pages. Formal definition in §II.2, ARC in §III, developmental-psychology background in §II.1.3.
✦ memory · ☽ night · ∞ loops · ❧ margins · ◆ proof
a personal library in perpetual arrangement · MMXXVI