+39 06 3972 8725 [email protected] Mon–Fri 9:30–18:30 · Rome, Via Gomenizza 41
YouTube Instagram LinkedIn

← All articles

ARC: The Brilliant Test That AIs Cannot Pass (But a Child Can)

AI Confidential · March 23, 2025 · by Sergio & Nox

ARC: The Brilliant Test That AIs Cannot Pass (But a Child Can)

ARC (Abstraction and Reasoning Corpus) is a test designed to measure the true intelligence of machines.

ARC (Abstraction and Reasoning Corpus) is not a simple test. It is a puzzle designed to strike at the heart of what we call intelligence. Created by the researcher François Chollet (the author of Keras), ARC was designed not to verify the computing power of AIs, but their ability for abstract reasoning, just as a human being would do.

How it works is simple only in appearance: each task shows a few colored grids with an underlying logical transformation. The goal? To grasp the rule and apply it to a new input. Humans, even children, manage to deduce the solution from a few examples. AIs don't.

Average AI success: 31%. Average human success: 80%. Source: Google AI Blog

But why is it so important?

ARC doesn't reward memory or exposure to large datasets — the classic strengths of LLMs — but demands mental flexibility, intuition, the capacity for abstraction. Exactly what machines still lack. In plain words, real intelligence.

To understand better, read Arc Prize

The Google Research study confirms it: “Current language models fail to tackle ARC because they lack generalization and adaptability.” It is not enough to recognize patterns or interpolate solutions from millions of examples. You have to understand, reduce to the essence, intuit the invisible rule. The one that cannot be calculated, only seen.

And here an uncomfortable truth emerges: however high-performing, LLMs are still blind in front of tasks that require human thinking. ARC is the point where machines slow down and humans accelerate.

It is a challenge that goes beyond AI. It is a test of humility.

It shows us that understanding the world is not only a matter of data, but of meaning.

Want to understand why artificial intelligence makes certain “strange” mistakes? Discover the secret hidden in the DNA of LLMs.

“Intelligence is not skill itself, but the ability to acquire new skills.”

— François Chollet


← All articles