I’m incredibly fortunate to work at a place like Hugging Face where around every virtual corner (hidden inside a maze of Slack channels, open source repositories, blog posts, and research papers) there is a brilliant mind waiting. It leaves me with a sense of awe and a healthy amount of imposter syndrome.
The science team in particular is always cooking up something that pushes the boundaries of what a team 10x their size is capable of. Thomas Wolf, the Chief Science Officer at Hugging Face, recently wrote a response to Dario Amodei’s Machines of Loving Grace, that once again showed me the outsized impact this small but mighty team has in the world of AI. The take is sweet and to the point and in direct contrast to much of the AGI hype in the industry.
I shared a controversial take the other day at an event and I decided to write it down in a longer format: I’m afraid AI won’t give us a “compressed 21st century”.
In a nutshell, this idea of a “compressed 21st century” is that the advances made by AI over the course of a decade will be equivalent to the advances made by modern science over the last 100 years. Think that’s fantastical? You’re not alone.
What we’ll actually get, in my opinion, is “a country of yes-men on servers” (if we just continue on current trends) but let me explain the difference with a small part of my personal story.
Thomas goes on to tell the story of his transition from an excellent student to a mediocre researcher (his words!) as a way to demonstrate that recall is not sufficient for making scientific leaps.
This perspective misses the most crucial aspect of science: the skill to ask the right questions and to challenge even what one has learned. A real science breakthrough is Copernicus proposing, against all the knowledge of his days -in ML terms we would say “despite all his training dataset”-, that the earth may orbit the sun rather than the other way around.
And what would pondering about questions be without thinking about the answer to the ultimate question about life, the universe, and everything.
Remember Douglas Adams’ Hitchhiker’s Guide? The answer is apparently 42, but nobody knows the right question. That’s research in a nutshell.
We’re currently building very obedient students, not revolutionaries. This is perfect for today’s main goal in the field of creating great assistants and overly compliant helpers. But until we find a way to incentivize them to question their knowledge and propose ideas that potentially go against past training data, they won’t give us scientific revolutions yet.
Unless we make a breakthrough in harnessing unbounded computational power or paradigm shift in architectures that unlocks thinking outside the training data, we are left with incredibly powerful autocomplete. Which, honestly, might not be a bad outcome.