Gary Marcus? Sam Altman? Apocryphal?
Question for Quote Investigator: The progress achieved in artificial intelligence (AI) during the 21st century has been remarkable. Billions of dollars have been spent to build supercomputers and to train AI systems. Yet, the rate of future progress is uncertain. A skeptic stated:
Deep learning is hitting a wall.
An enthusiast replied:
There is no wall.
Would you please help me to identify the people in this exchange and find precise citations?
Reply from Quote Investigator: In 2022 cognitive scientist and AI researcher Gary Marcus published an article in the popular science magazine “Nautilus” with the following provocative title:1
Deep Learning Is Hitting a Wall
The piece argued that the field of artificial intelligence was filled with hype and bravado. Also, current AI systems did not “genuinely understand human language” and displayed “nowhere near the ordinary day-to-day intelligence of Rosey the Robot” (from “The Jetsons” animated TV series of the 1960s).
The article said that the strategy of building larger systems with more chips and more data was “perhaps already approaching a point of diminishing returns”, and the AI field should pursue a neurosymbolic approach.
In November 2024 Sam Altman, the CEO of the leading AI company OpenAI tweeted the following riposte:2
there is no wall
Later, Gary Marcus tweeted the following reply:3
1. Multiple media reports from multiple companies are reporting diminishing returns, exactly as I warned in 2022 in “deep learning is hitting a wall”.
2. If I am wrong, where is GPT-5?
From 2018 to 2024 OpenAI announced a series of models called GPT-1, GPT-2, GPT-3, GPT-3.5, GPT-4 and GPT-4o. But a model named GPT-5 was not released in 2024.
Beginning in September 2024 OpenAI announced a different series of models called o1 and o3. These models used more computation to ruminate about complex tasks. The o3 model achieved new high scores on difficult benchmark tests for mathematics, programming, and fluid intelligence.
Below are additional selected citations in chronological order.
The notion that the latest surge of progress in AI might stall has been expressed by others. For example, in August 2018 the website “Popular Science” published a piece titled:4
Another AI winter could usher in a dark period for artificial intelligence
In October 2018 the “Financial Times” published the following article title and subtitle:5
Artificial intelligence: winter is coming
Today’s AI is not much better at solving real world problems than its ancestors
In 2019 a commentator on the “Medium” website published a piece titled “The next AI winter is coming, and that’s great” which contained the phrase “AI is hitting a wall”:6
However, that doesn’t explain why the current boom is ending. The answer, I’d propose, is that AI is hitting a wall because it has to. The boom was never going to deliver all the goods that were promised.
In conclusion, the phrase “Deep Learning Is Hitting a Wall” appeared as the title of an article by Gary Marcus in March 2022. The phrase “there is no wall” appeared as a tweet from Sam Altman in November 2024.
Image Notes: Abstract image representing AI from Gerd Altmann at Pixabay. The image has been cropped and resized.
Acknowledgement: Great thanks to the anonymous person whose inquiry led QI to formulate this question and perform this exploration.
- Website: Nautilus, Article title: Deep Learning Is Hitting a Wall, Article author: Gary Marcus, Date on website: March 10, 2022, Website description: Science magazine and website. (Accessed nautil.us on Jan 1, 2024) link ↩︎
- X-Tweet, From: Sam Altman @sama, Timestamp: 1:06 AM, Nov 14, 2024, Text: there is no wall. (Accessed on x.com on Jan 1, 2025) link ↩︎
- X-Tweet, From: Gary Marcus @GaryMarcus, Timestamp: 8:40 AM, Nov 14, 2024, Text: Multiple media reports from multiple companies are reporting diminishing returns. (Accessed on x.com on Jan 1, 2025) link ↩︎
- Website: Popular Science, Article title: Another AI winter could usher in a dark period for artificial intelligence, Article author: Eleanor Cummins (Contributor), Date on website: Aug 30, 2018, Website description: Science news. (Accessed popsci.com on Jan 2, 2025) link ↩︎
- Website: Financial Times FT, Article title: Artificial intelligence: winter is coming, Article author: Zia Chishti, Date on website: October 17, 2018, Website description: Financial news. (Accessed ft.com on Jan 2, 2025) link ↩︎
- Website: Medium: productOps, Article title: The next AI winter is coming, and that’s great, Article author: Alexander Lamb, Date on website: Feb 19, 2019, Website description: Medium is a platform for writers and readers; productOps says “Software product development in Santa Cruz, CA. We cover strategy, development, operations, and marketing”. (Accessed medium.com/productops on Jan 2, 2025) link ↩︎