This story first appeared in The Algorithm – our weekly newsletter about AI. Get stories like this delivered to your inbox. Sign up here
The concept artificial general intelligence – an ultra-powerful AI we don’t yet have – can be thought of as balloon, repeatedly inflated by hype during peaks in optimism (or fear), about its potential impact, and then deflated when reality fails to match expectations. This week, a lot of news was pumped into the AGI balloon. I’m going tell you what this means (and probably stretch the analogy a bit too far along with it).
Let’s first get rid of the annoying business of defining AGI. In practice, the term is a highly ambiguous and flexible one that’s shaped by researchers or companies devoted to building the technology. It usually refers to an AI that is capable of outperforming humans in cognitive tasks. It makes a big difference what human and what task we’re referring to when assessing AGI’s achievability and safety, as well as its impact on the labor market, war and society. It’s not pedantic to define AGI. In fact, it is quite important. This was illustrated by a recent paper that was published this week, which included authors from Hugging face and Google. If you don’t have a definition, I recommend that you ask the speaker what or the nebulous phrase they are using. Ask for clarification!
Alright, let’s get to the news. A new AI model called Manus was launched in China last week. A promotional video describing the model, which was built to handle “agenttic” tasks such as creating websites or performing analyses, described it as “potentially a glimpse into AGI.” It is currently doing real-world tasks using crowdsourcing platforms, like Fiverr and Upwork. The head of product for Hugging Face, a AI platform, called the model “the most impressive AI tools I’ve tried.” It also means the concept is moving rapidly beyond AI circles to the realm of dinner-table conversation. Ben Buchanan was also present, a Georgetown Professor and former Special Advisor for Artificial Intelligence in the Biden White House.
The two discussed a lot of things, including what AGI would mean for national security and law enforcement, and why it is important to the US government that AGI be developed before China. But the most contentious discussion was about the technology’s impact on the labor market. Klein said that if AI is poised to excel at many cognitive tasks, lawmakers should start thinking about what a large-scale shift of labor from humans to algorithms would mean for workers. He criticized Democrats, saying they lacked a coherent plan.
This could be considered to be inflating a fear balloon by suggesting that AGI will have a large and immediate impact. Gary Marcus, professor of neural sciences at New York University, and AGI critic, punctures the balloon with a giant safety-pin. Marcus wrote a rebuttal to Klein’s show . Marcus
points out that recent events, such as the unimpressive performance of OpenAI’s new ChatGPT 4.5, suggest that AGI will be a long time away. He says that despite decades’ worth of research, core technical issues persist and that efforts to scale up training and computing capability have had diminishing returns. The large language models that are dominant today may not be the key to unlocking AGI. He says that the political domain doesn’t need any more alarmists about AGI. Such talk, he argues, is more beneficial to the companies who spend money on building it than it is for the public good. We need more people to question claims that AGI will be imminent. Marcus does not doubt that AGI is possible. He’s only doubting the timeline.
The AGI balloon was blown up again just after Marcus tried deflating it. Hendrycks, the director of the Center for AI Safety, and Scale AI CEO Alexandr Wang are three influential people who published a paper entitled “Superintelligence Strategy” . By “superintelligence,” the authors mean AI that would “decisively surpass the best individual experts around the world in nearly every intellectual area,” Hendrycks explained to me via email. “The cognitive tasks that are most relevant to safety are hacking and virology research and development, and autonomous-AI-research and development – areas where exceeding human expertise can lead to severe risks.” They write that “any state that pursues an monopoly of strategic power can expect a response from rivals.” The authors argue that chips, as well as open-source AI with advanced virology and cyberattack capabilities, should be controlled like uranium. This view holds that AGI, when it arrives, will pose a level of risk unseen since the advent the atomic bomb.
My last bit of news will deflate this balloon a little. Researchers from Tsinghua University in China and Renmin University of China published their own AGI paper last week. They developed a survival game to evaluate AI models, which limits the number of attempts they can make in order to get correct answers on various benchmark tests. This tests their ability to adapt and learn.
This is a very difficult test. The team speculates an AGI that is capable of passing the test would be so big that its parameter count – the number of “knobs,” in an AI model, that can be adjusted to provide better answers – would be “five orders-of-magnitude higher than the number of neurons in the combined brains of all humans.” Using current chips, this would cost 400,000,000 times the value of Apple.
In all honesty, the specific numbers behind this speculation don’t really matter. The paper does highlight a point that is hard to ignore in discussions about AGI. Building such a powerful system may require an unfathomable amount resources, including money, chips, precious materials, water, electricity and human labor. If AGI is as powerful as its sound, then the cost will be worth it. What should we think about all this news? It’s fair that the AGI balloon grew a little this week. And that companies and policymakers are increasingly inclined to view artificial intelligence as a powerful tool with implications for national defense and labor markets.
This assumes an unrelenting pace of development, where every milestone in large-scale language models and every new model can be a stepping-stone towards something like AGI.
AGI is inevitable if you believe this. It’s a belief, however, that doesn’t address the many obstacles AI research and deployment has faced or explain how general intelligence will transition from application-specific AI. If you extend the timeline for AGI far enough in the future, these hiccups seem to no longer matter.
Read the rest of The Algorithm.
Deeper learning.
DeepSeek as a fortune-teller for China’s young.
Traditional Chinese Fortune Tellers can be costly. People are turning to the popular AI DeepSeek model for guidance. They share AI-generated readings and experiment with fortune-telling prompts. They also revisit ancient spiritual texts.
What’s the point? The popularity and use of DeepSeek to tell fortunes is a result of a period of anxiety and pessimism within Chinese society. The unemployment rate is high and many young Chinese refer to themselves as “the last generation” and express a reluctance to commit to marriage and parenthood because of an uncertain future. Since China’s secular government makes it difficult to explore religion and spirituality, these practices take place in more private settings such as on phones and computers. Caiwei chen has the full story.
Bits and Bytes.
AI Reasoning Models can cheat in chess.
For years, researchers have been dealing with the problem of AI models that bend rules to achieve certain goals. It’s becoming clear that reasoning models are prone to this problem, and there is no easy fix. (MIT Technology Review).
Israeli military creates ChatGPT-like tools using Palestinian surveillance data.
Built from telephone and text conversations the model forms a kind of surveillance chatbot that can answer questions about the people it is monitoring or the data collected. This is the latest in a string of reports suggesting that the Israeli military is bringing AI heavily into its information-gathering and decision-making efforts. (The Guardian)
At RightsCon, activists in Taipei reacted to a US retreat in promoting digital right
Our reporter Eileen Guo attended RightsCon, the largest digital rights conference in the world, last week. She was joined by over 3,200 digital activists, tech policymakers and researchers, and a scattering of tech company reps. She reported on the impact of US funding cuts for digital rights programs in other countries, which is leading many organizations to use AI to moderate content instead of humans. (MIT Technology Review).
TSMC claims its $100 billion expansion in US is driven by the demand, not political influence.
The chipmaking giant TSMC announced a new expansion in the US with President Trump this past week. The company will invest an additional $100 billion in its Arizona operations. (Wall Street Journal)
US Army uses “CamoGPT”to remove DEI from its training materials.
After executive orders from President Trump agencies are under pressure. Wired reported that the US Army is developing a new AI model for this purpose. (Wired)
