In a policy document published on Wednesday, former Google CEO Eric Schmidt and Scale AI CEO Alexandre Wang, as well as Center for AI Safety director Dan Hendrycks, said that the U.S. shouldn’t pursue a Manhattan Project style push to develop AI systems of “superhuman” intellect, also known by AGI.
In the paper titled ” Superintelligence Strategy” asserts that a U.S. aggressive bid to control superintelligent artificial intelligence systems could prompt fierce retaliation by China, possibly in the form a cyberattack which could destabilize the international relations.
[A] Manhattan Project ( [for AGI] ) assumes that rivals are more likely to accept an imbalance or omnicide than to take action to prevent it,” write the co-authors. “What starts as a push for superweapons and global control could lead to hostile countermeasures and escalated tensions, undermining the stability that the strategy purports secure.” U.S. Secretary Chris Wright recently stated that the U.S. was at ” The start of a Manhattan Project” on AI, while standing in front a supercomputer site with OpenAI cofounder Greg Brockman.
In the Superintelligence Strategy paper, the authors challenge the idea that government-backed programs pursuing AGI are the best way to compete against China.
According to Schmidt, Wang, & Hendrycks the U.S. and China are in a standoff over AGI that is similar to Mutually assured destructionSchmidt and his coauthors argue that, just as global powers don’t seek monopolies on nuclear weapons – which could trigger preemptive strikes from an opponent – the U.S. shouldn’t rush to dominate extremely powerful AI systems.
While comparing AI systems to nuclear weapons sounds extreme, world leaders consider AI as a top advantage in military affairs. Pentagon officials have already stated that AI helps speed up the military kill chain.
Schmidt et al. They introduce a concept called Mutual Assured AI Malfunction, in which governments can disable threatening AI project proactively rather than waiting for opponents to weaponize AGI. Schmidt, Wang, Hendrycks, and others propose that the U.S. focus its efforts on developing methods to prevent other countries creating superintelligent AI, rather than “winning the race towards superintelligence”. The co-authors say the government should “expand its [its] arsenal to disable threatening AI project” controlled by other countries as well as limit their access to advanced AI models and open source chips.
In their article, the co-authors describe a dichotomy in AI policy. The “doomers” believe that AI development will lead to catastrophic outcomes and advocate countries slowing AI advancement. On the other hand, there are “ostriches” who believe that nations should accelerate AI and essentially hope it will all work out.
This paper proposes a ‘third way’: a measured, defensive-oriented approach to AGI development.
This strategy is especially notable, coming from Schmidt who has previously spoken out about the need for America to compete aggressively against China in developing advanced artificial intelligence systems. Schmidt wrote an op/ed a few months back stating that DeepSeek was a turning-point in the AI race between America and China.
It seems that the Trump administration is determined to push ahead with America’s AI research. The co-authors point out that America’s AGI decisions are not made in a vacuum. Schmidt and his coauthors suggest that it might be better to take a defensive stance as the world watches America push AI to its limits. Maxwell Zeff, a senior reporter for TechCrunch who specializes in AI and emerging technology, was born in 19659014. Zeff covered the rise and fall of AI, as well as the Silicon Valley Bank Crisis, for Gizmodo and MSNBC. He is based out of San Francisco. When he is not reporting, you can find him hiking, biking and exploring the Bay Area food scene.
View Bio
