Seeing Silicon: The prophesies of AGI

A week ago, James Cameron, the director of science fiction movie Avatar announced that the reality of AGI coming into being was scarier to him than the fiction around it. The same week researcher Miles Brundage who worked as a Senior Advisor for AGI Readiness at OpenAI resigned from his position, announcing on his Substack that no one was ready for AGI.
Not a day goes by before someone – in the government, civil society or corporate industry mentions artificial general intelligence or AGI – and perhaps with good reason. In the last decade, AI as a technology has dramatically changed much of how every discipline works and it shows no sign of slowing. This year, two Nobel Prizes went to AI scientists. At a pub in Mountain View where I live, someone joked that the Nobel committee might give the Nobel Prize in Peace to someone working in AI policy (It didn’t).
As the field of AI matures and we get adjusted to seeing this technology roll out in our phones, browsers and life, conversation is shifting to a somewhat nebulous idea of achieving AGI – a vague singularity vision of the technology where AI will exceed human intelligence. AGI is also known as “strong AI”, “full AI” or the ability of a machine to perform “general intelligent action.”
Imagine a single human intelligence, say AGI advocates, now put it together with millions of intelligences working in a hivemind to solve humanity’s biggest problems. That’s the moment when proponents of AGI – academicians, venture capitalists, founders, billionaires, and researchers amongst them – start sounding like prophets or science fiction authors. These tech-first capitalistic imaginations of the world have always been popular with technologists – including Effective Altruists, Effective Accelerationism – perhaps harking back to billionaire Andreessen Horowitz’s philosophy around a technology-led society, titled The Techno-optimist Manifesto.
In the last few weeks, technocratic visions of AGI utopias have flooded the internet written by CEOs, investors and billionaires – this philosophy’s prophets. A few weeks ago, Sam Altman reiterated his oft quoted vision of AGI through ‘The Intelligence Age’ where he says “the future is going to be so bright that no can do it justice by trying to write about it now” – a call out to a new “Intelligence Age”. He adds that near-limitless intelligence and abundance energy will fix the climate, establish a space colony and discover all physics.
Dario Amodei, CEO and co-founder of Anthrophic published an idea titled Machines of Loving Grace in where he describes AGI as creating a “country of geniuses in a data center” that will “solve very difficult problems, very fast”.
In a similar vein venture capitalist Vinod Khosla shared his philosophy titled AI: Dystopia or Utopia? which again imagined what a world will look like post achieving superintelligence. “We’re on the cusp of a near-infinite expansion of brain power that can serve humanity,” he says, adding that AGI will bring in “unparalleled abundance” and a “post-scarcity economy” where every human has robots, e-tutors, self-flying aircrafts, personal assistants, synthetic physicians in a society of “boundless abundance”.
These ideas are different from narrow definitions of AGI that businesses came up with in the last 12 months. Last year, Google DeepMind listed six levels of AGI in a scientific paper, which introduced a measurement framework for levels of AGI performance, generality and autonomy. In July, OpenAI followed with its five-level system to achieve AGI that included starting at conversational AI and progressing through reasoners, agents and innovators into AI systems capable of functioning as entire organisations.
Unlike the step-by-step narrowly defined evolution of AI as a technology, the prophesies by billionnaires are more focused on what the world will look like once this technology is up and functioning. Some of the ideas seem to have popped out of the iconic science fiction novels, the Culture series by author Iain M Banks, where AI and humanoid species live in a ‘superabundant civilization’ of space socialism. But the prophesies themselves are not socialist, but inherently capitalist in nature.
Most of us will lose jobs and money will be made by “a small concentration of world changing companies”, writes Khosla, who will then share this capital to benefit society through a “universal basic income”. All these visions want humanity to ride through the short-term disruptions to their lives and want governments (particularly the US government as most of these manifestos are coming from US billionaires) provide boundless energy, compute resources and massive data to allow the representatives of these technologies to build them to their ultimate vision.
On the other hand, a flood of these manifestos also seems to suggest that something might be amiss in the AI world. It’s been two years that ChatGPT was launched, bringing an era of AI-optimism and an infusion of billions of dollars into the technology. Now the market’s wondering where the adoption of the technology is. Though AI has no doubt helped scientific advancement and the field is here to stay, the startups won’t keep getting investments on the basis on just this adoption. The public doesn’t seem to have warmed to the technology like they did to social media or the smartphones. Perhaps these manifestos are a way for these owners of AI capital to build a grand world-changing narrative around their products to obfuscate the near-future rising cost of compute, energy and the massive need for resources and data. After all, if we don’t believe that AGI or superintelligence is at the end of this technology, why would we invest in it?
But the author in me keeps wondering that if we build this superintelligence and ask it to find a solution to all our problems, will the answer, as author Douglas Adams imagined it, still be the number 42?
———————
Shweta Taneja is an author and journalist based in the Bay Area. Her fortnightly column will reflect on how emerging tech and science is reshaping society in the Silicon Valley and beyond. Find her online with @shwetawrites. The views expressed are personal.