Future of Life Institute

AIs are inching ever-closer to a critical threshold. Beyond this threshold lie great risks—but crossing it is not inevitable.

Intelligence explosion, singularity, fast takeoff… these are a few of the terms given to the surpassing of human intelligence by machine intelligence, likely to be one of the most consequential – and unpredictable – events in our history.

For many decades, scientists have predicted that artificial intelligence will eventually enter a phase of recursive self-improvement, giving rise to systems beyond human comprehension, and a period of extremely rapid technological growth. The product of an intelligence explosion would be not just Artificial General Intelligence (AGI) – a system about as capable as a human across a wide range of domains – but a superintelligence, a system that far surpasses our cognitive abilities.

Speculation is now growing within the tech industry that an intelligence explosion may be just around the corner. Sam Altman, CEO of OpenAI, kicked off the new year with a blog post entitled Reflections, in which he claimed: “We are now confident we know how to build AGI as we have traditionally understood it… We are beginning to turn our aim beyond that, to superintelligence in the true sense of the word”. A researcher at that same company referred to controlling superintelligence as a “short term research agenda”. Another’s antidote to online hype surrounding recent AI breakthroughs was far from an assurance that the singularity is many years or decades away: “We have not yet achieved superintelligence”.