Wake up and smell the coffee
Deep Mind co-founder and Microsoft AI CEO, Mustafa Suleyman, raises big questions on UK national radio regarding AI consciousness, the future of work and more generally the two sided coin of AI’s rapid progress.
“If you’re not a little bit afraid at this moment, then you’re not paying attention” is Mustafa Suleyman’s opening stance when recently interviewed on Radio 4. The 15 minute interview starting around 2hrs 12mins into the Dec 29th Today programme is well worth a listen. If anyone has trouble accessing that time-bombed recording, let me know.
Suleyman, CEO of Microsoft AI, is a self-professed techno-optimist. Nonetheless, this interview makes a very compelling case for the two sided nature of the AI coin. Although not a hardcore AI researcher like his Deep Mind co-founder and Nobel Laureate Sir Demis Hassabis, Suleyman is nonetheless absolutely in the small cadre of commercial and technical leaders well placed to judge the current state and trajectory of AI towards Artificial General Intelligence. This deeply knowledgeable cadre mostly see the long term outcome of the race towards AGI as high stakes. If it goes well, AGI will transform human civilisation with enormous benefits. If it goes really, really badly it will end humanity. But more realistically on the downside, even if it only goes a bit badly, the ride will be very, very bumpy. Amongst his peers, Suleyman provides perhaps the most balanced perspective on the pros and cons of the whole endeavour.
As guest editor of the 3 hour Today programme linked above, Suleyman unsurprisingly gave AI substantial airtime, including conversations with academics on the possibilities of AI ever becoming conscious. I find the definition of consciousness extremely slippery, and I am supremely unqualified to contribute to such a philosophical debate, so I mostly focus on more tangible questions.
However, AI and its potential consciousness turns an arcane 5,000 year philosophical debate about what it is to be human into a matter of real political urgency.
The deeper essence of consciousness remains hugely debated in philosophical circles, although Suleyman usefully simplifies it as the capacity to suffer. Despite not agreeing what consciousness really is, at least most non-biologic philosophies agree that indirect observation is our only tool for assessing whether another human, an animal or a rock is actually conscious. We cannot know what is truly going on inside another body, but we can observe how they react to the world around them. If they yelp, limp into a corner or run for the hills when electrocuted we surmise they feel something approximating to pain from an electric shock. We know that we feel pain from electric shocks and know by definition that we are conscious. If we observe enough other reactions to the external world which are similar enough to our own in a very fuzzy sort of way, then we reach consensus that this other category of body is also conscious. Simplistically, if it looks enough like me, smells enough like me and behaves enough like me then, like me, it must be conscious. And if it is conscious then it naturally has moral rights.
The scope of “Enough similarity” changes over time. Historically, slavery, maltreatment of other primates, arbitrary hunting of corvids and boiling of live lobsters have all been justified by an overly small sphere of similarity around the observer. So maybe we should some day extend the sphere to include AGI.
But whether AGI could be conscious is not just a philosophical debate. It could have indirect existential consequences for humanity. As Suleyman points out, “if we allow anyone to declare these things conscious they will quite quickly make the claim that they deserve rights. We cannot spawn a new species of conscious beings that have a right to not suffer or to not be turned off. We do not want these things to be conscious let's just start there, whether they are or they aren't. There should be widespread collective agreement that this would be completely undesirable. If we could spawn billions of digital minds at zero marginal costs that could self-replicate and learn from infinite amounts of data relative to what a human could learn, how on earth could we possibly control and contain something like that. […] If we carry on behaving the way that we have with respect to governance and regulation over the last 70 years then we will end up shooting ourselves in the foot in a major way. We have to do something very different.”
Exactly how we cast AI’s lack of consciousness into concrete in perpetuity through international treaty or another fiat mechanism is a thorny issue, not just politically but practically. In the coming weeks I’ll revisit some of the practical challenges in legislatively constraining the consciousness of such an emergent and fast developing technology – not least given that we don’t even know what it (consciousness) is we are trying to constrain.
Suleyman also confronts inevitable AI driven changes to the labour market starting with white collar jobs. This is already happening in call centres and he expects very capable AI powered replacements for many project managers will emerge in the next couple of years. Paraphrasing from an earlier 2025 interview, “adaptability will be the most valuable skill for anyone entering the workforce today, and Autodidacts will be most resilient [to AI powered career erasure]”. So if you don’t know if you’re an autodidact, best start by teaching yourself what it means. Linguistic jokes aside, developing new skills by generalising from past experiences is often cited as a key barrier on the journey to Artificial General Intelligence. So this does feel like some sort of race between ultra adaptable autodidactic humans and ever more adaptable and autodidactic AGI. We will be constantly re-inventing ourselves to either swerve into the jobs AI cannot quite yet do, or to grab one of those allegedly soon to emerge, albeit currently unknown, uniquely human, erasure resistant, jobs of the future.