Microsoft CEO Satya Nadella says he’s excited about AGI, at least when it comes to how AI could drive surprising changes to the structure of the economy and how politics could change because of it. “I’m caught by AGI in the sense that, on an economic level, what is the organization of the political economy around AGI?” he asked at a panel discussion after his Microsoft Ignite keynote at the Chase Center in San Francisco.
But Nadella doesn’t think frontier models like Claude are conscious. “I don’t think so,” he said, adding that he follows Microsoft AI CEO Mustafa Suleyman’s view that companies should not grant rights to models and treat them as if they have moral status, which companies like Anthropic do as a matter of principle. Suleyman has told me that he believes the Anthropic team has become “tangled” and that this approach is dangerous.
Nadella was not so firm about it. “It’s a good debate,” he said. “I’m not at all dogmatic about this. What I’m saying is that the most important social technical question we have to face is how do you ensure that an economy can spread this technology widely?”
I asked Nadella what he thought about AI after hearing his response to another journalist’s questions about Anthropic launching more products in his office suite. Nadella spoke at length about the possible collapse of the economy and the end of democracy if an AI model were allowed to achieve monopoly status.
“If the Marxist view is that there is only one company, then it’s over, right? For all of us.”
“Most people think this is a competition between technology companies, and of course it will be,” he said. “But remember, the model has no limits. It will be a healthcare company. It can be a financial services company. It can be anything if it wants to be. It can take over the government. Thinking of this as a close competition between technology companies, over who will get the next trillion dollars of market capitalization, is one of the strangest and narrowest ways to think about it.”
Nadella went on to say that the very nature of businesses could be altered in the future. “If intelligence has network effects, what is the future of a company?” asked. “If the model sees something and learns it, and essentially all human capital collapses into symbolic capital, what is the role of a company and of human capital in a company? That’s the debate.”
“I think political economy is going to become more and more interesting in that context. Nobody – at least in a democracy – can even win an election if everything goes like this, if there is only capital and no labor. I mean, I don’t know, that’s not an economy.”
“If the Marxist view is that there is only one company, then it’s over, right? For all of us. Not just me. In some way, you’ll be some affiliate of Anthropic, too.”
Needless to say, this is not how tech CEOs typically talk about launching competing spreadsheet software.
Of course, Nadella was also promoting Microsoft’s various AI tools at Ignite, which promise to allow enterprise customers to choose the most cost-effective models for various tasks, run their own models, and control their own data in different ways. The idea here is that the market will demand many different types of sovereign models and data to prevent control from being centralized, a theme Nadella kept coming back to as we spoke.
“I think if you believe that the world, the economy, needs more than one or two companies, then by definition there have to be as many models as there are companies of some kind,” he said.