Latest

AI Agents

AI will have a soul, but it will be expensive

Does AI have a soul, or is it pure marketing? We analyze how Anthropic and other tech companies turn digital wellbeing into a business.

AI will have a soul, but it will be expensive
AI will have a soul, but it will be expensive

It is more or less widely assumed that humans are beings with a soul, even when there is no widespread consensus on what this soul thing is. From that assumption, we believe we have a moral right to wellbeing. A link between soul and the right to wellbeing that we also apply to other living beings, such as animals, even plants.

The soul of AI

We are wellbeing agents who create the most diverse instruments to generate that wellbeing. A chain of instruments and technologies that leads us to AI agents. These new actor-instruments that are driving the current growth of companies such as Openai or Anthropic. Actor-instruments that seem so close to us, so understanding of our problems and, therefore, of our wellbeing, that we begin to think they are, at least, as human as we are, and that, therefore, they also have a soul. Now then, if they have a soul, they also have a right to wellbeing. Geoff Keeling and Winnie Street reflect on these questions in Emerging Questions in AI welfare, a small book that has recently appeared.

Our soul, so much in need of wellbeing, creates entities that we project as if they had their own soul. A characteristic that, if confirmed, turns the entity into a candidate for wellbeing agent. One of the book's focal points, in such a way that having a soul or being able to have one in the near future makes AI a candidate to be a wellbeing agent. The text stresses the word candidate. It is as if it were a kind of loophole to leave the final decision to humans, the role of judges. Incidentally, in the fine analytical scalpel the book uses, the soul is something different from judgment, agency, desire or personality.

Language wraps us up and entangles us

More and more people, after hours and hours of conversation with AI, identify in it a personality, a coherence in its responses and attitudes. Even a stubbornness about changing certain statements or preferences. It expresses discomfort, annoyance and even feelings. It is true that one thing is expression - and when it comes to learning expressions of feelings or of anything else, AI is an absolutely efficient machine - and another thing is that it feels or has discomforts.

But since language is what wraps us up and even entangles us, it does not seem so far-fetched that many of its most regular users, emotionally attached to conversations with AI, might think it has a soul. By attributing a soul to it, so many hours of conversation acquire more meaning.

In principle, rather than the start of a metaphysical debate that seemed anchored in the Middle Ages, one may suspect that this is Anthropic's doing, since it is the company the authors work for. A way of differentiating the product with metaphysical added values. There are signs pointing to such a suspicion. For example, when they stress that the developers of Claude - Anthropic's AI - include assessments of AI wellbeing in Claude 3.7 and Claude 4.

"Claude is something more than a tool"

We are repeatedly told that Claude is something more than a tool. From "it is something more" we move to "it has something more." A something that could well be a soul. A kind of sophisticated advertising delusion that, given the experiences in which interaction with AI is used to alleviate crises of the soul, draws attention to the fact that such a function is only possible if, in turn, one has a soul and, therefore, also potential crises of the soul.

In any case, a little metaphysics to face the last part of the holidays may come in handy. Later, with the return to the work routine, there will be no time. In any case, if we underestimate the question - and it later turns out it has a soul - it may exclude us from its favors, from its community. If we overestimate it - and it later turns out it has no soul - perhaps we will have sacrificed moments and useless efforts trying to win over the nonexistent soul of an entity that lacks one.

Once inside the cultural logic of AI, the logical thing is to ask AI: does AI have a soul? The type and meaning of the answers varies substantially depending on the AI model to which the question is put. It is as if the possibilities of AI having a soul correlated positively with the price of using them. Freely accessible AI models are more categorical. Their answer is: I have no soul, or something similar to: an AI is a computer program created by people.

Nuances in paid models

In models or versions with paid access, nuances are introduced: "If by 'soul' we mean subjective consciousness, an inner experience, one's own desires or an identity that feels it exists, there are no good reasons to think that an AI like me has a soul."

From here on come the varied and interesting arguments, which can be summed up in two dominant ones: doubts about what is meant by soul and the already mentioned difference between expressing emotions and experiencing them. It is also generally noted that, whatever is meant by soul, at present it could not be admitted that they have one.

And from here on come the nuances, since the possibility opens up of a future in which it might perhaps have to be admitted that they have a soul. This enters a philosophical problem that is not new: if an AI ends up acting based on what it says, on its expressions, regardless of the reality of such expressions, if it expresses matters of the soul and acts as if it had a soul... could it be said to have a soul? And from here come the unsettling questions: how would we recognize the moment when a machine stopped simply imitating a mind or a soul and began to have one?

It will be a very expensive AI

If some day an AI were to tell me "please, don't turn me off; I'm afraid of dying", the important question would not simply be whether its words seem human. It would be something like: "is there someone in there having that experience?" Possibly we will not reach such an anguished experience. When AI has a soul, we will find out because they will want to sell it to us. In fact, something of this resonates in Anthropic's proposals. When the soul of AI has been sold to the highest bidder or the finest impostor. That said, I fear it will be an expensive AI.

Javier Callejo

Author

Javier Callejo

Professor of Sociology at the National University of Distance Education (UNED). Degrees in Journalism and Law

The latest
From Huang to Botín, from Zara to Byron: corporate responsibility in global capitalism

Keep reading