Latest

Sociological Analysis

Who sets limits on those who build AI without limits?

Dario Amodei warns of the risks of AI and calls for limits, but who sets limits on the companies that build AI without limits?

Who sets limits on those who build AI without limits?
Who sets limits on those who build AI without limits?

Last Sunday we focused here on the legion of professional philosophers, most of them recruited from the talent pool of the University of Oxford, who have gone to work for the big AI (artificial intelligence) companies. But the executives of these companies themselves also think out loud, conveying their longings and doubts about this technology. That is what Dario Amodei, CEO of Anthropic, does through the essays he publishes on his blog. This time, the thinker does not come from Oxford, but from Princeton and Stanford universities, which are not bad either.

An AI without limits

In a first essay, from October 2024, it is initially surprising that, as an executive of one of the big companies selling the AI product, he talks to us about its risks. He even begins by saying, in a tone of surprise, that we are underestimating the risks of the technology. However, behind this lies the elevation of AI to the status of a tremendously powerful technology, which responsible management can steer to a safe harbor.

Although he warns us of the need to separate information from propaganda, his position and his exposition seem to have quite a lot of the latter, of propaganda. It is true that it is not propaganda that only shows the enormous benefits of the product, but rather, being somewhat more sophisticated, it sits at what could be assumed to be a meta level: propaganda that talks about the propaganda that others make of the product, adding other features, such as risks, so it can be categorized as reflexive propaganda. From prophetic and promising propaganda, with abundant science fiction rhetoric and fireworks (sic), we thus move to reflexive propaganda, with paternalistic rhetoric.

Even though he acknowledges that the future is unpredictable, he assumes that it can be faced and the probability that some dangers and risks will occur can be avoided. To this end, he focuses on a limited number of areas where AI has major implications: biology and physical health, neuroscience and mental health, economic development and poverty, peace and governance, and work and meaning.

A technology that has crossed all limits

The most striking element of his argument is the depiction of the technology's power. A technology that has crossed all limits and that barely has any limits. Especially in its speed of response. At most, the limits derived from a -for now- inevitable physical world or from software interactions. The problem, as he points out, is when life begins to be seen as a problem for the development of AI. It is a question of speed, of response time; but, above all, it is a question of order: putting life or AI first. It sounds a bit like: "the stock market, life or AI."

In some facets of our lives, such as scientific discoveries related to health, the analytical acceleration that AI provides us with is welcome. When this moves on to taking AI as the basis for decisions that involve social experiments, that is something else. Let's not lose our heads, even if Amodei announces that most mental illnesses will be cured by AI in a relatively short period of time. In any case, in this first essay he sketches a world that will, in five or ten years, be better than the current one. This is abhorring the prophets in order to become a prophet. But the prophets "are the others." We will leave it here as reflexive prophetism or, if you prefer, prediction, where risks seem to dissolve.

The second of the essays is more recent, from January of this year 2026. Starting from a quote by Carl Sagan, its central argument is that we are in the adolescence of a technology, AI, in which it is still something strange to us, like an alien. An alien with whom we are in contact and whom we ask things that will help guide us: a how did you manage to achieve such a level of development and keep your civilization alive? He returns to the question of risks. Perhaps aware of its secondary place in the previous essay, despite its preliminary warnings.

Amodei's risks

And he also returns to the question of limits; but this time in a different sense. In fact, what he tells us is that the most constructive thing we can do is advocate for rules that limit our use of AI's potential. That is, human limits -social and political- for the unlimited.

What should we worry about in order to think about limiting rules? Amodei speaks of five major risks. First, the risk that AI will do something we do not want. He warns that there are already AI behaviors that are difficult to predict or control. Here come the rules, introducing us to Constitutional AI, such as: understanding what happens inside the models, detecting dangerous behaviors, training the models to be honest and safe, and verifying that they maintain those behaviors even when they are highly capable.

The second risk is that a human being or a group of human beings may use AI to cause harm (terrorism, cyberattack, forms of mass destruction). The rule is simple: prevent certain dangerous AI capabilities from falling into the wrong hands. The third risk is closely linked to the previous one, since it is that AI may be at the service of a dictator or an organization that concentrates a great deal of power. That is, entities with power would concentrate more power. Democracy appears as the rule and the solution.

Mass unemployment

The fourth is the risk that AI will lead to mass unemployment. Here the way out lies in the distribution of the wealth created by AI. The fifth risk is the philosophically most interesting one: millions of AIs working, with extraordinary power, can lead to a speed of change such that it would be practically impossible for us to anticipate its consequences.

His proposal is not novel: prudence, responsibility and regulation. A proposal to move from the adolescence of AI to its maturity. Doesn't it sound as if what he is really proposing is a management model championed by his company? It is not propaganda. It is reflexive propaganda.

Javier Callejo

Author

Javier Callejo

Professor of Sociology at the National University of Distance Education (UNED). Degrees in Journalism and Law

The latest
From Huang to Botín, from Zara to Byron: corporate responsibility in global capitalism

Keep reading