The illusion of AI understanding and creativity

Today’s generative Artificial Intelligence (AI) applications are impressive. Large Language Models (LLMs), such as ChatGPT, easily pass the Turing Test and are thus indistinguishable from humans in an online text conversation. They are used in professional settings to handle customer inquiries, draft legal texts, and a variety of tasks that, until recently, only humans could manage. Other generative AIs produce high-quality images, music, and video, often with high artistic value, based on simple descriptions or ‘queries.’ It has thus become difficult for the average educated person to avoid the conclusion that today’s AIs actually understand the questions or tasks posed to them, and even have artistic sensitivity.

Yet, nothing could be farther from the truth. For AIs—yes, even today’s AIs—do not understand anything; nothing at all. And they have no creativity in any sense of the word that could be even remotely related to human creativity. Allow me to elaborate.

Let’s take LLMs as our example, for the feats of ChatGPT tend to be regarded as the most compelling when it comes to attributing understanding and creativity to generative AIs. LLMs are transformers (a technical term), meaning that they take input text, apply a series of geometric transformations to it, and spit out the resulting text one word at a time. The answer ChatGPT gives you is a ‘transformation’ of your question.

The particular parameters of the geometric transformations applied are defined during a so-called ‘training’ phase, when the LLM is exposed to an enormous database of human-written text. Its parameters are then iteratively adjusted—calibrated, fine-tuned—so to represent how words tend to appear together in human-produced text. Once the training is complete and the parameters set, the LLM can then infer, given an input sentence (i.e. the question or query), what the most likely word is to appear next in the output sentence (i.e. the answer). Once that is done, the new output sentence—with one more word appended to its end—is fed back to the LLM, which then infers the next word, and so on, until the answer is complete. This so-called ‘inference’ phase is what we users get exposed to when we interact with ChatGPT online.

From the point of view of the LLM, the text database used during training contains merely a gigantic collection of signs. These signs happen to be English words—or some parts of words—but they might as well be squiggles; it doesn’t matter, for the LLM does not know the meaning of the words; it doesn’t know that e.g. the word ‘tree’ corresponds to something that looks like a tree out there in the world. Indeed, it has no access to the world. All it is trained to do is to capture and represent the statistical regularities with which words occur together, or follow one another, in the human-written text of the training database. If squiggles were used during training—instead of words—the LLM would still capture the regularities with which the squiggles tend to appear; from its point of view, it’s all the same thing. The LLM has no understanding of the meaning of the text it is trained on. It merely deals with how signs—squiggles, words—relate to one another in the training database, not what the signs denote or connote in the real world.

Once the statistical regularities with which words tend to occur are captured in the LLM’s parameters, the LLM can start inferring which words to use in response to a query. From its own point of view, its answer is thus just a series of squiggles whose meaning it does not know; it only knows that these are the squiggles that are most likely to appear following your query, given the way squiggles appeared in the training database. That’s all there is to it. At no point does understanding or creativity come into the picture.

So why does it seem to us as though the LLM really did understand our query, and produced a fully understandable answer? How does the LLM produce such coherent outputs if it has no understanding of language, in the sense of knowing what words denote and connote in the world? The answer is quite simple: it's because the LLM was trained on human-written text, and it is always a human who interprets its outputs. Now, humans do know what the words mean! The intelligence involved here is thus always human intelligence, as embedded in both the training database and the interpretation of inferred answers. The meaning we discern in an answer produced by ChatGPT is (a) the meaning imparted on the training database by the humans who wrote the corresponding texts, and (b) the meaning we impart on the answer when reading and interpreting it. ChatGPT itself only ever sees squiggles and the statistical regularities with which they tend to occur; it understands nothing; it creates nothing; it only rearranges—‘transforms’—meaningless squiggles. All meaning is imparted and projected on the squiggles by us humans.

The same goes for generative AI art: all artistic creativity involved is that of the human artists who composed the images used in the training database. All the AI ever does is rearrange—‘transform,’ combine—elements of those images based on a query. Generative AIs thus merely recycle the products of human intelligence and creativity, nothing else. The only reason why ChatGPT can pass a bar examination is that it was trained on text written by capable human lawyers. If there weren’t human lawyers, ChatGPT would produce gibberish in a bar examination. The only reason it can tell you what Analytic Idealism is, is that it was trained on text written by me; it has no understanding of Analytic Idealism. The only reason other generative AIs can produce beautiful art is that they were trained on beautiful art produced by creative, sensitive people. If you take human input out of the equation, generative AIs can do nothing; they have no understanding or creativity of their own; they just transform—recycle—human understanding and creativity.

That’s why there is a strong sense in which the output of generative AIs is always a—sophisticated, complex—form of plagiarism. AIs can never produce something whose building blocks weren’t first produced by human beings. At best, AIs can find associations—connections—across different products of human creativity and insight that would, otherwise, be difficult for humans to find on their own, since AIs operate on much larger training databases than humans can accommodate in their minds. But the building blocks are always human-produced; no exceptions. The meaning is always human-imparted; no exceptions. There is no such thing as AI creativity or understanding.

The problem, however, is that the plagiarism is so sophisticated and nuanced that a PhD in computer science and engineering is necessary for one to understand what is truly going on. And things will only get worse as larger and larger AIs—with more and more parameters—are trained on larger and larger databases. The illusion of artificial understanding and creativity, already so compelling, will become irresistible for the vast majority of people. This is a great danger, for we risk losing sight of our own value and dignity by projecting all of it onto electronic mechanisms. This is a form of ‘kenosis,’ an emptying out of ourselves, wholly unjustified by the facts.

Businesses see so much value in generative AI because of its effectiveness in recycling, adapting, and re-using human output. If a few lawyers somewhere managed to write very clever legal texts, an AI trained on those texts can produce clever legal texts for your business on demand, without your having to pay new lawyers to do the same kind of creative, intellectual work again; someone else, somewhere else, already paid for the originals. If clever artists have produced a large database of effective illustrations somewhere, you don’t need to pay new artists to do it for you again; an AI can cleverly re-use and adapt that previous human output to your particular needs. Economically, this is incredibly efficient. But it requires no understanding or creativity beyond those already embedded in the training database and the minds of the people who contemplate the outputs of the AI. The latter simply rearranges things, without having a clue about what they mean.

It is critically important for us to understand that AI does not replace human creativity and understanding; on the contrary, it entirely relies on them. Its value resides solely in stretching, leveraging the re-use potential of human production, not replacing it. AI amplifies the reach of human productivity; it doesn’t render it redundant. All meaning and all creativity discernible in the outputs of AIs are human meaning and human creativity. Without human input in the form of training databases, AIs are wholly useless. Without the understanding projected by humans onto their outputs, AIs are only capable of spitting out meaningless squiggles. Artificial Intelligence ultimately is human intelligence.