Are Large Language Models knowledgeable?

There is a lot of chatter about LLMs such as Gemini and ChatGPT. Some people are excited by the potential of these tools to take the drudgery out of some human jobs. Some think that these tools can actually perform some tasks better than they can. Others worry that LLMs will end up possessing more knowledge than human beings and therefore will take over the world. These worries bring into sharp relief the central question of ToK: what is knowledge and are LLMs knowledgeable?

AI has been a hot topic since the release of the first Large Language Models at the end of 2022. Of course AI has been around since the 1950's and, as an area of study, is a major contributor to the field of Cognitive Science, which also includes disciplines such as philosophy, psychology, neuroscience, anthropology, and linguistics. But what is getting people in a funk is a special kind of AI that generates streams of text which appears to be similar to text produced by human beings. Indeed, in many cases it is difficult to distinguish between the two - most LLMs pass the so-called Turing test - the test devised in 1950 by the British mathematician and philosopher to determine whether a machine is truly intelligent. Turing argued that if we cannot distinguish between the output of a computer system and the text production of a human being then the system is intelligent. It is this aspect of LLMs that is exciting people and also worrying them. If they are indistinguishable then how can we tell if the work of a student is their own rather than that of ChatGPT? More worrying to some is the thought that if these systems are able to generate human-like text, does that mean that they have human-like knowledge. What happens, then, when these systems have more knowledge than human beings? Does that mean that they are more powerful than humans and will bend them to their will. The cognitive scientist Marvin Minsky wrote in 1970 that “Once the computers got control, we might never get it back. We would survive at their sufferance. If we’re lucky, they might decide to keep us as pets.”

But do the digital computers that run LLMs really have knowledge? This question hangs deeply upon what we count as knowledge and is therefore of vital interest to us in ToK. One of the main characteristics of knowledge that most experts think is important is the notion of truth. And human beings on the whole take truth-telling to be a social norm. We do not output random stuff independent of its truth value. We try to tell the truth and hold others to the same standard. A lot hangs on this not least our acceptance and inclusion in social cooperation that is essential to our survival. But LLMs do not care one jot about truth. They can't because they do not have cares.

Sure, these systems seem to mimic having knowledge quite well but at the same time these systems are vastly different to any organism that we traditionally think of as knowing beings - including humans. They do not have bodies, they do not have emotions, they do not have wishes or desire, they are not moulded by evolution to want to survive or reproduce, as indicated above, they do not care about whether their output is true or not. So the question is whether a system can still have knowledge while being so different. Is knowledge something so abstract that a digital computer system can have it?

Of course, this is a hard question and I do not pretend to know the answer. All we can do is to review the continuing debate. The group who are most sympathetic to the idea that LLMs have knowledge are likely to be those who think that knowledge is essentially a disembodied linguistic activity. This group includes researchers working on designing artificial intelligence systems on digital computers in the 1970.s and 1980's - what the philosopher John Haugeland ironically calls GOFAI - Good Old Fashioned Artificial Intelligence. However, this approach hit a wall in the late 1980's and since then there has been something of a revolution. New approaches have emphasised the essential embodied nature of human and animal intelligence and reject the idea that intelligence can be implemented on a disembodied system. Unlike LLMs, human beings generally care about truth because on the whole it gets us out of trouble in the real world and allows us to survive and have children. We use knowledge to do things in the world and it really helps us get it right if this really is knowledge - that it is true - rather than being just a set of fanciful beliefs. This latter group, embodied and enactive theorists, have no worries about ChatGPT taking over the world, but worry more about human beings putting too much trust in these systems.

Of course not all knowledge is propositional, that is, collections of true statements. Some knowledge might not be about language at all and we investigate these different kinds of knowledge elsewhere on the website Kinds of knowledge . However, the question of whether LLMs like chatGPT actually possess knowledge is an interesting one and we explore it in a new Student Handbook page (Generative AI and knowledge)  with a new set of activities to go with it (Does ChatGPT actually know anything?) . Enjoy!