Close Encounters of the LLM Kind
On the alien-ness of large language models
You are today the total sum of all the experiences you have had in your life. They have shaped your identity, your belief system, and your cognitive outlook on the world. What I particularly want to emphasize here is that you have had a singular trajectory through life: one path through time and space that has shaped who you are today. As a consequence, anyone else interacting with you can assume that you have a singular opinion, even if nuanced, about anything of importance: there are foods you like and dislike, perhaps you’re an early riser, or a glass half-full person. You have a unique perspective on the world, and a largely consistent set of beliefs and values. With a few judicious questions, someone can probably get a sense of where you land on the political spectrum, what your religious beliefs are, where your interests lie, and expect them to remain both singular and consistent over time.
LLMs do not have that singular path through life that shapes a unique belief system. In some sense, they have had all the paths through life. They’re a plumber from Alabama, an 18th century poet, a 15 year old Reddit troll, and everyone else that has ever contributed data to the model. They are all these at once, not some average over them. They don’t have one identity, they have all the identities. Not one religion, not one political view, but all the beliefs and values ever expressed on the web. And you can trigger each of these personas by clever prompting: ask a question that would typically raise the interest of one particular group, and the LLM will respond in the voice of that particular internet constituency. Ask a home improvement question, and it’s the web’s DIY hackers responding. Ask about a disease, and you’re likely to get the medical experts (and maybe some hypochondriacs as well.) There isn’t a guarantee that you will consistently elicit a unique voice from the model. It can meander and diverge, which is why prompt injection attacks are often so successful. But to a large extent, this ‘multi-polarity’ is a desirable feature: you want the model to speak in the voice of the expert and give well-informed advice on any topic.
This is not an original idea. Engineers working on this technology have all internalized this concept to some degree. However, many fail to realize that, by and large, the rest of the world has not. A huge problem we face in the public perception of LLMs today lies in the fact that it is extremely challenging for us humans to comprehend that we’re talking to an entity that is not a singular entity, with a unique life experience and set of views.
We talk to a chatbot and expect that the intelligence behind it to have a personality, an association encouraged by the chatbot format. That expectation is so ingrained in the public’s imagination that much of the media chatter around LLMs centers on: this one AI is acting Nazi, or that other AI is “woke.” But the truth is they’re some shade of both, and everything in between. They are not shaped by a unique past.
There is nothing fundamental about intelligence requiring having a unique worldview. Even our attempts at portraying similar swarm intelligence in science fiction fall short: the Borg of Star Trek have their queen, clearly the embodiment of some sort of emergent consistent persona, even when they are meant to be a “collective.” Brandon Sanderson’s “Legion” has its Stephen Leeds, whose personal narrative is the single thread around which all the lives of the several personas that inhabit his brain are woven. We can’t seem to imagine or relate to a distributed intelligence without giving it an identity, even though our experience of LLMs suggest that this may not be a requirement.
What LLM providers attempt to do is make their systems give the feeling of having a consistent identity with pro-social and generally middle-of-the-road values to their users. But much of what they’re actually doing boils down to upweighting the latent personas in the system that correlate with these attributes, thereby fueling the illusion of AIs having an opinion or worldview, without fundamentally changing the fact that this is not possible for this class of AIs.
I find it fascinating how much we humans care and rely on being able to attach such labels to the entities we interact with. LLMs are the first alien intelligence we encounter that is “swarm-like,” made up of a multiplicity of perspectives fused in a way that does not seek consensus, but reflects the diversity of its constituents. It makes me question whether sentience does have such a requirement, as we have never been confronted with a sentient entity that hasn’t had a unique path through life.
More pragmatically, while I understand that much of the work aiming at curbing the personality traits that these models display has immediate functional value in making them safer to use, I increasingly see the problem as having more to do with compensating for our own cognitive flaws. I do wonder how we may be able to achieve similar goals without working against what makes LLMs so rich in their diversity of experiences.
