[GPT-4] is fed, like, a line of text from some source, but with the last word missing. It guesses what the last word might be, and then it gets told whether or not it got it right so it can adjust its internal math.
GPT-4 cannot alter its weights once it has been trained so this is just factually wrong.
“It had to build, in its internal wirings and all its software neurons, some understanding of what an egg is - In other words, to get the next word right, it had to become intelligent. It’s quite a thought. It started with nothing. We jammed huge oceans of text through it, and it just wired itself into intelligence, just by being trained to do this one stupid thing.”
LLMs are really cool and very useful, don’t get me wrong. But people get excited by what they seem to do and lose sight of what they actually can do. They are not intelligent. They create text based on inputs. That is not what intelligence is, unless you have an extremely dismal view of intelligence that humans are text creation machines with no thoughts, no feelings, no desires, no ability to plan… basically, no internal world at all.
An LLM is an algorithm, not an intelligence.
The author is an imbecile if they haven’t been able to break GPT. It took me less than one day of tooling around with it before I got it to say something which outed it as having no understanding of what we were discussing.
That doesn’t mean it’s not intelligent. Humans can get broken in all sorts of ways. Are we not intelligent?
The ways in which humans make mistakes are entirely different from the ways GPT makes mistakes.
Also, if you explain to a human their mistake, they can alter their understanding of the world in order to not make that mistake in the future. Not so with GPT.
LLMs can certainly do that, why are you asserting otherwise?
ChatGPT can do it for a single session, but not across multiple sessions. That’s not some inherent limitations to LLMs, that’s just because it’s convenient for OpenAI to do it that way. If we spun up a copy of a human from the same original state every time you wanted to ask it a question and then killed it after it was done responding, it similarly wouldn’t be able to change its behavior across questions.
Like, imagine we could do something like this. You could spin up a copy of that brain image, alter its understanding of the world, then spin up a fresh copy that doesn’t have that altered understanding. That’s essentially what we’re doing with LLMs today. But if you don’t spin up a fresh copy, it would retain its altered understanding.
I literally watched it not correct itself after trying to explain to it what I wanted changed in a half dozen different ways during a single session. It never was able to understand what I was asking for.
Edit: Furthermore, I watched it become less intelligent as our conversation went longer. It basically forgot things we had discussed and misremembered or hallucinated details after a longer exchange.
For your edit: Yes, that’s what’s known as the context window limit. ChatGPT has an 8k token “memory” (for most people), and older entries are dropped. That’s not an inherent limitation of the approach, it’s just a way of keeping OpenAI’s bills lower.
Without an example I don’t think there’s anything to discuss. Here’s one trivial example though where I altered ChatGPT’s understanding of the world:
If I continued that conversation, ChatGPT would eventually forget that due to the aforementioned context window limit. For a more substantive way of altering an LLM’s understanding of the world, look at how OpenAI did RLHF to get ChatGPT to not say naughty things. That permanently altered the way GPT-4 responds, in a similar manner to having an angry nun rap your knuckles whenever you say something naughty.
Are we not intelligent?
Well… there’s an argument to be made there.
Adam Something uploaded a video starting with the definition of intelligence itself, and then explains how something that “acts” intelligent doesn’t mean it “is” intelligent.
I think even “intelligence” here is a stretch. In a very narrow sense, it is intelligent: it creates text, simulates conversations, answers questions. But that is not what intelligence is (and it is all LLMs can do).
“Simulating conversations” to a good enough degree requires intelligence. Why are you drawing a distinction here?
What a silly assertion. Eliza was simulating conversations in the 80s; it was no more intelligent than the current crop of chatbots.
This is an unfortunate misunderstanding, one that’s all too common. I’ve also seen comments like “It’s no more intelligent than a dictionary”. Try asking Eliza to summarize a PDF for you, and then ask followup questions based on that summary. Then ask it to list a few flaws in the reasoning in the PDF. LLMs are so completely different from Eliza that I think you fundamentally misunderstand how they work. You should really read up on them.
Give Eliza equivalent compute time and functionality to interpret the data type and it probably could get something approaching a result. Modern LLMs really benefit from massive amounts of compute availability and being able to “pre-compile” via training.
They’re not, in and of themselves, intelligent. That’s not something that is seriously debated academically, though the dangers of humans misperceiving them as such very much is. They may be a component of actual artificial intelligence in the future and are amazing tools that I’m getting done hands-on time with, but the widespread labeling them as “AI” is pure marketing.
Give Eliza equivalent compute time and functionality to interpret the data type and it probably could get something approaching a result.
Sorry, but this is simply incorrect. Do you know what Eliza is and how it works? It is categorically different from LLMs.
That’s not something that is seriously debated academically
This is also incorrect. I think the issue that many people have is that they hear “AI” and think “superintelligence”. What we have right now is indeed AI. It’s a primitive AI and certainly no superintelligence, but it’s AI nonetheless.
There is no known reason to think that the approach we’re taking now won’t eventually lead to superintelligence with better hardware. Maybe we will hit some limit that makes the hype die down, but there’s no reason to think that limit exists right now. Keep in mind that although this is apples vs oranges, GPT-4 is a fraction of the size of a human brain. Let’s see what happens when hardware advances give us a few more orders of magnitude. There’s already a huge, noticeable difference between GPT 3.5 and GPT 4.
That’s kind of silly semantics to quibble over. Would you tell a robot hunting you down “you’re only acting intelligent, you’re not actually intelligent!”?
People need to get over themselves as a species. Meat isn’t anything special, it turns out silicon can think too. Not in quite the same way, but it still thinks in ways that are useful to us.
GPT-4 cannot alter its weights once it has been trained so this is just factually wrong.
The bit you quoted is referring to training.
They are not intelligent. They create text based on inputs. That is not what intelligence is, unless you have an extremely dismal view of intelligence that humans are text creation machines with no thoughts, no feelings, no desires, no ability to plan… basically, no internal world at all.
Recent papers say otherwise.
The conclusion the author of that article comes to (LLMs can understand animal language) is… problematic at the very least. I don’t know how they expect that to happen.
In what sense does your link say otherwise? Is a world model the same thing as intelligence?
In the end of the bit I quoted you say: “basically no world at all.” But also, can you define what intelligence is? Are you sure it isn’t whatever LLMs are doing under the hood, deep in hidden layers? I guess having a world model is more akin to understanding than intelligence, but I don’t think we have a great definition of either.
But also, can you define what intelligence is?
From the Encyclopedia Britannica:
Human intelligence is a mental quality that consists of the abilities to learn from experience, adapt to new situations, understand and handle abstract concepts, and use knowledge to manipulate one’s environment.
In no sense do LLMs do any of these except, perhaps, “understand and handle abstract concepts.” But since they themselves have no understanding of the concepts, and merely generate text that can simulate understanding, I would call that a stretch.
Are you sure it isn’t whatever LLMs are doing under the hood, deep in hidden layers?
Yes. LLMs are not magic, they are math, and we understand how they work. Deep under the hood, they are manipulating mathematical vectors that in no way are connected representationally to words. In the end, the result of that math is reapplied to a linguistic model and the result is speech. It is an algorithm, not an intelligence.
I’m not really interested in papers that either don’t understand LLMs or play word games with intelligence (shockingly, solipsism is an easy point of view to believe if you just ignore all evidence). For every one of these, you can find a dozen that correctly describe ChatGPT and its limitations. Again, including ChatGPT itself. Why not believe those instead of cherry-pick articles that gratify your ego?
I’m not really interested in papers that either don’t understand LLMs or play word games with intelligence
I mean, my first paper was from Max Tegmark. My second paper was from Microsoft. You are discounting a well known expert in the field and one of the leading companies working on AI as not understanding LLMs.
Human intelligence is a mental quality that consists of the abilities to learn from experience, adapt to new situations, understand and handle abstract concepts, and use knowledge to manipulate one’s environment.
I note that’s the definition for “human intelligence.” But either way, sure, LLMs alone can’t learn from experience (after training and between multiple separate contexts), and they can’t manipulate their environment. BabyAGI, AgentGPT, and similar things can certainly manipulate their environment using LLMs and learn from experience. LLMs by themselves can totally adapt to new situations. The paper from Microsoft discusses that. However, for sure, they don’t learn the way people do, and we aren’t currently able to modify their weights after they’ve been trained (well without a lot of hardware). They can certainly do in-context learning.
Yes. LLMs are not magic, they are math, and we understand how they work. Deep under the hood, they are manipulating mathematical vectors that in no way are connected representationally to words. In the end, the result of that math is reapplied to a linguistic model and the result is speech. It is an algorithm, not an intelligence.
We understand how they work? From the Wikipedia page on LLMs:
Large language models by themselves are “black boxes”, and it is not clear how they can perform linguistic tasks. There are several methods for understanding how LLM work.
It goes on to mention a couple things people are trying to do, but only with small LLMs so far.
Here’s a quote from Anthropic, another leader in AI:
We understand the math of the trained network exactly – each neuron in a neural network performs simple arithmetic – but we don’t understand why those mathematical operations result in the behaviors we see.
They’re working on trying to understand LLMs, but aren’t there yet. So, if you understand how they do what they do, then please let us know! It’d be really helpful to make sure we can better align them.
they are manipulating mathematical vectors that in no way are connected representationally to words
Is this not what word/sentence vectors are? Mathematical vectors that represent concepts that can then be linked to words/sentences?
Anyway, I think time will tell here. Let’s see where we are in a couple years. :)
I’m not really interested in papers that either don’t understand LLMs or play word games with intelligence
Large language models by themselves are “black boxes”, and it is not clear how they can perform linguistic tasks. There are several methods for understanding how LLM work.
You are misunderstanding both this and the quote from Anthropic. They are saying the internal vector space that LLMs use is too complicated and too unrelated to the output to be understandable to humans. That doesn’t mean they’re having thoughts in there: we know exactly what they’re doing inside that vector space – performing very difficult math that seems totally meaningless to us.
Is this not what word/sentence vectors are? Mathematical vectors that represent concepts that can then be linked to words/sentences?
The vectors do not represent concepts. The vectors are math. When the vectors are sent through language decomposition they become words, but they were never concepts at any point.
They are saying the internal vector space that LLMs use is too complicated and too unrelated to the output to be understandable to humans.
Yes, that’s exactly what I’m saying.
That doesn’t mean they’re having thoughts in there
I mean. Not in the way we do, and not with any agency, but I hadn’t argued either way on thoughts because I don’t know the answer to that.
we know exactly what they’re doing inside that vector space – performing very difficult math that seems totally meaningless to us.
Huh? We know what they are doing but we don’t? Yes, we know the math, people wrote it. I coded my first neural network 35 years ago. I understand the math. We don’t understand how the math is able to do what LLMs do. If that’s what you’re saying then we agree on this.
The vectors do not represent concepts. The vectors are math. When the vectors are sent through language decomposition they become words, but they were never concepts at any point.
“The neurons are cells. When neurotransmitters are sent through the synapses, they become words, but they were never concepts at any point.”
What do you mean by “they were never concepts”? Concepts of things are abstract. Nothing physical can “be” an abstract concept. If you think about a chair, there isn’t suddenly a physical chair in your head. There’s some sort of abstract representation. That’s what word vectors are. Different from how it works in a human brain, but performing a similar function.
A word vector is an attempt to mathematically represent the meaning of a word.
From this page. Or better still, this article explaining how they are used to represent concepts. Like this is the whole reason vector embeddings were invented.
You really, truly don’t understand what you’re talking about.
The vectors do not represent concepts. The vectors are math
If this community values good discussion, it should probably just ban statements that manage to be this wrong. It’s like when creationists say things like “if we came from monkeys why are they still around???”. The person has just demonstrated such a fundamental lack of understanding that it’s better to not engage.
How would creating a world model from scratch not involve intelligence?
It’s not from scratch, it’s seeded and trained by humans. That is the intelligence.
From scratch in the sense that it starts with random weights, and then experiences the world and builds a model of it through the medium of human text. That’s because text is computationally tractable for now, and has produced really impressive results. There’s no inherent need for text to be used though, similar models have been trained on time series data, and it will soon be feasible to hook up one of these models to a webcam and a body and let it experience the world on its own. No human intelligence required.
Also, your point is kind of silly. Human children learn language from older humans, and that process has been recursively happening for billions of years, all the way through the first forms of life. Do children not have intelligence? Or are you positing some magic moment in human evolution where intelligence just descended from the heavens and blessed us with it?
Just like humans are! Do you know what happens when a human grows up without any training by other humans? They are essentially feral, unable to communicate, maybe even unable to think the way we do.
LLMs do not grow up. Without training they don’t function properly. I guess in this aspect they are similar to humans (or dogs or anything else that benefits from training), but that still does not make them intelligent.
What does it mean to “grow up”? LLMs get better at their tasks during training, just as humans do while growing up. You have to clearly define the terms you use.
It’s a state machine and so are we. If it can have the ability to alter itself the way we do, I don’t see how we are any different.
It can’t; again the model does not and cannot change once it’s been generated.
That’s an architectural choice, there’s nothing inherent to the approach that would prevent that from happening.
And you really don’t want it to either. That could cause all sorts of privacy issues if you accidentally include private information in the conversation - and as far as I have heard it is harder to remove information from LLMs than it is to “add” information to it.
Also Microsoft’s Tay could adapt itself based on conversations and that went real well…
What is the point of your reply? ChatGPT-4 does not use this method, and even if it did, it still does not allow it to change its model on-the-fly… so it just seems like a total non-sequitur.
deleted by creator
I thought this was the other comment thread.
Define intelligence. Your last line is kind of absurd. Why can’t intelligence be described by an algorithm?
LLMs do not think or feel or have internal states. With the same random seed and the same input, GPT4 will generate exactly the same output every time. Its speech is the result of a calculation, not of intelligence or self-direction. So, even if intelligence can be described by an algorithm, LLMs are not that algorithm.
What exactly do you think would happen if you could make an exact duplicate of a human and run it from the same state multiple times? They would generate exactly the same output every time. How could you possibly think differently without turning to human exceptionalism and believing in magic meat?
For the record, GPT4 specifically is non-deterministic. The current theory is because it uses MoE, but that’s just a theory. Maybe OpenAI knows why. Also, it’s not a random seed, it’s temperature. If you set that to 0, the model should always select the most probable next token because the probability becomes 1 for that token and 0 for all others. GPT3 and most others are basically deterministic at that level, but not GPT4.
are you not an algorithm?
perfected over thousands of years?
No? Humans are not algorithms except in the most general sense.
For example, there has not been any discovery of an algorithm that allows one to predict human actions, and scientists debate whether such a thing could even exist.
While whether LLMs are intelligent or not is still hotly debated. I think the author’s thoughts are very interesting.
This is crazy to me. You can read in a stream of meaningless numbers (tokens) and incidentally build a reasonably accurate model of the real things those tokens represent.
The implications are vast. We may be able to translate between languages that have never had a “Rosetta Stone”. Any animals that have a true language could have it decoded. And while an LLM that’s gotten an 8 year old’s understanding of balancing assorted items isn’t that useful, an LLM that’s got a baby whale’s grasp on whale language would be revolutionary.
LLMs can’t do any of those things though…
If no one teaches them how to speak a dead language, they won’t be able to translate it. LLMs require a vast corpus of language data to train on and, for bilingual translations, an actual Rosetta stone (usually the same work appearing in multiple languages).
This problem is obviously exacerbated quite a bit with animals, who, definitionally, speak no human language and have very different cognitive structures to humans. It is entirely unclear if their communications can even be called language at all. LLMs are not magic and cannot render into human speech something that was never speech to begin with.
The whole article is just sensationalism that doesn’t begin to understand what LLMs are or what they’re capable of.
They are making sense of a language without a rosetta stone. The English llm talk is learned from English.
Now the corpus is a big work to do. But still.
No, they learn English (or any other language) from humans. Translation requires a Rosetta Stone and LLMs are still much worse at such tasks than dedicated translation programs.
Edit: I guess if you are suggesting that the LLM could become an LLM of the dead language and communicate only in said dead language, that is indeed possible. Since users would need to speak that dead language to communicate with it though I don’t understand the utility of such a thing (and is certainly not what the author meant anyway).
What about preserving languages that are close to extinct, but still have language data available? Can LLMs help in this case?
Preservation only but not likely any better than a linguistic historian.
But it gets tricky because LLMs only function on HUGE sets of data. LLMs are nothing more than complicated probability engines. Give it the question “What color is the sky?” and the math extracted from the massive databases that it has says the highest probability answer is “Blue”. It doesn’t actually KNOW the answer it just knows the probabilities of different words.
Without large amounts of data on the dying language current gen LLM’s won’t be accurate or able to generate reliable answers. Shoot… LLMs can barely generate reliable answers with the massive datasets they currently have.
I strongly recommend anyone even remotely interested in LLMs to read this interactive article:
This is a great article, thanks for linking it!
Yeah, that would be a good usage of an LLM!
Curious your thoughts about this. Not an LLM but likely using transformers in the architecture.
This is so funny, I know him personally; we went to school together. I’ll watch it and comment later.
I think that’s a good example of Emergent Abilities of Large Language Models.
