cross-posted from: https://lemmy.ml/post/2811405
"We view this moment of hype around generative AI as dangerous. There is a pack mentality in rushing to invest in these tools, while overlooking the fact that they threaten workers and impact consumers by creating lesser quality products and allowing more erroneous outputs. For example, earlier this year America’s National Eating Disorders Association fired helpline workers and attempted to replace them with a chatbot. The bot was then shut down after its responses actively encouraged disordered eating behaviors. "
You must log in or register to comment.
The real issue is people need to realize how LLMs work. It’s just a really good next word generator that sounds plausible to a human. Accuracy and truth isn’t part of consideration for the most part. The AI doesn’t even see words, it just breaks words down to numbers and treats it like a giant math problem.
It’s an amazing tool that will massively boost productivity, but people need to know its limitations and what it’s actually capable of. That’s where the hype is overblown.
I work on AI research. I’ve been trying to explain it to people as an improv actor that takes suggestions from the audience. It just plays along with the prompt you give it. It’s not an expert, it’s just an actor playing a role.
Ironically, I think you also are overlooking some details about how LLMs work. They are not just word generators. Stuff is going on inside those neural networks that we’re still unsure of.
For example, I read about a study a little while back that was testing the mathematical abilities of LLMs. The researchers would give them simple math problems like “2+2=” and the LLM would fill in 4, which was unsurprising because that equation could be found in the LLM’s training data. But as they went to higher numbers the LLM kept giving mostly correct results, even when they knew for a fact that the specific math problem being presented wasn’t in the training data. After training on enough simple addition problems the LLM had actually “figured out” some of the underlying rules of math and was using those to make its predictions.
Being overly dismissive of this technology is as fallacious as overly hyping it.
No. Just… No. The LLM has not “figured out” what’s going on. It can’t. These things are just good at prediction. The main indicator is in your text: “mostly correct”. A computer that knows what to calculate will not be “mostly correct”. One false answer proves one hundred percent that it has no clue what it’s supposed to do.
What we are seeing with those “studies” is that social study people try to apply the same rules they apply to humans (where “mostly correct” is as good as “always correct”) which is bonkers, or behavioral researchers try to prove some behavior they attribute to the AI as if it was a living being, which is also bonkers because the AI will mimic the results in the training data which is human so the data will be biased as fuck and its impossible to determine if the AI did anything by itself at all (which it didn’t, because that’s not how the software works).These things are just good at prediction.
Indeed, and it turns out that in order to predict the next word these things may be thinking about stuff.
There’s a huge amount of complex work that can go into predicting stuff. If you were to try to predict the next word that a person you’re speaking with was going to say, how would you go about it? Developing a mental model of that person’s thought processes would be a really good approach. How would you predict what the next thing that comes after “126+118=” is? Would you always get it exactly correct, or might you occasionally predict the wrong number?
I think you’re starting from the premise that these things can’t possibly be “thinking”, on any level, and are trying to reinterpret everything to fit that premise. These things are largely opaque black boxes, just like human brains are. Is it really so impossible that thought-like processes are going on inside both of them?
Yes, it is impossible. There are no “thoughts”. The bloody thing doesn’t know what an Apple is if you ask it to write a 500 page book about them. It just guesses a word, then from there guesses the next one and so on. That’s why it will very often confidently tell you aggravating bullshit. It has no concept of the things it spits out. It’s a “word calculator” so to speak. The whole thing is not “revolutionary” or “new” by any stretch. What is new is the ability to use tons and tons and tons of reference data which makes the output halfway decent and the GPU power that will make it’s speed halfway decent. Other than that, LLMs are.not.“thinking”.
A rather categorical statement given that you didn’t say anything with regards to how you think.
Maybe wait until we actually know more what’s going on under the hood - both in LLMs and in the human brain - before stating with such confident finality that there’s absolutely no similarities.
If it turns out that LLMs aren’t thinking, but they’re still producing the same sort of interaction that humans are capable of, perhaps that says more about humans than it does about LLMs.
sees a plastic bag being blown by the wind
Holy shit that bag must be alive
They produce this kind of output because they break doen one mostly logical system (language) onto another (numbers). The irregularities language has get compensated by the vast number of sources.
We don’t need to know more about anything. If I tell you “hey, don’t think of an Apple”, your brain will conceptualize an Apple and then go from there. LLMs don’t know “concepts”. They spit out numbers just as mindlessly as your Casio calculator watch.
I would argue that what’s going on is that they are compressing information. And it just so happens that the most compact way to represent a generative system (like mathematical relations for instance) is to model their generative structure. For instance, it’s much more efficient to represent addition by figuring out how to add two numbers, than by memorizing all possible combinations of numbers and their sum. So implicit in compression is the need to discover generalizations. But, the network has limited capacity and limited “looping power”, and it doesn’t really know what a number is, so it has to figure all this out by example and as a result will often come to approximate versions of these generalizations. Thus, it will often appear to be intelligent until it encounters something that doesn’t quite fit whatever approximation it came up with and will suddenly get something wrong that seems outside the pattern that you thought it understood, because it’s hard to predict what it’s captured at a very deep level and what it only has surface concepts of.
In other words, I think it is “kind of” thinking, if thinking can be considered a kind of computation, but it doesn’t always capture concepts completely because it’s not quite good enough at generalizing what it’s learned, but it’s just good enough to appear really smart within a certain distribution of inputs.
Which, in a way, isn’t so different from us, but is maybe not the same as how we learn and naturally integrate information.
I’ve been making the same or similar arguments you are here in a lot of places. I use LLMs every day for my job, and it’s quite clear that beyond a certain scale, there’s definitely more going on than “fancy autocomplete.”
I’m not sure what’s up with people hating on AI all of a sudden, but there seems quite a few who are confidently giving out incorrect information. I find it most amusing when they’re doing that at the same time as bashing LLMs for also confidently giving out wrong information.
I suspect it’s rooted in defensive reactions. People are worried about their jobs, and after being raised to believe that human thought is special and unique they’re worried that that “specialness” and “uniqueness” might be threatened. So they form very strong opinions that these things are nothing to worry about.
I’m not really sure what to do other than just keep pointing out what information we do have about this stuff. It works, so in the end it’ll be used regardless of hurt feelings. It would be better if we get ready for that sooner rather than later, though, and denial is going to delay that.
Can you give examples of that?
The engineers of ChatGPT-4 themselves have stated that it is beginning to show signs of general intelligence. I put a lot more value in their opinion on the subject than a person on the Internet who doesn’t work in the field of artificial intelligence.
That wasn’t the engineers of GPT-4, it was Microsoft who have been fanning the hype pretty heavily to recoup their investment and push their own Bing integration and then opened their “study” with:
“We acknowledge that this approach is somewhat subjective and informal, and that it may not satisfy the rigorous standards of scientific evaluation.”
An actual AI researcher (Maarten Sap) regarding this statement:
The ‘Sparks of A.G.I.’ is an example of some of these big companies co-opting the research paper format into P.R. pitches. They literally acknowledge in their paper’s introduction that their approach is subjective and informal and may not satisfy the rigorous standards of scientific evaluation.
It’s PR by Microsoft. I am beginning to doubt the intelligence of many humans rather than that of ChatGPT considering these kinds of comments.
A computer program is just a series of single bits activating and deactivating. That’s what you’re saying when you say a LLM is simply predicting words. You’re not thinking at the appropriate level of abstraction. The whole point is the mechanism by which words are produced and the information encoded.
No, you’re wrong. All interesting behavior of ML models is emergent. It is learned, not programmed. The fact that it can perform what we consider an abstract task with success clearly distinguishable from random chance is irrefutable proof that some model of the task has been learned.
No one said anyhting about “learned” vs “programmed”. Literally no one.
OP is saying it’s impossible for a LLM to have “figured out” how something it works, and that if it understood anything it would be able to perform related tasks perfectly reliably. They didn’t use the words, but that’s what they meant. Sorry for your reading comprehension.
“op” you are referring to is… well… myself, Since you didn’t comprehend that from the posts above, my reading comprehension might not be the issue here. <insert trollface>
But in all seriousness: I think this is an issue with concepts. No one is saying that LLMs can’t “learn” that would be stupid. But the discussion is not “is everything programmed into the LLM or does it recombine stuff”. You seem to reason that when someone says the LLM can’t “understand”, that person means “the LLM can’t learn”, but “learning” and “understanding” are not the same at all. The question is not if LLMs can learn, It’s wether it can grasp concepts from the content of the words it absorbs as it it’s learning data. If it would grasp concepts (like rules in algebra), it could reproduce them everytime it gets confronted with a similar problem. The fact that it can’t do that shows that the only thing it does is chain words together by stochastic calculation. Really sophisticated stachastic calculation with lots of possible outcomes, but still.
“op” you are referring to is… well… myself, Since you didn’t comprehend that from the posts above, my reading comprehension might not be the issue here.
I don’t care. It doesn’t matter, so I didn’t check. Your reading comprehension is still, in fact, the issue, since you didn’t understand that the “learned” vs “programmed” distinction I had referred to is completely relevant to your post.
It’s wether it can grasp concepts from the content of the words it absorbs as it it’s learning data.
That’s what learning is. The fact that it can construct syntactically and semantically correct, relevant responses in perfect English means that it has a highly developed inner model of many things we would consider to be abstract concepts (like the syntax of the English language).
If it would grasp concepts (like rules in algebra), it could reproduce them everytime it gets confronted with a similar problem
This is wrong. It is obvious and irrefutable that it models sophisticated approximations of abstract concepts. Humans are literally no different. Humans who consider themselves to understand a concept can obviously misunderstand some aspect of the concept in some contexts. The fact that these models are not as robust as that of a human’s doesn’t mean what you’re saying it means.
the only thing it does is chain words together by stochastic calculation.
This is a meaningless point, you’re thinking at the wrong level of abstraction. This argument is equivalent to “a computer cannot convey meaningful information to a human because it simply activates and deactivates bits according to simple rules.” Your statement about an implementation detail says literally nothing about the emergent behavior we’re talking about.
Can we stop giving out copium like this? You are fact free.
How does behaviour that is present in LLMs but not in SLMs show that an LLM can “think”?`It only shows that the amount of stuff an LLM can guess increases when you feed it more data. That’s not the hot take you think it is.
I think this is downplaying what LLMs do. Yeah, they are not the best at doing things in general, but the fact that they were able to learn the structure and semantic context of language is quite impressive, even if it doesn’t know what the words converted into tokens actually mean. I suspect that we will be able to use LLMs as one part of a full digital “brain”, with some model similar to our own prefrontal cortex calling the LLM (and other things like vision model, sound model, etc.) and using its output to reason about a certain task and take an action. That’s where I think the hype will be validated, is when you put all these parts we’ve been working on together and Frankenstein a new and actually intelligent system.
Here, here. We need legislation to limit this, and we need it YESTERDAY.
Someone make an AI that replaces CEO’s. Seriously, I’m not kidding. This is the answer.
Stopping math is never a good idea. By limiting your own constituents, you set their progress back from what other governments’ constituents can achieve.
Also, effectively replacing a CEO requires AGI level capabilities. We’re closer to that than ever before, but LLMs in their current state aren’t it.
yea, more like: ceos and white collar hurt consumers and workers. those jobs need to be eliminated and corps need to be heavily taxed so that universal basic income becomes ubiquitous. idk if any of this makes sense but i think this how things should be going
At least people are coming around to why it’s called AI. Artificial Intelligence is called that because it’s a facsimile of intelligence. It acts intelligent, but has no intellect. It’s an algorithm, usually one designed in a black box so no one can analyze exactly how the output occurred
The human brain is itself still largely a black box as far as our reasoning capabilities are concerned.
We don’t need to develop tech that can’t be analyzed directly. AI can and has been developed in a way that can be easily analyzed, like why an output was given.
We’ve been trying to do that approach for decades and progress has been slow and disappointing.
When we finally decided “screw it, just build a giant black box and throw terabytes of text at it to see what happens” we got GPT3 and now the world is about to be revolutionized.
Start revolutionizing, we’ve been waiting for months now…
Gosh, months.
If it’s supposed to be the labor extinguisher of the future, yes I expect something in the order of months
Your expectations are unrealistic. I am a programmer and I find tools like ChatGPT and copilot to be fantastic, but the company I work for has banned use of them until the legal department has figured out what the heck (and they won’t figure out what the heck until the judicial system figures out what the heck, and the legislative layer above that). It takes time for these sorts of massive shifts in well-established systems to happen.
The black box isn’t being done because it’s a new idea, it’s actually the other way around. The newer idea is actually the method for easier analysis. There’s a few reasons that they aren’t doing that though.
- It’s a newer idea, not everything has been studied so methods will be experimental.
- It’s in the company’s interest to make the AI harder to analyze, because they don’t want open the door on a better algorithm from a different company/government/group.
- It’s cheaper up front to build a black box and then do statistical analysis the hard and expensive way. Companies would much rather spend money doing things the wrong way instead of saving money long term doing things the right way.
If doing it the “wrong way” is cheap and works well, then perhaps it’s not the “wrong way.”
There are many companies (and researchers and hobbyists now) who are doing this stuff other than OpenAI, at this point. They just broke the ice and showed what was possible.
I just explained that it’s not cheap. It costs far more to buy a cheap car and do constant maintenance than it is to buy the mid tier car without much maintenance. That’s what’s happening with AI right now, we’re buying the cheap car and paying for it in labor and development costs. I’m saying that the right way is to buy the more expensive one, which will be cheaper in the long run.
There is no agent on the planet who is intentionally choosing to make their models harder to analyze. This is a ridiculous idea that you could only believe if you didn’t understand where the complexity comes from in the first place. Creating ML models that can be efficiently and effectively trained and interpreted is an extremely hard and unsolved problem, and whomever could solve it would be rolling in cash.
Isn’t it called AI because marketing people don’t understand the difference?
What do you think “it’s an algorithm” is supposed to imply? Can nothing deterministic be considered intelligent?
Also, “designed in a black box” is misleading. It’s opaque because it’s emergent behavior, not because it was obfuscated or designed in secrecy or something. The algorithm itself is simple. All the interesting data is encoded in the billions to trillions of input parameters. These parameters aren’t designed at all, they are learned.
We should call it: “Algorithmic Intelligence” or A.I. for short.
How about Algorithmic Sentence Shaper?
You can’t regulate automation to stop it. You need to learn to adapt just like everyone else who has been automated out of a job.
I bet the prison cells and the fines feel all the same, AI or no
Let’s see…
They may create text which appears to human eyes like the result of thinking, reasoning, or understanding, but it is in fact anything but.
For generation of fictional text and images that’s fine.
There is a pack mentality in rushing to invest in these tools, while overlooking the fact that they threaten workers
Like any other case of automation in the history of society.
[…] and impact consumers by creating lesser quality products
That sounds very subjective.
and allowing more erroneous outputs.
Large language models should not be used as a source of facts, that’s why they all warn you about their limitations. LLMs are tools and should be used properly. A blow torch can get your balls burnt if used improperly.
I’m reminded of a phenomenon in the 70s and 80s the computer is never wrong in which pricing mistakes and bank errors were expected to be impossible since there was a computer involved.
As an aside, I wonder if this is in any way related to the rush of patents in the 90s and aughts, for things humans obviously do, but on a computer or on the web like transferring money or making transactions. We still have lawsuits like that.
Also related, the predictive policing software that some US counties bought, unvetted, and is used to justify longer sentences for poor and nonwhite convicts so that no judge has to attach his name to bigoted rulings.
We humans seem to imagine that since there’s a magic box involved in the computation of our answers that the answer is automatically more precise. Perhaps it’s related to the notion that were considering more factors, but that only works if we’ve properly measured those factors and applied them appropriately to the model. Otherwise, as the saying goes (also from early computing) Garbage in; garbage out.
