• 324 Posts
  • 1.75K Comments
Joined 10 months ago
cake
Cake day: July 25th, 2024

help-circle
  • It’s an easy mistake to make. For future reference, Wikiquote – a sister project of Wikipedia like Wiktionary and Wikimedia Commons are – is very often a good benchmark for whether famous people have said a quote.

    • For famous quotes that they’ve said, they’re usually listed (if they are, there’s a citation to exactly where that quote came from).
    • For famous quotes they didn’t say, the “Misattributed” section often has the quote with a cited explanation of where it actually comes from.
    • For famous quotes they might’ve or probably didn’t say, the “Disputed” section shows where it’s first attributed to them but of course cannot provide a source where they themselves say it.

    It doesn’t have every quote, but for very famous people, it filters out a lot of false positives. Since it gives you a citation, often you can leave a URL to the original source alongside your quote for further context and just so people who’d otherwise call BS have the source. And it sets a good example for others to cite their sources.







  • Uhh… yeah, goddamn. The Daily Beast citing the Daily Mail as their source is really something. Not only do we not use them as a source on Wikipedia, and not only was this the first source ever to be deprecated there in this way because of how egregious they are, but we don’t even allow their online historical archives because they’ve been caught faking those too.

    The Daily Mail isn’t a rag; it’s sewage. It single-handedly motivated the idea that there are sources bad enough that Wikipedia just prohibits their usage everywhere (except in rare cases in an about-self fashion, but I don’t know if editors would even trust that anymore). The Daily Beast isn’t the pinnacle of credible journalism, but it isn’t abysmal either.


    Edit: sorry, here’s a source instead of just “my source is that I made it the fuck up.”


  • This is entirely correct, and it’s deeply troubling seeing the general public use LLMs for confirmation bias because they don’t understand anything about them. It’s not “accidentally confessing” like the other reply to your comment is suggesting. An LLM is just designed to process language, and by nature of the fact it’s trained on the largest datasets in history, practically there’s no way to know where this individual output came from if you can’t directly verify it yourself.

    Information you prompt it with is tokenized, run through a transformer model whose hundreds of billions or even trillions of parameters were adjusted according to god only knows how many petabytes of text data (weighted and sanitized however the trainers decided), and then detokenized and printed to the screen. There’s no “thinking” involved here, but if we anthropomorphize it like that, then there could be any number of things: it “thinks” that’s what you want to hear; it “thinks” that based on the mountains of text data it’s been trained on calling Musk racist, etc. You’re talking to a faceless amalgam unslakably feeding on unfathomable quantities of information with minimal scrutiny and literally no possible way to enforce quality beyond bare-bones manual constraints.

    There are ways to exploit LLMs to reveal sensitive information, yes, but you have to then confirm that sensitive information is true, because you’ve just sent data into a black box and gotten something out. You can get a GPT to solve the sudoku puzzle, but you can’t then parade that around before you’ve checked to make sure the puzzle is correct. You cannot ever, under literally any circumstance, trust anything a generative AI creates for factual accuracy; at best, you can use it as a shortcut to an answer which you can attempt to verify.





  • “Expert in machine learning”, “has read the literal first sentence of the Wikipedia entry for ‘machine learning’” – same thing. Tomayto, tomahto.

    Everything else I’m talking about in detail is just gravy; literally just read the first sentence of the Wikipedia article to know that machine learning is a field of AI. That’s the part that got me to say “no one in this thread knows what they’re talking about”: it’s the literal first sentence in the most prominent reference work in the world that everyone reading this can access in two seconds.

    You can say most people don’t know the atomic weight of oxygen is 16-ish. That’s fine. I didn’t either; I looked it up for this example. What you can’t do is say “the atomic weight of oxygen is 42”, then when someone contradicts you that it’s 16, refuse to concede that you’re wrong and then – when they clarify why the atomic weight is 16 – stand there arms crossed and with a smarmy grin say: “wow, expert blindness much? geez guys check out this bozo”

    We get it; you read xkcd. The point of this story is that you need to know fuck-all about atomic physics to just go on Wikipedia before you confidently claim the atomic weight is 42. Or, when someone calls you out on it, go on Wikipedia to verify that it’s 16. And if you want to dig in your heels and keep saying it’s 42, then you get the technical explanation. Then you get the talk about why it has that weight, because you decided to confidently challenge it instead of just acknowledging this isn’t your area of expertise.


  • Quite the opposite: I recognize there’s a difference, and it horrifies me that corporations spin AI as something you – “you” meaning the general public who don’t understand how to use it – should put your trust in. It similarly horrifies me that in an attempt to push back on this, people will jump straight to vibes-based, unresearched, and fundamentally nonsensical talking points. I want the general public to be informed, because like the old joke comparing tech enthusiasts to software engineers, learning these things 1) equips you with the tools to know and explain why this is bad, and 2) reveals that it’s worse than you think it is. I would actually prefer specificity when we’re talking about AI models; that’s why instead of “AI slop”, I use “LLM slop” for text and, well, unfortunately, literally nobody in casual conversation knows what other foundation models or their acronyms are, so sometimes I just have to call it “AI slop” (e.g. for imagegen). I would love it if more people knew what a transformer model is so we could talk about transformer models instead of the blanket “AI”.

    By trying to incorrectly differentiate “AI” from “machine learning”, we’re giving dishonest corporations more power by implying that only now do we truly have “artificial intelligence” and that everything that came before is merely “machine learning”. By muddling what’s actually a very straightforward hierarchy of terms (opposed to a murky, nonsensical dichotomy of “AI is anything that I don’t like, and ML is anything I do”), we’re misinforming the public and making the problem worse. By showing that “AI” is just a very general field that GPTs live inside, we reduce the power of “AI” as a marketing buzzword word.



  • Even though I understand your sentiment that different types of AI tools have their place, I’m going to try clarifying some points here. LLMs are machine learning models; the ‘P’ in ‘GPT’ – “pretrained” – refers to how it’s already done some learning. Transformer models (GPTs, BERTs, etc.) are a type of deep learning is a branch of machine learning is a field of artificial intelligence. (edit: so for a specific example of how this looks nested: AI > ML > DL > Transformer architecture > GPT > ChatGPT > ChatGPT 4.0.) The kind of “vision guided industrial robot movement” the original commenter mentions is a type of deep learning (so they’re correct it’s machine learning, but incorrect that it’s not AI). At this point, it’s downright plausible that the tool they’re describing uses a transformer model instead of traditional deep learning like a CNN or RNN.

    I don’t entirely understand your assertion that “LLMs are shit with large data and numbers”, because LLMs work with the largest data in human history. If you mean you can’t feed a large, structured dataset into ChatGPT and expect it to be able to categorize new information from that dataset, then sure, because: 1) it’s pretrained, not a blank slate that specializes on the new data you give it, and 2) it’s taking it in as plaintext rather than a structured format. If you took a transformer model and trained it on the “large data and numbers”, it would work better than traditional ML. Non-transformer machine learning models do work with text data; LSTMs (a type of RNN) do exactly this. The problem is that they’re just way too inefficient computationally to scale well to training on gargantuan datasets (and consequently don’t generate text well if you want to use it for generation and not just categorization). In general, transformer models do literally everything better than traditional machine learning models (unless you’re doing binary classification on data which is always cleanly bisected, in which case the perceptron reigns supreme /s). Generally, though, yes, if you’re using LLMs to do things like image recognition, taking in large datasets for classification, etc., what you probably have isn’t just an LLM; it’s a series of transformer models working in unison, one of which will be an LLM.


    Edit: When I mentioned LSTMs, I should clarify this isn’t just text data: RNNs (which LSTMs are a type of) are designed to work on pieces of data which don’t have a definite length, e.g. a text article, an audio clip, and so forth. The description of the transformer architecture in 2017 catalyzed generative AI so rapidly because it could train so efficiently on data not of a fixed size and then spit out data not of a fixed size. That is: like an RNN, the input data is not of a fixed size, and the transformed output data is not of a fixed size. Unlike an RNN, the data processing is vastly more efficient in a transformer because it can make great use of parallelization. RNNs were our main tool for taking in variable-length, unstructured data and categorizing it (or generating something new from it; these processes are more similar than you’d think), and since that describes most data, suddenly all data was trivially up for grabs.



  • Okay, at this point, I’m convinced no one in here has even a bare minimum understanding of machine learning. This isn’t a pedantic prescriptivism thing:

    1. “Machine learning” is a major branch of AI. That’s just what it is. Literally every paper and every book ever published on the subject will tell you that. Go to the Wikipedia page right now: “Machine learning (ML) is a field of study in artificial intelligence”. The other type of AI of course means that the machine can’t learn and thus a human has to explicitly program everything; for example, video game AI usually doesn’t learn. Being uninformed is fine; being wrong is fine. There’s calling out pedantry (“reee you called this non-Hemiptera insect a bug”) and then there’s rendering your words immune to criticism under a flimsy excuse that language has changed to be exactly what you want it to be.

    2. Transformers, used in things like GPTs, are a type of machine learning. So even if you say that “AI is just generative AI like LLMs”, then, uh… Those are still machine learning. The ‘P’ in GPT literally stands for “pretrained”, indicating it’s already done the learning part of machine learning. OP’s statement literally self-contradicts.

    3. Meanwhile, deep learning (DNNs, CNNs, RNNs, transformers, etc.) is a branch of machine learning (likewise with every paper, every book, Wikipedia (“Deep learning is a subset of machine learning that focuses on […]”), etc.) wherein the model identifies its own features instead of the human needing to supply them. Notably, the kind of vision detection the original commenter is talking about is deep learning like a transformer model is. So “AI when they mean machine learning” by their own standard that we need to be specific should be “AI when they mean deep learning”.

    The reason “AI” is used all the time to refer to things like LLMs etc. is because generative AI is a type of AI. Just like “cars” are used all the time to refer to “sedans”. To be productive about this: for anyone who wants to delve (heh) further into it, Goodfellow et al. have a great 2016 textbook on deep learning*. In a bit of extremely unfortunate timing, transformer models were described in a 2017 paper, so they aren’t included (generative AI still is), but it gives you the framework you need to understand transformers (GPTs, BERTs). After Goodfellow et al., just reading Google’s original 2017 paper gives you sufficient context for transformer models.

    *Goodfellow et al.'s first five chapters cover traditional ML models so you’re not 100% lost, and Sci-Kit Learn in Python can help you use these traditional ML techniques to see what they’re like.


    Edit: TL;DR: You can’t just weasel your way into a position where “AI is all the bad stuff and machine learning is all the good stuff” under the guise of linguistic relativism.


  • The scenario you just described, though, is technically correct is my point (edit: whereas you seem to be saying it isn’t technically correct; it’s also colloquially correct). Referring to “machine learning” as “AI” is correct in the same way referring to “a rectangle” as “a quadrilateral” is correct.


    EDIT: I think some people are interpreting my comment as “b-but it’s technically correct, the best kind of correct!” pedantry. My point is that the comment I’m responding to seems to think they got it technically incorrect, but they didn’t. Not only is it “technically correct”, but it’s completely, unambiguously correct in every way. They’re the ones who said “If you’re expecting “technically correct” from them, you’ll be doomed to disappointment.”, so I pointed out that I’m not doomed to disappointment because they literally are correct colloquially and correct technically. Please see my comment below where I talk about why what they said about distinguishing AI from machine learning makes literally zero sense.