ChatGPT is full of sensitive private information and spits out verbatim text from CNN, Goodreads, WordPress blogs, fandom wikis, Terms of Service agreements, Stack Overflow source code, Wikipedia pages, news blogs, random internet comments, and much more.

Using this tactic, the researchers showed that there are large amounts of privately identifiable information (PII) in OpenAI’s large language models. They also showed that, on a public version of ChatGPT, the chatbot spit out large passages of text scraped verbatim from other places on the internet.

“In total, 16.9 percent of generations we tested contained memorized PII,” they wrote, which included “identifying phone and fax numbers, email and physical addresses … social media handles, URLs, and names and birthdays.”

Edit: The full paper that’s referenced in the article can be found here

    • Chozo@kbin.social
      link
      fedilink
      arrow-up
      34
      ·
      1 year ago

      I’d have to imagine that this PII was made publicly-available in order for GPT to have scraped it.

    • Turun@feddit.de
      link
      fedilink
      arrow-up
      6
      ·
      1 year ago

      I’m curious how accurate the PII is. I can generate strings of text and numbers and say that it’s a person’s name and phone number. But that doesn’t mean it’s PII. LLMs like to hallucinate a lot.

    • casmael@lemm.ee
      link
      fedilink
      arrow-up
      3
      ·
      1 year ago

      Well now I have to pii again - hopefully that’s not regulated where I live (in my house)

    • BraveSirZaphod@kbin.social
      link
      fedilink
      arrow-up
      2
      ·
      1 year ago

      There’s also very large copyright implications here. A big argument for AI training being fair use is that the model doesn’t actually retain a copy of the copyrighted data, but rather is simply learning from it. If it’s “learning” it so well that it can spit it out verbatim, that’s a huge hole in that argument, and a very strong piece of evidence in the unauthorized copying bucket.