A student in America asked an artificial intelligence program to help with her homework. In response, the app told her "Please Die." The eerie incident happened when 29-year-old Sumedha Reddy of Michigan sought help from Google’s Gemini chatbot large language model (LLM), New York Post reported. The program verbally abused her, calling her a “stain on the universe.” Reddy told CBS News that she got scared and started panicking.  “I wanted to throw all of my devices out the window. I hadn’t felt panic like that in a long time to be honest,” she said.

Ouch.

  • pixxelkick@lemmy.worldEnglish
    34·
    1 month ago

    On the original thread of questions, it went on for a long time and had multiple questions about psychological, emotional, and physical abuse.

    LLMs get more and more off the rails as their context gets longer (longer convo), most folks have prolly at this point noticed every now and then a long running convo gets a little… schizophrenic feeling as it drags on.

    The combination of a very long convo with a lot of tokens, and its subject being that of discussing and defining types of abuse, and I can see how eventually the LLM will generate a response like that randomly when it goes off the rails.

    • IninewCrow@lemmy.caEnglish
      14·
      1 month ago

      This happened to me and my friends this summer. The three of us were talking about AI technology and one friend who is an engineer wanted to demonstrate all this so he turned on ChatGPT on his phone and we started asking random questions. The three of us were just having fun and taking turns asking about food, birds, geology, houses, construction, math equations, medicine, the meaning of life, and a bunch of other silly things … after about half an hour it went off the rails and started giving bizarre answers that tried to create responses that tried to combine everything we had been asking about up to that point. Completely crazy responses that tried to give a meaning of life explanation that included birds, peanuts and how a bicycle works. We wanted to record the responses because they were so off the wall but by the time we started recording the audio, we were disconnected, the conversation reset and everything went back to normal.

      • bane_killgrind@slrpnk.netEnglish
        14·
        1 month ago

        There is a new conversational space beyond which is known to man. It is a space as vast as your mom and as timeless as corporate greed. It is the middle ground between light and shadow, between the observed and deducted, and it lies between the pit of man’s assumptions and the summit of his hubris. This is the dimension of hallucination. It is an area which we call, “The Twilight Zone.”

  • Nougat@fedia.io
    13·
    1 month ago

    The easy part is making a program that can pretend to be human. The hard part is getting it to not be an asshole.

    • LifeInMultipleChoice@lemmy.dbzer0.comEnglish
      2·
      1 month ago

      Did you read through it, it was a remarkable answer by Gemini, but it was also cool to see how they were utilizing the LLM to minimalize putting any thought into the work.

      … put in paragraphs, add more, add more, add these key terms, put back in paragraphs, add more.

      Okay, I guess I know all about this subject now.

  • dumbass@leminal.spaceEnglish
    10·
    1 month ago

    How bad at doing homework is she that the ai had a mental breakdown trying to teach her!?

  • desktop_userEnglish
    7·
    1 month ago

    they really should have shared the entire token context, I get hating on llms, but context matters.