• copygirl
    link
    fedilink
    English
    arrow-up
    35
    ·
    23 days ago

    Great, so it’s still wrong 1 out of 20 times, and just got even more energy intensive to run.

    • kippinitreal@lemmy.world
      link
      fedilink
      English
      arrow-up
      8
      ·
      22 days ago

      Genuine question: how energy intensive is it to run a model compared to training it? I always thought once a model is trained it’s (comparatively) trivial to query?

        • hitmyspot@aussie.zone
          link
          fedilink
          English
          arrow-up
          1
          ·
          21 days ago

          How much energy does it take for the PC to be on and the user to type out that email manually?

          I assume we will get to a point where energy required starts to reduce as the computing power increases with moores law. However, it’s awful for the environment in the mean time.

          I don’t doub that rather than reducing energy, instead they will use more complex models requiring more power for these tasks for the foreseeable future. However eventually it will be diminishing returns on power and efficiency will be more profitable.

      • DavidGarcia@feddit.nl
        link
        fedilink
        English
        arrow-up
        6
        ·
        22 days ago

        For the small ones, with GPUs a couple hundred watts when generating. For the large ones, somewhere between 10 to 100 times that.

        With specialty hardware maybe 10x less.

        • Pennomi@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          22 days ago

          A lot of the smaller LLMs don’t require GPU at all - they run just fine on a normal consumer CPU.

          • copygirl
            link
            fedilink
            English
            arrow-up
            3
            ·
            22 days ago

            Wouldn’t running on a CPU (while possible) make it less energy efficient, though?

            • Pennomi@lemmy.world
              link
              fedilink
              English
              arrow-up
              3
              ·
              22 days ago

              It depends. A lot of LLMs are memory-constrained. If you’re constantly thrashing the GPU memory it can be both slower and less efficient.

          • DavidGarcia@feddit.nl
            link
            fedilink
            English
            arrow-up
            1
            ·
            21 days ago

            yeah but 10x slower, at speeds that just don’t work for many use cases. When you compare energy consumption per token, there isn’t much difference.

      • 4am@lemm.ee
        link
        fedilink
        English
        arrow-up
        3
        ·
        22 days ago

        Still requires thirsty datacenters that use megawatts of power to keep them online and fast for thousands of concurrent users

  • riplin@lemm.ee
    link
    fedilink
    English
    arrow-up
    13
    ·
    23 days ago

    LLM’s will never achieve much higher than that simply because there’s no reasoning behind it. It. Won’t. Work. Ever.

  • Lugh@futurology.todayOPM
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    23 days ago

    I still see even the more advanced AIs make simple errors on facts all the time…