• copygirl
    link
    fedilink
    English
    arrow-up
    3
    ·
    3 days ago

    Wouldn’t running on a CPU (while possible) make it less energy efficient, though?

    • Pennomi@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      3 days ago

      It depends. A lot of LLMs are memory-constrained. If you’re constantly thrashing the GPU memory it can be both slower and less efficient.