• orca@orcas.enjoying.yachts
    link
    fedilink
    arrow-up
    53
    ·
    27 days ago

    From my experience with ChatGPT:

    1. It will NEVER consistently give you only the value in the response. It will always eventually add in some introductory text like it’s talking to a human. No matter how many times I tried to get it to just give me back the answer alone, it never consistently did.
    2. ChatGPT is terrible with numbers. It can’t count, do math, none of that. So asking it to do byte math is asking for a world of hurt.

    If this isn’t joke code, that is scary.

    • BluesF@lemmy.world
      link
      fedilink
      arrow-up
      4
      ·
      edit-2
      26 days ago

      I know a guy who was working on something like this, they just had the call to the model loop until the response met whatever criteria the code needed (e.g. one single number, a specifically formatted table, viable code, etc) or exit after a number of failed attempts. That seemed to work pretty well, it might mess up from time to time but it’s unlikely to (with the right prompt) do so repeatedly when asked again.

      • orca@orcas.enjoying.yachts
        link
        fedilink
        arrow-up
        2
        ·
        26 days ago

        That’s a good approach. I think for my use case the struggle was trying to not use a ton of tokens (upper management was being stingy on that front). I kept trying to push to make it more robust but you know how those things go. Axed ahead of their time or zombified.

    • Sotuanduso@lemm.ee
      link
      fedilink
      English
      arrow-up
      4
      ·
      27 days ago

      For 1, that’s why you say “Format your answer in this exact sentence: The number of bytes required (rounded up) is exactly # bytes., where # is the number of bytes.” And then regex for that sentence. What could go wrong?

      Also, it can do math somewhat consistently if you let it show its work, but I still wouldn’t rely on it as a cog in code execution. It’s not nearly reliable enough for that.

    • mindaika@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      4
      ·
      26 days ago

      Response:

      Observation 1: ChatGPT is designed to provide context for responses to enhance clarity for human users. Requests for answers without accompanying text may result in inconsistent behavior due to its conversational model. It is not optimized for providing pure data outputs without context.

      Observation 2: ChatGPT is not inherently equipped to perform complex mathematical operations with high reliability. Numerical inaccuracies or rounding errors may occur due to the model’s structure. While capable of basic arithmetic, it is not a specialized tool for precise calculations, particularly in domains like byte math, where accuracy is critical.

      Statement acknowledged.

    • addie@feddit.uk
      link
      fedilink
      arrow-up
      11
      ·
      27 days ago

      True. Although given how easy it is to cast void pointers to the wrong damn thing, it would be nice if you did, makes refactoring much easier. Makes me appreciate std::any all the more.

    • Subverb@lemmy.world
      link
      fedilink
      arrow-up
      5
      ·
      edit-2
      26 days ago

      This isn’t malloc though. I have to assume the cast is because the user has experience with the output from an LLM being untrustworthy.