Despite being primarily trained on text, these models achieved remarkable compression rates on image and audio data.
  • Tibert@compuverse.ukEnglish
    12·
    1 year ago

    Well from the article a dataset is required, but not always the heavier one.

    Tho it doesn’t solve the speed issue, where the llm will take a lot more time to do the compression.

    gzip can compress 1GB of text in less than a minute on a CPU, an LLM with 3.2 million parameters requires an hour to compress

    • rubikcuber@programming.devEnglish
      1·
      1 year ago

      I imagine that the compression is linked to the dataset, so if you update or retrain then you maybe lose access to the compressed data.