LLM scrapers are taking down FOSS projects’ infrastructure, and it’s getting worse.

  • sudo@programming.dev
    link
    fedilink
    arrow-up
    3
    ·
    18 hours ago

    Proof of Work is a terrible solution because it assumes computational costs are significant expense for scrapers compared to proxy costs. It’ll never come close to costing the same as residential proxies and meanwhile every smartphone user will be complaining about your website draining their battery.

    You can do something like only challenge data data center IPs but you’ll have to do better than Proof-of-Work. Canvas fingerprinting would work.

    • refalo@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      3 hours ago

      Proof of Work is a terrible solution

      Hard disagree, because:

      it assumes computational costs are significant expense for scrapers compared to proxy costs

      The assumption is correct. PoW has been proven to significantly reduce bot traffic… meanwhile the mere existence of residential proxies has exploded the availability of easy bot campaigns.

      Canvas fingerprinting would work.

      Demonstrably false… people already do this with abysmal results. Need to visit a clownflare site? Endless captcha loops. No thanks