WorkingLemmy@lemmy.world to Open Source@lemmy.ml · 2 months agoFOSS infrastructure is under attack by AI companiesthelibre.newsexternal-linkmessage-square22linkfedilinkarrow-up1371cross-posted to: technology@beehaw.orglinux@programming.devopensource@jlai.lutechnology@lemmy.worldOpenSource@europe.pubtechnology@lemmy.world
arrow-up1371external-linkFOSS infrastructure is under attack by AI companiesthelibre.newsWorkingLemmy@lemmy.world to Open Source@lemmy.ml · 2 months agomessage-square22linkfedilinkcross-posted to: technology@beehaw.orglinux@programming.devopensource@jlai.lutechnology@lemmy.worldOpenSource@europe.pubtechnology@lemmy.world
minus-squareStrawberrylinkfedilinkarrow-up5·2 months agoThe bots scrape costly endpoints like the entire edit histories of every page on a wiki. You can’t always just cache every possible generated page at the same time.
minus-squarejagged_circle@feddit.nllinkfedilinkEnglisharrow-up1·edit-22 months agoOf course you can. This is why people use CDNs. Put the entire site on a CDN with a cache of 24 hours for unauthenticated users.
The bots scrape costly endpoints like the entire edit histories of every page on a wiki. You can’t always just cache every possible generated page at the same time.
Of course you can. This is why people use CDNs.
Put the entire site on a CDN with a cache of 24 hours for unauthenticated users.