Tea@programming.dev to Technology@lemmy.worldEnglish · 10 days agoCloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.blog.cloudflare.comexternal-linkmessage-square133fedilinkarrow-up11.41Kcross-posted to: technik@feddit.orgaboringdystopia@mander.xyztechnology@lemmy.mlnottheonion@lemmy.mltechnology@lemmy.zip
arrow-up11.41Kexternal-linkCloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.blog.cloudflare.comTea@programming.dev to Technology@lemmy.worldEnglish · 10 days agomessage-square133fedilinkcross-posted to: technik@feddit.orgaboringdystopia@mander.xyztechnology@lemmy.mlnottheonion@lemmy.mltechnology@lemmy.zip
minus-squarewingiee@lemm.eelinkfedilinkEnglisharrow-up19·10 days agoI assume a crawler which adheres to robots.txt
minus-squarelily33@lemm.eelinkfedilinkEnglisharrow-up5·10 days agoI would love to think so. But the word “verified” suggests more.
minus-squarekilleronthecorner@lemmy.worldlinkfedilinkEnglisharrow-up2·10 days agoIP verification is a not uncommon method for commercial crawlers
I assume a crawler which adheres to robots.txt
I would love to think so. But the word “verified” suggests more.
IP verification is a not uncommon method for commercial crawlers