Has kbin defederated or is federation bugging? - Lemmy
lemmy.ml
I’m asking since I can’t seem to access some kbin communities on lemmy.ml [http://lemmy.ml] but they are available from other instances. I tested this a few days ago so things might have changed. I also have the same question for fedia.io [http://fedia.io]. I can see it from other instances but not lemmy.ml [http://lemmy.ml]. Are there issues? I’m asking since there was no response to previous posts.

Apparently one of the lemmy.ml admins was overzealous in banning all User-Agent strings that contained the string “bot”. Bans were entered for all of the individual strings containing that word which were observed in their webserver logs, which impacted kbin’s reported agent of “kbinBot”.

The issue has been fixed, and I observed that one of my kbin posts to a lemmy.ml community was successfully pushed to the original instance.

  • blightbow@kbin.socialOP
    1·
    1 year ago

    There are plenty of high volume, non-malicious bots that do. robots.txt is a thing for a reason, and we can see here that lemmy.ml has implemented it. Not all bots that ignore robots.txt are malicious though, just poorly designed. You can basically lump them into three categories:

    • Well-behaved bots that announce that they’re bots in the User-Agent header and obey robots.txt (note that they may still slam the server even if they obey it)
    • Mediocre bots that announce that they’re bots in the User-Agent header but ignore robots.txt (or vice versa)
    • Bad bots (malicious or otherwise) that announce their User-Agent as other things, often pretending to be other software.

    Their logs told them they had a lot of traffic from stuff identifying itself as bots. Throwing that traffic out wouldn’t break lemmy but would help them deal with the capacity problems that all of the mainstream lemmy/kbin instances had to deal with shortly after the Reddit exodus began. They fucked up and tagged kbin in the process, which definitely would have been one of the highest volume ActivityPub consumers matching their criteria.