All our servers and company laptops went down at pretty much the same time. Laptops have been bootlooping to blue screen of death. It’s all very exciting, personally, as someone not responsible for fixing it.

Apparently caused by a bad CrowdStrike update.

Edit: now being told we (who almost all generally work from home) need to come into the office Monday as they can only apply the fix in-person. We’ll see if that changes over the weekend…

  • tibi@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    ·
    4 months ago

    Crowdstrike is not a monopoly. The problem here was having a single point of failure, using a piece of software that can access the kernel and autoupdate running on every machine in the organization.

    At the very least, you should stagger updates. Any change done to a business critical server should be validated first. Automatic updates are a bad idea.

    Obviously, crowdstrike messed up, but so did IT departments in every organization that allowed this to happen.

    • h0rnman@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      8
      ·
      4 months ago

      You wildly underestimate most corporate IT security’s obsession with pushing updates to products like this as soon as they release. They also often have the power to make such nonsense the law of the land, regardless of what best practices dictate. Maybe this incident will shed some light on how bad of an idea auto updates are and get C-levels to do something about it, but even if they do, it’ll only last until the next time someone gets compromised by a flaw that was fixed in a dot-release

    • Excrubulent@slrpnk.net
      link
      fedilink
      English
      arrow-up
      1
      ·
      4 months ago

      Monopolies aren’t absolute, ever, but having nearly 25% market share is a problem, and is a sign of an oligopoly. Crowdstrike has outsized power and has posted article after article boasting of its dominant market position for many years running.

      I think monopoly-like conditions have become so normalised that people don’t even recognise them for what they are.