All our servers and company laptops went down at pretty much the same time. Laptops have been bootlooping to blue screen of death. It’s all very exciting, personally, as someone not responsible for fixing it.

Apparently caused by a bad CrowdStrike update.

Edit: now being told we (who almost all generally work from home) need to come into the office Monday as they can only apply the fix in-person. We’ll see if that changes over the weekend…

  • foremanguy
    link
    fedilink
    English
    71 year ago

    Bahaha 😂😂 continue using proprietary software, that’s all you are going to get in addition to privacy issues… Switch to Linux.

      • @[email protected]
        link
        fedilink
        English
        241 year ago

        OK, but people aren’t running Crowdstrike OS. They’re running Microsoft Windows.

        I think that some responsibility should lie with Microsoft - to create an OS that

        1. Recovers gracefully from third party code that bugs out
        2. Doesn’t allow third party software updates to break boot

        I get that there can be unforeseeable bugs, I’m a programmer of over two decades myself. But there are also steps you can take to strengthen your code, and as a Windows user it feels more like their resources are focused on random new shit no one wants instead of on the core stability and reliability of the system.

        It seems to be like third party updates have a lot of control/influence over the OS and that’s all well and good, but the equivalent of a “Try and Catch” is what they needed here and yet nothing seems to be in place. The OS just boot loops.

        • @[email protected]
          link
          fedilink
          English
          31 year ago

          banks wouldn’t use something that black box. just trust me bro wouldn’t be a good pitch

        • @[email protected]
          link
          fedilink
          English
          8
          edit-2
          1 year ago

          It’s not just Windows, it’s affecting services that people that primarily use other OS’s rely on, like Outlook or Federated login.

          In these situations, blame isn’t a thing, because everyone knows that a LSE can happen to anyone at any time. The second you start to throw stones, people will throw them back when something inevitably goes wrong.

          While I do fundamentally agree with you, and believe that the correct outcome should be “how do we improve things so that this never happens again”, it’s hard to attach blame to Microsoft when they’re the ones that have to triage and ensure that communication is met.

          • @[email protected]
            link
            fedilink
            English
            4
            edit-2
            1 year ago

            I reckon it’s hard to attach blame to Microsoft because of the culture of corporate governance and how decisions are made (without experts).

            Tech has become a bunch of walled gardens with absolute secrecy over minor nothings. After 1-2 decades of that, we have a generation of professionals who have no idea how anything works and need to sign up for $5 a month phone app / cloud services just to do basic stuff they could normally do on their own on a PC - they just don’t know how or how to put the pieces together due to inexperience / lack of exposure.

            Whether it’s corporate or government leadership, the lack of understanding of basics in tech is now a liability. It’s allowed corporations like Microsoft to set their own quality standards without any outside regulation while they are entrusted with vital infrastructure and to provide technical advisory, even though they have a clear vested interest there.

        • @[email protected]
          link
          fedilink
          English
          151 year ago

          AFAICT Microsoft is busy placing ads on everything and screen logging user activity instead of making a resilient foundation.

          For contrast: I’ve been running Fedora Atomic. I’m sure it is possible to add some kernel mod that completely breaks the system. But if there was a crash on boot, in most situations, I’d be able to roll back to the last working version of everything.

  • @[email protected]
    link
    fedilink
    English
    711 year ago

    I’m so exhausted… This is madness. As a Linux user I’ve busy all day telling people with bricked PCs that Linux is better but there are just so many. It never ends. I think this is outage is going to keep me busy all weekend.

  • @[email protected]
    link
    fedilink
    English
    201 year ago

    Irrelevant but I keep reading “crowd strike” as “counter strike” and it’s really messing with me

  • umami_wasabi
    link
    fedilink
    English
    191 year ago

    No one bother to test before deploying to all machines? Nice move.

    • @[email protected]
      link
      fedilink
      English
      21
      edit-2
      1 year ago

      This outage is probably costing a significant portion of Crowd strike’s market cap. They’re an 80 billion dollar company but this is a multibillion outage.

      Someone’s getting fired for this. Massive process failures like this means that it should be some high level managers or the CTO going out.

        • @[email protected]
          link
          fedilink
          English
          9
          edit-2
          1 year ago

          They’re already down ~9% today:

          https://finance.yahoo.com/quote/CRWD/

          So I think you’re late to the party for puts. Smart money IMO is on a call for a rebound at this point. Perhaps smarter money is looking through companies that may have been overlooked that would be CrowdStrike customers and putting puts on them. The obvious players are airlines, but there could be a ton of smaller cap stocks that outsource their IT to them, like regional trains and whatnot.

          Regardless, I don’t gamble w/ options, so I’m staying out. I could probably find a deal, but I have a day job to get to with nearly 100% odds of getting paid.

            • @[email protected]
              link
              fedilink
              English
              2
              edit-2
              1 year ago

              Nice. The first comment is basically saying, “they’re best in class, so they’re worth the premium.” And then the general, “you’ll probably do better by doing the opposite of /r/wallstreetbets” wisdom.

              So yeah, if I wanted to gamble, I’d be buying calls for a week or so out when everyone realizes that the recovery was relatively quick and CrowdStrike is still best in class and retained its customers. I think that’s the most likely result here. Switching is expensive for companies like this, and the alternatives aren’t nearly as good.

            • @[email protected]
              link
              fedilink
              English
              2
              edit-2
              1 year ago

              52 Week Range 140.52 - 398.33

              They’re about where they were back in early June. If they weather this, I don’t see a reason why they wouldn’t jump back to their all-time high in late June. This isn’t a fundamental problem with the solution, it’s a hiccup that, if they can recover quickly, will be just a blip like there was in early June.

              I think it’ll get hammered a little more today, and if the response looks good over the weekend, we could see a bump next week. It all depends on how they handle this fiasco this weekend.

  • YTG123
    link
    fedilink
    English
    1691 year ago

    >Make a kernel-level antivirus
    >Make it proprietary
    >Don’t test updates… for some reason??

  • Encrypt-Keeper
    link
    fedilink
    English
    1151 year ago

    Yeah my plans of going to sleep last night were thoroughly dashed as every single windows server across every datacenter I manage between two countries all cried out at the same time lmao

  • @[email protected]
    link
    fedilink
    English
    12
    edit-2
    1 year ago

    A lot of people I work with were affected, I wasn’t one of them. I had assumed it was because I put my machine to sleep yesterday (and every other day this week) and just woke it up after booting it. I assumed it was an on startup thing and that’s why I didn’t have it.

    Our IT provider already broke EVERYTHING earlier this month when they remote installed" Nexthink Collector" which forced a 30+ minute CHKDSK on every boot for EVERYONE, until they rolled out a fix (which they were at least able to do remotely), and I didn’t want to have to deal with that the week before I go in leave.

    But it sounds like it even happened to running systems so now I don’t know why I wasn’t affected, unless it’s a windows 10 only thing?

    Our IT have had some grief lately, but at least they specified Intel 12th gen on our latest CAD machines, rather than 13th or 14th, so they’ve got at least one win.