Maximizing shareholder value by using tried-and-true industry-standard systems services is going just great.
Let's see if "nobody ever got fired for choosing Windows" still holds a week from now.
Crowd Strike thing is basically an "Ever Given stuck in Suez Canal" of IT industry.
All the techies losing hair, sleep, and family time trying to get this un-stuck are the excavator operator trying to get things un-fucked.
Cannot wait for the first tech media galaxy-brained piece that finds a way to blame this on "hackers", somehow.
Because obviously: computer go bad? Hackers!
This kind of failure is *systemic*, but of course it will get blamed on some lowly techie somewhere whose name is on the commit message.
> It was all Steve.
> We have now fired Steve, thus solving the problem once and for all.
> Bonuses to all management for a job well done!
Yet another example why techies might want to consider unionizing.
Crowdstrike:
> The fix is to delete C-00000291*.sys
Google:
> quick, we need to call ICANN and get .sys gTLD registered, stat!
"The buck stops with me!" – tech CEO says, head held high, pocketing the profit while firing Steve the intern, whose name happened to be on the commit message.
A gentle reminder that very recently the broader FLOSS community avoided a potentially massively problematic security issue with an attempted sneaky xz back-door affecting OpenSSH, because a Microsoft developer was curious about a weird slowdown.
I mean, screw Big Tech and all that ecosystem, but we might want to take this opportunity to consider our own potential failure modes.
CrowdStrike is a small, local, struggling, resource-constrained mom-and-pop infosec shop which should not be regulated because that would kill it, and also is a globally-recognized security vendor of advanced AI-based EDR tools that you should definitely use because the company is massive and has all the resources in the world that they can put to making their tools top-of-the-line, not like those FLOSS amateurs.
Obviously.
So, does anyone have any reasonably reliable info on what actually happened?
Hearing things from "the CDN CrowdStrike uses done goofed" to "someone at CrowdStrike decided to push a quickfix outside of the standard testing-staging-QA pipeline."
I'm sure I am not the only one who would really appreciate something solid on this.
Also, how did this get installed on the end systems? Aren't CrowdStrike's updates signed?
@rysiek From what I've gathered so far from the ever reliable Social Media Feeds™ it appears that it wasn't a driver update, but rather a update data file that then caused the unchanged driver to crash when it tried to parse and apply it, which would effectively bypass the signing requirement.
@rysiek Well, you linked to someone who claimed it wasn't. How trustworthy that report is? You tell me! 😁
@dos I don't know, which is why I am asking for further confirmation.
> wouldn't make any difference if they were as that just validates what you sent is what you got.
I have seen things that suggest the CDN that CrowdStrike uses b0rked the files. I have also seen people mentioning that on different b0rked systems the files that were installed as part of the b0rked update were all *different* garbage.
If these are true, signing and verifying these signatures could have prevented both of these.
Hence me asking about signing.
@dos right. But one would assume that the update was at least signed by CrowdStrike?