Botched CrowdStrike security update breaks Windows worldwide, causing BSOD and crashes

Steve C · Jul 19, 2024

What an epic software engineering failure and a management failure in many organisations not to have reversionary methods of operation.

RD Mercer · Jul 19, 2024

Can't wait to see what CrowdStrike does with all of the electronic voting machines on election day here. I wonder is this a test run for the "UPDATE” they’re going to install on the voting machines this November?

5

Senecio · Jul 19, 2024

flhthemi said:
This is why I wish there was a simple way, like there used to be, to turn off automatic updates. I know, you can pause them, but I don't want them until I see they're not problematic.

But would you really want to turn off automatic updates for your antivirus/anti ransomware software?

I'm sure that conscientious network managers who look after thousands of machines might want to check each and every update, but I wonder if their CEOs would consider that a cost-effective use of staff time.

Brink · Jul 19, 2024

How to fix CrowdStrike BSOD issue on Windows PCs [Update]

We now have the official fix from CrowdStrike on how to deal with the Falcon Sensor for Windows update error which caused a critical issue that is currently affecting tons of PCs worldwide.

www.neowin.net

Winuser · Jul 19, 2024

TraderGary said:
I was live, online, with Interactive Brokers Trader Workstation from 8:30 AM to 5:00 PM EDT. I was also active online with Microsoft 365 and OneDrive throughout that time. I did not experience any problems.

The only thing I lost last night was access to my OneDrive account.

thecaretaker · Jul 19, 2024

Brink said:
How to fix CrowdStrike BSOD issue on Windows PCs [Update]

We now have the official fix from CrowdStrike on how to deal with the Falcon Sensor for Windows update error which caused a critical issue that is currently affecting tons of PCs worldwide.

www.neowin.net

At least that is more constructive than Microsoft's fix. They suggested tuning it off and on again 15 times. Microsoft Azure Status

That's got to be stalling tactics :lmao:

pparks1 · Jul 19, 2024

antspants said:
They immediately took responsibility. They immediately set out to rectify it, all in the same day and first few hours of the incident.
If anyone needs to be held responsible for anything debilitating it should really be company and services system administrators for not having backup systems and processes in place.
Just my opinion of course.

People in my organization have been up for past 10 hours dealing with fallout from this debacle. While there are backups, and policies and procedures, it's a huge impact on the business and we are still on the calls and working through the problems. Fortunately, our non-windows portions of our environment were not impacted.....but this still hit between 700 and 1000 servers, and countless workstation from end-users. It certainly wasn't the quiet Friday most people were hoping for.

RD Mercer · Jul 19, 2024

pparks1 said:
People in my organization have been up for past 10 hours dealing with fallout from this debacle. While there are backups, and policies and procedures, it's a huge impact on the business and we are still on the calls and working through the problems. Fortunately, our non-windows portions of our environment were not impacted.....but this still hit between 700 and 1000 servers, and countless workstation from end-users. It certainly wasn't the quiet Friday most people were hoping for.

Wow! that's terrible, hopefully this issue is rectified soon.

kelper · Jul 19, 2024

A relative spent six hours at London Gatwick and was then told to go home after half the passengers had boarded. But for many of us in the UK it has not had any impact. The media has exaggerated again.

Cliff S · Jul 19, 2024

https://twitter.com/x/status/1814227312721768805

Almighty1 · Jul 19, 2024

Silvio said:
Only Windows 10 machines with CrowdStrike installed (typically Business owned), are at risk.

skype.com was down too and showed nothing but a string of characters for a few hours when this thread was started.

Almighty1 · Jul 19, 2024

neves said:
CrowdStrike might really take a hit cause of this. To bad - since it's been deemed a top security solution in the corporate world - most popular recommendation (it's what I've been recommending as well...). Even Intel went for Crowdstrike (tho, maybe not paired with Windows - so they're probably safe).

They might or the company might be worth investing in as it shows how much everyone depended on them.

Cromax · Jul 19, 2024

When i see stuff like this i laugh at notion that ai will drive me maybe somebody else will but not me

glasskuter · Jul 19, 2024

While this is all over my head, what I take from this is this shows the weakness of companies worldwide because they have put all their eggs in one basket by using total end-point protection provided by only one company and nothing but Windows on their computers. Though Crowd Strike service is and has been exemplary, stuff happens.... and it did. End users have to be prepared for it when it does.
I don't claim to understand extensive corporate IT. I'm just a simple person who uses basic thinking. Even I have 2 computers sitting on my desk, one which dual boots Linux so all my stock is not in Windows. Looks to me like essential services like 911 centers or hospitals or airports would have some sort of contingency system in place, however basic that system might be, rather than leaving themselves open to sit wringing their hands when all their computers go down at one time.

pparks1 · Jul 19, 2024

kelper said:
A relative spent six hours at London Gatwick and was then told to go home after half the passengers had boarded. But for many of us in the UK it has not had any impact. The media has exaggerated again.

LOL, this is not an exaggeration for those of us dealing with this. This is legit a substantial debacle.

kelper · Jul 19, 2024

The banks were not affected because they don't roll out untested updates. Major companies will be asking their IT gurus why they didn't use similar precautions. This will sort the men from the boys.

pparks1 · Jul 19, 2024

glasskuter said:
While this is all over my head, what I take from this is this shows the weakness of companies worldwide because they have put all their eggs in one basket by using total end-point protection provided by only one company and nothing but Windows on their computers. Though Crowd Strike service is and has been exemplary, stuff happens.... and it did. End users have to be prepared for it when it does.

We are talking about massive enterprise IT solutions. It's not really possible to build one system that let's say runs on Microsoft SQL server, uses a 3rd party vendor application, and runs on Windows servers...and then just for redundancy and emergency fallback have another implementation that runs on MySQL, with another 3rd party app that runs on Linux that would be ready to go in the event of a catastrophic outage like we are seeing. While it may seem logical from a common sense standpoint, it's not feasible to actually try to implement.

We have a lot of stuff that stayed up, that doesn't run on Windows. But we have a lot of stuff that did run on Windows, and lots of time and effort has been dedicated to mitigating the issue and getting back online. The biggest issue in our case now, is thousands of employee workstations that may not be booting successfully, stuck in bitlocker recovery, having to to through a manual procedure to work around the issue, who work remote. It's a slow go, and we are working through it. But these employees certainly don't have a Windows laptop and a Mac laptop just in case something goes wrong with their Windows machine. It's not cost effective or feasible to support.

mecanicogolf · Jul 19, 2024

My toilet still won't flush, but I guess it's got nothing to do with this.

Edit: Just found out that MS tried to update my toilet float. When they did, the float sank to the bottom.

Trying to get tickets at the airport is nothing compared to my float not floating. There is nothing worse than non floating float. And if it doesn't float, it is not.

:lmao:

pparks1 · Jul 19, 2024

kelper said:
The banks were not affected because they don't roll out untested updates. Major companies will be asking their IT gurus why they didn't use similar precautions. This will sort the men from the boys.

yeah, these types of scenarios are always in a "grey" area. We have strict change control procedures and everything is scrutinized, discussed, reviewed and if anything goes wrong with anything, it's always discussed and we learn from our mistakes.

However, with endpoint protection systems, updates can and often do protect against 0 day exploits and such which puts your business at risk. One has to weigh the potential risk of not deploying something while you test internally and slowly roll out and then get clobbered with a 0 day exploit to the machines/systems that don't yet have the update.

In this particular case, it's a matter of blue screening a system...which is probably a more "acceptable" risk than say letting all of your customer data get compromised while "thoroughly testing" an update because a 0 day exploit ran wild on your network.

snerd · Jul 19, 2024

glasskuter said:
While this is all over my head, what I take from this is this shows the weakness of companies worldwide because they have put all their eggs in one basket by using total end-point protection provided by only one company and nothing but Windows on their computers. Though Crowd Strike service is and has been exemplary, stuff happens.... and it did. End users have to be prepared for it when it does.
I don't claim to understand extensive corporate IT. I'm just a simple person who uses basic thinking. Even I have 2 computers sitting on my desk, one which dual boots Linux so all my stock is not in Windows. Looks to me like essential services like 911 centers or hospitals or airports would have some sort of contingency system in place, however basic that system might be, rather than leaving themselves open to sit wringing their hands when all their computers go down at one time.

Yep! Centralization is a killer! Corporations have hijacked the entire internet and tied most of it to centralized servers.

Guess what service was not affected? Bitcoin!!

Botched CrowdStrike security update breaks Windows worldwide, causing BSOD and crashes

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computers

Administrator

My Computers

Well-known member

My Computers

Well-known member

My Computer

Well-known member

My Computers

Well-known member

My Computer

Well-known member

My Computers

ꓠꓳꓪꓱꓷ ꓶꓯꓕꓲꓨꓲꓷ ꓘꓤꓯꓷ

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

aka Mama Glass

My Computers

Well-known member

My Computers

Well-known member

My Computers

Well-known member

My Computers

Well-known member

My Computer

Well-known member

My Computers

Well-known member

My Computer

Similar threads