By Peter Hesse
Peter Hesse is a partner and EVP at 10Pearls, a global digital transformation company helping businesses innovate, digitalize, and scale.
Lessons from CrowdStrike: Speed vs Safety in Mission-Critical Systems
Blog contents
Cybersecurity bug disrupts travel industry
The news over the last week or two has been dominated by a CrowdStrike bug that took down many Microsoft Windows-based systems early in the morning of July 19, 2024. This had especially wide-ranging impacts in the travel industry, with Delta Airlines facing days of delays and cancellations as a result of the issue.
CrowdStrike has now released their preliminary post-incident report which describes how the error came to pass and how they missed it — including steps they could take in the future to reduce or eliminate similar issues.
What caused the crash
It may be hard to believe, but CrowdStrike admitted in their report a lack of end-to-end testing of their solution. Specifically, CrowdStrike deployed an update that passed an initial test through their content validator. The content validator itself had a bug in it, allowing data through that caused Windows-based computers to crash.
This is actually not uncommon. Software developers can be lulled into a false sense of security when a number of changes or updates are made without having any impact. The changes worked before, so why would anything be any different next time?
By relying only on the unit test through the content validator, and never testing their solution end-to-end, to ensure that the update didn’t impact the end systems after deployment, they failed themselves and their customer base.
The need for rapid response
On the flip side, one could argue that CrowdStrike must release updates as quickly as possible. Consider the scenario of a new “zero-day” strain of ransomware spreading rapidly across the internet, encrypting systems and demanding ransom. As seen in the Change Healthcare and Ascension Health attacks earlier this year, such incidents can be catastrophic, not only affecting organizations but also the patients, citizens, members, and customers who depend on them.
Unlike traditional antivirus solutions that typically receive daily updates, CrowdStrike clients benefit from continuous updates on indicators of compromise. This rapid distribution and kernel-level access enable their solution to respond almost instantaneously to spreading threats.
For those advocating for more extensive testing before deployment, the urgency may become clear when ransomware devastates your entire corporate infrastructure.
Striking the balance between speed and safety
It’s possible to balance the speed of quick updates with the safety of comprehensive testing by embracing automated testing. CrowdStrike, along with other mission-critical systems, must invest in automated testing across their entire deployment spectrum.
-
Accelerated testing
workflows -
Reduced human
error -
Enhanced test
coverage
Test automation can save time, reduce human error, enhance test coverage, and improve testing capabilities. It can also handle batch operations and execute parallel processes simultaneously. This approach should be part of a holistic test strategy encompassing test planning, test case development and execution, goal setting, and reporting.
An automated testing infrastructure that thoroughly tests every part of the process end-to-end builds confidence in the resilience of mission-critical systems. CrowdStrike could implement a checkpoint in their update process where updates are first applied to a series of representative systems and rigorously tested. Properly automated, this system could perform end-to-end testing in minutes. Updates would only be deployed once the process confirms no system failures, unresponsiveness, or performance issues.
How 10Pearls can help
If you’re interested in adding security robustness and automated testing to your mission-critical infrastructure, contact us.
We support DevSecOps and automated testing in mission-critical systems across key organizations like energy & industrials, healthcare, and finance. Let us know if we can help you with an automated testing strategy to build resilience and safety without sacrificing speed.
Self-Healing Test Automation
Related articles
AI/ML
Top AI-Powered Software Testing Companies
Compare the top AI-powered software testing services and learn how automation, self-healing tests, and AI-augmented QA de-risk enterprise releases.
AI/ML
Corporate AI Implementation Failure – Role of Leadership
Most enterprises now share the same models and tools, so why does AI still fail? The gap is leadership: prioritization, ownership, and governance, not better technology.
Mobile app development
In-House vs Outsource Mobile App Development: What’s Best?
Every mobile app starts with one decision: build in-house or outsource. We break down the real costs, tradeoffs, and the questions that determine which model is right for you.
AI/ML
Outcome Based Pricing in Digital Engineering
AI is making outcome-based pricing more viable, but success still depends on clear metrics, shared accountability, and disciplined execution.
AI/ML
The Hidden Risk Behind AI-Powered Software Development
AI coding tools are accelerating development, but code review, governance, and quality assurance aren't keeping pace. Learn how enterprises can scale AI development responsibly.
Software development
Top 10 custom software development companies delivering innovative solutions
This article compares the top 10 custom software development companies for 2025. The rankings are based on the average review rating on Clutch
Software development
Software Development Challenges and Solutions for Modern Enterprises
Discover the top software development challenges enterprises face, legacy systems, security, talent shortages, and proven solutions to overcome them.
Software development
How AI is transforming Agile in 2026
Modern Agile evolves with AI, cloud, and governance, reshaping team workflows and delivery for scalable, secure enterprise software in 2026.
Software development
Explore the tech driving software development innovation
Our top software development technologies in 2026 overview covers AI tools, low code & cloud development tools, programming languages, and databases.
Software development
Top sectors using custom software in Saudi Arabia
Saudi Arabia is quickly shifting away from an oil-based economy for the first time in approximately 8 decades. As Saudi Arabia diversifies, the need for adaptable, sector-specific digital solutions is driving a rise in custom...