Are Your Third-Party SOC Providers Actually Doing Their Job?

Since joining Spartans Security, I've noticed a disturbing trend: a significant number of the penetration tests we've conducted reveal that third-party SOC (Security Operations Centre) providers aren't flagging or alerting on critical security events. These aren't subtle or advanced attacks, but clear, malicious activities that should trigger immediate SOC responses. Yet, in too many cases, our clients are left vulnerable, unaware of the lurking threats because their SOC vendors failed to detect them.

In today's complex cyber landscape, organisations rely heavily on outsourced SOC services to keep a vigilant eye on their environments. These providers claim to offer 24/7 monitoring, threat detection, and incident response. However, how often are these promises validated? And more importantly, how can you, as a customer, ensure that these claims are not just sales pitches but actual performance?

The Disconnect Between Vendor Promises and Reality

Many organisations outsource their security operations to save costs, expecting SOC providers to actively monitor and respond to threats. But our penetration tests tell a different story. In some instances, no alerts were generated for simulated attacks that should have triggered a high-priority response.

This isn't just an oversight—it's a significant failure of one of the most critical security layers.

What’s causing this gap? Here are a few potential reasons:

1. Overreliance on Automation: Many SOC providers rely on automated tools, and while automation can be powerful, it lacks the nuance to detect more complex attacks.
2. Poor Tuning and Configuration: Out-of-the-box SOC solutions often aren't tailored to a specific client's environment, leading to missed detections. Sometimes even no detections at all.
3. Overconfidence in Threat Intelligence Feeds: Threat intel feeds can only be as good as their updates, and in fast-evolving threat landscapes, lagging or irrelevant feeds can create blind spots.
4. Lack of Active Engagement: Many SOC services focus more on reporting metrics than actively hunting for threats in a client’s environment.

Testing SOC Effectiveness: The Role of Atomic Testing

Atomic testing is an effective method to evaluate whether SOC services are catching what they should. These are controlled, simple test attacks, or simulations of specific techniques that an adversary might use, derived from frameworks like MITRE ATT&CK. The goal of atomic tests is to isolate and trigger certain detection capabilities of a SOC provider, allowing an organization to see which behaviours are being missed and which are properly flagged.

By conducting these tests, you can:

• Validate SOC Claims: Does the SOC really detect lateral movement or malicious PowerShell executions as claimed?
• Identify Gaps: Which attack techniques are going unnoticed in your environment?
• Tailor Detection Rules: Ensure that your SOC provider adjusts detection rules to meet the specific risks faced by your organization.

Steps to Validate Your SOC Providers

1. Run Tests with Atomic Attacks: Simulate real-world attack scenarios using atomic techniques and observe whether they trigger alerts in your SOC’s monitoring system.
2. Red Team/Blue Team Exercises: Collaborate with your SOC provider in coordinated exercises where a red team simulates attacks, and the blue team defends. This real-time feedback loop exposes weaknesses in detection and response.
3. Leverage Threat Hunting: Test the SOC’s active threat-hunting capabilities by deploying unknown attack patterns in your network and evaluating how well they can identify and mitigate them.
4. Regularly Test, Not Just Once: Security testing isn’t a one-time event. Continually assess your SOC provider with regular tests, and push for improvements in their performance.
5. Hold Your SOC Providers Accountable: Set clear SLAs (Service Level Agreements) for detection and response times, and ensure that vendors meet them.

Conclusion: Don’t Assume, Test

Relying on third-party SOC providers without validation is a gamble. The disturbing trend of missed alerts during our penetration tests has proven that trust isn’t enough—verification is essential. Atomic testing and ongoing validation should be standard practice for any organisation relying on outsourced SOC services.

The next time your SOC vendor promises to keep your environment secure, ask yourself—how do you know they will? Test, validate, and hold them accountable.