You ran a purple team exercise last week. Your red team executed a flawless attack chain: initial access via phishing, lateral movement via Kerberos relay, privilege escalation, and data exfiltration. Picture perfect.
Then you got the debrief: your EDR caught nothing. Your SIEM was silent. Your firewalls didn't alert. Your detection engineers looked at you and said, "Our tools are misconfigured."
They're wrong. And you probably are too.
The problem isn't your tools. It's that your red team tested an adversary that doesn't exist in your environment.
The Fundamental Mismatch
Here's what I see happen in most purple team programmes:
- Red team is given a playbook. Usually MITRE ATT&CK or some vendor-blessed "attack pattern." Test everything: initial access, lateral movement, persistence, exfil. Be creative.
- Blue team sits and watches. They monitor alerts. When nothing fires, they assume detection is broken. They tune. They add rules. They adjust thresholds.
- Repeat. Next quarter, red team runs the same tests. Still nothing fires (or things fire randomly, drowning in false positives). Conclusion: "We need better tooling."
Here's what's actually happening: You're testing threat capabilities your environment doesn't support.
An Example
Let's say you're a mid-sized financial services firm. Your threat model is: targeted APTs (FIN7, Lazarus), occasional script kiddies, insider threats.
Your environment: Windows 10/11 workstations, hardened build. Modern domain controllers with Kerberos hardening. Network segmentation: finance backend on isolated VLAN. EDR deployed globally. No plaintext credentials stored on endpoints. Conditional Access enforces MFA.
Your red team shows up and runs Mimikatz, tries Kerberos relay attacks, attempts lateral movement across network segments, and exfiltrates via unencrypted channels.
Result: EDR blocks the binaries. Lateral movement fails (segmentation). Nothing detectable because nothing succeeds.
Your blue team concludes: "EDR is misconfigured. We need better visibility into Kerberos traffic."
Wrong. Those attacks don't fit your threat model. An actual FIN7 intrusion into your environment would:
- Use living-off-the-land techniques (no Mimikatz)
- Move laterally via legitimate admin tools (RDP, WinRM) already allowed in your firewall rules
- Steal credentials from memory over time, not via flashy in-memory attacks
- Exfil via HTTP/HTTPS, not SMB or DNS tunneling
So your EDR should miss those attacks, because they're not your risk. Your tools are configured for your actual adversary, not the MITRE ATT&CK checklist.
The Real Cost of Environment-Agnostic Testing
When you run tests that don't align with your threat model:
- You get false confidence. Red team "successfully" executes techniques. You feel like you're being tested thoroughly. You're not.
- You waste blue team time. Detection engineers spend weeks building rules for attacks that will never happen to you.
- You create alert fatigue. All those rules for non-existent threats? They generate false positives. Your SOC tunes them down or disables them. Then an actual threat uses a legitimate technique you didn't monitor.
- You fund vendor marketing. Tool vendors love environment-agnostic testing. It justifies upgrading to the newest EDR feature. None of which you needed if your testing was threat-model-driven.
What Actually Works
Stop testing "everything." Start testing what you're actually at risk from.
Step 1: Define your threat model. Ask yourself: Who targets companies like mine? What are they after? What access do they typically start with? What's their environment awareness? Write this down. Three paragraphs, maximum.
Step 2: Map your environment against that threat. What attack paths does your threat model realistically have access to? What techniques are actually possible? What detection gaps exist on those paths?
Step 3: Test only the relevant paths. Your red team doesn't run the MITRE ATT&CK playbook. They run your threat playbook.
If FIN7 uses spear-phishing and living-off-the-land lateral movement, test that. If they exfil via HTTPS, test that — and tune your detection to spot anomalous HTTPS traffic from workstations. If they're blocked by your segmentation, celebrate the segmentation and move to the next gap.
The Honest Conversation
"Your tools are configured correctly for the threats you actually face. When red team tests succeed using techniques that wouldn't work in a real intrusion, that's not a detection failure — that's a sign your test wasn't threat-driven. Let's refocus on the adversaries who can actually hurt us."
This conversation is uncomfortable because it implies your purple team programme might be theatre. But it's the one that matters.
What's Next
For your next purple team exercise:
- Spend time defining threat model, not technique selection
- Have blue team validate which attack paths are realistic in your environment
- Red team tests those paths, trying to evade detection on actual adversary techniques
- Blue team tunes for those specific gaps
The tests will be less flashy. You won't check every MITRE ATT&CK technique. But when red team succeeds, it means something: you've found a real gap against a real threat. That's worth a lot more than catching Mimikatz on a hardened system that would never run it in the first place.
Next: How to actually scope a red team exercise to your threat model. Plus: Real detection gaps found in a financial services engagement.