Reading an alert at 3am: what good triage actually looks like
The difference between a mature SOC and an expensive alert forwarder shows up in the first four minutes of an incident. Here is the sequence.
SOC console, live detection view
Real console screenshot with a genuine timeline and alert queue, identifiers removed. Do not fabricate alert volumes.
2400×1350 · PNG · @2x
An alert fires at 03:12. Impossible travel: the same account authenticated from Richmond and from a hosting provider in another country eleven minutes apart. A tool can tell you that. What happens next is entirely about people and process.
Minute one — is this real?
Impossible travel is one of the noisiest detections in existence. A VPN, a corporate proxy, a mobile carrier routing through another region, a scheduled task on a cloud VM using a service account. Before anything else, the analyst establishes whether the second location is explainable. If the "impossible" leg comes from your own egress range, the alert is a tuning problem, not an incident.
Minute two — what did the session do?
This is where most alert-forwarding services stop and most real investigations begin. A login is not an incident. A login followed by an inbox rule that forwards everything to an external address and deletes the sent copy is an incident, and the two events live in different logs.
- New inbox or mail-flow rules, especially forwarding or auto-delete
- MFA method added, changed, or registered from the new location
- OAuth application consent granted
- Bulk file access or download against document storage
- Any privilege change on the account or on others
A login is not an incident. A login plus a new forwarding rule is a breach in progress with a fifteen-minute head start.
Minute three — scope before you act
The instinct is to disable the account immediately. Sometimes that is right. But an attacker who loses access abruptly knows they have been seen, and if they have established a second foothold you have just told them to use it. Two minutes establishing whether this account is the only one affected is usually two minutes well spent — and it is a judgement call that requires an analyst, not a playbook branch.
Minute four — contain, then wake someone
Revoke sessions and refresh tokens, not just the password — a password reset alone leaves an existing token valid until it expires. Remove the malicious rules and record them first. Then make the call. At 3am, waking the right person for a genuine account compromise is correct, and a SOC that will not make that call to avoid annoying anyone is not providing the service you bought.
What happens the next morning
The part that separates a service from a subscription: the detection gets tuned. If the alert was noise, the rule is adjusted so the same false positive does not fire again. If it was real, the gap that allowed it — no conditional access on that account, no token protection, no alert on OAuth consent — becomes a written recommendation with an owner. An incident you do not learn from is an incident you have agreed to have again.