In my various news feeds, I find an intriguing article by Jason Hanson titled “Spy Secrets That Can Save Your Life” Given certain life experiences, this caught my attention. It is a quick read, and I recommend it.
For MY readers, and this missive, we will examine the first section, “Situational awareness is the basic first step that enables you to execute all other survival skills effectively.“
Hanson lays out four conditions of situational awareness, White, Yellow, Orange, and Red, each corresponding to a state of alertness. I will review each of these and place them in the context of I.T. Operations.
This is a state of total obliviousness and is where most Ops and DevOps live. Heads down on a project, rushing to meet a deadline, worrying about deployments, and unaware of anomalies occurring around them.
- Servers that seem to have no owner
- Databases that appear
- Logins from strange IP addresses
- Unusual Git Commits
- Off-hours events (Weekends, Holidays.)
- Unusual DNS queries
- Unusual Traffic spikes
All of the above are indications of reconnaissance activities, actual compromise, or data exfiltration.
Under the Condition White “Rules of Engagement,” these are missed, or if noticed, ignored.
This is NOT the place to be.
Is calm alertness. It is not a state of hypervigilance that tries to anticipate a threat at every turn. It is rather an ongoing data collection process that takes stock of people and surroundings. It’s living with your eyes open.
In the world of IT operations we have a few things going on:
- Operational baselines have been established.
- Logins; success and failure are logged.
- User creation and deletions are logged.
- High-value events are noted and alerted.
- Anomalous events are flagged alerted and investigated.
Operations and security groups should strive to live here
A state of alert where a series of anomalous events have occurred. Hypervigilance kicks in, and as such, all events get a closer look:
- Administrative or power user logins are confirmed via a backchannel.
- New resource allocations are reviewed.
- Traffic patterns are examined.
- Logging, monitoring, and alerting are turned up a notch.
- Anything that does not “Smell Right”, gets examined and verified.
Condition Orange is very tiring on administrators, paranoia will run quite high. Try to stay out of the orange.
This is a state of crisis and confrontation. You are ready to fight. Your tactical team was at the ready as of Condition Orange, but now you’re calling them in.
- All logins are examined and verified.
- Unnecessary operations are halted.
- Unsuccessful logins are banned with little mercy.
- Unusual events are investigated and persecuted.
- Network and security modifications are communicated and agreed upon by the tactical team.
- Access credentials are rotated and communicated by a secure channel.
- Questionable resources are restored from known good backups.
- Anomalous communications are terminated with prejudice.
- A total perimeter check is conducted
In this condition, you are reacting to threats, not proactively remediating them. You HAVE LOST CONTROL of your assets. This truly is not a place to be.
The takeaway, “The more time spent in a state of situational awareness, the less you will spend in fighting fires.”