I woke up and couldn’t get back to sleep, so I just read a good post by Richard Bejtlich on his site TaoSecurity.blogspot.com.

In his post, Richard talks about the investigative process he follows when he sees an alert on his system. While it is a thorough and very successful process, he’s right to assume that there will be someone who says ‘that takes too long’ because it does. I see the following problems with such a manual and in-depth investigation, at least on a relatively insignificant alert (two syn packets to a non-critical port that nothing is listening on):

1. With such an intense process, its human nature that it won’t be followed exactly the same every time. All people are, at some point and to some degree lazy. The easier and quicker the task, the more likely we are to do it.
2. Time spent on insignificant incidents such as this take valuable time from more relevant incidents. Richard did say it was a trivial example, but surely he could have used a more relevant attack. As it stands, his example does not support his argument.

When you do security right, you build a platform that you can trust to a high degree. For example; create a predictable, reliable system when building and maintaining your firewalls. Build and maintain all of them using the same process. Document the process and then routinely audit the firewall policies, the firewall software and/or firmware and your firewall maintenance process itself. Then, when you see an attack on a port that is insignificant to your environment, you can trust your system and save valuable cycles for another, more relevant attack, say on port 80 against a web server.

Where Richard and I differ in our security ideology is Richard is reactive and I am proactive. Even the name of his ideology implies reaction; Network Security Monitoring. Now take a breath and don’t let your mind start formulating your flame against me for dismissing logging and monitoring. That’s not what I’m saying. I’m saying that highly effective security practices focus a majority of your time and energy on proactively preventing or mitigating attacks, while of course monitoring for those you can’t prevent, all of which is the very heart of risk management.

In my case, I trust my process of firewall and IPS building and tuning. I trust my process of end-system hardening and maintenance. I trust my process of containment. Therefore I greatly reduce the amount of energy and effort required to investigate alerts such as this. I know that I have already done my due diligence and addressed the risks my assets face and what I have left to worry about are the risks I must accept.

For example; I have a web server available to the public. Of all the risks that system faces, I have explicitly chosen to accept the risk of certain web-based attacks, simply by allowing port 80 access to that system. However, before I ever put the system into production I have done and documented my due diligence; I patched and hardened the OS, I installed and actively tune HIPS software, etc, etc. That process has eliminated a large portion of attack opportunities and has left me with a small and focused area of concern; the web service itself. I know with certainty (because I trust my system process) that for that system to be compromised, the web service itself must first be compromised.

Following my security ideology, the investigative process I’d follow if I received the exact same alert on my system would be as follows:

Here’s the alert taken from Richard’s post:

Count:1 Event#1.78890 2007-01-15 03:17:36
BLEEDING-EDGE DROP Dshield Block Listed Source
203.113.188.203 -> 69.143.202.28
IPVer=4 hlen=5 tos=32 dlen=48 ID=39892 flags=2 offset=0
ttl=104 chksum=57066 Protocol: 6
sport=1272 -> dport=4899

My process:

1. I know nothing can respond to a packet on port 4899 because my firewall policies implicitly block access to that port. This specific alert is quickly becoming irrelevant and my concern level is actively reduced.
2. I check to see what all my firewalls logged for this source IP. Again from Richard’s example, a total of two inbound syn packets and two outbound resets were logged (I chopped off the time info to fit my blog):

1 203.113.188.203 -> 69.143.202.28 TCP 1272 > 4899
 [SYN] Seq=0 Len=0 MSS=1460
2 69.143.202.28 -> 203.113.188.203 TCP 4899 > 1272
 [RST, ACK] Seq=0 Ack=1 Win=0 Len=0
3 203.113.188.203 -> 69.143.202.28 TCP 1272 > 4899
 [SYN] Seq=0 Len=0 MSS=1460
4 69.143.202.28 -> 203.113.188.203 TCP 4899 > 1272
 [RST, ACK] Seq=0 Ack=1 Win=0 Len=0

3. Lastly I check the web server and HIPS logs on the target system. This will show if there were any web service attacks that made it past my border protection. Since Richard’s example only stated the two packets, the logs on the end-system will be clean.

At this point my investigation has shown me that my firewalls did their job and blocked whatever this was and I’m now able to move on to the next incident.

I don’t need to do a ‘whois’ on the source IP because I never reached a point where I was concerned with who was poking at me. That isn’t actionable information at this stage (and it has been my experience that these Snort rules that block entire subnets based on Dshield and/or other collaborative data mining are over-kill and counter productive for a global enterprise that expects to do business with, say Chinese clientele). I could at this point file this incident in a ‘look deeper later’ file so that I can be Curious George if I have spare time. However, with the sheer volume of attacks public-facing systems are exposed to day in and day out, compounded by the fact that few shops have the staffing necessary to address every single incident, its as important to know what to investigate as is it to how you investigate it.

Triage is one of the best skills an incident handler should have. Without it you risk wasting time on irrelevant attacks over here while your web server gets pwned over there.

I find it interesting that the Richard’s whole process was initiated by an alert, but Richard is arguing against alert-based systems. Even more interesting is Joe’s comment that states;

“Relying on alerts only would not allow you to notice that this guy is trying out some freshly compiled 0-days against your box.”

1. This example didn’t show some guy trying zero-day exploits. It showed two syn packets destined for port 4899. In fact, it doesn’t even indicate an attack. It could be a mis-configured system on the source end.
2. Even more to the point; I argue that this specific alert in this particular example is a waste of time because a) there are no systems listening on port 4899 and b) the Snort alert amounts to a ‘guilty by association’ alert that in many cases could be counter productive

Yes, its great to be inquisitive and explorative. Its better to be efficient and effective.

For more information about this topic

• Windows DNS/RPC Vulnerability
• My Black Tuesday Routine
• Micro defense in depth
• Evaluating malware from a network perspective
• ANI exploits
• ANI Vulnerability: big fuggin’ deal
• Why system hardening is critical