Inline...
On 2/20/06, Ivan Arce <ivan.arce@coresecurity.com> wrote:
Aaron Turner wrote:
[snip]
Generally speaking, tcpreplay is better when one or more of the
following is true:
1) Trying to do comparative analysis and you want to make sure each
device sees exactly the same thing
Hmm, why is that harder to accomplish with Metasploit than with tcpreplay?
Because metasploit, other tools and exploits incorporate PRNGs and
other methods of altering the attack so that it isn't exactly the same
each time. Sure it's the same "exploit", but the bits on the wire
are different. That makes "each device sees exactly the same thing"
really difficult. Of course some tools allow you to control these
via seeds but others don't.
2) Need to automate or do a lot of regression testing and want a
stable and relatively simple lab environment
same as above....
Again, less complex (no 2nd box and vmware to maintain/automate) and
100% repeatable data streams. Also what about attacks that Metasploit
doesn't have? What if you want background traffic?
3) Already have a library of pcap's (either from customers, the wild
or capturing traffic of real tools like Metasploit)
Yeah, but that is an entirely different kind of testing. Replaying the
packets captured from the execution of an exploit is not the same as
executing the same exploit again.
If you're testing a vulnerable application then I agree. but if you
are testing an IDS/IPS, then I would argue that it is for all intents
and purposes it's the same thing. If you believe otherwise, then
please explain.
4) Don't want to worry about re-installing or fixing target systems
after they've been 0wn3d. VMware of course helps, but there is still
a lot more administrative overhead.
Hmm, maybe or maybe not... Actually you can pretty much automate the
entire process (or a big part of it):
1. set up of the proper VMware images (specially if you're using GSX or
a similar virtualization server that lets you manage images
programatically and from remote)
2. run a set of exploits in the appropriate order
3. generate reports or other output with the results
4. correlate output with IDS/IPS alerts/logs/etc.
The point being you have to put forth the effort to automate
vmware/metasploit is a lot more work then just tcpreplay. Not to
mention, if you want to add other tools other then metasploit now you
have more automation work to do. tcpreplay is exploit/tool agnostic
and provides a simple and unified method of generating all kinds of
traffic- not just exploits.
[snip]
In general, tcpreplay isn't all that useful IMHO when you're first
starting off and "want to do some IDS/IPS testing" or only intend to
run a few tests or tests only once or twice unless you already happen
to have a nice pcap library.
Ahh that's interesting, I see it in exactly the opposite way: tcpreplay
is ok when you want to scratch the surface of your IDS capabilities or
perhaps more appropriate for stress or throughput testing or very basic
regression testing. However, if you truly want to check if the IDS
recognized real attacks you need to test with real exploit runs not with
a replay of their captured traffic.
Why? What is the different between "real exploit runs" vs. "replaying
real exploit runs" from the viewpoint of an IDS/IPS/UTM/etc?
Obviously the biggest limitation of tcpreplay is it doesn't come with
a library of pcaps. Maybe one of these days I can figure out the
In my view, the biggest limitation is that replaying captured packets an
overly simplified manner of modeling real world attacks. Today's exploit
code is a lot "smarter" than simple PoC that send the same fixed data on
each run. modern exploits make runtime decisions based on the state of
the target system/application and several other things. To successfully
simulate the execution of real exploits you need to maintain state about
both endpoints (target and attacker's systems) and properly simulate the
meaningful state changes in them that would change exploit-code's
execution flow and elicit different traffic patterns that those from
previous runs.
Perhaps you're not aware, but tcpreplay fully emulates both client and
server state, from protocol setup, to the actual exploit and all the
way through getting a shell/whatever if that is what you captured. An
IPS/IDS has absolutely no way of knowing if traffic was generated by
tcpreplay or not, because it is EXACTLY like the traffic that was
originally generated/captured. If you want to run the same exploit
500 times with different parameters with Metasploit, that's great.
Just capture each of those 500 attacks and replay each of them against
10 different IDS/IPS's.
The point is that with tcpreplay, your test cases are 100% repeatable.
You *know* that the all those runtime decisions that happened in the
original are replayed exactly the same way each and every time.
Why is this important? Well consider this:
Say you have two IPS's you want to test. You an send an "attack" with
Metasploit against the first one and it detects it. You run it again
against the second one and it doesn't. Does that mean the first one
is better? Well unless the byte stream was exactly the same in both
cases (because Metasploit chose different runtime parameters based on
a PRNG, time of day or whatever) or maybe you upgraded Metaspolit
since the first time and things changed, you don't know. That's why
people should use tcpreplay.
--
Aaron Turner
http://synfin.net/
------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------
