Well, now that I've seen this thread going on for a while, I've finally just
read the subject heading and taken notice...=)
I have actually tested the Top Layer boxes against a signature-based
solution like Tipping Point and Mazu Networks. In short, the Tipping Point
box died because it couldn't handle the gigs of traffic we threw at
it--exactly what Tim is describing below. The Mazu box worked similarly.
These boxes could detect anomalies well when traffic was low, but could not
handle the real-life environment of gigabit speed DDoS (TCP SYNs).
On the other hand, the Top Layer box successfully mitigated all the traffic
we sent at it. I am now a true-believer in rate-based IPS and have thus
implemented the Top Layer solution in our high-traffic E-Commerce
environment. From a technical perspective, I love these boxes. They are
truly an effective means of protection against many of today's DDoS threats.
That being said, I must admit that pretty much all my experiences with Top
Layer on the administrative side (i.e. technical support, sales support,
etc.) have been some of the worst experiences of my IT career.
Non-responsiveness, non-professionalism, etc, has been the mark of my
experiences with this company. Also, as Tim may probably agree, the HA
solution needs some major work...
Tom, you guys have a good product, but now need to start spending resources
on your internal resources...particularly customer service.
My $.02,
Ed
-----Original Message-----
From: THolman@toplayer.com [mailto:THolman@toplayer.com]
Sent: Tuesday, June 07, 2005 4:55 AM
To: ghalleen@cisco.com; THolman@toplayer.com; focus-ids@securityfocus.com
Subject: RE: IDS\IPS that can handle one Gig
Hi Gary,
I disagree with your first point. Test conditions are not clearly stated in
any publicly available Cisco literature - if you can offer me a publicly
available link (non-CCO) then you win! :)
I am not contending your performance figures - 5000 connections per second
is quite a reasonable amount to assume on your average enterprise network,
but is certainly not sufficient for large enterprises, data centres and
ISPs.
Even in a small network, when worms decide to attempt propagation and
initiate a few hundred connections per second from each workstation - it
would only take 10-20 such infected machines to breach your 5000 connection
per second limit and start causing problems.
Also, any DDOS attempt against a network protected by a device that is only
capable of 5000 connections per second will succeed. A botnet of 1-200
devices would have a field day!
This is why it is important for an IPS to have rate-based, and not just
content-based protection. The content-based stuff works fine in most
networks, but as soon as any critical events occur, network administrators
don't give a toss as to the precise taste and colour of individual packets,
and want PROTECTION.
There is no slam intended in any of my posts, but I would like to see
vendors be a little more 'open' about their product shortfalls so that
customers at least get the chance to supplement the solution with other
protective measures.
There is just too much mis-selling going on. Customers are being sold IPS's
as an all-in-one security solution, only to find a few weeks or months later
that this is not the case. These salesman should be shot, as they're giving
us ALL a bad name ! :)
Regards,
Tim
-----Original Message-----
From: Gary Halleen [mailto:ghalleen@cisco.com]
Sent: 05 June 2005 09:22
To: THolman@toplayer.com; focus-ids@securityfocus.com
Subject: RE: IDS\IPS that can handle one Gig
If you Google as you've suggested, it's quite obvious that your message is
intended as a slam against our (Cisco's) products.
1.) Cisco bases our performance test on industry accepted standards
following the stringent NSS Group test criteria as well as our own analysis
of live network traffic indicative of typical enterprise networks. We
clearly state the test conditions under which we reach our performance
metrics and they are legitimate and representative of real-world situations.
2.) The statement that 5000 cps equates to only 10 Mbps of throughput is
flawed and assumes that each newly established session only has a delivery
of 250 bytes of total payload per session. This would be equivalent to only
establishment and teardown of the session with no useful communication. Our
research indicates that an average session contains between 10,000 and
25,000 bytes of information transferred. From these numbers (if you do the
math) you will find that the throughput of these useful sessions are between
500 Mbps and 1 Gbps supporting Cisco's reported performance claims.
3.) Cisco never disables "vital security features" such as fragment
reassembly, TCP stream reassembly, or HTTP deobfuscation when testing,
validating and reporting our IPS performance. We don't take shortcuts as
implied in this thread.
The author of the original email is using inappropriate math to attempt to
make a self-serving statement around ASIC based technology and TopLayer's
performance supremacy.
Gary
-----Original Message-----
From: THolman@toplayer.com [mailto:THolman@toplayer.com]
Sent: Thursday, May 26, 2005 1:47 AM
To: focus-ids@securityfocus.com
Subject: RE: IDS\IPS that can handle one Gig
Hi Randall,
Throughput is unimportant when it comes to choosing an IDS/IPS, and to be
honest, a bit of a bun fight when you place two vendors side by side and
start scouring their datasheets for practical information.
What is important, however, is the number of packets per second the device
can process, the maximum number of connections that such a device keeps
state for, and last but not least, the latency that such a device will
introduce into your network if placed inline.
The smaller the packets used in a test, the smaller the performance in terms
of megabits. The larger the packets, the bigger the performance in terms of
megabits. Unreliable, and totally abused by most vendors on their
datasheets. It's quite easy to say 'we support 1000 Mbps', only to say in
small print the average packet size is 595 bytes. You only need to search
Google for '1000 Mbps 595 bytes' and you'll soon find out what I mean.. ;)
The vendor in question, although claiming Gigabit performance, can only
setup TCP connections at a rate of 5,000 per second - if you do the math,
you'll soon find out that this represents less that TEN MEGABITS per second
in 'throughput' terms.
Is it ethical to claim Gigabit performance, only for the potential end user
to run a number of tests with small packets sizes and find out this is not
the case?
The moral of the plot is to never trust a datasheet - either thoroughly test
the products before purchase, or look toward an independent testing house,
such as NSS (www.nss.co.uk), whom have the resources and experience to
regularly generate test results that count.
At TopLayer, we regularly deploy into Gigabit environments, and encourage
the customer to test (using Smartbits, Ixia or Spirent) for piece of mind.
Rest assured, each time they do this, we pass with flying colours, and this
is what makes us one of the top market leaders in Gigabit IPS solutions.
Regards,
Tim
-----Original Message-----
From: Randall Jarrell [mailto:rgj@msn.com]
Sent: 19 May 2005 16:28
To: focus-ids@securityfocus.com
Subject: IDS\IPS that can handle one Gig
Greetings,
We are currently evaluating IDS\IPS vendors. We have tried two vendors, whom
I will not name unless you ask me, that have made claims that they can
handle a Gig of through put but actually start to fail around the 300-500MB
range.
Could anyone share a success story of a vendor they are using that is
handling this type of traffic?
Thanks in advance,
-RGJ
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE
IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE
IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE
IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE
IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
