Ethical Hacking

Learn to find vulnerabilities before the bad guys do! Gain real world hands on hacking experience in our state of the art hacking lab. Course designed and taught by expert instructors with years of penetration testing experience. 12 student maximum in every class. Certification attempt included in every package.
Computer Forensics Training at InfoSec Institute

Gain the in-demand skills of a certified computer examiner, learn to recover trace data left behind by fraud, theft, and cybercrime perpetrators. Discover the source of computer crime and abuse at your organization so that it never happens again. All of our class sizes are guaranteed to be 12 students or less to facilitate one-on-one interaction with one of our expert instructors.




Network Security Pen-Test
[Top] [All Lists]

Re: Spidering

Subject: Re: Spidering
Date: Mon, 21 Jan 2008 08:38:05 +0100
Salut,

On Thu, 17 Jan 2008 17:58:41 +0000 "me me" <securityoneoone@googlemail.com> 
wrote:
Whilst I don't expect it to get everything (JavaScript etc is going
to take manual intervention, so is a number of other possible
technologies), I have never really found a tool that I consider to be
the defacto spidering tool from this perspective. One of the biggest
problems is a lot of the spiders seem to choke on really big sites,
or go into infinite loops  etc etc.

Yes, Microsoft Passport is very evil there, as an example. My trick to
solve the Microsoft Passport Problem is to search every link if it
contains an URLencoded version of the current URL and if it does,
ignore it. That appears to avoid deadloops.

I haven't yet seen other deadloops as far as I remember, but then again
I didn't index very much yet.

                                Tonnerre
-- 
SyGroup GmbH
Tonnerre Lombard

Solutions Systematiques
Tel:+41 61 333 80 33            Güterstrasse 86
Fax:+41 61 383 14 67            4053 Basel
Web:www.sygroup.ch              tonnerre.lombard@sygroup.ch

Attachment: signature.asc
Description: PGP signature

<Prev in Thread] Current Thread [Next in Thread>