Ethical Hacking Learn to find vulnerabilities before the bad guys do! Gain real world hands on hacking experience in our state of the art hacking lab. Course designed and taught by expert instructors with years of penetration testing experience. 12 student maximum in every class. Certification attempt included in every package. | Computer Forensics Training at InfoSec Institute Gain the in-demand skills of a certified computer examiner, learn to recover trace data left behind by fraud, theft, and cybercrime perpetrators. Discover the source of computer crime and abuse at your organization so that it never happens again. All of our class sizes are guaranteed to be 12 students or less to facilitate one-on-one interaction with one of our expert instructors. |

| Subject: | Re: Spidering |
|---|---|
| Date: | Mon, 21 Jan 2008 08:38:05 +0100 |
Salut, On Thu, 17 Jan 2008 17:58:41 +0000 "me me" <securityoneoone@googlemail.com> wrote:
Whilst I don't expect it to get everything (JavaScript etc is going to take manual intervention, so is a number of other possible technologies), I have never really found a tool that I consider to be the defacto spidering tool from this perspective. One of the biggest problems is a lot of the spiders seem to choke on really big sites, or go into infinite loops etc etc.
Yes, Microsoft Passport is very evil there, as an example. My trick to
solve the Microsoft Passport Problem is to search every link if it
contains an URLencoded version of the current URL and if it does,
ignore it. That appears to avoid deadloops.
I haven't yet seen other deadloops as far as I remember, but then again
I didn't index very much yet.
Tonnerre
--
SyGroup GmbH
Tonnerre Lombard
Solutions Systematiques
Tel:+41 61 333 80 33 Güterstrasse 86
Fax:+41 61 383 14 67 4053 Basel
Web:www.sygroup.ch tonnerre.lombard@sygroup.ch
signature.asc
Description: PGP signature
| <Prev in Thread] | Current Thread | [Next in Thread> |
|---|---|---|
| ||
| Previous by Date: | Re: Oracle URL SQL Injection issue, Todd Manning |
|---|---|
| Next by Date: | WifiZoo v1.3 released (minor release), Hernan Ochoa |
| Previous by Thread: | RE: Spidering, Thor (Hammer of God) |
| Next by Thread: | RE: Spidering, merigoth |
| Indexes: | [Date] [Thread] [Top] [All Lists] |