Hi Ory,
I don't think the issue here is about "quality" items such as page load
time, broken links, etc. I think he's talking about having irrelevant
files junking up his webroot to avoid things like all sorts of source
code, config files, stack traces and other such items from being browsed
or picked up in a scan.
This is a really interesting subject to me: I've always recommend
keeping your webroot clean, and I suggest that SA's list the dir's and
check with the Dev group on any unusual files ("Hey Bob, is this
login.bob page really supposed to be there ? " ) .
First of all I don't think crawlers would cut it ( or other client side
products). You'd have to look at the source code to gather the
dependencies, because you can have obvious client side stuff like href's
/ scripts /images, but you can have plenty of server side dependencies
that won't be visible to the client, like SSIs, class libs, modules,
HTML files read from a filestream (but no href), pages referenced
through parameter ( showpage.jsp?pageid=123) etc. A quick example of
some page1.html being dependent, but not visible to the client:
<a href=page.asp?pageid=1>Read Report</a><br>
<%
Set fso = server.createobject("Scripting.FileSystemObject")
Select case request.querystring("pageid")
case 1 ' Read a text report imported from a nightly batch
transaction
Set file = fso.OpenTextFile(Server.MapPath("\") &
"/page1.html")
do until file.atendofstream
response.write file.readline
...
End select
%>
Unless I did something lame in this sample, I don't see how a crawler
would know where the content is actually coming from page1.html and not
page.asp.
Given the server side dependencies issue, I see two approaches short of
finding an actual tool for this (which, btw, is more likely to be found
in a Dev group than an AppSec on IMHO)
1. Grep the code for various file patterns and then list those as
"needed." The problem is you'd have to anticipate the patterns
2. List the existing files, then grep for THOSE. If the file exists
but doesn't match a grep, then carefully remove it.
By carefully remove it I mean wait until a config window, take that box
out of service on any load balancers, rename it, test as much of the app
as possible and then finally delete it or move it to a 'deleteme'
folder, or whatever other processes you take to ensure career longevity
when playing with production systems.
This is an area that I think is pretty darned important (having done
really keen stuff against multiple sites based on fluff files). I would
imagine the tool would end up being a development tool (non security
related). There are products that will perform code-to-UML mapping
(vice versa) that may help with this. May.
I'd really be interested in hearing about it if anyone finds a good
tool / technique but at this point I really don't see how it could be
sufficiently performed from any client sided product such as crawlers,
scanners, accessibility testers etc.
- Matt @ SPI
http://www.spidynamics.com
WebInspect, DevInspect,
QAInspect, SecureObjects, AMP
Start Secure. Stay Secure.
-----Original Message-----
From: Ory Segal [mailto:osegal@watchfire.com]
Sent: Thursday, April 14, 2005 4:27 AM
To: Jarmon, Don R; webappsec@securityfocus.com
Subject: RE: webapp dependencies
Hello,
Watchfire ( http://www.watchfire.com ) has a platform for
automating scanning, analysis and reporting of online
businesses. The platform is called WebXM, and it includes
many modules, some of which you mentioned.
For example:
1) Scanning for web application security
2) Scanning for site accessibility problems
3) Scanning for web site quality issues (what you have mentioned)
4) Manage web site privacy
And many more.
The platform is installed as a server, with a web interface
acting as a reports dashboard, which will also present
regression testing.
Thank you,
Ory Segal,
Watchfire
-----Original Message-----
From: Jarmon, Don R [mailto:Don.Jarmon@Intergraph.com]
Sent: Thursday, April 14, 2005 2:23 AM
To: webappsec@securityfocus.com
Subject: webapp dependencies
I looking for a tool that will analysis content hosted on a
web site, identify all the webapp dependencies, and report on
any non-essential content. The tool would run from the
server. Does such a tool exist?
