Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.networking > #2129
| From | Sandman <mr@sandman.net> |
|---|---|
| Newsgroups | comp.os.linux.networking, comp.os.linux.security, comp.infosystems.www.servers.unix |
| Subject | wpad.dat attack on Linux Apache server |
| Date | 2013-05-24 11:22 +0200 |
| Message-ID | <mr-983E88.11221424052013@News.Individual.NET> (permalink) |
Cross-posted to 3 groups.
I don't actually know if it's an attack.
My Apache server was slow - symptoms being slow responses (after PHP had
processed and sent to client) and dropped connections, looked in the log
files, found 12GB of accesses to /wpad.dat on my catch-all vhost.
I got thousands of requests per minute from hundreds of different hosts,
and a sample of these showed that they all seemed like legit end-user
hosts, not a tor proxy botnet at least.
I googled some, and found that wpad is some form of auto-discover proxy
settings. Problem is, it should be done to the local network. So if my
machine is on the "example.com" network, my browser will send a request
to "wpad.example.com" to find proxy settings. not
"wpad.remoteinternetsite.com".
So, maybe it is an attack after all?
I added a wpad.dat file to the server, with this content:
function FindProxyForURL(url, host) { return "PROXY 127.0.0.1:445"; }
Which just tells these clients to look for a proxy on localhost. Nothing
changed. Thousands of thousands of requests.
In /server-status for Apache, my queue is filled with /wpad.dat requests
with the "K" status (Keep-Alive), so that sounds like why it's slow.
Ok, so I'll block it. Blocking it in apache seemd stupid, it would still
process the requests, so to iptables:
iptables -I INPUT -p tcp --dport 80 -m string --to 70 --algo bm \
--string "GET /wpad.dat" -j REJECT --reject-with tcp-reset
Yes, I know this analyzes *every* request, and wastes CPU cycles, I may
get around to chain this into a seperate iptables chain and only act on
correct parts. In about an hour, this has blocked 45k requests, or about
750 per minute
Either way, the requests are gone, the log file is clean(er) but the
server is still slow and still drops connections.
So, troubleshooting some more. CPU is at 0.8% usage, Memory is 80% free.
Asking my hosting ISP, my bandwidth capacity is at 0.62%
ethtool -S eth0 shows no errors (but a shitload of packets, of course)
Someone suggested that it could have something to do with my using a
wildcard DNS. So, my server hosts some 100+ virtual hosts. All my
clients are told to use a CNAME pointer for their servers. So:
www.client.com -> CNAME -> cluster.mydomain.com -> A -> 123.123.123.123
Which means that every visitor to my sites has their web browser first
look up www.client.com to find cluster.mydomain.com which in turn points
to my IP.
No, the mydomain.com had a wildcard setting, so if and when they would
access "wpad.mydomain.com" my DNS would point that to
cluster.mydomain.com and then that wold point to the IP. So supposedly,
all the request could channel to my server this way.
I have now removed wildcard for mydomain.com, and also added a wpad host
for all my domains that points to 127.0.0.1. I'm waiting to see that
propagate and see if it makes any difference. It hasn't so far.
Do any of you guys have any ideas what this might be? Or rather - how
do I trouble shoot this some more?
I have:
Slow transfer speeds on apache
Super fast on other ports (SFTP for instance)
Thousands of requests per minute that are now being blocked
Super low CPU usage
Super low RAM usage
No reported ethernet errors
--
Sandman[.net]
Back to comp.os.linux.networking | Previous | Next — Next in thread | Find similar
wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 11:22 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 10:37 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 11:44 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 11:04 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 12:13 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 11:20 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 12:23 +0200
Re: wpad.dat attack on Linux Apache server J G Miller <miller@yoyo.ORG> - 2013-05-24 11:07 +0000
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 14:43 +0200
Re: wpad.dat attack on Linux Apache server Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2013-05-24 13:05 +0000
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 15:14 +0200
Re: wpad.dat attack on Linux Apache server Joe Beanfish <joebeanfish@nospam.duh> - 2013-05-24 13:39 +0000
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 16:23 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 15:13 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 16:26 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 17:38 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 19:02 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 18:45 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 21:12 +0200
Re: wpad.dat attack on Linux Apache server Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2013-05-24 16:47 +0000
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 18:53 +0200
Re: wpad.dat attack on Linux Apache server J G Miller <miller@yoyo.ORG> - 2013-05-24 17:15 +0000
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 21:20 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-25 08:45 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-25 09:54 +0200
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-25 10:04 +0200
Re: wpad.dat attack on Linux Apache server Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2013-05-26 10:12 +0000
Re: wpad.dat attack on Linux Apache server "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2013-05-24 13:50 -0400
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 19:10 +0100
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-24 19:15 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 21:33 +0200
Re: wpad.dat attack on Linux Apache server Richard Kettlewell <rjk@greenend.org.uk> - 2013-05-25 08:38 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 21:25 +0200
Re: wpad.dat attack on Linux Apache server Whiskers <catwheezel@operamail.com> - 2013-05-25 15:52 +0100
Re: wpad.dat attack on Linux Apache server Roger <invalid@invalid.invalid> - 2013-05-25 17:19 +0100
Re: wpad.dat attack on Linux Apache server J G Miller <miller@yoyo.ORG> - 2013-05-25 17:22 +0000
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-25 19:41 +0200
Re: wpad.dat attack on Linux Apache server Chris Davies <chris-usenet@roaima.co.uk> - 2013-05-24 12:31 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 14:29 +0200
Re: wpad.dat attack on Linux Apache server Chris Davies <chris-usenet@roaima.co.uk> - 2013-05-24 23:45 +0100
Re: wpad.dat attack on Linux Apache server Sandman <mr@sandman.net> - 2013-05-24 14:49 +0200
Re: wpad.dat attack on Linux Apache server jcharth@gmail.com - 2013-05-28 12:45 -0700
csiph-web