Dealing with Intruders? 656
drakyri asks: "I've been running a server for a small company for a few months. Recently, the number of attempted intrusions has jumped from about one every week to several per day - and these are only the really obvious attempts, like idiots who try to log in as root from the outside.
The problem is that I'm not sure what to do about this. I've got their IP addresses and can usually tracert their ISP's - is there an accepted type of letter to send them without seeming like one of the corporate cease-and-desist gnomes?"
Abuse@ (Score:5, Informative)
http://www.arin.net
or lookup the RADB abuse contact
http://www.dnsstuff.org
I agree (Score:1, Informative)
The site is not a high-profile site by any means but rather a home for some personal projects. I just wrote it of as the script-kiddy attemt de jour but it's interesting to see that others experience the same thing.
ask your company lawyer (Score:1, Informative)
let your lawyers deal with crap like this
My Advice (Score:3, Informative)
Snort + Guardian (Score:4, Informative)
Well... (Score:5, Informative)
So far, all of these "cease and desist" letters has resulted in action on the ISPs part, and in 50% of the cases, their admins write me back and give me feedback on the problem.
Ofcourse, I don't do this for every attempt (all depending on my mood
The worst (or craziest?) attempt yet was by some nut who portscanned the system, port by port from start to finish. I actaully managed to get hold of the owner of the computer system that was scanning me and phoned him. Quite a hilarious experience. Needless to say, the portscanning stopped
In my experience (Score:4, Informative)
I had confidence in my setup, and no server I had control over was, to my knowledge, ever compromised.
We never had any sensitive data outside the firewall, anyway.
On two occasions it got serious (if an easily beaten DOS attack can be called serious) and even then it was only for 20 minutes or so. Our ISP (being a large telecom) was champing at the bit to go after people we had even a small scrap of evidence against, so on those two occasions we simply handed what information we'd gleaned to them, and they let out the dogs.
At some stage, you've got to stop worrying and learn how to love the internet!
Yes, there are several good ways. (Score:5, Informative)
I tend to report viruses. I grep my logs daily for viruses from various norwegian ISPs, to the mailserver I admin for my company. During the last five months I've sent daily virus reports to the largest ISP in norway, and they tend to reply within one business day - having notified their customer about the infection. If the customer gets several 'heads up' messages from the ISP without removing the virus, they get their port 25 access filtered until they've confirmed that they've removed the virus.
I tend to send emails such as this.
"
Hi there.
I've got several viruses from your customers today, and would appreciate it if you could notify your customers about the virus infections they probably have.
Here are the relevant snippets from my logs:
Virus: Netsky.B
Received: from at
Virus: Bagle.C
Received: from at
All timestamps on the server are NTP-sync'ed against
Thanks for your time
"
Recently I've also included a more personalized
"Oh, and I have to commend your ISPs efficiency, as since march - you've managed to reduce the number of virus sending users to us from about per day, to this
You could probably just adapt what I'm writing to something saying that a customer of theirs probably has been cracked, and that they are currently scanning for
If it's the actual cracker that's stupid enough to use his own computer, he'll get scared enough if they contact him telling him that his computers has been abused by others to scan people -- and will probably quit doing it.
at some level you have to ignore it.... (Score:5, Informative)
Ignore it? (Score:5, Informative)
get an auto reporting tool (Score:2, Informative)
the system automatically sends a warning to the isp
Maybe related to this? (Score:3, Informative)
threads on the full disclosure mailing list archives [netsys.com] and dslreports forums [dslreports.com] about that
wonder if this is what the topic poster was encountering?
don't forget logfiles & date/time (Score:3, Informative)
You'd be better off configuring your security better though.
Re:Corporate Gnome (Score:2, Informative)
Keep in mind that those IP's could be spoofed however, so without something a little more substantial than an IP addy, you are likely to be ignored by most major isp's.
well, if you decide to write to ISP, don't write letter in which you accusing but ask ISP for help to investigate and be polite.
Re:Well... (Score:4, Informative)
Many times the ISP has responded and usually their customer has a zombie box.
Always include a log if possible so they know the time and the IP-address. Remember to tell them what timezone the timestamps are from.
WHOIS links
http://ws.arin.net/cgi-bin/whois.pl
http:
http://www.apn
Re:Snort + Guardian (Score:2, Informative)
Re:Maybe set up a honeypot for a bit (Score:2, Informative)
firewall - allow only certain IPs access port 22 (Score:2, Informative)
You said, YOU are running a server for ONE client. Who is it that needs SSH access to the machine - YOU. What i would do is limit access to port 22 to IP adresses I am going to use. Add your normal internet adresses to the list (like your ISPs IP-block, work, girlfriends isp, ...) And of course you need to add a machine that is alwas up and has no such firewall restrictions (i.e. shell access to your server at home, i know you have one ;-)). This way you can login to the server from your most common locations, and login indirectly to the server using another box as "proxy" in case you are on vacation sitting in an internet-cafe.
i think it's also good practice to generally disallow direct root-logins in ssh-config and only allow shell users having group wheel to su to root.
Re:Somewhat offtopic, but how do people deal with (Score:5, Informative)
Re:Very Easy (Score:5, Informative)
Remote logins? Are you insane? (Score:5, Informative)
So you've got a machine sitting on the internet, home to a million and one active worms, and are surprised that it gets scanned constantly?
Don't bother with the abuse reports -- more than likely it's just worm activity from computers whose clueless owners don't realize have been infected. A more recent one attempts SSH logins, which may be what you're seeing.
It it was a _real_ crack attempt then you:
1: Wouldn't know about it.
2: Would be unable to pin it down. It would be bounced through several victim networks, so your ability to see where it's "coming from" is really just the last victim machine in the chain.
Third possibility is script kiddies, in which case you would know about it and where they were coming from, but they would have no chance of success unless you are unwilling to keep up on patches and follow basic security practices like decent passwords.
Best would be to close off remote-login ports altogether. If you need remote login then block for all but the address range you'd be coming from. If you need remote access from random locations, then at least consider using a heavily locked down system (e.g.: OpenBSD) or work _really hard_ to get your systems firewall/logging/etc. set up well.
One OpenBSD/pf feature you might be interested in (also available from other systems) is the ability to tie Snort into the pf ruleset so that remote scanners, once detected, are ignored.
This is more fun! (Score:5, Informative)
Re:Yes, there are several good ways. (Score:3, Informative)
Nah. We only get around 50 viruses per day, and I've made a list of the responsive ISPs. I tend to email the responsive ISP's one email per day, containing nothing but the relevant headers.
The ISPs just receives an email with the name of the virus, and the Received: from header(s) they need to track down the person with that virus.
Most is automatically generated by my scripts. I just paste it into my mail client and send it off with a few nice words on top of the list - and if I'm very pleased with the ISPs responsiveness in the past -- some nice words of encouragement for their great work.
The cool thing is that I'm seeing an actual reduction in viruses received from the responsive ISPs, and when they're bogged down - I've gotten my "IMPORTANT!" emails moved quickly up the queue. One particular instance with someone that was pounding our mailserver several times per minute - I got a response from the ISP within 20 minutes.
More good advice ... (Score:3, Informative)
This is really good advice, but you can do more. :-)
Most ISPs really appreciate the complete header of the mail, and sometimes even the body in case of spam. First of all it adds to the authenticity, and second they'll be able to forward your complaint to the responsible ISPs if you had too much beer while reading a spoofed header (more so for spam than virus mails). Some ISPs are quite helpful in this regard.
To aid in identifying the correct abuse addresses I can recommend the hinfo utility as a complement to whois. Oh and if you're stuck with a standard whois, consider replacing it with the one made by Marco d'Itri - it's the default in Debian, and has the ability to guess the correct whois hosts to ask.
zTwo things (Score:5, Informative)
Re:Firewall? (Score:5, Informative)
I don't subscribe to it. I look at it like this:
To drive a car, you need a licence. You have to follow rules. You drive on the correct side of the road. You don't drink and drive. You obey the speed limit. And why do we have to follow the fules? It's because there are other people who also want to use the road, and therefore all drivers have a responsibility to ensure that the safety of others is protected.
Sounds like common sense, right? Well the same should apply to placing computers on the internet. If you want to have viruses and backdoors and worms etc running on your home PC, then fine. Whatever. But if you put your home PC on the internet and take absolutely no fucking responsibility for what you are doing then you are waiving all rights you have over the the safety of your computer. If your computer now pisses me off, I'll 'smbdie' it off the internet. If you're fine with all the rest of the shit that's infecting your PC, then you don't really have any right to complain about me rebooting it once every 5 minutes. And yes I'm doing everyone a service. Firstly, the computer is on the internet for less time than it otherwise would have been, so there's less chance of others being infected. Also, the idiot who owns the computer will be far more likely to do a complete re-install, or at least get a god-damned virus checker and get Windows up-to-date.
Do you know how many people come bitching and complaining to me about their PC being rooted, and when I boot it up find that they're running Windows 2000 SP1 and NO virus protection at all? It's not good enough. And the only ways to get them to take responsibility for their computer are:
a) Legislate. No-one wants legislation covering their computer. It will screw things up for the responsible among us and have no effect on the rest.
b) Make it so unconfortable to run an unprotected computer that they get the hint and protect it.
Having said all this, I know most people will still disagree with me. That's fine. Be angels. Just keep your damned computer secure and you've got nothing to worry about.
Re:Why not seem like a cease and desist gnome? (Score:2, Informative)
Re:This is more fun! (Score:5, Informative)
The problem with these two (most common) scenarios that the person who owns the computer isn't the real perpetrator, and the ability to track the perp down requires much more work than a simple whois lookup of the offending IP.
Most attacks you see are going to be automated and launched on a wide scale. There are thousands and thousands of compromised Windows machines out on the net that are being used by people such as spammers and crackers for their dirty work.
Lock your box down.
Don't allow root to log in on SSH.
Lock SSH and other sensitive services down to specific IP address blocks if you can. If you can't, investigate port knocking [portknocking.org] if you can do that. If you can't even go that far, investigate implementing a lockout policy for failed login attempts.
Unless you see a single host being the source of a large pile of offensive behavior, chances are these are machines in a zombie hoarde. If it is limited to a single IP or a few IP's in a single C class, contact the ISP's abuse department *politely* (remember these are folks like you in jobs like yours, if you go in with guns blazing, they're less likely to help) and provide as much information as you can regarding the nature of the attack. Then firewall off the offending IPs.
I used to aggressively track intrusion attempts and spam. I had a little PHP/MySQL tool I wrote where I could log these things, dumping in offending logs (or spam source), and it'd extract the culprit IP address, and once a day go through, looking up abuse addresses on whois and mailing a digest of the day's activities for that ISP to them.
Ultimately I probably got about a 1% response rate from the ISP's (excluding auto-responses). After ~6 months of this, and about 40,000 records in my database, I started some statistical analysis. It turns out that there were no significant outliers for abusive activity from any given ISP (considering the size of that ISP's net blocks). Basically every intrusion attempt was some kind of zombie. There were probably a few by-hand attempts, but these are typically so low profile that there's no easy way to distinguish them from the hoardes.
Some time later I was the recipient of a DDoS attack. Someone's zombie hoarde decided to repeatedly visit a page on my website that turns out to be a bit resource intensive to generate (my code is open source, so whoever devised this probably knew that). Every day, ~25,000 IP's each requested the same page every 4 minutes (+/- a few seconds I suppose for network latency). 375,000 hits an hour = 9,000,000 bogus hits a day. Day to day this number fluctuated, and the ISP's involved in the attack kept changing. It was obvious to me that whoever was driving the attack wasn't exposing the entire zombie hoarde to me at any given point because of how the ISP's involved kept shifting around. I figured he probably had a script set up to launch X number of zombies every day, and they probably had commands to execute for ~24 hours. The number was always pretty close to 25,000, never over, but usually more than 24,500.
Ultimately the attack lasted about a month. I figured out a simple way to distinguish the zombie computers from legitimate users based on an error in the request headers, and I could just exit() at the top of my site for those who exhibited this error. I also logged the attempts I blocked, and was left with over 900,000 distinct IP addresses once the attack finally stopped.
My point in all of that is that there *are* zombie hoardes out there, and it's the zombie hoardes that are most likely to compromise you. There's little you can do about it because getting a single IP from a hoarde firewalled off or cleaned up won't slow down your real attacker who was going to use a different zombie the next day anyhow.
Re:Well... (Score:1, Informative)
These are some of the servers speaking it:
whois.arin.net
whois.ripe.net
whois.apnic.
whois.lacnic.net
If you have to use a webinterface, you might as well use one that doesn't give you the runaround:
http://www.iks-jena.de/cgi-bin/whois
Human responses to intrusion don't scale (Score:3, Informative)
The problem with your suggestion is that human response doesn't scale. At her average low of 15 mins per day dealing with the problem manually or socially, the rate of intrusions only has to increase 32-fold before it takes up an entire 8-hour normal working day. How many thousands of network admins are you going to hire to handle a DDoS attack from 100K sources? There is no limit to the number of owned Windows boxes out there.
It doesn't scale and it doesn't help. It is far better to spend your network admin's time on making your systems ever more impervious to attack, and if she has any time left over, to teach others how to do likewise. Ultimately, if all sites are securely tied down then it doesn't matter what the cracker kiddies are doing.
Re:Easy (Score:3, Informative)
Agreed.
A few years ago in my last job, we got hit with one of those viruses that hacks your web site (IIS of course) and modifies the home page to include bad words. We actually got the FBI involved.
I got to talking with the agent and he basically said, unless someone actually intrudes into your system, you have no recourse. Atempts are one thing, actual intrusions are something else. Also, most likely, the activity you're seeing is viruses, not someone actively trying to break-in. Just keep your systems secure and patched and keep an eye on them.
Re:This is more fun! (Score:4, Informative)
Document Everything (Score:5, Informative)
1) Make notes about what you've found
2) Report the the abuse as per the WHOIS info for the offenders
3) Block their IPs at your border
If you're using a firewall, great. If not--get one.
If you haven't read Frisch's "Essential System Admnistration" read it:
http://www.oreilly.com/catalog/esa3/index.html
If you haven't read Stephen Northcutt's "Network Intrusion Detection" you should probably give it a good read as well:
http://www.amazon.com/exec/obidos/tg/detail/-/073
There are some good articles all over the web regarding Linux security. A few google searches will help uncover them.
Patch. It's not just for Windows.
Limit services with ACLs and host restriction.
Harden your system by partitioning read/write slices away from static mountpoints where your binaries are by mounting the read only ones as read only.
chattr +i on your binaries--makes it tougher for skript kiddies.
Talk to other admins--every day is a school day.
AND
Face the fact that you're not as smart as the crackers so you just have to create layers of security that keep you from being an easy target.
Post IPs! (Score:4, Informative)
Aug 12 05:08:28 pokey sshd[7534]: Illegal user test from
Aug 12 05:08:31 pokey sshd[7534]: Failed password for illegal user test from
Aug 12 10:51:33 pokey sshd[7615]: Illegal user test from
Aug 12 10:51:35 pokey sshd[7615]: Failed password for illegal user test from
Aug 12 10:51:39 pokey sshd[7617]: Illegal user guest from
Aug 12 10:51:41 pokey sshd[7617]: Failed password for illegal user guest from
Aug 12 10:51:48 pokey sshd[7619]: Illegal user admin from
Aug 12 10:51:50 pokey sshd[7619]: Failed password for illegal user admin from
Aug 12 10:51:54 pokey sshd[7621]: Illegal user admin from
Aug 12 10:51:57 pokey sshd[7621]: Failed password for illegal user admin from
Aug 12 10:52:01 pokey sshd[7623]: Illegal user user from
Aug 12 10:52:03 pokey sshd[7623]: Failed password for illegal user user from
Aug 12 10:52:10 pokey sshd[7625]: Failed password for root from
Aug 12 10:52:16 pokey sshd[7627]: Failed password for root from
Aug 12 10:52:23 pokey sshd[7629]: Failed password for root from
Aug 12 10:52:27 pokey sshd[7631]: Illegal user test from
Aug 12 10:52:29 pokey sshd[7631]: Failed password for illegal user test from
Aug 12 11:01:41 pokey sshd[7659]: Illegal user test from
Aug 12 11:01:44 pokey sshd[7659]: Failed password for illegal user test from
Aug 12 11:01:48 pokey sshd[7661]: Illegal user guest from
Aug 12 11:01:50 pokey sshd[7661]: Failed password for illegal user guest from
Aug 12 11:01:54 pokey sshd[7663]: Illegal user admin from
Aug 12 11:01:57 pokey sshd[7663]: Failed password for illegal user admin from
Aug 12 11:02:01 pokey sshd[7665]: Illegal user admin from
Aug 12 11:02:03 pokey sshd[7665]: Failed password for illegal user admin from
Aug 12 11:02:07 pokey sshd[7667]: Illegal user user from
Aug 12 11:02:10 pokey sshd[7667]: Failed password for illegal user user from
Aug 12 11:02:16 pokey sshd[7669]: Failed password for root from
Aug 12 11:02:22 pokey sshd[7671]: Failed password for root from
Aug 12 11:02:29 pokey sshd[7673]: Failed password for root from
Aug 12 11:02:33 pokey sshd[7675]: Illegal user test from
Aug 12 11:02:35 pokey sshd[7675]: Failed password for illegal user test from
Aug 12 12:23:19 pokey sshd[7703]: Illegal user test from
Aug 12 12:23:22 pokey sshd[7703]: Failed password for illegal user test from
Aug 12 12:23:26 pokey sshd[7705]: Illegal user guest from
Just Use Dshield (Score:1, Informative)
Breaking in... (Score:3, Informative)
What a lot of you don't know, which I learned via hard knocks, was that unless you are a large corporate entity with gross yearly earnings in excess of $500k, there is NOTHING that you can do with any judge, law enforcement, or the FBI. They simply tell you to "deal with it".
This is why the issues of hacking and open spam relays, and all the other jazz will never go away, because it's not profitable or should I say; "chargable" under current statutes.
Good luck!
Companies don't care. (Score:5, Informative)
So, being a good guy, I never respond in kind (I could, but 1) it's wrong, 2) it affects more than just the target and 3) I don't feel like going to pound-me-in-the-ass prison), I just log every single packet I can, and when the attack is over find the worst offenders (typically the packets are not spoofed) and use Spamcop and whois to find the responsible parties for each one, and send them all an email.
Many (most?) emails elicit an automatic response.
Perhaps 10% get a personalized response, but usually this response says that I should contact the ISP of the offender (when in fact that's exactly what I'm doing.) Perhaps half of the responses I do get say they'll do something about it, which is good -- usually these are compromised drone/zombie machines, and need cleaning anyways.
Quite often, the attacker is stupid enough to ping my machine from his home machine (so he can see how it's going), not thinking I'll notice that. When this happens, I can also email his home ISP, the people who really know who he is, and the people who can really hit him where it hurts. Except that they ignore my email too, and if they do email me back, they just tell me that the attack did not come from their ISP so they can't do anything, or there's no proof that the pinging is related to the attack.
Phone calls are much more effective than emails, but you really need to make them during the attack for them to take them seriously. And often the attacks happen outside of business hours, so there's nobody to call. And they're very time consuming.
Though I did succeed in nailing at least one guy. He was in Romania, and he messaged me a few weeks after the attack basically pleading with me that it wasn't him, but his brother using his computer. Apparantly the police (in Romania) were questioning him, and one of the things they showed him was my email. The police had never contacted me -- I'm guessing that my email was just one of many pieces of evidence they had against the guy. I felt a bit bad for him, but not that bad. Not that I had any control over what was happening to him at that point -- it was out of my hands the moment I sent my email.
So, if it happens again, I'll do the same thing. I know it's not likely that anything substantial will come from my emails, but there's still a chance. Every time it happens, I know I nail at least some of his compromised machines, and have a chance at getting him. I'll win eventually -- either that, or he'll hit puberty, in which case we both win.
hosts files (Score:2, Informative)
hosts.allow
#
# hosts.allow This file describes the names of the hosts which are
# allowed to use the local INET services, as decided
# by the '/usr/sbin/tcpd' server.
#
# Prevent those with no reverse DNS from connecting.
ALL : PARANOID : RFC931 20 : deny
# Allow anything from localhost. Note that an IP address (not a host
# name) *MUST* be specified for portmap(8).
ALL : 127.0.0.1 : allow
# internal ip
ALL : 192.168.1.100 : allow
ALL : 192.168.1.200 : allow
ALL : 192.168.1.201 : allow
ALL : 192.168.1.202 : allow
ALL : 192.168.1.203 : allow
ALL : 192.168.1.204 : allow
ALL : 192.168.1.205 : allow
ALL : 192.168.1.206 : allow
ALL : 192.168.1.207 : allow
ALL : 192.168.1.208 : allow
ALL : 192.168.1.209 : allow
ALL : 192.168.1.210 : allow
# other people you like go here
ALL : 00.000.000.00 : allow
# You need to be clever with finger; do _not_ backfinger!! You can easily
# start a "finger war".
fingerd : ALL \
: spawn (echo Finger. | \
: deny
hosts.deny
#
# hosts.deny This file describes the names of the hosts which are
# *not* allowed to use the local INET services, as decided
# by the '/usr/sbin/tcpd' server.
#
# The portmap line is redundant, but it is left to remind you that
# the new secure portmap uses hosts.deny and hosts.allow. In particular
# you should know that NFS uses portmap!
# The rest of the daemons are protected.
ALL : ALL \
: severity auth.info \
: twist
Re:Abuse@ (Score:2, Informative)
Gibraltar
Monserrat
British Virgin Islands
British Indian Ocean Territory
Pitcairn Island
Ascension Island
Falkland Islands
South Georgia
WON"T WORK (Score:2, Informative)