What Kind Of Logs Should ISPs Keep? 176
Effugas asks: "An engineer at a rather large ISP recently asked me a rather simple question that I didn't have a particularly good answer for: What logs should they be storing? He wasn't asking about the simple question of whether their own servers should be watched closely--that's obvious. He was asking about his routing infrastructure. I told him they of course musn't record the actual data being routed through their network; however, endpoint to endpoint route logs(since the establishment of those routes is the ISP's raison d'etre) did seem viable. But now, I'm not so sure--if there's one thing we learned from Kenneth Starr's subpeona of Lewinsky's book purchase records, it's that Barnes and Noble stored such records in the first place! But on the flip side, I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either. So I ask, without passing judgement in either direction: What options does a network administrator have for retaining forensic evidence in case of abuse, which ones are ethically justified, and what are the actual router configurations which implement such ethical systems?"
Everything (Score:2)
Okay, that might be a bit extreme, but it seems the only workable and enforceable policy. If you choose any other criterion, there will always be some unscrupulous ISPs which ignore it, and it gives people a false sense of security.
You asked for it... (Score:4)
To help keep their logs dry, they should purchase a log rack, or simply arrange the logs on top of a makeshift support system so that the logs do not directly contact the ground.
Moist logs tend to attract bugs and decompose much faster.
Its matter of ethics anyway. (Score:1)
Most of logs lose their information for administrator after some one week. In case you spot a hacker attack, that is more than 2 weeks old, you can as well forget to try to track anything in logs. Unless its real amateur, you get no informations there.
Therefore I think that there should be two cathegories of logs. One that is periodically thrown away each 1-2 weeks and one that is kept for longer time.
Well, let's see now, (Score:1)
For example:
(are you ready for this?)
Captain's log, Stardate 2433, We found several of our subscribers logging into a known rebellious website called Slashdot.com. They were summarily executed. (Etc. Etc.)
Cool, eh? YOU BETCHA!!
logging should be kept down (Score:4)
And when this does spiral out of control, efforts to redress the wrongs that have been committed, no matter how good-intentioned or extensive, will never fully wipe out the harm that has been caused within the lifetimes of those who have really been hurt most. Once you go too far, you can never truly come back.
So I would definitely urge keeping logging to an absolute minimum if you can't eliminate it entirely. If you can't really appreciate the wisdom of not logging, I strongly urge you to take a hike.
And then, after you come back from your tromp through tree-lined trails, to reconsider.
Time Frame (Score:3)
E-mail, routing information, and the like, should have a relatively short lifespan, if a person is being harassed, they should report it quickly. You should allow them a week or 2 for turnaround in such cases, and burn the necessary information to a CD or other storage media for any followup needed, when there is a report. You shouldn't, however, keep a long papertrail on your users, this only invades their privacy. If there is a legitimate need for such logs, it will arise relatively quickly.
Attack logs should be kept longer. All attack logs should be analyzed and damage should be evaluated. Appropriate individuals should be informed of the attack based on what has been compromised. Even these, however, should be trashed after a period of time. Do you really care about an unsuccessful attack 2 years ago? Probably not, you might, however, care about someone who root-kitted your server a year ago, since they probably still have the passwords of at least a few of your users.
Re:Yea (Score:1)
You're joking, right?
Re:Everything (Score:1)
There is a difference between security of data, and privacy. If I use https (or whatever secure protocol) the information that is being transmitted may be secure (or maybe not if a governmental security agency is interested in it), but why should the ISP log where I connected to, and for how long, and how often? I've paid for their services - specifically the ability to connect to the internet. Unless I specifically agree to them logging every transaction I make I do not see why they should.
(I realise that back in the real world that they do log things - without such logs many net criminals could not ever be caught. I also realise that with some ISPs, when you sign up you do agree to allow them to log stuff.)
Re:Everything (Score:1)
WTF? (Score:1)
I never knew that! What the hell is the US legal system doing prying into that kind of thing? One of these days it'll be illegal to buy books without some kind of ID.
Crypto (Score:5)
Personally, I would encrypt them all using public-private key crypto. The "public" key is what is used to feed the data into syslog, and the private key can be used to decrypt it if you need it. If your systems are physically or otherwise compromised, the attacker still cannot derive the private key as long as you maintain due diligence in maintaining the security of the logging host(s). This means you can log everything to your hearts content and not worry about privacy concerns, as much. Just make sure to put the standard disclaimers in your AUP.
I suspect, however, that wasn't quite the answer you were looking for. Honestly, in order to compromise most people's privacy requires an ungodly large harddrive to store all that information. Simply monitoring a T1 with a packetsniffer doing decent filtering can easily trash a fast 30GB HDD. The security industry is replete with stories of how crackers were caught because their packet sniffers went amok trying to log everything, and crashed the system trying.
I'd recommend logging the source and destination of mail, and when it was retrieved. If you are using RADIUS servers, log the times they signed on and off, and keep the system clock religiously on-time. Have the facilities to monitor each user (ie, be familiar with how to use a packetsniffer, and have a box on standby if you need to use it). A quick cheat would be to configure the RADIUS server to tell $SUSPECT connection to only use $MONITORED_IP and then tell the packetsniffer to dump everything from $MONITORED_IP to disk. It's simple, but it works.
As far as advice on law enforcement.. it depends on your situation. If you have been compromised, it still may do you more harm than good to report it due to the administrative overhead involved in prosecuting them. Generally, however, they are quite helpful on getting you the information you need to prosecute. Don't expect them to get too involved though unless your SMTP logs say that a message was sent from l335h4x0r@yourisp.com to president@whitehouse.gov with a subject line mentioning what he's going to do with a box of cigars and a can of surgical lubricant. In that case, you probably won't have any choice but to cooperate. :)
Hope this helps,
Elm (Score:2)
This has been a question for a while. (Score:1)
Cisco ROCKS!!!!
A log here a log there (Score:1)
logs... (Score:1)
I guess this should be logged:
Dial-in.
Excessive protocol floods.
Connects made to customers on ports that are used for trojan programs like netbus ect.
Troubleshooting (Score:1)
Those who argue that ISPs should not keep any logs are not being realistic.
Without logs, the ISP can only shrug its shoulders when a customer calls about email being dropped. With logs, the ISP has a chance to narrow the problem and fix it.
Logging (Score:4)
Assume they log nothing, for purposes of maintaining your own documentation.
Because the fact is, they probably don't log what you need them to log, and log all sorts of crap you wouldn't want them to.
What they should log, IMHO, is everything they can, but only keep it for a couple of weeks.
Having made use of everything from error logs to snooped IRC traffic to bust intruders on my systems, I recognize both the value of such logs, and the potential for abuse.
--
Attack logs (Score:1)
Re:You asked for it... (Score:2)
Freeze! (Score:1)
Nonetheless, it IS right. The ISP is providing Internet service (duh, that's what ISP means). Period, end of story. If they want to keep (or get back) "common carrier" status, they CANNOT log packet contents.
In my view it should go like this:
Harrasee: Hey, Mr ISP--your user BlahBlah keeps sending me threatening emails, please kick him off
Mr ISP: I have no way of checking the contents of incoming or outgoing emails so I can't verify what you say is true. Furthermore, even if I could, I am not a law enforcement agency and can't take action against this person.
--
Re:You asked for it... (Score:1)
To help keep their logs dry, they should purchase a log rack, or simply arrange the logs on top of a makeshift support system so that the logs do not directly contact the ground.
They should also be wary of the 'loghost', a terrible creature that eats away at your logs and stores vast quantities for food, thus making them all useless.
Re:Logs.... (Score:1)
and in reference to my prior post, i forgot one thing, "First Post", ha.
Logs? We don't need no schtinking logs! (Score:1)
In a much simpler time, I had an experience with an ISP that logged every ounce of traffic that made its way through their Annex portserver. They had enormous filesystems devoted entirely to violating their customers privacy. "Just in case"
I strongly disagree with this form of Orwellian observation, while at the same time, understand the need as an administrator to keep a certain number of logs to keep certain the system is running smoothly and that your users aren't taking down space shuttles, etc...
While more sophisticated users are aware that every keystroke can be logged, and have various facilities at thier disposal to conduct point to point encryption, the bulk of the people are unaware of this blatant violation of their most fundamental right to privacy, or more importantly, how to avoid invasions on said privacy.
In summary, I say that any logs in excess of what are necessary to continue the smooth operation of the server (which obviously vary from place to place) are entirely too many.
gitm
Re:Everything (Score:1)
It is not about the content, but about the logs what connections your have made.
In my opinion, the ISP should log everything that it needs to ensure that their own system is not being hacked/attacked and a reasonable amount of information to help their customers track down hostile attackers, and keep this data for a reasonable amount of time. Let's say a week (or two).
After this period the data should be deleted (not retired on tape). If you haven't found out you have been attacked by someone after two weeks you deserve it.
it's subjective (Score:2)
What it really comes down to, IMHO, is that information itself is rarely bad. Having information is neither good nor bad in itself.
Consider a widespread DDOS attack--in this case tracking down the origin is difficult enough, and having profuse logs would be a real plus not just for the ISP, but for the net at large.
On the other hand, logging routing traffic which shows that users X,Y, and Z downloaded metallica songs which they did not own, thereby making it possible to prosecute and put them in jail for a long time would come under the heading of a Very Bad Thing.
Notice that in each of these cases, having the data in itself is not bad--it depends entirely on what is done with it. The real question which should determine what logs should be kept is, how likely is it that this information will be abused?
disclaimer: I don't think that people will really go to jail for downloading metallica MP3's-that was just an example to illustrate a point-that if the existence of logs in a given situation, in this case a police state situation, were this likely to be abused, it would be a consciencious netizens duty to come up with a convincing reason why logging was impossible. Something about the data bandwidth of (n-1)^10000 exceeding possible logging potential of network based systems under primary load conditions. Impossible to argue with that, now, isn't it?
Re:Everything (Score:1)
Yup they are. Man tcpdump, man libpcap.
One NetAdmin at my former univ. did log every IP packet header of connections involving "outside" to a (by then) huge partition. And that wasn't a specially small-traffic site...
Logs to keep (Score:1)
Re:You asked for it... (Score:1)
Ethics and Pragmatics (Score:2)
Whatever policy is adopted, a breach of ethics would not arise from the maintenance of logs, but rather from the failure to inform customers that such logs are being maintained. By informing the customers, each customer is on notice to take steps to assure the security of any information sent in the clear or over the wire.
Re:Everything (Score:1)
Just for fun I tried tcpdump -i ppp0 on my linux box, and it totally flooded the screen, even with nothing downloading. Is this normal? I've never done this before...
--
When the bacon comes a knockin (Score:1)
The process of digging through hundreds of megs of logs to find out the proper sender IP from a webmail interface is quite possibly less entertaining then counting drops of water on your farhead in a chinese POW camp. And thats WITH the leather whip. I though i was going to go into epileptic shock. I ended up having to pass the job on to a coworker. yuk.
Just my two kronors....
Re:Everything (Score:2)
"Sweet creeping zombie Jesus!"
Keeping logs.... but what logs and why. (Score:2)
At least one belgian ISP got his password file very often cracked. So, if you can't track the connexion via the phone company, the information is useless. Even more, accesses are often pirated using tools like Back Orifice and such. So the information of what user connected is useless by itself...
Connexions made should definitively not be logged, for privacy and practical reasons. The people who do craking/pirating visit many web/ftp sites, connect to many machines each time they use internet. Those who only make 2 or 3 connexions are those who log on the net, connect to IRC and check their mail. Without forgetting about those web sites with so many ad-banners/counters/... that to visit one page, about 10 different IP are accessed !
Bad formed packet could be logged in order to spot people trying DoS, spoofing and such. again, how long is the question. If you can't track the real people connecting, it's useless.
Mail server use should also be tracked. but no mail content. (remind me of the FIDOnet time when many unscrupulous Sysops spend their time reading the mail going through their machine)
For the rest, AFAIK, log files can be modified at will. So I can't see how they could be used as legal evidence. IMHO, they could only be used as a tool to spot problems. But nothing more. So I think that all what is not needed for such purpose should not be logged.
Re:Attack logs (Score:1)
Re:Elm (Score:1)
Logging at our ISP... (Score:3)
... consists of tcplog and our RADIUS log.
Essentially that gives us a list of which IPs are in communication with our own, and a list of who was on what IP at the time the communication occurred. We keep our RADIUS log for 6 months for billing purposes and dispute settlements over billing, and our tcplogs are kept for one month.
Re:You asked for it... (Score:1)
It's big its heavy its wood!
It's Log! Log!
It's better than bad, it's good!
'basic' loggin. (Score:1)
Re:Everything (Score:1)
I don't necessarially agree with this. Just for an example, I'm going to point out e-mail. I don't think that either POP3 or SMTP are encrypted protocols. A lot of people who would rather not have an ISP keep a log of all their private e-mails use these protocols to transfer mail. In addition, AOL web-mail is not over a secure connection. When entering your password, you are directed to a secure connection, then back to an insecure connection when you actually read your mail.
However, I suppose, that if these logs are for the purpose of tracking down criminals, for example, the child pornographers mentioned in an above post, than keeping logs of people's e-mails might be desirable. Mind you, I would not approve of this policy, but then again, I'm not running an ISP.
./configure
make comment
make post
Re:When the bacon comes a knockin (Score:2)
. o O ( I wonder which the most popular cookie in my ONE GIG of httpd ERROR logs was? )
~Tim
--
Limits I feel should be placed on ISPs (Score:1)
I do know it is possible using anonimity services to retrieve just about everything you would want off the net. This however is slow, awkward and inconvenient but the biggest downside to these services is that they are only is use by the technically ept.
I feel that 'ordinary' users should be protected by law so that no more information is gathered from them than is gathered from a hardcore conspiracy theorist that routes all his traffic through unregulated offshore servers.
Just because PGP exists and is freely available (well almost freely) doesn't mean that those who are not savvy enough to use it should be punished and have their communications needlessly intercepted.
Re:Crypto (Score:1)
And how do you gurantee that the hacker with root-access can't get at the secret key actually used for encrypting the logs? After all, unless you have a few spare Crays taking up space, you wont be able to actually use (secure) asymetric encryption for data, only for secret keys ... AFAIK that is.
I say run important logs to a printer, and BURN them after a while. Then it'll be quite gone (unless something sits arund in your printspools or something).
All I'm saying is that if there is to be a point to crypto it has to be part of a carefully crafted system/strategy. After all - the only reason pgp makes sense for mail, is because we assume no (such ;) agency has put a camera right over our keyboards and/or screens.
Be afraid. Be very afraid
Re:Logs.... (Score:3)
Yeah, suuure. Get yourself a copy of "1984", read it, and learn why total surveillance is a bad thing. The very existence of such data is a danger in itself, because it can be used to commit crimes, and you can never be sure who eventually gets hold of it.
logx n = 0 (Score:1)
Bradford L.
Re:Logs.... (Score:2)
I think you will find that most states have document retention laws, which specify for how long you are able to keep certain kinds of documentation. Lawsuits have been lost because companies did not comply with these laws, i.e. kept logs/documents for too long.
You might want to recheck the laws in your state before you start keeping stuff "indefinitely".
Privacy is not in the logs... (Score:1)
Avioding faked emails. (Score:4)
What options does a network administrator have for retaining forensic evidence in case of abuse
This also ties into the carnivore question about faked emails. I've gotten some harrassing emails and considered forwarding them back to the sys admin of the jerk in question. However, realisticly, I could send anything I wanted with FWD in the title, and without digital signatures, they wouldn't know if I was forwarding a real email or not. But what kind of logs could they keep that they could confirm the authenticity of a message without invading the privacy of the user?
Now, bearing in mind that I don't do this for a living, wouldn't it be possible to set up a logging program that ran a metric on each message that came through, based on date, to and from and message content, that could not be reversed to actually produce that data, but would have an astronomically improbable chance of being reproduced by a fake message?
That way, the logs kept, just looking at them (even by the ISP) would tell them nothing but how many messages had gone to and fro from the whole ISP. But if someone came to them with an "incriminating" or "harrassing" email they could (at their discretion or under warrent) confirm the authenticity of that message actually having been sent by their service. If each ISP used their own metrics and kept them private, it would be very difficult for anyone to fake email evidence. This would be useful for both law enforcement/people being harrassed and the innocent but framed.
So, is this kind of log possible, and would it satisfy privacy advocates, since you couldn't even tell "how aften and when used" for any given user?
-Kahuna Burger
Re:logging should be kept down (Score:1)
UK RIP (snooping) Bill (Score:2)
Alot of ISPs are _threatening_ to pull of the UK because of this.
Perhaps a more interesting question.... (Score:2)
While most Slashdotters will answer a resounding no to those questions, what happens when child pornography comes into play? Should a police officer, or the FBI even, be able to demand an ISP hand over their logs, and examine them for people who have downloaded child porn? (Not exactly the easiest search, but I suppose doable none the less).
I think that determining who has access to the logs is perhaps even more important than determining what to log in the first place.
./configure
make comment
make post
Re:WTF? (Score:1)
it was a very legitimate query. people got freaked out because they heard starr was looking into her purchases and assumed it was part of his "reckless" investigation.
Re:Crypto (Score:1)
Well, it's "guarantee", and you guarantee that a hacker with root can't decrypt the data by never providing him the opportunity to get the key in the first place. I said this system would be using public/private key crypto, right? Okay, public encrypts.. so private....
And the private key isn't on the system, because it needs to remain secure.
I say run important logs to a printer, and BURN them after a while. Then it'll be quite gone (unless something sits arund in your printspools or something).
The CIA and NSA are well versed in recovering data after they were burned. Infact, this is how we have emoticons/smileys now - originally they were used as a code. But they killed the professor that created them, sealed the documents in magnesium binders, burned it, and then threw it in the ocean. Unfortunately some enterprising university kids got wind of this, went off-shore, recovered it, and reconstructed most of the data. This was AFTER a government agency burnt it to a crisp. So the idea of "using a printer" to secure your logs is one of the stupidest ways to do it - both in terms of space, and in terms of security.
After all - the only reason pgp makes sense for mail, is because we assume no (such ;) agency has put a camera right over our keyboards and/or screens.
Ugggghhhnnn. And here you prove the very point you're trying to dispute - PGP uses public/private key crypto.. the same solution that I was advocating be used to prevent your hypothetical cracker from getting access to my hypothetical system. I think I'll stop short of getting sarcastic here and hit the submit button...
Re:Time Frame (Score:1)
If only it were. In order to decide that something is a request, attack, whatever, you would need to log and analyze it.
Maybe a better way to look at the situation is by defining layers of logging.
Of course, you could always stick to a set of rules, ie: a SYN followed by the next X packets on tcp port 80 is an HTTP request, and should be logged.
I think this would fail unacceptably for attack logging though, as a frequent attribute of an attack is playing outside the traditional rules.
This also leaves you to play perpetual catch-up with new protocols.
Pretend it like a car! (Score:1)
they are using. Then they can go anywhere
on the internet and do what they want. If
someone with a warrant comes and says:
who did you give that plate to, then you
give it to them. And only what the warrant
specifies.
If you are going to keep more information than
that, then you should inform your cusotmers that
you are keeping tabs on them. But I dont think
you will keep your customers for very long
if you do.
Remember that a lawyer can get access to almost
anything for any reason. And it can be a civil,
not criminal matter: Custody, Divorce, Libel.
Do you really want to be the vessel used by
an unscrupulous lawyer doing a character assasination for his client on your customer?
And do you really want to get supoenaed every
day for access to your logs?
Nothing (Score:2)
The ISP should log nothing, out of their own self-interest. Anything they need to log, for their own purposes, should be destroyed after use.
Although it may be useful to Starr to find Lewinsky's book buying history, it's not good press for Barnes & Noble to have the existence of this log disclosed. Similarly it's never going to be in the ISPs interests to be at risk of having logs subpoenaed. The only legally-secure defence against this is to not have the logs in the first place (and this may require a traceable and provabel process to show that any that did exists have been destroyed).
In the UK... (Score:4)
To be honest, I don't think the harm is in the logging - it's what is done with the logs. Disclosure to third parties is definitely illegal and unethical, but the use of this sort of data within an organisation can also be dubious. How much would your marketing department like to know about the 'real' (read 'secret') interests of all of your customers?
I say you guys have got it pretty easy in the US, but at least we're now getting clear legislation (even if it is b0rked) saying what we can and can't do over here in the UK. To easily answer this question in the UK though, does require a few hours with a copy of the Data Protection Act, the Interception of Communications Act and the Regulation of Investigatory Powers bill. Even then, you're probably wrong.
As far as we what we do is concerned (as an ISP) - we log enough for billing, and we have some machines running an IDS in promisc. mode to pick up scans, viruses, etc. going across the network. Apart from that, it's all pretty standard syslog-out-of-the-box.
--
Spammers/child porn/death threats/etc (Score:3)
With the hash, the data can not be retrieved as such, but it is possible to verify objectionable content as genuine and not forged. This would be in the "kiddie porn/death threat/Metallica song" category.
These logs should be expired in a reasonable period of time. Any sufficiently serious death threat could not fail to be investigated within 30 days. Any behavior which is not repeated within that period of time can be considered at an end. Tough for the slowpoke.
Otherwise, no content logging, and no intrusive logging such as unauthorized snooping on what software is being used and how.
Re:You asked for it... (Score:1)
--
Your friendly neighborhood mIRC scripter.
if (ismoderator(reader)) hidemessage(this);
Transparent Web proxies: are you being logged? (Score:5)
Naturally, my ISP keeps logs for that traffic (Inktomi boasts that its Traffic Server can write many different log formats), in part to deal with abuse.
As you might also expect, the privacy policy does not directly cover these logs. It makes promises about some very specific types of information, but does not make any general statements that obviously pertain to types of information not covered in the enumerated, specific types. Result: I think most lawyers would say my ISP could sell access to DoubleClick, the FBI, or anyone else.
Checking your system
So are you using a proxy, but don't know it? You can check pretty quickly (though I should warn you, while a positive/proxy result is conclusive, a negative/no-proxy result may be a result of the proxy configuration, as the systems can be set up to bypass the proxy for certain sites, or to only use the proxy for certain sites, etc.).
Step 1: what's your address?
Check your current address for whatever network adapter (ethernet card, PPP/dialup device, etc.). In Unix or Linux, something like '/sbin/ifconfig eth0' will do; in Windows 9x, run 'winipcfg'; in Windows NT, 'ipconfig'.
Step 2: what address do web sites see?
Go to a URL that will show you the environment variables passed to CGI scripts, like http://www.cgihost.com/cgi-bin/env.cgi [cgihost.com] or http://www.ualberta.ca/htbin/dumpenv.pl [ualberta.ca] . Look at REMOTE_ADDR. Reload several times. Does it change? You might see some other proxy-specific variables like HTTP_CLIENT_IP and HTTP_VIA, depending on the proxy server's configuration.
Step 3: interpreting the results
If you ever see a REMOTE_ADDR value in Step 2 that doesn't match the local address from Step 1, yet you don't have a Manual or Automatic proxy configured in your browser, then congratulations, you're behind a transparent proxy, and should assume that all your Web traffic is being logged.
http:// vs https:// For regular HTTP, there's a lot they can conceivably record. The URL. Your cookies. Where you came from. Etc. For https:// it's a bit better. All they can do is record where you connected to, and when. Even this information might be deemed valuable, e.g., someone frequently connecting to many banking sites probably isn't eligible for low income tax credits. https:// is somewhat like encrypting your email: they can't tell what you're doing, but they can tell who you're contacting.
I've complained via email a few times, and received a couple polite emails from the technical staff. But nothing has changed in the official policy, so my ISP is still free to share my complete Web usage history with whomever they wish. Highest bidder? Most pushy government agency? I can't say.
-Peter
Keeping logs (Score:1)
I don't know for ISPs since I'm administering the Unix domain of a hosting company. Since we are usually the victim of an attack I know that I'd report to an ISP whitin a couple of days of the attack. If I find that the security of a server/subnet have been compromised for a longer while than 3 days, we usually resoft everything in that area and restore the latest safe backup of the content, the system and binary that are executed with this content are usually replaced with a later version. I seldom find any use in reporting such an incident to an ISP based on the logs that are kept, because they can be pretty different from what happened (as in tangeled with).
I know of a safer method for logging even tho I don't use it. Using a serial port for logging for instance, not beeing able to mess up the log from the server that actually generates the log is pretty secure.
Log ip use, nothing else (Score:1)
For web servers: The standard log, with referers, is a nice thing.
--Mike--
Re:Logs, none, incorrect (Score:5)
These logs should include:
* Radius logs - username, port, and time, (Caller ID or npanxx info if you can get it), and IP assignment.
* SMTP logs - SMTP ID. Actual copies of emails would require too much space than available to any ISP.
* NNTP logs - again ID information only (NNTP post ID, date, time, etc).
* Accounting logs as relevant to specific devices - for instance, shell and web servers which allow for telnet/ssh access, ftp servers, etc. This is not spying, this is good system administration.
* DNS - knowing about those lame delegations is a big help. Especially when your customers routinely register domain names with your name servers as authoritative but fail to alert you!
* Most important, accounting logs for root level commands as executed by the system's administrators. This can be a sore spot with some admins, but logging into a machine as root or su'ing immediately to root after login does not present accurate data as to what the admins are doing on a box. Using sudo or one of the other packages and maintaining an adherence policy to its' use should be expected. (Yes, yes there are ways around it..).
Most of these things are standard practices for any of you who have worked for an ISP. I could care less what people were doing online unless they were violating our TOS/AUP and generated complaints. At that point, we needed to know who was doing what in order to fufill our contractual obligations to all of our customers.
Re:Everything (Score:1)
You probably did. Ain't fine print wonderful. My ISP (RoadRunner) admits to keeping full logs for as long as you are a customer, or 15 years after you stop becoming a customer. "For Billing purposes". WTF??? It's a monthly bill, not traffic
Re:Everything (Score:2)
Re:Freeze! (Score:2)
What course of action the ISP takes is subjective. However, most abusers go quietly or make up some wildly outrageous story that nobody believes.
Re:Keeping logs (Score:1)
Yes there is. It's great marketing information. Unfortunately, that means we get the shaft.
Full lyrics :) (Score:1)
What rolls down stairs,
Alone or in pairs,
Rolls over your neighbor's dog?
What's great for a snack,
And fits on your back,
It's Log...Log...Log!!
It's Lo-og, Lo-og,
It's big, it's heavy, It's wood!
It's Lo-og, Lo-og,
It's better than bad, It's good!!!
Everyone wants a Log!
You're gonna love it, Log!
Come on and get your Log!
(Everyone needs a... Come on and get your... You're gonna love it, Log!)
Log, from Blammo!
--
Your friendly neighborhood mIRC scripter.
if (ismoderator(reader)) hidemessage(this);
Re:Avioding faked emails. (Score:1)
I guess the only problem would be if different destination email systems mangled the contents to a point where the contents couldn't be verified anymore by the key, not to mention that if this were to be widely implemented, the sizes of the average email sent would begin to increase, and add additional load to email servers and their bandwidth. (although I'm guessing that would be negligible)
Micro$oft(R) Windoze NT(TM)
(C) Copyright 1985-1996 Micro$oft Corp.
C:\>uptime
Re:Freeze! (Score:1)
Mr ISP: I have no way of checking the contents of incoming or outgoing emails so I can't verify what you say is true. Furthermore, even if I could, I am not a law enforcement agency and can't take action against this person.
It's not a question of being "a law enforcement agency". If the harassor is violating the AUP he agreed to then the ISP have every right ot kick him off. This assumes that the harassee checked the ISP's AUP to make sure the harassor was breaking it of course.
Re:You asked for it... (Score:1)
Re:You asked for it... (Score:1)
Re:You asked for it... (Score:1)
You can also use pine sap to paint little figures on the sides of logs and then have fire people. Keeps the kids amused...
Re:Logs.... (Score:3)
Regards
Re:You asked for it... (Score:1)
Re:Crypto (Score:1)
This sounds remarkably like... (Score:2)
What do you log? As has been said, packet sniffing content would take ungodly amounts of storage, and if you're an ISP, you really shouldn't be doing it. It's Just Wrong (tm). Once again, it depends how tyrannical you want to be, but I think that just monitoring what IP's are hitting your boxes when is sufficient for most security concerns. At the most I'd say take note of traffic patterns, just incase a customer's box has been broken into and is doing things it didn't normally do.
Should logs be permanent? We all should be able to come up with one real simple example of a corporation that was burned by e-mail leaking out that honestly shouldn't have. Corporations are now beginning to take a policy of purging e-mail stores often, so it doesn't come back to bite them in the ass. Is this ethical? Probably not. Which is why you have every right to be dumping your logs too. If corporation XYZ comes to you looking to see if the maintainer of corporationxyzsucks.com is one of your customers... sorry, you dumped the log. Don't get me wrong here, I'm not saying that ISP's shouldn't help big evil corporations if someone from them DoS'd them. I'm just saying that ISP's have a right to 'lose' information just like corporations do. Things are much less of a hassle that way.
Legal issues. If I were a customer of an ISP that suddenly decided to start logging everything, they damn well better tell me that their terms of service are changing. Anonymity is something I value, and is a key factor in my ISP choice. What with all the DoubleClick-ish privacy things going on right now, I would not get yourself into that mess. Let your customers know exactly what you're logging, they have every right to know.
Perhaps this is all remarkably obvious, and the opinions have been karma whored up by now, but I just thought I'd offer my two cents.
I still maintain... (Score:2)
Of course, what they do with this information is the important part.
All data logged (Score:2)
And don't kid yourself that many ISPs are not. And unless you are administering the ISP yourself, don't kid yourself that YOU are not having all your network traffic recorded.
It is like Microsoft or Real sending pings of your net traffic back to home base across the net. There is little motivation for an ISP to abstain from such activity. It is very tough to get caught. And some people will pay for your data, especially if you preprocess it properly.
Whatever you keep, destroy them soon (Score:2)
Forged Packets (Score:2)
Re:Yea (Score:2)
No. Nice try, though. Its the emergency backup heavy duty LART stick [winternet.com].
Someone else.. (Score:3)
Should you use encryption, to keep your data secure? *YES* absolutely.
Should your ISP be forced to keep your surfing habits private? *ABSOLUTELY*
Should they be allowed to log as much data as they want for their own analysis later? *ABSOLUTELY*. Why? Because they *can*. It's *THEIR* network. If we say 'they can't' they can just put it in the contract; you want to use @home, you agree that we may log as much information about packet flow as we want. Period.
Re:Freeze! (Score:2)
After all, my ISP has a transparent proxy on my web traffic. That's not 'common carrier'. That's interfering.
Common carrier means that they simply move data, and have 0% responsibility as to what kind of data, or where it goes.
@home telling me *don't run a server* and scannign me for servers is *NOT* the behavior of a common carrier, as they are dictating what types of traffic I may produce.
my 0.02 (Score:2)
First, I do not currently work at an ISP, but I have done. I also have administered arrangements for remote access at educational estabishments, thereby effectively being an ISP for the students and staff. This was a VERY thorny question for us in all those cases. We recorded who connected when, with what IP, and who accessed the services we provided, again recording the source IP. Those logs were kept for a few months. Logs of suspected probes were kept for a few weeks, overt attacks for longer. That was it. With this info we were able to pin down the account associated with any abuse reports and spot a few compromised user accounts (usually because somebody used the same password for everything and it got cracked somewhere else) by seeing the same user pop up twice from different locations at the same time.
The logs we kept on OURSELVES though were much more thorough. Anything one of our machines did was watched somewhere and whilst most of those logs were short-term and verbose enough to require scripted assistance to scan in any meaningful manner we made damn sure that we looked into everything that poked up above the background noise level there.
Privacy was important too - in all cases it was clearly understood that discussing logged info with anyone outside the admin team apart from the customer who owned a suspect account was cause for getting fired immediately. To even discuss it with the customer required written authorisation. If anyone else wanted the info it had to go through the head of the admin team. Marketing folks, the billing dept, top level management (by their own request) or support staff did not have access to that raw data and it would only be turned over to anyone outside the company with a court order.
Other guys at the company sometimes accused us (the admin team) of being anal about it and I guess we were, but the complaints sure dried up when the policy saved us from getting our ass sued.
# human firmware exploit
# Word will insert into your optic buffer
# without bounds checking
Lions and tigers and bears, oh my! (Score:2)
I don't think a lot of people really think about this stuff when they do it. Yes we all want to be safe. We want our friends to be safe. Sometimes we even want those we despise to be safe as well.
But where do we draw the line? This type of thinking is as dangerous as blanket "log everything no matter what!" As the story suggests, Barnes and Noble learned a very important lesson when they kept track of everything.
Remember, the moment you give up just one of your rights to privacy is the moment you have given them all up. Also remember that "protecting the little children", as the religious right likes to say all the time, does not mean that MY rights as an adult should be erroded because of whatever draconian law they want passed.
None whatsoever. (Score:2)
They should log what someone could be sued over (Score:2)
/* Wayne Pascoe
Sniffer log overflow? (Score:3)
A T1 is 1.5 megabits/sec. To fill up 30 gigabytes recording _all_ data sent across the T1 (no filtering) would take about 44 hours. If a cracker leaves a sniffer unattended for that long, I have little sympathy for them.
Overflowing a user account I can believe, but I would be amazed if drive overflow was a significant problem for the vast majority of packet-sniffing crackers. Heck, cut out HTTP and take only the first few packets of an FTP or POP session's data and you've reduced your data load by a factor of 100 or more, while keeping the information you're interested in (passwords).
In summary, I don't think that drive space is a problem for a half-way competent sniffer.
Re:You asked for it... (Score:2)
Re:Full lyrics :) (Score:2)
yes you need logging (Score:2)
Yes you must respect privacy, but you shoudl also state clearly and in laymans terms what your privacy policy is, and stick with it even when times are tough.
What you should log depends on what your needs are and also what services you provide. Remeber though that you may be held responsibe for someone abusing your network, so it may be wise to keep track of who is on it and from where.
send flames > /dev/null
Re:Full lyrics :) (Score:2)
Re:Nothing (Score:2)
We keep logs of connections to our boxes, the ip number given to that connection, the login name of the person connecting, what they connected at and for how long. We do not log where they went after they connected just that they connected and got a good solid IP number for the machines dns resolution.
This not only helps us with router and dailup box diagnoses incase of trouble, but it also allows us to better help a customer who is having connection problems because it shows, when, how, and how fast they connected, and how they disconnected, timeout or dropped carrier.
IMHO this is really all the log needed kept as far as what our customers are doing internal system logs on the other hand we keep very extensible because of the high number of hack attempts that go on against larger ISP's.
Re:Transparent Web proxies: are you being logged? (Score:3)
It's in the Echelon Users Manual (Score:2)
Good ol' USA!
Rejected packets (Score:2)
Today, if you did that, you'd be overwhelmed. But it's useful to have the capability and to log such stuff during a peak period now and then, just to get an indication of what junk is out there. Most denial-of-service attacks will show up in such logs, of course.
Not quite analogous... (Score:2)
That's a different issue. Lewinsky's purchase history was based on *financial* records. Financial records MUST MUST MUST be kept to help eliminate errors, and correct them when they do arise. Not to mention for tax purposes...
-JF
Re:Crypto (Score:4)
Because the public key only encrypts a 128 bit (or whatever) session key every ten minutes or so, it's fairly quick, and two-way crypto is very quick, easily enough to dump logs through.
If you ever implement a log system and don't want them modified, keep an ID # in each packet of logs, along with a MD5 hash of the previous packet of logs (including the previous-packet hash of the log file before it.) This way if a log is modified, the attacked has to change all logs after that point.
Ideally you'd also have the log catcher dumping logs to a write-only media, like CDs. Preferably in a session-based way, so it didn't have to wait too long between getting logs and writing them.
Re:Logs, none, incorrect (Score:2)
Re:Crypto (Score:2)
Simple logs, based on multiple users, such as bandwidth usage, number of connections to various services, etc, should all be plaintext to make them easier to use. But logs of individual connections, when someone picked up email, what sites people went to, MD5 hashes of outgoing mail, etc, shouldn't be plaintext.
And the specifics of many of these logs would be unimportant, you rarely need to prove a user did or didn't mail something, so if it sits encrypted on a CD for a year, and then takes ten minutes to decrypt and view, no big deal. Much better that it can't be easily accessed by someone unauthorized.
Of course, if your random number generation was flawed, all of your session keys could be compromised, but ideally you'd use fairly strong methods. And yeah, there are swap issues and all to deal with, but I'd left the details to the user's discretion.
And as for the trashing, that's why you'd offload them to CD or hardcopy frequently. Perhaps you'd dump the MD5 hash to a lineprinter every time a log bundle came in and dump it to a CD every time you got a few MBs... (Depending if you can write multisession CDs.)
But, to summarize. Not only are logs for catching bad guys, but they're also private info, which if you collect sensitive stuff, needs to be guarded properly.