Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
News

What Kind Of Logs Should ISPs Keep? 176

Posted by Cliff from the stuff-to-think-about dept.
Effugas asks: "An engineer at a rather large ISP recently asked me a rather simple question that I didn't have a particularly good answer for: What logs should they be storing? He wasn't asking about the simple question of whether their own servers should be watched closely--that's obvious. He was asking about his routing infrastructure. I told him they of course musn't record the actual data being routed through their network; however, endpoint to endpoint route logs(since the establishment of those routes is the ISP's raison d'etre) did seem viable. But now, I'm not so sure--if there's one thing we learned from Kenneth Starr's subpeona of Lewinsky's book purchase records, it's that Barnes and Noble stored such records in the first place! But on the flip side, I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either. So I ask, without passing judgement in either direction: What options does a network administrator have for retaining forensic evidence in case of abuse, which ones are ethically justified, and what are the actual router configurations which implement such ethical systems?"
This discussion has been archived. No new comments can be posted.

What Kind of Logs Should ISPs Keep? More

What Kind of Logs Should ISPs Keep?

Comments Filter:
  • The ISP should feel free to log anything. Anyone who wants their data to be secure will be using https or ssh anyway.

    Okay, that might be a bit extreme, but it seems the only workable and enforceable policy. If you choose any other criterion, there will always be some unscrupulous ISPs which ignore it, and it gives people a false sense of security.

  • You asked for it... (Score:4)

    by Trinition ( 114758 ) on Thursday July 13, 2000 @02:49AM (#937640) Homepage
    Anyone, including ISPs, should always keep dry logs. The type of wood varies with personal preference. For example, Pine has a strong, well... piney odor -- especially when combusted.

    To help keep their logs dry, they should purchase a log rack, or simply arrange the logs on top of a makeshift support system so that the logs do not directly contact the ground.

    Moist logs tend to attract bugs and decompose much faster.

  • Since they virtually can get the information, its anyway also just a matter of ethics. In my opinion they should keep all logs about all connections, but the question should be asked: "How fast they should throw them away?"

    Most of logs lose their information for administrator after some one week. In case you spot a hacker attack, that is more than 2 weeks old, you can as well forget to try to track anything in logs. Unless its real amateur, you get no informations there.

    Therefore I think that there should be two cathegories of logs. One that is periodically thrown away each 1-2 weeks and one that is kept for longer time.

  • All ISP's should be held accountable for log entries that hold up to the industry prescribed format, which begins with the reporting parties official rank, followed by universal date sequence followed by a detailed analysis of his or her findings.

    For example:

    (are you ready for this?)

    Captain's log, Stardate 2433, We found several of our subscribers logging into a known rebellious website called Slashdot.com. They were summarily executed. (Etc. Etc.)

    Cool, eh? YOU BETCHA!!

  • logging should be kept down (Score:4)

    by Anonymous Coward on Thursday July 13, 2000 @02:53AM (#937643)
    Logging in this country has gone way too far and is an abuse that cannot be permitted to progress any further. Not only does abuse of this cost many of us what they view as their birthright, but it also scares the hell out of those who haven't lost anything due to it yet. Sure, there are definitely some political and corporate interests who benefit by letting this sort of thing run rampant, but can we really afford it? And who's this world for, anyway -- the corporations or the people?

    And when this does spiral out of control, efforts to redress the wrongs that have been committed, no matter how good-intentioned or extensive, will never fully wipe out the harm that has been caused within the lifetimes of those who have really been hurt most. Once you go too far, you can never truly come back.

    So I would definitely urge keeping logging to an absolute minimum if you can't eliminate it entirely. If you can't really appreciate the wisdom of not logging, I strongly urge you to take a hike.

    And then, after you come back from your tromp through tree-lined trails, to reconsider. :)

  • Time Frame (Score:3)

    by Dungeon Dweller ( 134014 ) on Thursday July 13, 2000 @02:55AM (#937644)
    It's really not a question of what logs to keep, but how long to keep them. You should keep logs of requests, attacks, e-mails, routing information, anything that you might actually need, but only keep them for the appropriate period of time. You really don't want to have to dedicate a tape changer to this anyway, do you?

    E-mail, routing information, and the like, should have a relatively short lifespan, if a person is being harassed, they should report it quickly. You should allow them a week or 2 for turnaround in such cases, and burn the necessary information to a CD or other storage media for any followup needed, when there is a report. You shouldn't, however, keep a long papertrail on your users, this only invades their privacy. If there is a legitimate need for such logs, it will arise relatively quickly.

    Attack logs should be kept longer. All attack logs should be analyzed and damage should be evaluated. Appropriate individuals should be informed of the attack based on what has been compromised. Even these, however, should be trashed after a period of time. Do you really care about an unsuccessful attack 2 years ago? Probably not, you might, however, care about someone who root-kitted your server a year ago, since they probably still have the passwords of at least a few of your users.
  • But why would in ISP need logs?

    You're joking, right?

  • Playing devils advocate:

    There is a difference between security of data, and privacy. If I use https (or whatever secure protocol) the information that is being transmitted may be secure (or maybe not if a governmental security agency is interested in it), but why should the ISP log where I connected to, and for how long, and how often? I've paid for their services - specifically the ability to connect to the internet. Unless I specifically agree to them logging every transaction I make I do not see why they should.

    (I realise that back in the real world that they do log things - without such logs many net criminals could not ever be caught. I also realise that with some ISPs, when you sign up you do agree to allow them to log stuff.)
  • I admit that logging everything is probably the best open policy- but while using https or ssh will gaurantee users that their exchanges will be kept private, it still means that who/what/where they connect to can still be kept, logged, profiled, etc.
  • ... Kenneth Starr's subpeona of Lewinsky's book purchase records ...

    I never knew that! What the hell is the US legal system doing prying into that kind of thing? One of these days it'll be illegal to buy books without some kind of ID.

  • Crypto (Score:5)

    by Signal 11 ( 7608 ) on Thursday July 13, 2000 @02:56AM (#937649)
    That depends. You could get in trouble for taking this advice, depending on what form of tyranny^H^H^H^Hgovernment you happen to live under...

    Personally, I would encrypt them all using public-private key crypto. The "public" key is what is used to feed the data into syslog, and the private key can be used to decrypt it if you need it. If your systems are physically or otherwise compromised, the attacker still cannot derive the private key as long as you maintain due diligence in maintaining the security of the logging host(s). This means you can log everything to your hearts content and not worry about privacy concerns, as much. Just make sure to put the standard disclaimers in your AUP.

    I suspect, however, that wasn't quite the answer you were looking for. Honestly, in order to compromise most people's privacy requires an ungodly large harddrive to store all that information. Simply monitoring a T1 with a packetsniffer doing decent filtering can easily trash a fast 30GB HDD. The security industry is replete with stories of how crackers were caught because their packet sniffers went amok trying to log everything, and crashed the system trying.

    I'd recommend logging the source and destination of mail, and when it was retrieved. If you are using RADIUS servers, log the times they signed on and off, and keep the system clock religiously on-time. Have the facilities to monitor each user (ie, be familiar with how to use a packetsniffer, and have a box on standby if you need to use it). A quick cheat would be to configure the RADIUS server to tell $SUSPECT connection to only use $MONITORED_IP and then tell the packetsniffer to dump everything from $MONITORED_IP to disk. It's simple, but it works.

    As far as advice on law enforcement.. it depends on your situation. If you have been compromised, it still may do you more harm than good to report it due to the administrative overhead involved in prosecuting them. Generally, however, they are quite helpful on getting you the information you need to prosecute. Don't expect them to get too involved though unless your SMTP logs say that a message was sent from l335h4x0r@yourisp.com to president@whitehouse.gov with a subject line mentioning what he's going to do with a box of cigars and a can of surgical lubricant. In that case, you probably won't have any choice but to cooperate. :)

    Hope this helps,

  • I think that ELM has a much longer burn time, and a stronger, prouder smell, that reminds me of my heritage.
  • I've found myself asking the same question. How much info do you really need to log? Unfortunately, many ISP's never even think about this, until something bad happens. Right now, I make sure that basic routing info getts logged, and anything that is blocked via our access lists.

    Cisco ROCKS!!!!
  • A logAlog everywhere and the FBI search for Mitnick in there
  • ISPs should log to a certain point. It shouldnt become the task with the highest priority.

    I guess this should be logged:

    Dial-in.
    Excessive protocol floods.
    Connects made to customers on ports that are used for trojan programs like netbus ect.
  • In my opinion, the only valid use for keeping logs is to help troubleshoot any problems that may occur in the network. The details recorded and length of time the logs are kept should be consistent with this purpose.

    Those who argue that ISPs should not keep any logs are not being realistic.

    Without logs, the ISP can only shrug its shoulders when a customer calls about email being dropped. With logs, the ISP has a chance to narrow the problem and fix it.

  • Logging (Score:4)

    by Syberghost ( 10557 ) <syberghostNO@SPAMsyberghost.com> on Thursday July 13, 2000 @03:00AM (#937655)
    Assume they log everything, for purposes of guaranteeing your own privacy.

    Assume they log nothing, for purposes of maintaining your own documentation.

    Because the fact is, they probably don't log what you need them to log, and log all sorts of crap you wouldn't want them to.

    What they should log, IMHO, is everything they can, but only keep it for a couple of weeks.

    Having made use of everything from error logs to snooped IRC traffic to bust intruders on my systems, I recognize both the value of such logs, and the potential for abuse.

    --
  • Since most attacks are perpetrated by people who I can only refer to as true amatures, I said that attack logs should hang around for a while given that the security was successfully breached, whereas an unsuccessful attack, or some moron who is trying to "bitchslap" your webserver (can you believe that there are people who try that on boxen that obviously won't respond to such an attack), should be tossed after a few weeks, given that no repeat attempt has been made. If there has been a repeat attempt, older logs may be of use.
  • Pine leaves alot of sap residue in your fireplace. Honestly, I would recommend using dry oak as it burns dry and hot. You can also turn the dampener down quite abit and it will happily burn for hours and hours. The only wood I wouldn't recommend is poplar. Up here in Minnesota, that wood is very common, however it burns very crappy. Just avoid it, trust me.

  • "I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either."

    Nonetheless, it IS right. The ISP is providing Internet service (duh, that's what ISP means). Period, end of story. If they want to keep (or get back) "common carrier" status, they CANNOT log packet contents.

    In my view it should go like this:

    Harrasee: Hey, Mr ISP--your user BlahBlah keeps sending me threatening emails, please kick him off
    Mr ISP: I have no way of checking the contents of incoming or outgoing emails so I can't verify what you say is true. Furthermore, even if I could, I am not a law enforcement agency and can't take action against this person.
    --

  • To help keep their logs dry, they should purchase a log rack, or simply arrange the logs on top of a makeshift support system so that the logs do not directly contact the ground.

    They should also be wary of the 'loghost', a terrible creature that eats away at your logs and stores vast quantities for food, thus making them all useless.

  • Perhaps in clarification, i should state that i'm speaking more in terms of hoping for ISPS to be more like FreeNet, etc. Than an industry, as far as what is kept on file. Granted, with Centralized servers and the laws being passed, one should lean more towards logs of intrusions to the system, ftp, perhaps telnet (i'm refering to connections directly to the server, not beyond that, etc.

    and in reference to my prior post, i forgot one thing, "First Post", ha.

  • In a much simpler time, I had an experience with an ISP that logged every ounce of traffic that made its way through their Annex portserver. They had enormous filesystems devoted entirely to violating their customers privacy. "Just in case"

    I strongly disagree with this form of Orwellian observation, while at the same time, understand the need as an administrator to keep a certain number of logs to keep certain the system is running smoothly and that your users aren't taking down space shuttles, etc...

    While more sophisticated users are aware that every keystroke can be logged, and have various facilities at thier disposal to conduct point to point encryption, the bulk of the people are unaware of this blatant violation of their most fundamental right to privacy, or more importantly, how to avoid invasions on said privacy.

    In summary, I say that any logs in excess of what are necessary to continue the smooth operation of the server (which obviously vary from place to place) are entirely too many.

    gitm

  • It is not about the content, but about the logs what connections your have made.

    In my opinion, the ISP should log everything that it needs to ensure that their own system is not being hacked/attacked and a reasonable amount of information to help their customers track down hostile attackers, and keep this data for a reasonable amount of time. Let's say a week (or two).

    After this period the data should be deleted (not retired on tape). If you haven't found out you have been attacked by someone after two weeks you deserve it.

  • just like anything else in life, it's a subjective question. different ISP will have different needs.

    What it really comes down to, IMHO, is that information itself is rarely bad. Having information is neither good nor bad in itself.

    Consider a widespread DDOS attack--in this case tracking down the origin is difficult enough, and having profuse logs would be a real plus not just for the ISP, but for the net at large.

    On the other hand, logging routing traffic which shows that users X,Y, and Z downloaded metallica songs which they did not own, thereby making it possible to prosecute and put them in jail for a long time would come under the heading of a Very Bad Thing.

    Notice that in each of these cases, having the data in itself is not bad--it depends entirely on what is done with it. The real question which should determine what logs should be kept is, how likely is it that this information will be abused?

    disclaimer: I don't think that people will really go to jail for downloading metallica MP3's-that was just an example to illustrate a point-that if the existence of logs in a given situation, in this case a police state situation, were this likely to be abused, it would be a consciencious netizens duty to come up with a convincing reason why logging was impossible. Something about the data bandwidth of (n-1)^10000 exceeding possible logging potential of network based systems under primary load conditions. Impossible to argue with that, now, isn't it?

  • >Are per packet logs even possible with IP?

    Yup they are. Man tcpdump, man libpcap.

    One NetAdmin at my former univ. did log every IP packet header of connections involving "outside" to a (by then) huge partition. And that wasn't a specially small-traffic site...
  • IMHO, an ISP should feel free to keep logs of all traffic that crosses their network. The use of the logs afterwards is really what matters as well as how long the logs are kept. In general, I'd feel comfortable saying that a year worth of logs is a good time frame. However, if that's too short you might want to keep the logs for as long as you can based on available storage. As for what to do with the logs after you have collected them, that's pretty easy. Nothing. Unless of course they are needed by 'the authorities' for an investigation. In that case a method of reporting is needed that will only report on the desired target so that you don't end up looking at EVERYONE with suspicion.
  • Nothing better than shag-bark hickory period
  • In view of the competing interests and liabilities of the ISP, it is probably pragmatically necessary for the ISP to maintain as comprehensive a set of logs as possible.

    Whatever policy is adopted, a breach of ethics would not arise from the maintenance of logs, but rather from the failure to inform customers that such logs are being maintained. By informing the customers, each customer is on notice to take steps to assure the security of any information sent in the clear or over the wire.
  • If you log everything, it would take quite a bit of room, as well as system resources.

    Just for fun I tried tcpdump -i ppp0 on my linux box, and it totally flooded the screen, even with nothing downloading. Is this normal? I've never done this before...
    --

  • I work at a large internet portal in sweden, and, amoung other things, we have a free ISP and mail service. A few days ago a letter was put on my desk from the police asking for a mail that was sent from our service to be traced.

    The process of digging through hundreds of megs of logs to find out the proper sender IP from a webmail interface is quite possibly less entertaining then counting drops of water on your farhead in a chinese POW camp. And thats WITH the leather whip. I though i was going to go into epileptic shock. I ended up having to pass the job on to a coworker. yuk.

    Just my two kronors....
    /nutt
  • But I think for the most part, one would not have the expectation that those would be kept secret. The phone company has the same information about the calls that you make. What is important then is to make sure that there are policies in place that determines who gets that information, and for what purpose. If you are going to keep any sort of logs or records of activity, who/where/when is probably one of the most basic ones, and I would expect most people know it will be logged. Just important to 1)make sure people know that sort of information is subject to logging 2)Clearly define the circumstances when the logs will be made available, and make sure your customers know that too.

    "Sweet creeping zombie Jesus!"
  • Keeping logs of who connected when may be interresting (in order to help finding those who used their access to hack someone else)... but are useless after the phone company purged their log.

    At least one belgian ISP got his password file very often cracked. So, if you can't track the connexion via the phone company, the information is useless. Even more, accesses are often pirated using tools like Back Orifice and such. So the information of what user connected is useless by itself...

    Connexions made should definitively not be logged, for privacy and practical reasons. The people who do craking/pirating visit many web/ftp sites, connect to many machines each time they use internet. Those who only make 2 or 3 connexions are those who log on the net, connect to IRC and check their mail. Without forgetting about those web sites with so many ad-banners/counters/... that to visit one page, about 10 different IP are accessed !

    Bad formed packet could be logged in order to spot people trying DoS, spoofing and such. again, how long is the question. If you can't track the real people connecting, it's useless.

    Mail server use should also be tracked. but no mail content. (remind me of the FIDOnet time when many unscrupulous Sysops spend their time reading the mail going through their machine)

    For the rest, AFAIK, log files can be modified at will. So I can't see how they could be used as legal evidence. IMHO, they could only be used as a tool to spot problems. But nothing more. So I think that all what is not needed for such purpose should not be logged.

  • Anymore, attacks are rarely through some exploit, but more by using a lot of bandwidth to shut the site down. DDOS coms to mind. For example, when yahoo got shut down, it was because DDoSers were costantly throwing search requests at it, and it finally buckeled. Honestly, you log everything, and then sort it out when you have to.

  • Just don't burn my mutt. Thats just cruelty to animals.

  • Logging at our ISP... (Score:3)

    by tzanger ( 1575 ) on Thursday July 13, 2000 @03:16AM (#937674) Homepage

    ... consists of tcplog and our RADIUS log.

    Essentially that gives us a list of which IPs are in communication with our own, and a list of who was on what IP at the time the communication occurred. We keep our RADIUS log for 6 months for billing purposes and dispute settlements over billing, and our tcplogs are kept for one month.

  • It's Log! Log!
    It's big its heavy its wood!
    It's Log! Log!
    It's better than bad, it's good!
  • Well, really, to save their ass, they should log when data is transferred, but not where or how much, or what it is, for that matter. Sort of like how phone calls are recorded at the phone company, except this won't keep the destination, because that can easily saturate a log. (imagine the logs from multiplayer games where you contact other computers for peer.. or someone after one night of Gnutella.. yeeesh). On the flip side, they could always just keep track of things like user login/logoff, any server accesses they did locally (say, to DNS, mail, news, etc) which would do about the same.
  • Anyone who wants their data to be secure will be using https or ssh anyway.

    I don't necessarially agree with this. Just for an example, I'm going to point out e-mail. I don't think that either POP3 or SMTP are encrypted protocols. A lot of people who would rather not have an ISP keep a log of all their private e-mails use these protocols to transfer mail. In addition, AOL web-mail is not over a secure connection. When entering your password, you are directed to a secure connection, then back to an insecure connection when you actually read your mail.

    However, I suppose, that if these logs are for the purpose of tracking down criminals, for example, the child pornographers mentioned in an above post, than keeping logs of people's e-mails might be desirable. Mind you, I would not approve of this policy, but then again, I'm not running an ISP.

    ./configure
    make comment
    make post

  • Quite so. I wonder what interest the police have in seeing your webmail logs and IP#s and stuff, anyway. OTOH, I would maintain a log of TCP connections when/where and list of who was logged in when, that you could use to restrict it all down a bit. Letting the pragmatic work in favour of the "let it all be private" ethics...

    . o O ( I wonder which the most popular cookie in my ONE GIG of httpd ERROR logs was? )
    ~Tim
    --
    .|` Clouds cross the black moonlight,
  • I feel that ISPs should be limited to logging who was using a particular IP at a particular point in time.

    I do know it is possible using anonimity services to retrieve just about everything you would want off the net. This however is slow, awkward and inconvenient but the biggest downside to these services is that they are only is use by the technically ept.

    I feel that 'ordinary' users should be protected by law so that no more information is gathered from them than is gathered from a hardcore conspiracy theorist that routes all his traffic through unregulated offshore servers.

    Just because PGP exists and is freely available (well almost freely) doesn't mean that those who are not savvy enough to use it should be punished and have their communications needlessly intercepted.
  • Oh ye of strong faith in crypto...

    And how do you gurantee that the hacker with root-access can't get at the secret key actually used for encrypting the logs? After all, unless you have a few spare Crays taking up space, you wont be able to actually use (secure) asymetric encryption for data, only for secret keys ... AFAIK that is.

    I say run important logs to a printer, and BURN them after a while. Then it'll be quite gone (unless something sits arund in your printspools or something).

    All I'm saying is that if there is to be a point to crypto it has to be part of a carefully crafted system/strategy. After all - the only reason pgp makes sense for mail, is because we assume no (such ;) agency has put a camera right over our keyboards and/or screens.

    Be afraid. Be very afraid

  • Re:Logs.... (Score:3)

    by -brazil- ( 111867 ) on Thursday July 13, 2000 @03:26AM (#937681) Homepage
    They shouldn't unless they're committing a crime.

    Yeah, suuure. Get yourself a copy of "1984", read it, and learn why total surveillance is a bad thing. The very existence of such data is a danger in itself, because it can be used to commit crimes, and you can never be sure who eventually gets hold of it.

  • The ISP is responsible to maintain a log simply by the fact that they want to service their customers well (assuming that is the case). There is currently no legislation or laws of any kind (that I am aware of) that FORCE an ISP to keep logs of it's users. With this in mind, the logs needed to maintain one's own network, and making logs of abuse issues, as they arise, is probably the best way to go. Now, if you are a large ISP, you should seriously get your bigwigs to consider this little puppy (pdf file): InverseIP Insight [inversenet.com] It is not exactly "cheap" from what I hear, but this is the best ISP utility I have ever seen and there are many large ISPs deploying/using it. Not only for technical (testing the network and KNOWING when there is an issue), but for technical *support* as well, as this program tells every last detail down to what initialization string their modem is using. You can know what a customer's issue is and how to fix it before they even call. Perhaps ISPs will start calling their customers before they call the ISP?
    Bradford L.
  • They should all be archived indefinitely.

    I think you will find that most states have document retention laws, which specify for how long you are able to keep certain kinds of documentation. Lawsuits have been lost because companies did not comply with these laws, i.e. kept logs/documents for too long.

    You might want to recheck the laws in your state before you start keeping stuff "indefinitely".
  • Let ISP's log as much as they can. They probably hire a cheap MCSE-dude who will never be able to do anything with 'em anywayz... I mean: information is only information if it's interpreted by someone who knows what he's doing...otherwise it's just data. And the more data around, the bigger the chances on certain data getting lost..or untraceable... It's time to start worrying once ISP's start hiring skillfull personell! Not when they start glogging up their own server-disks with logs.

  • Avioding faked emails. (Score:4)

    by KahunaBurger ( 123991 ) on Thursday July 13, 2000 @03:30AM (#937685)
    But on the flip side, I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either.

    What options does a network administrator have for retaining forensic evidence in case of abuse

    This also ties into the carnivore question about faked emails. I've gotten some harrassing emails and considered forwarding them back to the sys admin of the jerk in question. However, realisticly, I could send anything I wanted with FWD in the title, and without digital signatures, they wouldn't know if I was forwarding a real email or not. But what kind of logs could they keep that they could confirm the authenticity of a message without invading the privacy of the user?

    Now, bearing in mind that I don't do this for a living, wouldn't it be possible to set up a logging program that ran a metric on each message that came through, based on date, to and from and message content, that could not be reversed to actually produce that data, but would have an astronomically improbable chance of being reproduced by a fake message?

    That way, the logs kept, just looking at them (even by the ISP) would tell them nothing but how many messages had gone to and fro from the whole ISP. But if someone came to them with an "incriminating" or "harrassing" email they could (at their discretion or under warrent) confirm the authenticity of that message actually having been sent by their service. If each ISP used their own metrics and kept them private, it would be very difficult for anyone to fake email evidence. This would be useful for both law enforcement/people being harrassed and the innocent but framed.

    So, is this kind of log possible, and would it satisfy privacy advocates, since you couldn't even tell "how aften and when used" for any given user?

    -Kahuna Burger

  • Says log everything and forward it to the police on request (well, over simplification of the truth but thats about what it amounts to).

    Alot of ISPs are _threatening_ to pull of the UK because of this.
  • Instead of commenting on what you should log, or as is mentioned in many of the other posts her, how long you should keep your logs, I'd like to pose another question. Who should have access to these logs. Obviously this is going to have something to do with the law in the area that the ISP is located. However, should the RIAA have access to logs in order to prosecute people who have downloaded MP3's? Should individual artists have the right to examine ISP logs to determine who downloaded their songs? Should these parties, in fact, be able to demand that an ISP keep logs of this information?

    While most Slashdotters will answer a resounding no to those questions, what happens when child pornography comes into play? Should a police officer, or the FBI even, be able to demand an ISP hand over their logs, and examine them for people who have downloaded child porn? (Not exactly the easiest search, but I suppose doable none the less).

    I think that determining who has access to the logs is perhaps even more important than determining what to log in the first place.

    ./configure
    make comment
    make post

  • not true. starr subpoeaned her book purchases for a very simple reason: she said she'd purchased some books for president clinton that were related to their sexual liaisons and starr was looking for corroboration, in case (as had happened before) clinton denied ever seeing such a book.

    it was a very legitimate query. people got freaked out because they heard starr was looking into her purchases and assumed it was part of his "reckless" investigation.
  • And how do you gurantee that the hacker with root-access can't get at the secret key actually used for encrypting the logs?

    Well, it's "guarantee", and you guarantee that a hacker with root can't decrypt the data by never providing him the opportunity to get the key in the first place. I said this system would be using public/private key crypto, right? Okay, public encrypts.. so private....

    And the private key isn't on the system, because it needs to remain secure.

    I say run important logs to a printer, and BURN them after a while. Then it'll be quite gone (unless something sits arund in your printspools or something).

    The CIA and NSA are well versed in recovering data after they were burned. Infact, this is how we have emoticons/smileys now - originally they were used as a code. But they killed the professor that created them, sealed the documents in magnesium binders, burned it, and then threw it in the ocean. Unfortunately some enterprising university kids got wind of this, went off-shore, recovered it, and reconstructed most of the data. This was AFTER a government agency burnt it to a crisp. So the idea of "using a printer" to secure your logs is one of the stupidest ways to do it - both in terms of space, and in terms of security.

    After all - the only reason pgp makes sense for mail, is because we assume no (such ;) agency has put a camera right over our keyboards and/or screens.

    Ugggghhhnnn. And here you prove the very point you're trying to dispute - PGP uses public/private key crypto.. the same solution that I was advocating be used to prevent your hypothetical cracker from getting access to my hypothetical system. I think I'll stop short of getting sarcastic here and hit the submit button...

  • All too easy.
    If only it were. In order to decide that something is a request, attack, whatever, you would need to log and analyze it.
    Maybe a better way to look at the situation is by defining layers of logging.
    Of course, you could always stick to a set of rules, ie: a SYN followed by the next X packets on tcp port 80 is an HTTP request, and should be logged.
    I think this would fail unacceptably for attack logging though, as a frequent attribute of an attack is playing outside the traditional rules.
    This also leaves you to play perpetual catch-up with new protocols.
  • You issue the license plates for the vehicle
    they are using. Then they can go anywhere
    on the internet and do what they want. If
    someone with a warrant comes and says:
    who did you give that plate to, then you
    give it to them. And only what the warrant
    specifies.

    If you are going to keep more information than
    that, then you should inform your cusotmers that
    you are keeping tabs on them. But I dont think
    you will keep your customers for very long
    if you do.

    Remember that a lawyer can get access to almost
    anything for any reason. And it can be a civil,
    not criminal matter: Custody, Divorce, Libel.
    Do you really want to be the vessel used by
    an unscrupulous lawyer doing a character assasination for his client on your customer?

    And do you really want to get supoenaed every
    day for access to your logs?

  • The ISP should log nothing, out of their own self-interest. Anything they need to log, for their own purposes, should be destroyed after use.

    Although it may be useful to Starr to find Lewinsky's book buying history, it's not good press for Barnes & Noble to have the existence of this log disclosed. Similarly it's never going to be in the ISPs interests to be at risk of having logs subpoenaed. The only legally-secure defence against this is to not have the logs in the first place (and this may require a traceable and provabel process to show that any that did exists have been destroyed).

  • In the UK... (Score:4)

    by DrWiggy ( 143807 ) on Thursday July 13, 2000 @03:46AM (#937694)
    The new Regulation of Investigatory Powers bill is due to be passed in the UK soon, which means that on request (i.e. a warrant issued by the Secretary of State is produced), an ISP must be able to intercept all traffic that a particular customer sends or receives. If you haven't got such a warrant when you intercept traffic coming from or destined to a UK citizen, then you are in breach of the Interception of Communications Act, and so you shouldn't be doing any logging at all.

    To be honest, I don't think the harm is in the logging - it's what is done with the logs. Disclosure to third parties is definitely illegal and unethical, but the use of this sort of data within an organisation can also be dubious. How much would your marketing department like to know about the 'real' (read 'secret') interests of all of your customers?

    I say you guys have got it pretty easy in the US, but at least we're now getting clear legislation (even if it is b0rked) saying what we can and can't do over here in the UK. To easily answer this question in the UK though, does require a few hours with a copy of the Data Protection Act, the Interception of Communications Act and the Regulation of Investigatory Powers bill. Even then, you're probably wrong.

    As far as we what we do is concerned (as an ISP) - we log enough for billing, and we have some machines running an IDS in promisc. mode to pick up scans, viruses, etc. going across the network. Apart from that, it's all pretty standard syslog-out-of-the-box.

    --

  • Spammers/child porn/death threats/etc (Score:3)

    by muldrake ( 171275 ) on Thursday July 13, 2000 @03:47AM (#937695) Homepage Journal
    I think ISPs should keep point-to-point logs as well as log, say, an rc hash of certain content such as email and news.

    With the hash, the data can not be retrieved as such, but it is possible to verify objectionable content as genuine and not forged. This would be in the "kiddie porn/death threat/Metallica song" category.

    These logs should be expired in a reasonable period of time. Any sufficiently serious death threat could not fail to be investigated within 30 days. Any behavior which is not repeated within that period of time can be considered at an end. Tough for the slowpoke.

    Otherwise, no content logging, and no intrusive logging such as unauthorized snooping on what software is being used and how.

  • Isn't Minnesota west of Michigan? (i.e. over to MN as opposed to up)

    --
    Your friendly neighborhood mIRC scripter.
    if (ismoderator(reader)) hidemessage(this);

  • Transparent Web proxies: are you being logged? (Score:5)

    by peterw ( 88369 ) on Thursday July 13, 2000 @03:48AM (#937697)
    Last year, my ISP, without any announcement, began using a transparent Web proxy. Most of my outbound traffic to TCP port 80 gets re-routed through machines running some Inktomi transparent HTTP proxy software.

    Naturally, my ISP keeps logs for that traffic (Inktomi boasts that its Traffic Server can write many different log formats), in part to deal with abuse.

    As you might also expect, the privacy policy does not directly cover these logs. It makes promises about some very specific types of information, but does not make any general statements that obviously pertain to types of information not covered in the enumerated, specific types. Result: I think most lawyers would say my ISP could sell access to DoubleClick, the FBI, or anyone else.

    Checking your system

    So are you using a proxy, but don't know it? You can check pretty quickly (though I should warn you, while a positive/proxy result is conclusive, a negative/no-proxy result may be a result of the proxy configuration, as the systems can be set up to bypass the proxy for certain sites, or to only use the proxy for certain sites, etc.).

    Step 1: what's your address?
    Check your current address for whatever network adapter (ethernet card, PPP/dialup device, etc.). In Unix or Linux, something like '/sbin/ifconfig eth0' will do; in Windows 9x, run 'winipcfg'; in Windows NT, 'ipconfig'.

    Step 2: what address do web sites see?
    Go to a URL that will show you the environment variables passed to CGI scripts, like http://www.cgihost.com/cgi-bin/env.cgi [cgihost.com] or http://www.ualberta.ca/htbin/dumpenv.pl [ualberta.ca] . Look at REMOTE_ADDR. Reload several times. Does it change? You might see some other proxy-specific variables like HTTP_CLIENT_IP and HTTP_VIA, depending on the proxy server's configuration.

    Step 3: interpreting the results
    If you ever see a REMOTE_ADDR value in Step 2 that doesn't match the local address from Step 1, yet you don't have a Manual or Automatic proxy configured in your browser, then congratulations, you're behind a transparent proxy, and should assume that all your Web traffic is being logged.

    http:// vs https:// For regular HTTP, there's a lot they can conceivably record. The URL. Your cookies. Where you came from. Etc. For https:// it's a bit better. All they can do is record where you connected to, and when. Even this information might be deemed valuable, e.g., someone frequently connecting to many banking sites probably isn't eligible for low income tax credits. https:// is somewhat like encrypting your email: they can't tell what you're doing, but they can tell who you're contacting.

    I've complained via email a few times, and received a couple polite emails from the technical staff. But nothing has changed in the official policy, so my ISP is still free to share my complete Web usage history with whomever they wish. Highest bidder? Most pushy government agency? I can't say.

    -Peter

  • I don't think there is any purpose of having logs lying around for more than 2-4 weeks. You should log everything, do not look at the logs except if you suspect or know there is something wrong.
    I don't know for ISPs since I'm administering the Unix domain of a hosting company. Since we are usually the victim of an attack I know that I'd report to an ISP whitin a couple of days of the attack. If I find that the security of a server/subnet have been compromised for a longer while than 3 days, we usually resoft everything in that area and restore the latest safe backup of the content, the system and binary that are executed with this content are usually replaced with a later version. I seldom find any use in reporting such an incident to an ISP based on the logs that are kept, because they can be pretty different from what happened (as in tangeled with).
    I know of a safer method for logging even tho I don't use it. Using a serial port for logging for instance, not beeing able to mess up the log from the server that actually generates the log is pretty secure.
  • For internet access: All that needs to be logged is who had what IP at what time. So you can track the odd email threat back to it's source, etc.
    For web servers: The standard log, with referers, is a nice thing.
    --Mike--

  • Re:Logs, none, incorrect (Score:5)

    by thesparkle ( 174382 ) on Thursday July 13, 2000 @03:52AM (#937700) Homepage
    Maybe some of you have not worked at an ISP, but ISP's keeping logs is very important, if only to combat SPAM and other forms of abuse.

    These logs should include:

    * Radius logs - username, port, and time, (Caller ID or npanxx info if you can get it), and IP assignment.

    * SMTP logs - SMTP ID. Actual copies of emails would require too much space than available to any ISP.

    * NNTP logs - again ID information only (NNTP post ID, date, time, etc).

    * Accounting logs as relevant to specific devices - for instance, shell and web servers which allow for telnet/ssh access, ftp servers, etc. This is not spying, this is good system administration.

    * DNS - knowing about those lame delegations is a big help. Especially when your customers routinely register domain names with your name servers as authoritative but fail to alert you!

    * Most important, accounting logs for root level commands as executed by the system's administrators. This can be a sore spot with some admins, but logging into a machine as root or su'ing immediately to root after login does not present accurate data as to what the admins are doing on a box. Using sudo or one of the other packages and maintaining an adherence policy to its' use should be expected. (Yes, yes there are ways around it..).

    Most of these things are standard practices for any of you who have worked for an ISP. I could care less what people were doing online unless they were violating our TOS/AUP and generated complaints. At that point, we needed to know who was doing what in order to fufill our contractual obligations to all of our customers.
  • Unless I specifically agree to them logging every transaction I make I do not see why they should.

    You probably did. Ain't fine print wonderful. My ISP (RoadRunner) admits to keeping full logs for as long as you are a customer, or 15 years after you stop becoming a customer. "For Billing purposes". WTF??? It's a monthly bill, not traffic

  • I run a small ISP and I agree with most of the people here that trying to make ISPs log routing is a bit much. Do restaurant owners have to keep a log of what everyone of their customers eats everytime they visit? Does Walmart keep track of every product you buy when you are there? I don't think so. The logs I keep are to ensure everything is running correctly and to HELP my customers with problems...nothing more. My 2 cents worth
  • Yep, but Mr. ISP can check the headers of the suspected email, compare it with his SMTP logs and radius logs of Alledged Perp and go from there.

    What course of action the ISP takes is subjective. However, most abusers go quietly or make up some wildly outrageous story that nobody believes.

  • I don't think there is any purpose of having logs lying around for more than 2-4 weeks

    Yes there is. It's great marketing information. Unfortunately, that means we get the shaft.

  • All kids love Log!

    What rolls down stairs,
    Alone or in pairs,
    Rolls over your neighbor's dog?
    What's great for a snack,
    And fits on your back,
    It's Log...Log...Log!!

    It's Lo-og, Lo-og,
    It's big, it's heavy, It's wood!
    It's Lo-og, Lo-og,
    It's better than bad, It's good!!!

    Everyone wants a Log!
    You're gonna love it, Log!
    Come on and get your Log!
    (Everyone needs a... Come on and get your... You're gonna love it, Log!)
    Log, from Blammo!


    --
    Your friendly neighborhood mIRC scripter.
    if (ismoderator(reader)) hidemessage(this);
  • It seems as if a good solution would be to have the SMTP server attach some kind of PGPish key to each message which signs the contents of the message with the ISP's key, perhaps a string in the MIME headers. That way, you could keep a log of what keys went out from your service, and then match those up with what a supposed "incriminating" email at some later point if necessary. No storage of the actual content would need to take place.

    I guess the only problem would be if different destination email systems mangled the contents to a point where the contents couldn't be verified anymore by the key, not to mention that if this were to be widely implemented, the sizes of the average email sent would begin to increase, and add additional load to email servers and their bandwidth. (although I'm guessing that would be negligible)

    Micro$oft(R) Windoze NT(TM)
    (C) Copyright 1985-1996 Micro$oft Corp.
    C:\>uptime

  • Harrasee: Hey, Mr ISP--your user BlahBlah keeps sending me threatening emails, please kick him off
    Mr ISP: I have no way of checking the contents of incoming or outgoing emails so I can't verify what you say is true. Furthermore, even if I could, I am not a law enforcement agency and can't take action against this person.

    It's not a question of being "a law enforcement agency". If the harassor is violating the AUP he agreed to then the ISP have every right ot kick him off. This assumes that the harassee checked the ISP's AUP to make sure the harassor was breaking it of course.

  • Hedge is what your looking for. Little smoke, and will burn forever.
  • Definately they have to stay away from that rock maple stuff, it's terribly difficult to cut.
  • It's crappy, but useful to get oak started - a few small/medium sized poplar logs (properly dried) are rather useful to get a large Chunk-O-Oak(TM) burning...

    You can also use pine sap to paint little figures on the sides of logs and then have fire people. Keeps the kids amused...

  • Re:Logs.... (Score:3)

    by Wheely ( 2500 ) on Thursday July 13, 2000 @04:11AM (#937711)
    The argument that nobody should mind unless they are committing a crime always pops up when some invasion of privacy is being advocated. What constitutes a crime may well look very black and white when you mention child pornography but what happens when some future or in some case current laws start to gnaw away at basic freedoms.

    Regards
  • Ya, u becha! Only Canada and the U.P. are up from Michigan,eh?
  • Hmm, yes, but couldn't govt. officials still supeona the data and force the ISP to reveal its passphrase? Having the data in any form still constitutes possession no? Though, I think that is the closest to a good solution mentioned so far. The data must be available to make valid queries to troubleshoot potential problems or security threats, however there has to be some sort of system in place to make sure that the data is not abused and privacy risked. The only other alternative would be to develop a filter that would keep the bare minimum of data required, that is the only way to assure that it will not be abused. Crypto is a great idea, but I doubt it can protect users from the Govt. authorities. my 2c -[v]-
  • ...what corporations are going through. The issue of logging is definetly a double edged sword for them. It's all a question of what you want the logs to be used for, and then conversely what they could be used for. Basically it sounds like what you're asking is what's the best trade-off.

    What do you log? As has been said, packet sniffing content would take ungodly amounts of storage, and if you're an ISP, you really shouldn't be doing it. It's Just Wrong (tm). Once again, it depends how tyrannical you want to be, but I think that just monitoring what IP's are hitting your boxes when is sufficient for most security concerns. At the most I'd say take note of traffic patterns, just incase a customer's box has been broken into and is doing things it didn't normally do.

    Should logs be permanent? We all should be able to come up with one real simple example of a corporation that was burned by e-mail leaking out that honestly shouldn't have. Corporations are now beginning to take a policy of purging e-mail stores often, so it doesn't come back to bite them in the ass. Is this ethical? Probably not. Which is why you have every right to be dumping your logs too. If corporation XYZ comes to you looking to see if the maintainer of corporationxyzsucks.com is one of your customers... sorry, you dumped the log. Don't get me wrong here, I'm not saying that ISP's shouldn't help big evil corporations if someone from them DoS'd them. I'm just saying that ISP's have a right to 'lose' information just like corporations do. Things are much less of a hassle that way.

    Legal issues. If I were a customer of an ISP that suddenly decided to start logging everything, they damn well better tell me that their terms of service are changing. Anonymity is something I value, and is a key factor in my ISP choice. What with all the DoubleClick-ish privacy things going on right now, I would not get yourself into that mess. Let your customers know exactly what you're logging, they have every right to know.

    Perhaps this is all remarkably obvious, and the opinions have been karma whored up by now, but I just thought I'd offer my two cents.

  • It's a public network. Use encryption. ISP's should log as much as they want.

    Of course, what they do with this information is the important part.
  • ISPs should and probably do log ALL data coming through their routers. After all, that is very valuable data to some people.

    And don't kid yourself that many ISPs are not. And unless you are administering the ISP yourself, don't kid yourself that YOU are not having all your network traffic recorded.

    It is like Microsoft or Real sending pings of your net traffic back to home base across the net. There is little motivation for an ISP to abstain from such activity. It is very tough to get caught. And some people will pay for your data, especially if you preprocess it properly.
  • If I were running an ISP, whatever server logs I did decide to keep, I wouldn't keep them long; I'd be too concerned about potential abuse by overzealous law enforcement or litigants to want to retain them. If you consult a tort lawyer today, you'll be told to get rid of your company's old email fairly rapidly so that it can't be used against you in court. I think that this would be a smart strategy for server logs as well.
  • Drop forged packets (Essentially anything not inside your address range originating from within your address range) and log those too.
  • For the wood-burning emergency generator. Duh!

    No. Nice try, though. Its the emergency backup heavy duty LART stick [winternet.com].

  • Someone else.. (Score:3)

    by mindstrm ( 20013 ) on Thursday July 13, 2000 @04:53AM (#937740)
    pointed out (so I'm stealing their post to a degree) that there is a difference between security and privacy.

    Should you use encryption, to keep your data secure? *YES* absolutely.
    Should your ISP be forced to keep your surfing habits private? *ABSOLUTELY*
    Should they be allowed to log as much data as they want for their own analysis later? *ABSOLUTELY*. Why? Because they *can*. It's *THEIR* network. If we say 'they can't' they can just put it in the contract; you want to use @home, you agree that we may log as much information about packet flow as we want. Period.

  • I'm not sure that ISP's are really *common carrier*. I realize that they are generally assumed to be, but I'm not sure they have the legal designation.

    After all, my ISP has a transparent proxy on my web traffic. That's not 'common carrier'. That's interfering.

    Common carrier means that they simply move data, and have 0% responsibility as to what kind of data, or where it goes.
    @home telling me *don't run a server* and scannign me for servers is *NOT* the behavior of a common carrier, as they are dictating what types of traffic I may produce.

  • First, I do not currently work at an ISP, but I have done. I also have administered arrangements for remote access at educational estabishments, thereby effectively being an ISP for the students and staff. This was a VERY thorny question for us in all those cases. We recorded who connected when, with what IP, and who accessed the services we provided, again recording the source IP. Those logs were kept for a few months. Logs of suspected probes were kept for a few weeks, overt attacks for longer. That was it. With this info we were able to pin down the account associated with any abuse reports and spot a few compromised user accounts (usually because somebody used the same password for everything and it got cracked somewhere else) by seeing the same user pop up twice from different locations at the same time.

    The logs we kept on OURSELVES though were much more thorough. Anything one of our machines did was watched somewhere and whilst most of those logs were short-term and verbose enough to require scripted assistance to scan in any meaningful manner we made damn sure that we looked into everything that poked up above the background noise level there.

    Privacy was important too - in all cases it was clearly understood that discussing logged info with anyone outside the admin team apart from the customer who owned a suspect account was cause for getting fired immediately. To even discuss it with the customer required written authorisation. If anyone else wanted the info it had to go through the head of the admin team. Marketing folks, the billing dept, top level management (by their own request) or support staff did not have access to that raw data and it would only be turned over to anyone outside the company with a court order.

    Other guys at the company sometimes accused us (the admin team) of being anal about it and I guess we were, but the complaints sure dried up when the policy saved us from getting our ass sued.

    # human firmware exploit
    # Word will insert into your optic buffer
    # without bounds checking

  • I have to respond to this for one reason - namely the line "I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either."

    I don't think a lot of people really think about this stuff when they do it. Yes we all want to be safe. We want our friends to be safe. Sometimes we even want those we despise to be safe as well.

    But where do we draw the line? This type of thinking is as dangerous as blanket "log everything no matter what!" As the story suggests, Barnes and Noble learned a very important lesson when they kept track of everything.

    Remember, the moment you give up just one of your rights to privacy is the moment you have given them all up. Also remember that "protecting the little children", as the religious right likes to say all the time, does not mean that MY rights as an adult should be erroded because of whatever draconian law they want passed.

  • If it's your policy not to keep logs of any sort, then they can't be subpoena'd in court.
  • A friend of mine recently lost a database on his site by someone 'social-engineering' a username and password to the web interface from one of his staff. The site logged the ip that the person doing the deletions came from, but it was a proxy's IP. The ISP in question didn't store their cache logs for more than a day or two, and so could not tell him the account he was attacked from. They had backups of the data and were back up in minutes, and the pleb who leaked the password was beaten soundly, but it would have been nice to know who-dunnit :-)
    /* Wayne Pascoe

  • Sniffer log overflow? (Score:3)

    by Christopher Thomas ( 11717 ) on Thursday July 13, 2000 @06:29AM (#937767)
    Simply monitoring a T1 with a packetsniffer doing decent filtering can easily trash a fast 30GB HDD.

    A T1 is 1.5 megabits/sec. To fill up 30 gigabytes recording _all_ data sent across the T1 (no filtering) would take about 44 hours. If a cracker leaves a sniffer unattended for that long, I have little sympathy for them.

    Overflowing a user account I can believe, but I would be amazed if drive overflow was a significant problem for the vast majority of packet-sniffing crackers. Heck, cut out HTTP and take only the first few packets of an FTP or POP session's data and you've reduced your data load by a factor of 100 or more, while keeping the information you're interested in (passwords).

    In summary, I don't think that drive space is a problem for a half-way competent sniffer.
  • A couple of four-by-four posts work pretty well if you stack the logs right .... :)

  • Yeah, the only thing less stable then Netscape Communicator is Mozilla ... (which has an excuse, since it is still alpha... :)

  • It depends on what you are doing as an ISP> IF you are a web portal then you should log certain data so you can analyze who is comint to your traffic and possible why. If you have asearch engin then you need to log the search data so you can analyze it and find out what is being searched on most frequently so you can work at building a better search engine that returns better results. If you also do web hosting you may need to log IP addresses and more so that you can provide your customers with who is visitng their web site and let then have the information that they need.

    Yes you must respect privacy, but you shoudl also state clearly and in laymans terms what your privacy policy is, and stick with it even when times are tough.

    What you should log depends on what your needs are and also what services you provide. Remeber though that you may be held responsibe for someone abusing your network, so it may be wise to keep track of who is on it and from where.

    send flames > /dev/null

  • OT: Nick is going to start airing the original R&S episodes again soon. Keep the kids away from the TV! :-)
  • As someone who works at an ISP let me clarifiy somewhat about what logs we deem keepable...
    We keep logs of connections to our boxes, the ip number given to that connection, the login name of the person connecting, what they connected at and for how long. We do not log where they went after they connected just that they connected and got a good solid IP number for the machines dns resolution.

    This not only helps us with router and dailup box diagnoses incase of trouble, but it also allows us to better help a customer who is having connection problems because it shows, when, how, and how fast they connected, and how they disconnected, timeout or dropped carrier.

    IMHO this is really all the log needed kept as far as what our customers are doing internal system logs on the other hand we keep very extensible because of the high number of hack attempts that go on against larger ISP's.
  • Those of you who have used IP Masquerading (ipchains) on Linux will know what I'm talking about ... routers can log *EVERYTHING* that passes through them...no proxy is required.

  • What, he didn't see that section in the Echelon Users Manual that the FBI/CIA/NSA sent him when he started his ISP?

    Good ol' USA!
  • From a purely technical perspective, I used to log all "rejected packets". Packets dropped by TCP, IP, or UDP (we had our own TCP/IP stack) for any reason were logged, along with the reason. This included TCP packets that didn't advance anything, i.e. duplicates. Packets dropped due to congestion were logged as much as possible. (Only packet headers were ever logged; content belongs to the user.)

    Today, if you did that, you'd be overwhelmed. But it's useful to have the capability and to log such stuff during a peak period now and then, just to get an indication of what junk is out there. Most denial-of-service attacks will show up in such logs, of course.

  • <<But now, I'm not so sure--if there's one thing we learned from Kenneth Starr's subpeona of Lewinsky's book purchase records, it's that Barnes and Noble stored such records in the first place!>>

    That's a different issue. Lewinsky's purchase history was based on *financial* records. Financial records MUST MUST MUST be kept to help eliminate errors, and correct them when they do arise. Not to mention for tax purposes...

    -JF

  • Re:Crypto (Score:4)

    by WNight ( 23683 ) on Thursday July 13, 2000 @10:10AM (#937793) Homepage
    You don't use the public key to encrypt the logs, you randomly generate a session key, encrypt the logs with that, then use the public key to encrypt the session key. Rotate keys every few minutes and don't leave them sitting around. Ditto with the logs. Have a seperate machine which only accepts one incoming connection, that which dumps logs onto it. Then the log holding machine has *no* idea of the way the log was encrypted, nor, if the logs are removed (via console) to another machine, preferably laptop, for examination, would it know how to decrypt the logs.

    Because the public key only encrypts a 128 bit (or whatever) session key every ten minutes or so, it's fairly quick, and two-way crypto is very quick, easily enough to dump logs through.

    If you ever implement a log system and don't want them modified, keep an ID # in each packet of logs, along with a MD5 hash of the previous packet of logs (including the previous-packet hash of the log file before it.) This way if a log is modified, the attacked has to change all logs after that point.

    Ideally you'd also have the log catcher dumping logs to a write-only media, like CDs. Preferably in a session-based way, so it didn't have to wait too long between getting logs and writing them.
  • I agree with all of the above. We recently were slapped with a subpeona in a child pornography case. We are a small ISP in a small town, if we had nothing to give the police, it would have been very bad for business in this small town. The only time ANY info about any customer is given out is by subpoena, and it's happened twice since I started at this ISP. Some may cry that it's a violation of their freedom, but people pay to use a service that WE provide, on our equipment, and ISP's need to be able to protect themselves against things you might do 'in their name'. Our owner takes it quite personally when a user does illegal things on our service toting our e-mail address, etc. Granted, the logs we keep do not delve too much into someone's actual activity. Unless a user has static IP, it's nearly impossible to even see what webpages are being looked at in the logs on the cache machine. Sure, if I went to see what dynamic IP was assigned to a person at a specific time, then went to the cache logs to see what matches of the IP within the specific time frame there were, I could probably see, but it's too much trouble to even attempt without a court order involved. Radius logfiles in particular are very useful in terms of technical support. We can see why people get disconnected, etc, and we do have caller ID on all of our modems, and have caught people who were using other peoples' accounts, etc. with it. When someone calls and says that they can't get online, and we see that they are already logged in, but the phone number matches little Timmy's best friend's house, and that's where little Timmy is right now, parents get a tad upset ;-) The same people that would complain that these things are invasions will most likely be *helped* by these records at some point in the future.
  • The idea behind the crypto is that the logs, once public key encrypted, couldn't be decrypted, to gain information that could be used in further cracking attempts, or to lessen anyone's privacy.

    Simple logs, based on multiple users, such as bandwidth usage, number of connections to various services, etc, should all be plaintext to make them easier to use. But logs of individual connections, when someone picked up email, what sites people went to, MD5 hashes of outgoing mail, etc, shouldn't be plaintext.

    And the specifics of many of these logs would be unimportant, you rarely need to prove a user did or didn't mail something, so if it sits encrypted on a CD for a year, and then takes ten minutes to decrypt and view, no big deal. Much better that it can't be easily accessed by someone unauthorized.

    Of course, if your random number generation was flawed, all of your session keys could be compromised, but ideally you'd use fairly strong methods. And yeah, there are swap issues and all to deal with, but I'd left the details to the user's discretion.

    And as for the trashing, that's why you'd offload them to CD or hardcopy frequently. Perhaps you'd dump the MD5 hash to a lineprinter every time a log bundle came in and dump it to a CD every time you got a few MBs... (Depending if you can write multisession CDs.)

    But, to summarize. Not only are logs for catching bad guys, but they're also private info, which if you collect sensitive stuff, needs to be guarded properly.

Slashdot Top Deals

"The only way I can lose this election is if I'm caught in bed with a dead girl or a live boy." -- Louisiana governor Edwin Edwards

Close