What's The Problem With USENET? 32
Sir_Winston writes: "For many months now, several of the larger news servers--at least, the "premium" news servers who sell access to individuals, not just to other businesses--have been experiencing severe problems with retention and sometimes with propagation as well. I can remember subscribing to services like Altopia and Newsfeeds because my local ISP's feed had retention of only ~2 days, but now several of the commercial news servers are down from 8-14 days to around 2-4 days. (some offer retention stats on their Web pages, and others can be gleaned from talking with customers). Talking with users, some seem to blame broadband connections for allowing users to flood USENET beyond reasonable capacities. Is this the case, or are there other considerations? Given that USENET isn't a truly distributed system, is it in for increasing problems as more people keep dumping massive binaries that may be better posted somewhere else? (For example, almost every mp3 posted to USENET can be found on Napster, often at higher bitrate). So, is there a problem, and if so how can it be addressed, and which USENET providers are still doing well at retention and completeness?"
Oh, really? (Score:1)
Another obvious thing to do is to use deja [deja.com] (no binaries though AFAIK)
HTH
Wroot
Re:Death of Usenet Predicted! (Score:1)
USENET binaries of up to 1 month will be retained (Score:1)
Drop binaries (Score:3)
By the way, alt may be a sewer, but there are a couple of islands of very high quality groups. Most of them are high quality because they are patrolled by a cadre of old-timers who attack anybody who tries to lower the standards.
The problem is the Binary groups. (Score:2)
But I have been using usenet on and off since '91 and have no intention of giving it up.
The cure of the ills of Democracy is more Democracy.
Re: Use-Net, "real" system or not (Score:2)
Re:The problem... (Score:1)
Average daily traffic is around 175GB/day. It's broken 200GB on some peak days. It's been almost approximately a year since traffic broke the 100GB mark, and about 30GB the year before that, so I don't know what figures you were checking, unless that was a typo or you were referring to a non-binary feed. The last time a full Usenet feed was around 2GB was January of 96. That had already doubled by January of 97.
Too many crap groups (Score:2)
Agreed. alt.binaries.* is redundant and inefficient. Prune back problem groups and the rest work much better.
The problem with this simple and effective solution is that news providers (and ISPs in general) love to print group counts in their marketing. "Over 5 billion Usenet newsgroups! When you sign up for our service, we create a new group for you -- alt.cool.stuff.that.user.[foobar].likes !" etc.
For example, JHU has over 30,000 groups. They carry stuff like abg.* (Altenburg, Germany), uw.* (University of Waterloo), and realtynet.* (house selling). I know most of these groups have insignificant traffic, but it's symptomatic of the larger problem -- a willingness to carry every newsgroup ever created.
Too many groups is A BAD THING. For example, our feed has alt.education.*, k12.*, misc.education.*, and school.*, not to mention various local hierarchies and subgroups. If you have a pedagogy question, where the fuck do you post? If you want to hear other people's ideas, where do you read?
The clutter of excess groups make Usenet less valuable by segmenting the readership unnecessarily. Unfortunately the task of newsadmin is generally a minor duty tacked on to the schedule of a disinterested junior sysadmin who doesn't care about Usenet being actually usable.
Re:binaries on usenet. (Score:2)
While I agree encoding is a problem, otherwise usenet is a good model for binaries.
Instead of tieing up the entire internet to get a binary, you tie up just one subnet at your ISP. The servers get the binary once, and from a site closer then the orginial ftp server.
Really it is a perfect model for binaries of use within a few days of publication, but not much use latter, that a large number of people want to see.
Saddly it doesn't work that way. Few ISPs run their own news server meaning you tie up the backbone for each binary. Worse, instead of binaries being of use only during a certian time, most are reposts every few weeks of something you don't want anyway. (Word'97 parts 1-35,37-50,51-74 - of 75 seperate parts)
A simple solution (Score:2)
This isn't a maintenance-free solution - I don't believe your news server will properly process NEWGRP messages so you'll need to manually check that once a month or so, but it has several benefits:
<ul>
<li>You control the retention period,
<li>Your clients have extremely high responsiveness, since the server is on the same LAN and only serving local users, and
<li>You can run spam filters on the article spool.
</ul>
The importance of the last point can't be overemphasized if you're in the alt.* hierarchy. As an experiment I tried cleaning up some heavily polluted groups, and discovered that the bulk of the offensive spam is from a few <i>very</i> active spammers who are using a couple class-C IP address blocks. (I'm _S_U_R_E_ *Y*O*U*
Death of Usenet Predicted! (Score:1)
*Death of Usenet Predicted! Film at 11!*
I've been involved in Usenet and on-line communicications since about 1988. That doesn't make me an old-timer, but it does give me hopefully a little creadability.
In this time, I've seen people talk about how much trash has entered into Usenet, how spam is on the rise, how key backbone sites are dumping Usenet, etc. So far, Usenet has proven remarkably robust and resiliant to these kind of problems.
Yes, I browse Usenet with a very extensive SPAM filter. Just as in the old days, after about 500 messages in a thread, I tire of it and add it, also, to the SPAM filter. There are certain people who get the distinction of *plonk* as they are added to my kill files. But, life goes on.
In my opinion, Usenet is one of the last examples of community still left on the Internet (with some IRC channels and VERY FEW sites like Slashdot also there -- although I might argue that the community on slashdot isn't anything like the community on Usenet).
Adding to the problem (Score:2)
Simple Solution (Score:1)
If enough usenet administrators decide to drop/cancel the binaries, usenet could be saved.
Broadband? (Score:2)
What's more, Usenet-izens are much more strict about etiquette than anyone else on the Internet, as far as I've seen and a lot of Usenet rules of behaviour have to do with taking up as little bandwidth as possible.
Or maybe I've just been hanging out in the last bastions of politeness on Usenet...
Cyclopatra
"We can't all, and some of us don't." -- Eeyore
Re:The problem... (Score:2)
This traffic is almost all binaries. I'm currently running a news server for a company that I'm working for -- it's a SS2, 64mb ram, about 6gb of disk. It has a fairly full feed - most of the big seven groups (comp.*, rec.*, talk.*, sci.* etc.) and a few selected alt.* groups. No binaries. It has no problems keeping up with the incoming traffic and a few readers, and it has enough disk to keep most articles for at least a few days. I could upgrade it, but why? It's doing it's job just fine, even now ...
alt.* is a wasteland. Once you get away from it, Usenet is reasonably sane, even today. Which is a shame -- alt.* had such potential. at least until C&S got clever ...
Note that back when Usenet was designed, back in 1979, I'm not even sure T-1's existed. Back then, news and mail was shuttled around on dialup modems ( at 1200 bps or so) using UUCP and even the largest universities tended to not have connections to other places faster than 9600 bps.
It's all very different now ...
.uk (Score:1)
In .uk, Claranews [claranews.com] is regarded as one of the best pay-for news services (+ POP3 account). They just recently had to increase the cost from £20 to £29.99 *per year* to cover the cost of extra servers. It's still excellent value for money if you consider that Supernews costs £130 a year.
As for retention times, it's around 2 days in a.b.c.d.i., 2-3 in similar busy binary groups, 3-4 for "average" traffic groups, and 6+ (probably more than a week and a but; I haven't looked recently) for text-only groups - e.g. currently 7085 headers in comp.os.linux.misc & 11819 in .advocacy.
The news services provided with most ISPs seriously sucks - you'd be lucky to find any posts that have all the parts in binary groups.
HTH.
binaries on usenet. (Score:4)
Want to fix usenet? do away with binaries. How? damfino. I wish you luck. As someone else said, with things like Napster, there's no reason for usenet binaries any more, anyway. Except for the "I have a hammer, so this must be a nail" problem. It's there, it's used, and nobody can say "no".
The primary use for usenet these days, IMHO, is to be a place where the kiddies go so they don't annoy the people with a clue who left and moved onto real systems to get their stuff done....
The problem... (Score:3)
The easiest thing for most feeds to do is simply drop their alt.binaries.* groups (or at the very least, the binaries groups with the largest article counts); why bother having what effectively amounts to a local mirror of Napster? I fully expect to see most ISPs do something like this, if they haven't already.
The "nicer" providers might decrease the expire time on binaries groups to a day (I've seen half-day expires even) in order to raise expire times on other, more meaningful groups.
- A.P.
--
* CmdrTaco is an idiot.
Re:The problem... (Score:1)
Re:Broadband? (Score:1)
Re:binaries on usenet. (Score:3)
You might be surprised at how many fairly knowledgeable chums still poke around USENET. For example, if you want privacy and anonymity resources, there's no better place to go than USENET--the only things on the Web that can compare are those that rip off the FAQs and resources built up in some USENET groups.
The problem is certainly the binaries, though. Broadband connections have inspired too many twits to post hundreds of large files a day, which kills retention not just in binaries groups but in text groups as well since they're all typically stored in the same server pool. There are now binaries groups which were created specifically *for* broadband users, with no daily posting limits. Too many of these people fail to realize that what's posted in one group affects retention in all the others.
That's not to say that there shouldn't be binaries groups--I think there are some binaries groups in USENET which are great, because they're conversational and interactive, which you can't get from a Website or a Napster-type program. Some of them are very like the old BBS days, and it's nice to see "communities" not just file repositories.
For all the good stuff in some of the alt.security.* and comp.* and even occasionally in the alt.binaries.* hierarchies, I worry that a relative few people with broadband who are mostly posting stuff that would fit better elsewhere are going to drive USENET into the ground. It would be sad if USENET goes the way of BBSes--it'll go on, just like there are still a few BBSes here and there, but only as a shadow of its former self, catering mostly to nostalgics. That wouldn't be good.
Re:Broadband? (Score:1)
Most of the Usenet *groups* are text-based, but note that group != traffic. Last time I looked, binary groups consumed about 70% (if not more) of the total bandwidth in a typical news server.
uhhh, pr0n (Score:1)
Re:The problem... (Score:1)
>want to download binaries from USENET anyway...
>the web and FTP work just fine for me
Then you aren't much of a pornhound.
Only amateurs surf the web for pr0n; there are some great 'bots that will keep track of a given newsgroup and download everything automatically, keeping track of what has been previously posted so you don't waste time downloading duplicates.
There's a lot of spam etc. but you can killfile a few domains and zap the rest browsing with ACDSee. (Is there an image browser under *nix as good as ACDSee??)
You can get gigs of porn in a real hurry on Usenet. Some of it is bad; some of it is reallyreallygood.
Binaries, and the UseNet (Score:2)
Re:The problem with USENET: (Score:1)
Maybe... People like you?
The problem with USENET: (Score:1)
Now, anyone want to wager a guess as to what the problem with Slashdot is?
---
pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
Napster this, Napster that (Score:1)
1900 binary groups on my server and about 100 are for MP3s. You cannot, flat-out-cannot-I-don't-care-what-magic-tricks-vo
One word (Score:1)
They're not just on MUDs anymore.
Ours is a world of nuclear giants and ethical infants. -- General Omar N. Bradley
Re: NOT Adding to the problem (Score:2)
As for the issue of bandwidth consumed by this process, most people will only download the newsgroups they actually read. (I'm only grabbing all of comp.* because some friends asked for full archives for some research projects.) There is no additional overhead incurred in downloading articles that you would have downloaded anyway, it actually <i>helps</i> the availability because the access is spread out throughout the day instead of being clumped.
spammers, trolls, flooders, script-kiddies (Score:3)
Part of the problem is that there are a lot more assholes who delight in destroying newsgroups and filling them up with garbage.
One of these characters is Gary Burnore [mfn.org]. Some of you San Francisco Bay Area people might remember him as one who managed to piss off most of the ISPs around there and anyone heavily involved in database management. More of you might remember him as the one who harassed Jeff Burchell into shutting down the Huge Cajones anonymous remailer because someone fingered him for child molestation; a change for which he was subsequently convicted. In revenge, he had his now wife [mfn.org] impersonate a lawyer to get the nonexistant logs of Huge Cajones shortly after Helena Kobrin [demon.co.uk] harassed Jeff to get the same nonexistant logs. Jeff shut the remailer [inet-one.com] because of this.
Gary violated probation by leaving California for Raleigh, NC where he offers network services [databasix.com] primarily to trolls, flooders, spammers, and other vandals and shields them from complaints. He and his syncophants have been known to threaten lawsuits, make physical threats, launch denial of service attacks, and make up stories to get critics kicked off their ISPs.
Usenet is attracting many more psychopaths just like Gary. They don't care how many people they piss off, nor are they concerned about possible legal consequences or retaliations.
Desert Rat
Re:Drop binaries (Score:2)
There was a time when a LOT of MS-DOS executables were posted. That died down. Archives like SimTel just posted summaries of new stuff and people found that they could use ftp instead. If you only had Usenet and Mail access, there sprung up the FTP Mailer services that would mail you something from an FTP archive.
I also agree that there's still a lot of high quality discussion on Usenet. Some of the comp.lang groups are excellent (.moderated for c and perl, thank you). The comp.sys and comp.os groups can have some good topical news and help.
Hey, if you're looking for some good free content sometime, cruise over to DejaNews and search for dmr@bell-labs.com, bs@research.att.com or mash@mash.engr.sgi.com. This is just name-dropping on my part as there are other non-famous posters to Usenet that have a interesting things to say. If you've been avoiding Usenet, surf DejaNews on some topic that interests you. I think you'll be surprised at what you might learn.
---