Forgot your password?
typodupeerror
News

Contacting Network Admins Of Large Internet Companies? 327

Posted by Cliff
from the I'd-like-to-speak-to-your-superior-please dept.
lisa asks: "I work as a sysadmin for a national DSL ISP. Unfortunately, we've recently found that @Home.com is not allowing connections to port 25 from some of our primary mail servers: this of course means that our customers can't send mail to theirs. I've called and talked to people in their tech support, and only after several calls have we been able to get them to acknowledge there may be a problem. The trouble is, I can't seem to get in contact with any network admins there. Even the tech support person I spoke with expressed less than hopeful sentiments about being able to get this issue escalated. Has anyone had trouble like this with @Home or other simliar Internet companies?"

"What is the best way to get in touch with a Network Admin or someone who actually can do something about a network issue in cases like these? It would be nice to know that just writing root@home.com would get to their systems department, but I was told all of that mail goes through support first."

This discussion has been archived. No new comments can be posted.

Reaching Network Admins?

Comments Filter:
  • by Anonymous Coward
    Where I work, we hired a guy who previously worked at AT&T's @home tech support. He's really good at solving customer problems (we sell electronic sensors, not computers, software, internet service, etc). He walked in knowing virtually nothing about our products, but he read all the manuals in about a week and with suprisingly little hands-on help, he's able to use/install/troubleshoot almost every product we make. He seems to like the job... no timer running during the calls, and when it really looks like there's something wrong with the product or he's in over his head, he's free to walk over to engineering and get some of our time. He almost never does, and when he does, quite often it really is a problem with the product or an error in the install instructions.

    Anyways, he sometimes tells some stories about working at @home's tech support. Nearly all the calls were simple windows-based issues. The one type of tech call, which apparantly was pretty common that's pretty funny is people calling, completely outraged that their ping time went from 25 ms to 80 ms lately and they're getting killed! Something would change somewhere in a router and he'd spend hours, call after call, answering these pissed off gamers, who were insistant that AT&T had broken their network and it needed to be fixed. There are numerous stories about dumb users, which reportedly are the lion's share of all the tech calls. I asked once if he ever got a call from a real expert about a real problem... the answer: no.

    It can be frustrating calling places like @home with a real problem. It's well known that their tech support sucks. Based on what I've seen from this guy we hired (admittedly a sample of only one), it looks like @home's poor service is a function of their system and its rules (admittedly based on the large volume of unskilled end-user questions), and not a lack of talent in their staff.

    I'm posting this anonymously, even though I have a regular slashdot account (karma capped at 50), cause it just doesn't feel right to speak for my employer so much, but I thought maybe some of you reading these comments might like to hear a positive story about one of their (former) tech support guys.

  • by Anonymous Coward
    Of course, always try the normal channels. However, if that fails, dial up the heat. Contact, all via CC: on the same email:

    • the administrative and technical contacts listed by network solutions [networksolutions.com]
    • the contacts listed by ARIN [arin.net]
    • investor relations at the company (if publicly traded). visit their web page for IR contact.
    • try to find email addresses for higher ups (VP responsible for infrastructure, CIO/CTO, etc.)
    • postmaster@, security@, hostmaster@
    then let it rip. You'll get mixed results, but often this will get someone's attention. Keep the email polite and to the point, but remind them that, after all, you've been trying for weeks to get this resolved the simple way, but THEY haven't been keeping their end of the bargain.
  • by Anonymous Coward

    $ whois comp-u-geek.net -h whois.opensrs.net
    Registrant:
    Reliablehosting.com
    2227 Lake Tahoe Blvd.
    South Lake Tahoe, ca 96150
    US

    Domain Name: COMP-U-GEEK.NET

    Administrative Contact:
    Blancett, Phil phil@oakweb.com
    2227 Lake Tahoe Blvd.
    South Lake Tahoe, ca 96150
    US
    530-542-4209

    Technical Contact:
    Blancett, Phil phil@oakweb.com
    2227 Lake Tahoe Blvd.
    South Lake Tahoe, ca 96150
    US
    530-542-4209

    Billing Contact:
    Blancett, Phil phil@oakweb.com
    2227 Lake Tahoe Blvd.
    South Lake Tahoe, ca 96150
    US
    530-542-4209


    Record last updated on 20-Jan-2001.
    Record expires on 17-May-2001.
    Record Created on 17-May-2000.

    Domain servers in listed order:
    NS1.CALIFORNIA.NET 209.162.97.149
    NS1.OAKWEB.COM 209.233.101.2

  • by zztzed (279) on Saturday January 20, 2001 @09:09AM (#493650)
    You might want to look at this [nether.net]. It's a list of NOC contacts for many major providers.

    I don't know how up-to-date it is, though.

    --
  • I see. So in this company, service is what a stallion does to a mare.

    Which makes it just like 99.999999% of other ISP's. I do wish that people knew the meaning of the term "customer service", but must admit that it will not happen within my lifetime -- it's too easy to make money nowadays by being a complete asshole jerk. Just advertise like hell, make sure that potential competitors can't get into the market by using monopolistic practices such as exclusive contracts, and voila, you have a company like @Home. No need to have customer service -- in fact, the harder you make it for your customers to get service, the better, because then you don't have to pretend to care.

    If a competing ISP could get into my local cable drop, I'd switch ISP's in a minute. But by signing monopolistic exclusive contracts with local cable providers, @Home can continue providing lousy service while not giving a damn.

    I remember when you could go straight to the NOC's web site and find out what tickets were open. That headed off a lot of geeks calling in with "did you know you had a routing loop between router-xyz.dallas.net and router-zzzy.dallas.net?" You looked at the NOC's ticket list, said to yourself "Oh, they already know about that", and went your way. Today you can't do that, because the national service providers don't want you to know how crappy their service is. The sad thing is that this attempt at decieving us has convinced us that they have something to hide.

    Oh well. I guess the days when producing a good product at a good price in a friendly manner was the key to success are long gone. Today the goal appears to produce the shoddiest product for the highest price while providing the crappiest service -- then advertise the hell out of it, while using monopolistic tactics to drive the guys who do believe in producing good products out of business. Sort of like Microsoft did to folks like Digital Research, Quarterdeck (remember DesqView?), and (soon) Apple.

    -E

  • by Eric Green (627) on Saturday January 20, 2001 @04:15PM (#493652) Homepage
    In the old days, the major backbone NOC's kept a list of open trouble tickets available on the public Internet. So if you're doing a traceroute through Dallas and find that the Atlanta-Dallas route in the UUNET backbone is flapping (thus keeping all of your Shreveport customers from being able to reach any web site on the East Coast), you could actually go to noc.uu.net and find out whether they knew about it. If they knew about it, great. If not, you called up your local ISP, they contacted UUNET, and it got reported and fixed.

    Today, if something goes down, you have no idea whether anybody knows about it or not. None of the backbone NOC's post trouble tickets to the open Internet anymore. Apparently they don't want anybody to know how lousy their service is. The sad thing is that by keeping these secret, they've caused a thousand-fold escalation in the number of phone calls coming in saying "Hey, did you know your route between Dallas and Atlanta is flapping?". Aside from convincing the rest of us that they have something to hide, of course -- but if you're part of an oligarchy that has collectively decided (illegal cartel?) to screw the customer, there isn't much I can do about you deciding to be a deceitful scumbag (well, I could create a new backbone, but that isn't exactly cost-effective).

    -E

  • by defile (1059) on Saturday January 20, 2001 @10:16AM (#493653) Homepage Journal
    Most ISP technicians learn to assume that everyone has no idea what they're talking about unless they have proven otherwise. It's a safety mechanism. Failing to do so could result in hours of wasted time.

    Also, if you want to be taken seriously, don't mention that you use Linux if you can help it. 5 years ago it would've meant "hardcore programmer on the line escalate to admin", it now means "Windows dork trying to survive with Linux, much hand holding is about to occur. Shield busy admins from harm!"

    Don't just drop techno babble. If the technicians don't understand what you said, they'll assume (for their own safety) that you don't know anything. They will not escalate you.

    Flat out asking to speak to an admin will probably just make the technician feel insulted and less inclined to help you.

    For best results, if possible, work with the technician, try their suggestions (and tell them that they all failed), make him take out a trouble ticket so the whole spiel is recorded and doesn't have to be repeated. In most cases they'll escalate it when all of their suggestions fail.

    These are just my observations from the inside. *shrug*

  • As a system administrator at Earthlink, I am interested in looking into the problem you describe. Without more information I can't say definitively whether it is due to Pac Bell, us, or you. Please send details of you investigation to the above address.

    Note that this is primarily for personal interest. I may or may not be able to diagnose the problem.
    --
  • I can assure you that the system administrators at Earthlink are extremely concientious and try to err on the side of permissivity when trying to strike a balance between keeping our head above the spam and letting legitimate mail through. It is an extremely difficult task and we are fallible. It is clear however, that we can't afford to be hands off, nor can the rest of the net.
    --
  • I'd be interested in hearing the details on this. Email me if you care to.
    --
  • The average spammer uses a proggie to send hundreds of spam e-mails every hour, so why don't they just monitor the SMTP transfers per hour and then draw their own conclusions?

    Let me preface this by saying that while I am an Earthlink employee, the following is a personal opinion:

    The matter of port 25 blocking is disconcerting to me as a proponent of a free internet. However, spam generated by Earthlink customers dramatically affects other ISPs. There is a reason that if you look on Maps.vix.com, Earthlink's notes say something like "Formerly a prodigious source of spam."

    First of all, understand that the semantics of email are seemingly designed for DOS attacks. What other protocol is designed to allow a single message to be replicated many times by an intermediate server at no cost to the originating host. Left unchecked, spam is so bad that you would never get any mail. So we fight a vailiant battle at ISPs to keep our customers free while keeping the services they depend on running at a reasonable cost. At Earthlink we have no fewer than 4 separate independent spam managament tools that I can think of off hand.

    Every day I see the effects of being on the receiving end of networks which don't block port 25. While it would be eminently preferable to use traffic shaping at the router, rather than outright blocks, the protocol analysis required to identify and block spam is very involved and to the best of my knowledge can't be done at the router level except in very crude ways. For instance you can't simply monitor bytes sent, because a single message may have many recipients so the size multiplies. You can't measure connections since one SMTP connection can have multiple messages sent in it. Even at the application level it is difficult.

    Tell me of a better way and we will will most likely use it. I would like nothing better than to keep the internet as unfettered as possible.
    --
  • I propose that we have a secret geek codeword that can immediately identify each of us as a member of the geek commmunity

    We have. We will tell you when you are ready.
    __
  • It clearly said that Pac Bell wasn't blocking port 25, but that Earthlink was blocking Pac Bell's DSL users.
  • Do you have any idea how frightfully outnumbered we are? If every (seriously) technically competent dsl user dropped off the world completely, I doubt they would even notice.

    true, we are outnumbered, but there is nothing keeping the less technical from using the service if it proves superior.

  • According to http://www.isi.edu/in-notes/iana/assignments/port- numbers, SMTP runs on 25/TCP and 25/UDP. Although most traffic is on TCP, perhaps there's MTAs that would accept UDP.
    --
  • Following Lisa's posting about "@Home blocking PORT 25" I would like to add that AOL does the same thing.

    And good for them. AOL has been castigated for years as spammers used to grab a disposable AOL account, point to and open relay and spew until the plug got pulled. do {} while (1);

    So AOL filters outgoing port 25. You were able to work around it by using a different port for your SMTP. Excellent. But don't criticize them for taking an action.

    It's the tragedy of the commons.
  • Earthlink does not offer SMTP connection services. They offer email through their arrays of mail servers. This is not censorship; it's just a decision about what mechanism they choose to offer.

  • Actually, it's a GOOD way to get their attention. Since you can PROVE in court that their network architecture sucks, you can win. But you see, you let them sue you and THEN you get a lawyer contact their. Then you settle out of court, but now you have a contact ... "uh, we need to restore our slander against you, again, because your network is fucked up again ... fix it, again".

  • You wouldn't get those messages if your front line people responded correctly. In fact I have specifically dealt with ZoomNet [zoomnet.net] before, before it was part of Earthlink, and I had to call by phone and get one of the owners on the phone to get a DNS problem (incorrectly coded MX record) resolved. They did correct it within an hour. The thing is, sometimes it just takes finding someone who has AUTHORITY to get things done.

  • A lot of places get blocked for being open spam relays. Anyone finding themselves blocked should make sure their own house is clean.

  • The problem is, because the tech support guy you do get has read the first 3 chapters of the first book towards his MCSE, he thinks he has the Black Belt in networking. Unfortunately, you might not impress most of them (even if you know 1000 times as much as the sum total of them all ever could learn).

    The other unfortunate reality is that most of these companies do NOT want geeks as customers; they use the bandwidth too heavily :-(

  • Oh My God! That would mean we'd be back with an Internet the way it used to be about 10 years ago, with competent admins and engineers, and less crap and spam.

  • Given that @Home mail servers are open relays (since I get spam relayed through some of them, I know at least some are open), and given the lag of delivery through them, maybe someone has discovered it as a temporary storage device.

  • If the addresses are not listed in DUL [mail-abuse.org] then they may have blocked it on their own. I do that when I get spam that was relayed but not blocked by RBL/DUL/RSS. I check the ARIN records for the exact address the spam came from and I choose the most specific network involved. That gets blocked. However, it is still possible that spam was relayed from an address listed only with the broad SWIP record covering their whole network, even if they did put your addresses in at ARIN. If that is the case, you need to complain to them because their failure to SWIP **EVERYTHING** that might possibly relay or spew spam can end up affecting you even if they do SWIP yours. If they can't fix that policy then you need to run, not walk, to another ISP (and if you have a term agreement, pass it with a note to your lawyer that they are the ones to break the agreement for not providing proper service). If you tolerate bad ISPs, there will just be more bad ISPs.

  • You mean for mail not addressed to any domain which Earthlink is the ISP for, right?

  • BTW, I just realized that I have received relay spam from an Earthlink mail server at 207.217.121.12. That is one of yours, right? Did this one slip through your fingers? Did someone else set up that one? Was it hacked into? Did someone forget to test it against mail-abuse.org [mail-abuse.org] after making a config change? Why did no one respond to my spam abuse report?

    Give you a call, eh? You post as Anonymous Coward and don't list your phone number? At least you can email me if you are geek enough to understand my email address.

  • I've learned to NOT believe the obvious from so many people. I know what would be logically right. But so many people out there don't actually know, and just proceed on that basis. And then when you talk with them you're often not even talking the same issues. It's best to be 100% certain of what they are talking about and make NO assumptions about what they didn't actually say.

  • I wonder what would be said if the routers were to simply redirect port 25 (for packets addressed to any IP other than the local mail servers) to the local mail servers. Regardless of where the mail is intended to go, regardless of how the sending server is configured, it always goes through the ISP's mail proxy server. There, appropriate checks and controls can be applied as the above article suggest. The question is, would all the complaints about broken mail be reduced because now people would no longer be so incovenienced as to have to set their "SMTP host" address according to their ISP instructions?

  • Do you really think they are hiring for a position where someone will have the AUTHORITY to actually make the sweeping changes some people claim is needed? I personally don't know if AOL's network really sucks or not. I know a couple people whom I exchange email with that are on AOL, and it has worked OK. But if their network was fucked up as some people say, I'm assuming that it would take someone at the CTO level to fix it. Based on my experiences with other companies that do have networks in horrible or worse shape, anyone below CTO level just isn't going to have the AUTHORITY to get anything done, and won't even be listened to when they make the suggestions.

    If I do appply for the CTO job at AOL, do you think they would even reply to my resume? Unless the job really is open, I highly doubt it. And even if it is open, I suspect they would be more looking for some politcal wonk than a guru geek who knows what to do to scale a network up to universal proportions reliably.

  • That's fine. Make sure Pac Bell SWIPs your network. Make sure they SWIP every other network as well as dialup pools, so you network doesn't get blamed for spam. Or you can configure your mail server to just feed through the Pac Bell server. I find a less-mainstream ISP that doesn't seem to attract spammers.

  • I've been majorly *UN*impressed by the competency at Verizon. If they are rejecting mail coming in to their servers from their own IP space based on the FROM: address, then obviously they are paranoid more about the forgery aspect of it. And clearly, as you've determined, they are unaware of the fact that people within their access IP space can have legitimate email addresses outside of their network, and need one way or another (SMTP through the Verizon server, or SMTP around it) to get out. They can't do both without breaking things. They will have to decide which way they want to block mail. The way of forcing everything through their own mail servers would MAJORLY reduce spam originating in their access IP space, but they apparently haven't been clued into that concept, yet.

    Don't be glad they are not blocking port 25. Instead, be glad they are not blocking BOTH WAYS OUT at the same time. But you should be SAD they have chosen the one way which has virtually NO impact on spammers. The reason is, other networks will block the Verizon IP space when they start getting spam delivered from that spare.

    As for the "pay phone" analogy, consider that their view of the Internet is probably more like a television broadcast than even a pay phone. Big corporations want to feed, and control, information going to you. Be glad you even get to send mail at all.

    The contract thing may or may not help. They are probably reluctant to put it in because then they have to actually go enforce it, and they probably fear they can't be very successful at it. But I do agree that cancelling an account is pointless. Spammers know they can get 24 hours or so from an account, and more on weekends. They know accounts are sacrificial. That's why I don't focus on the disposable dialup accounts in my anti-spam measures. At least blocking outbound port 25 from dialups prevents the spam, which account cancelling after the fact never does.

  • It was about 2 years ago. I got past frontline because they were totally baffled (didn't know what an MX record should have). The guy who answered said he was one of the owners, and that he was there because they were replacing stuff that day. I told him the problem and he said it should be fine since that server had been back up for a few hours. He then talked to someone else and when he came back on the line, he said "Looks like they didn't convert everything right, give it an hour and I'll make sure they get it fixed". In an hour it was fixed. I didn't try earlier, so as far as I know, it could have been fixed in 5 minutes.

    As for throwing jargon at clueless techs, it usually is the quickest way to get a problem escalated. Often the problem is beyond their comprehension anyway, so what else is there to do since most won't escalate just because you say "save your time, just escalate this call now" (and I wouldn't ever expect them to).

    If your people love to blame the server/hardware, I wonder why that is.

  • If they use a collection agency, they usually get nothing unless the agency collects. OTOH, if they sell the debt to a buyer of bad debts (usually pennies on the dollar) they get money up front, but usually way less. But I doubt anyone would buy these debts, as they are usually to ficticious people with stolen credit cards (which can reverse the debt back to the original owner anyway).

  • Not all geeks do, and it is true that web sites are hitting up bandwidth providers for all customers more and more. But geeks do tend to stay online a lot longer, and download the latest FreeBSD, Linux, and MP3s. But perception plays a big role in this, too. The perception is probably more extreme than the reality.

  • When I was working at this one small ISP (sysadmin, not tech support), there were a shortlist of customers that tech support was authorized to forward direct to me on just their asking. One of them happened to be a CCIE working for a major telco doing their internet routers. After that first conversation where I asked why he was using us (because he didn't trust his own employer's network WRT privacy) he was actually useful because he would pin down exactly where problems were before calling, and I knew I didn't have to do the trackdown myself.

    But yes, you can get morons claiming to be gurus (because they installed a computer at work and it worked). I still think a direct line like that could be useful, but it should be given out sparingly, only when merited, and probably with some access code that could be revoked. OTOH, rot13 has been a reasonable filter, so far, on my email address.

  • Asking people to turn off those filters and just accept being spammed as a result is not really practical, either. The RFCs don't actually mandate that everything has to be one big internet. It's not unlike any other firewall filtering where someone makes a policy decision about what they do or do not want to do, or offer to their customers, or support.

    If blocking port 25 is brain dead (and I'm not really saying it isn't) then what is the alternative ..... that accomplishes as much spam reduction? Unfortunately, the design of the internet didn't really take into account the commercial proliferation we have today. We need some sort of secure mail transfer. Even with that, the problems won't go away. Even if we could authenticate exactly who sent every piece of mail at every hop with no chance of forgery, we'd still have the issue of having to decide who we want to accept mail from, and who we don't want to accept mail from. The concept of filtering is here to stay no matter what else we do.

  • Blocking spam is an imperfect art. Yes, legitimate mail is going to be blocked in some cases. But choosing methods to block spam which provides at least a way for legitimate mail to be sent around the blockade is preferrable, because then the sender can at least do something about it.

    People have suggested to me to use procmail to filter out spam. But when I prompt them for a good set of rules that work, so far 100% have balked, giving excuses like "every situation is different". I block dialups and relays not just to reduce spam to my mailbox, but to also reduce spam to the mailboxes of every customer. This in turn reduces complaints from customers. So far not a single customer has complained about losing legitimate mail, but I do have a means ready for any customer to opt out of the blockade and receive unblocked mail. It will be their choice, but they seem to be happy as is.

    If you do fire a sysadmin for subscribing to the DUL, send them to me. If I have an opening, I might just hire them. Ultimately the decision to do any filtering on any basis whatsoever is a decision to be made by those with the authority in the business. But based on my experience, it is a wise decision to use DUL. But I would not use ORBS.

  • There is some fallout blocking happening. You may be blocked as a result of someone else sending spam. Send me a private reply by email and tell me what IP your mail goes out from, and I can take a look and see what specifics I can discover. I am using RBL, DUL, RSS, and a blocking zone of my own. If you can get to me, then @Home is blocking on some other basis. But I can only guess without specifics (and may still only be able to guess then).

  • by Skapare (16644) on Saturday January 20, 2001 @09:45AM (#493713) Homepage

    No it doesn't take away the right to send email. It only NOT OFFERS the right to make SMTP connections. There's a difference. By blocking it, they force dialup/DSL/cable users to use the ISP SMTP server as first hop, where they can enforce (not all do, but at least they can) their no-spam policy.

  • by Skapare (16644) on Saturday January 20, 2001 @10:10AM (#493714) Homepage

    If Pacific Bell allowed the customers to connect port 25 directly, then it would create a massive headache, and high costs, for them to deal with the spam (and it would happen for certain, and probably has happened a lot in the past to get them to do this).

    When you sign up for service, you are told what SMTP server to use for outgoing mail. Use it. Or find whatever other way works for you. But they are not offering SMTP connection services to you. The solutions are easy, so deal with it.

  • by Skapare (16644) on Saturday January 20, 2001 @10:32AM (#493715) Homepage

    Have you made certain that your network never has sent out spam (and I mean EVERY machine on your network) and they your domain is not in one of the domain based anti-spam zones? You say you are running Exchange. Since Exchange has installed with relaying on by default (at least when I last checked it about 6 months ago) you may have been a spam conduit in the past (if not still one now). Test every mail server by getting on the machine and running telnet to mail-abuse.org [mail-abuse.org] (standard telnet port 23) and having it check to make sure you are not an open relay.

  • by Skapare (16644) on Saturday January 20, 2001 @10:49AM (#493716) Homepage

    reply... "Midnight will be when NTP says it is."

  • by Grit (18830) on Saturday January 20, 2001 @09:21AM (#493720) Homepage
    I have a Pacific Bell DSL line, running my own mail server with my own domain name (actually a subdomain of stanford.edu). The problem isn't on Pac Bell's side; my parents use Earthlink, and my email to them was bouncing. Some investigation showed that they had configured their mail servers to reject any mail traffic from Pac Bell IP addresses other than the Pac Bell mail servers. This was an explicit decision on their part, again with the motivation of "reducing spam."

    Fortunately, I was able to relay my SMTP traffic through Stanford's mail server (since I'm using a valid *.stanford.edu address) for each set of mail destinations that does this access control.

    I think it's pretty stupid to assume that a DSL line is going to be using the ISP's email services as well--- especially since Earthlink has no problem _delivering_ mail to that account.
  • Moderators

    Excuse me while I stoop to the newlevel of this post (No Score +1 Bonus off) so as not to disturb any self-respecting "Browse at 2" reader.

    This post is not flamebait. If any ISP had less than 15 hops in their network then I would surely consider them amateurs. The whole reason to use OSPF and BGP is because of lower convergence rates. If changes in the network occured you would not be able to access the page you are looking at now unless you ran OSPF and BGP.


    I have nothing more to say - except that I do not, personally, moderate topics that I know nothing about.


    What the hell, I'm turning Score + Bonus back on because I feel idiots are moderating me down. It doesn't matter anyway as I seem to have karma to burn.

  • by GC (19160) <giles@coochey.net> on Saturday January 20, 2001 @10:33AM (#493727)
    OSPF is explained in RFC1131, later replaced by OSPF V2 in RFC1247.

    Explaining the internals of OSPF is beyond the scope of this forum.

    You can, however, RTFM RFC1131 [landfield.com] and RFC1247 [landfield.com]. These are in Postscript.

    BGP is described in RFC1771 [landfield.com]

    Now, listen up, go get a life!
  • by Nicodemus (19510) on Saturday January 20, 2001 @09:06AM (#493728) Homepage
    At the time I was working for a web site, basicly, and the problem we were having is that @home customers in san francisco couldn't get to the site. After talking to a few of these customers, I had a couple do a traceroute to our server, and somewhere in the middle of @home's network a split horizon (i think that's what they are called) happened. It was where the packet just kept getting bounced between 2 of their routers back and forth until one of them finally dropped it. This only happened to traffic destined for our little network. I called @Home and was escalated to the top tech, who finally believed me. Then I was called back by a sysadmin there who required a lot of convincing. So he finally acknowledged the problem and said that they would get to it. Before I left that company I don't think it had been fixed, but it might have by now. It actually seems like it was a problem with their RIP or IGRP config, so maybe when a router was rebooted it would fix it's tables. Who knows. But the short of it is that I got ahold of a sysadmin and nothing was done. So good luck when you get that far. The journey may still not be over.

    -Nicodemus
  • Step 1. Reverse DNS lookup on domain name.

    And this does what exactly? What do you expect to see? Do you pattern match for something that looks like a dial up or what.

    Step 3. Compare IP address and MX address. If they are not equal, bounce mail.

    So you have now found a way of bouncing mail from *every* large ISP around - *no-one* running a decent sized installation uses the same machines for incoming and outgoing mail.

    And that comment just shows the real problem - every idiot thinks they know how to run an ISP. Go away and come back when you have sucessfully built a 1 million (or larger) user mail system - I have, and it needs plenty of experience and knowledge (and also some luck).

  • Much better, but futile. I connect through Telstra Big Pond Direct here (which is their permanent connection, static IPs, bandwidth resellers), as opposed to Big Pond Home (etc etc). This ISP provides bandwidth. They don't give you an email address, nor a mail server, or anything. They *do* give you MXing rights on their mail servers, but you need to have your own SMTP server. This would be mission impossible, much as I hate to say.
  • They block TCP Port 25, but they allow relaying through their mail servers for other domains IF you're part of their network. You can read about it at http://help.earthlink.net/port25 [earthlink.net]. I'd bet that they are putting some form of blocking / throttling intelligence into their mail servers. Sure, you could do this in filters with a sophisticated enough firewall, but I'm not aware of any products that have that level of sophistication AND can handle the kind of traffic they do at a reasonable expense. Mail servers are designed to understand mail, so it's much easier to put the intelligence there.
  • by RISCy Business (27981) on Saturday January 20, 2001 @10:44AM (#493736) Homepage
    Okay, first off, you're doing it wrong.

    You need to call their NOC, *NOT* tech support. Get their NOC number, which is according to my records, 650-556-5599. If that's not the NOC, you can get the NOC number from them.

    Once you get to the NOC, make them create a trouble ticket, and get ready to use your "I'm NOT HAPPY WITH YOU" techniques. The ONLY way anything will be done about it is if you ride them. Hard. They probably have the TT from Tech Support, so have that number ready, and give it to them. Start riding them hard. Demand supervisors, etcetera. Remember, the NOC is going to be setup with a front line defense (NOC techs), second line defense (NOC NetEng, NOC Unix Admin, etc), third line defense (NetEng, Unix Admins), and finally supervisors. That's NOT how it's managed, but how it's going to progress. Escalate often. Just keep calling them.

    That's the only way I've ever gotten anything done with Crack-Home or any other moronic overly large ISP. If they're big enough to have a NOC, then rest assured you'll only get things done if it gets to the NOC. The NOC will likely scream at Tech Support if they get TT's from them (I know we did when I worked in one) and generally have a fit, and ignore the ticket as much as possible. NOC and Tech Support typically do not get along.

    Hope this helps, and good luck.
    your company here. [fuckedcompany.com]
  • I have one email address on my ISP's POP box, and another on my own mail server hooked to my DSL line. If I want to send mail, then using my ISP's SMTP server works great.

    However, when my wife wants to use her email (at a university), then mail sent via my ISP's SMTP server is rejected. Why? Because they're doing really, really strict addressing rules. If it's not from ******@verizon.net (Verizon being my DSL ISP), then it gets rejected. So, thankfully, Verizon isn't [yet] blocking port 25, 'cause then my mail server would be worthless -- and my wife wouldn't be able to reply to any emails she receives.

    Yeah, she could use the reply-to (which is what I do so I can use my @acm.org address), but that'd mean folks would often reply to the wrong address, or CC the wrong address, and that I'd have to pay Verizon for another email address.

    As for @Home, I know around here that they scan for FTP, HTTP, and SMTP servers -- so you can't argue that it's an anti-spam campaign. Someone has decided that those are "commercial" activities, and that you thus must pay the extra $75/month (or whatever) for that priviledge.

    Without the ability to run your own servers, then @Home (and others) are essentially putting pay phones in our houses. We pay for outgoing calls, but we are unable to receive telephone calls.

    (And if that last part seems like a mystery to some folks, most of the pay phones I've seen won't accept incoming calls.)

    The real thing @Home and other ISP's ought to do is put a simple clause in the contract. If you cause a problem, then @Home gets to bill you for the expense of causing that problem. If you send out a million spam messages, then @Home gets to bill you for all the effort it took to deal with that problem. Most ISPs just cancel your account. That's not a deterrent.
  • I'm serious. I've run into a situation several times before when pay phones wouldn't accept calls. Best example was near Butte, Montana, when I had car problems, and had to phone a friend. I gave them the pay phone number, and they said they'd call me back (they were coming to help). After an hour, I called back, collect (having blown all my quarters calling them), only to be told they had been trying and trying. They finally called the phone company, who matter-of-factly said, "yeah, none of our payphones accept incoming calls."

    So, yeah, most pay phones accept incoming phone calls, but a few don't. Thankfully, I can afford a cell phone, and I now live in an area with cell service.
  • I mean, it's an unpaid bill. You then basically turn the whole thing over to a collection agency. Tell the agency they get half. The ISP then gets some money for its troubles, and someone else gets to be the pit bull. I mean, they already do that with some of the unpaid or underpaid bills.
  • What is really needed is a way for a geek to say "I have mad Kung Fu and have a Black Belt in Network Engineering" and they would say... "oh... excuse me... I will connect you to our third tier tech support right now". Of course that is not realistic. But what they could do is keep track of people with mad Kung Fu so they can go right through the line.

    Currently working for a major provider (not AOL, I wouldn't sell out) in tech support, I have to say I get a lot of callers who say they've got an MCSE and a CS degree and they've been in the field for 20 years calling me that make most AOL users look like Linus Torvalds.

    Best way to handle tech support is to tell them what you think the problem is, let them run through thier checklist so they can properly document the trouble ticket as required, and if you're cool and cooperative about it and try and be on the same level as the support geek, they'll escalate it for you.

    To be honest, I love it when someone outside the Winmac realm calls, because they're almost always the easiest calls for me, ie they're the first to notice a widespread network outage and are very cooperative in giving me details I need to document while I run my tests and document that, it gets escalated to NOC and the problem's fixed by the end of the day, or they just had to reinstall thier OS and lost thier TCP/IP settings and all I have to do is read them off the user ticket...

    One problem that slows down reporting of network issues a lot are technically illiterate people who get mad because they can't check thier email right now and won't cooperate with us making sure they're set up right or variations on that theme.

    --

  • Try just doing a whois on the domain name then call the adminsitrative contact. Ive done that a few times, and either its the correct people (NOC) or they transfer you to them. Saves me alot of time going through tech support.
  • Who says I don't? If you want to talk about Internet tradition, than you'll surely recognize the extremely common practice of having to use the SMTP of the provider you're connected to. I mean for Christ's sake, Sendmail is configured that way by default! Now if you want to talk about GUI e-mail clients, not mutt or pine, give me one example, one damned example of an e-mail client that acts as its own SMTP. Come on. Give me an example! You can't can you? I've been dealing with this spam problem for a very long time. The DUl is a damned good list. You should always use the SMTP server of the provider you're connected to, or POP-before-auth SMTP of another host if you have such a host at your command. Therefore no legit mail should ever EVER come from a host on the DUL.

    Let's switch this a bit. Give me an example of when legit mail should come from a host that could be in the DUL. Let's say you want to use Pine on your Linux box to send me a message. Ok, no problem. Configure Sendmail to direct all non-local SMTP traffic to your ISP's SMTP server. problem solved. Too much hassle? You use you Linux laptop at different locations on differnet networks? Convince your ISP to utilize POP-before-auth. The only other solution is to use an SMTP server that accepts traffic from your current location. If we didn't use lists such as DUL, than how else would we filter out all the non-legit e-mail (spam) that comes from those neworks, like uu.net and popsite.net? Well, in short, we can't unless we filter by content of messages. Now if you want to rant about accidentally deleting legit mail... You're fighting the wrong battle. Join akt.comp.sendmail or read the sendmail FAQs. It's enlightening.

    --

  • by tequila26er (46835) on Saturday January 20, 2001 @09:12AM (#493759)
    ...because it's a lot easier that trying to get anything useful out of @home. I've been an @home user for three years now and I can honestly say that if it weren't for the fact that I can't get high speed internet access from a competitor, I would be switching.

    I think part of your problem might be that the tech support staff are also kept in the dark. They can't help you if they aren't informed themselves. Maybe there's an @home techie out there who can answer this?

    I do tech support for another major ISP in this area and I am proud to say that our users don't suffer from this same problem.
  • @home was in my blacklist. Three of their mail servers were pounding one of my servers numerous times a second attempting to relay mail through the server. I tried every conventional (old school, so-to-speak; email the postmaster, or sysadm) method of contacting them to no avail. I tried calling and emailing abuse@home.com - nothing. It left me with the impression that the fucking bastards felt they were so big that they didn't give a shit whether they were a responsible member of the Internet community. So I blacklisted the entire domain until the relay attempts trickled down, then stopped, which was for months. Fuck You @Home!
  • It seems it would be pretty straightforward for a company to give their entry-level tech support people a list of keywords, that if the customer mentions enough of them, to send them on to the next level.

    Keywords might include:

    • arp (for DSL)
    • MAC address
    • RFC
    • traceroute, packet loss
    • port n
  • > Fully 50% of the SPAM I used to see originated from the dialpools of the gargantuan ISPs of the world. UUnet, Earthlink, PSInet etc.

    Since Earthlink went to port 25 blocking (actually, dialsprint.net leased to Earthlink customers), that's now 90% UUNET.

    I'm positive that the majority of abuse mail is piped to /dev/null

    For UUNET, I'd agree - they're wholly nonresponsive and deserve to be blocked.

    But I'll vouch for Dialsprint. It took a few months, but I did get a human response from one of the Dialsprint abuse staff. Of course, it was two days before they implemented port 25 blocking, so it was kind of a moot point, as spam from Dialsprint/Earthlink dropped from 30% of my spamload to "noise level" within a week of it.

    MAPS DUL rocks, but I'd like to see uu.net RBL'd.

    The main problem with that is that they don't have an "actionable" RBL nom for uu.net. I've got dozens of Telodigm (linkusnow.net and friends) spams and hundreds of UUNET ignorebot tickets. All I have to do is make the phone call. Sigh. Fuck Qwest for hosting Telodigm, and fuck UUNET for... well, being the world's biggest spamhaus.

    But the day UUNET blocks port 25 for its resellers (which I fear will require a full RBL of every netblock they own, with associated collateral damage) is the day we win a major battle in the spam wars. They're the only big dialup provider left.

  • We had this problem once. @Home blocked all email from our company's servers, claiming the sender was bad. This continued for a couple of hours before it mysteriously cleared up.

    I've been dealing with @Home tech-support for a couple of years now, and I've found the best way to get them to move is to threaten them with antitrust action. They have an interesting market position since in many areas they are the only company providing such a high-end service for such an affordable price. It's my opinion that they know this and are taking advantage of it to slack off.

    Here are just a few things that I've seen from them:
    • No backup power on their infrastructure (routers/switches/hubs)
    • Regular failures of their internal core routers
    • Regular failures of their mail servers
    • 64-KB limit on outgoing email attachments (which they claim doesn't exist)
    • Magic terms-of-service (now you see them, now you don't, now you see them again, ...)
    Back when I was an @Home customer I went as far as to track and log some of the above mentioned failures. I found on average they happened 2 to 3 times per day. I've never seen a major ISP have such critical issues so frequently. To me, this is a sign of incompetence.
  • by burtonator (70115) on Saturday January 20, 2001 @09:24AM (#493780)
    I feel your pain. I too have had huge problems with ISPs either not believing me or not listening to reason. I spent 7 hours (at least) on the phone with Earthlink (7 hours is much less than it takes to get another DSL provider) trying to fix a problem with their PPP servers. I was doing protocol analysis so I was *certain* what the problem was. The bad thing is that it was *very* technically complicated and not on one of their check sheets for their techs.

    The point is that there is nothing we can do about this. I am sure there are a lot of *really* smart people here. The problem is that tech support people have to deal with a lot of Microsoft Morons so they just assume we are in the same category.

    What is really needed is a way for a geek to say "I have mad Kung Fu and have a Black Belt in Network Engineering" and they would say... "oh... excuse me... I will connect you to our third tier tech support right now". Of course that is not realistic. But what they could do is keep track of people with mad Kung Fu so they can go right through the line.

    If an ISP would do this it would SERIOUSLY increase their business. All the geeks would subscribe to their services because they don't want to deal with other ISPs. It would also increase their reliability because they would have *really* smart people fixing their network problems for free! Open Source ISP! :)

    Somehow that is logical so I assume it will never happen. God forbid any Western country undertand Zen philosophy!
  • I've had this problem with Adelphia as well. Their support monkies form a wall that's very hard to get through. It's especially difficult since when you find someone who actually seems to know what they're doing, your replies go back into a 'support pool', not back to the person who originally replied to your message. I assume that the network admins have thier own email addresses like ts-jdoe@home.com. I believe your best chance would be to directly contact a known sysadmin through telephone or email.
  • Um...... no, this is not the same thing.

    The original poster stated that her company's smtp servers were/are blocked by @home's smtp servers. Earthlink is not doing this-- they are not allowing users dialed into them to use someone ELSE's smtp servers directly; the users must instead use EarthLink's smtp servers to send mail. Why this bothers people still manages to stump me-- IT DOESN'T MATTER WHO'S SMTP SERVER YOU USE TO SEND MAIL. It will all get to the same place regardless. The reason that EarthLink chose to do this was simple, to prevent people using their dualup lines from spamming via someone else's poorly-configured smtp server. It really amazes me how some technically-sound decisions made by a company are twisted into bloddy-murder when people who don't know what they are talking about gripe about free speech this or censorship that. This is nothing to do with censorship and its not at all the same as the original poster's problem, which is a legitimate gripe.
  • I use Netcom for most of my dialup, since they still have unlimited dialing for $19.95 per month,
    as opposed to most other companies that charge per hour after some limit like 20 or 100 hours. They're not who I use for my shell account, or web page, or incoming email, or outgoing email - I'm very happy with idiom.com , and sometimes I'll use my company-provided dialup instead of Netcom. The Real Netcom dial pops let me connect to Port 25 at my ISP, where I'm recognized as a customer and can send email. But the Mindspring-flavored Netcom dialups don't - the connection just hangs unless I'm using one of Netcom's email relays. Yes, I realize this blocks many spammers. But it also blocks many legitimate users, particularly of Unix systems.
  • Sure, there are limited-functionality mail clients that can't send email directly, and some people use them even on Unix machines. But standard Unix mail follows the RFCs and a decade and a half of Internet tradition, which is that you send mail directly to Port 25 of the destination, unless you're running some other mail protocol like UUCP bang-mail, in which case you find a "smart mailer" like sendmail to relay it for you, or unless the destination has an MX record saying somebody else will handle their mail for them. If I'm one of your dialup customers, I doubt I'll see many systems using you as their MX service?

    Furthermore, even using a Eudora client-oriented mail system, I don't want to have to reconfigure my client every time I dial in from somewhere different (e.g. take the laptop from home to the office or plug it into the DSL in the lab or a customer's LAN) - I should be able to send directly. If each ISP blocked port 25 except through its servers, laptops would be much lamer.

  • by BamaSlam (78998) <dav AT swbell DOT net> on Saturday January 20, 2001 @11:31AM (#493799) Homepage
    I work as a sysadmin in the NOC of a large very well known regional ISP. The only people who have the number to our NOC are the people above us and the managers of the front line support. Customers and the general public are not allowed to call us and I think it's a good thing. I saw a reply above that linked to a list of NOC phone numbers and checked it to make sure that my company wasn't on the list. Face it, if you were bombarded 24/7 by geeks who felt that their problem deserved your full undevided attention you wouldn't get much real work done (face it, there are people who feel that if their insignificant problem doesn't get fixed this second the world will collapse and anarchy will reign).
    For the problem that was listed at the top of this thread, I would suggest contacting the abuse@ email address for the domain in question. I do know that our company's email admins do get those messages as well as a few higher-ups. Other than that, route your mail thru your isp's SMTP server and save all of us headaches.
  • Hah. I used to do technical support. All the people who called in claiming to be "kung-fu engineers" were usually
    UTTERLY
    FUCKING
    CLUELESS.
    I remember this one lady who called in. Hot-shot MCSE didn't want me to tell her where in a connectoid to put in DNS information. After 3 minutes of silence she finally let me tell her where it was. What the fuck?

    Then there was this guy who called up during an authentication outage to ask why we didn't use "a BDC, a backup domain controller?" I told him we didn't use NT. Duh. Why the fuck would we trust NT to authenticate millions of dialup users? What a laugh. What the fuck were we, a mom-and-pop shop? I think not. Anyway, a million backups wouldn't have worked... the central auth database was down because the machine running it had a hardware failure and it usually takes a couple minutes to fully switch to the alternate machine.

    A "SUPER-SMART ENGINEERS ONLY !!!!111" line would be constantly inundated by clueless morons who have HEARD OF APPLESCRIPT or something TOTALLY INANE and suddenly think they're SUPER-WIZARD-GENIUS-OF-THE-INTERNET.

  • "Customer facing" is an industry-standard term. Everything is either customer-facing or non-customer-facing. A development server that is firewalled from the outside world is non-customer-facing. So is the sysadmin that runs it. You don't route calls to sysadmins. They are the (ostensibly) bright people on whom you rely to keep the show running. Encouraging them all to quit by forcing them to deal with cluebies *AND* do a full-time SA job is a great way to have them all say "Fuck you", quit, and then you have to re-hire them as "contractors" for 2x the wage you were paying them before because they're the only ones who know enough about the infrastructure to keep it alive.
  • Your mail server should be relaying through the SMTP server that PacBell assigned to you. Period.

    Why is that ? Home DSL users cannot be trusted to configure an SMTP server ? As it is, when you get a static IP address, it is MXd to pacbell anyway. You cannot relay your email server through Pacbell. You need to host your own domain name to receive email(because of the MX problem). If you want to have reverse DNS work, you also need to pay another $100 to Pacbell, the fee they charge for adding 5 lines to their named configuration files - blind highway robbery.

    A much easier solution, if you can configure a box, is to host your own DNS and SMTP. You save $100.

    If Pacbell has a problem with that tough. The DSL line is a monopoly, but there are about 10 providers I could use other than them.

    If my ISP is concerned with spam I invite them to check me for relaying, and port scan me too. Go for it.

  • I understand that you are a complete guru/Linux god, and are incapable of fucking up configuration of your box. If you want to host your own DNS and mail, then use a service that allows you to do that.


    To put it mildly - it is pretty difficult to screw up and leave relaying open if you are not using sendmail as your SMTP. In qmail, for example, no relaying is the default.

    As for DNS, it doesn't cause an issue with relaying/spam. You only list your box, you test it forwards and backwards, and you generally will know very quickly if it doesn't work properly.

    Many professional sysadmins accidently leave open smtp relays on the internet

    It is uncertain they should be called professionals then, Bare minimum, the longest O'Reilly book deals with sendmail conf at length.

  • How do they manage to keep track of the IPs used by major ISP's mail servers? Lucky guesses? Laborous investigation?

    Step 1. Reverse DNS lookup on domain name.
    Step 2. Check domain name entry for IP address
    for MX value
    Step 3. Compare IP address and MX address. If they are not equal, bounce mail.

  • Administrative Contact, Technical Contact:
    Kiewlich, Daniel (DKF336) abuse@HOME.COM
    @Home Network
    425 Broadway St
    Redwood City, CA 94063 US
    650-556-5399 650-556-6666

    I doubt that you'll get much value from the abuse@home.com address, but you may be able to find a useful path at the phone numbers.

    Billing Contact:
    Du, Trung (TD2157) trung@CORP.HOME.NET
    @Home Network
    425 Broadway Street
    Redwood City, CA 94063-3126
    650-569-5437 (FAX) 650-569-5100

    Going through the accounting department may not be as bad an idea as it looks like on first glance. Everybody talks to accounting. They should be able to point you to someone in Networking with purchasing authority. That's also someone who can pull strings to get things done.

    If you're not a good people person, you may want to find a techie who is. This path is probably going to take a little bit of schmoosing.

    Just because the front door's the only obvious way in, doesn't mean it's the only way in.
    `ø,,ø!

  • I think that I actually managed to do that with my ISP. Every once in a while I'll call them when there's a problem, and give them pointers to fix things. One time, I called them and the Tier 1 person I was talking to was quite vague about the answers he was giving me. Then he asked for my ID and looked it up in the database.

    I must have some sort of flag, because about 5 seconds after I heard him hit the enter key, his tone completely changed. Needless to say, I'm happy with the service I'm getting.
    `ø,,ø!

  • I CANNOT BELIEVE MY FELLOW GEEKS ARE OUT OF THEIR MINDS!!!

    Call the NOC, the sysadmins or network admins of a major ISP for a firewall change?!?!?! Are you OUT OF YOUR MINDS?!?!?

    "Hello, are you the Sr. Network Admin for @Home"?

    "Uh...yes?"

    "Can you please open up port 25 so people outside your network can send email to your mail servers?"

    "Oh sure, we take firewall change requests over the phone from strangers outside our network all the time. We don't even bother putting in change control, or discussing it with our manager. Afterall, our manager and his directors and the CEO don't care about all our millions of dollars of firewalls and security systems. We don't have any corporate policies or security procedures. This means we here at the NOC can do anything we want. I'll open up that port for you in a jiffy."

    "Wow great! I'll you call back for more network change requests!"

    "Great idea! Be sure to write down our number. We love answering calls from strangers, as well as users in our own company. Admins like us love helping users. Its been a pleasure serving you. Is there anything else my team of overworked network admins, sysadmins, and security experts can do for you?"

    If you believe that the above scenario has any basis in reality, perhaps you deserve to find a sysadmin or network admin somewhere. He or she, after hearing what you have to say, will most likely take a shotgun and blow your brains out.

  • i work for an isp that resells several nationwide networks. most big networks, such as uunet, psi, c&w, etc all block port 25 traffic to all but allowed hosts. on uunet, we can unblock 25 traffic by sending radius attributes. but, in general, people who use our dialup service and need to send mail through their third party web hosting or mail hosting providers need to simply use our mail servers for outgoing. i recommend you tell your users to simply plug in mail.home.com, or whatever it is, as their outgoing mail server. all in all, this is less of a problem and more of a security issue.
  • What is really needed is a way for a geek to say "I have mad Kung Fu and have a Black Belt in Network Engineering" and they would say... "oh... excuse me... I will connect you to our third tier tech support right now".

    A few things like that exist. Mostly from Microsoft. There are numbers that MCSEs can call for tech support; whether they're any good I don't know. Microsoft also has those 900 numbers that cost $95 to call, where you get your money back if you can prove they have a bug. You actually get somebody competent if you call those numbers, and they really do credit you back if they have a bug.

    And there's that banner ad on Slashdot for the hosting service that gives discounts to Slashdot readers.

  • Forget it. Pretend no one mentioned it. Because it really isn't part of the question. The question is: "What is the best way to get in touch with a Network Admin or someone who actually can do something about a network issue in cases like these?"

    Now pretend you're the network operator for a large company. Do you really want to be dealing with customers when you could be playing Starcraft? And even if you're not playing Starcraft all day on your carefully crafted network, chances are, you have better things to worry about than your company's customers. No, you have secretaries and underlings who take your calls and check your email. They sort it. They send it to you.

    Imagine you're sitting there, happily flirting with your co-worker when suddenly your beeper beeps... "zerg0 down." Bloody hell, why'd the web server crash? And why isn't it back up? Lemme go check on it... Suddenly every phone in the fucking office lights up with angry customers demanding to know what happened. Some of them are probably the helpful sort who'd like to explain to you in minute detail what happened. Well, would you rather be working on the problem or dealing with customers? Well?

    Therefore, the problem isn't how to contact the network operator, but to convince the underlings/secretaries that there is indeed a problem that can only be solved by having them put you in touch with an admin. Money helps. Lots and lots of money. A legal contract entitling you to contact the network admin when you need to (which no sane service provider would sign, but you never know) might also help.
    --
    Peace,
    Lord Omlette
    ICQ# 77863057
  • by patter (128866) <pat@@@sluggo...org> on Saturday January 20, 2001 @09:46AM (#493847) Homepage Journal
    I just left a job where I was doing front line support for one of the big US national ISPs.

    The problem for us was two fold:

    1 - front line tech support is staffed by people who have to learn that if the problem can't be fixed by them, it may never get fixed. In our case Tier 2 support was staffed by a bunch of idiots. Whenever I'd get an issue that I believed was legitimately our problem, we would try to escalate to them. Sometimes they wouldn't even understand the nature of the problem (most of them don't have any formal training, and don't have a clue what TCP/IP is never mind have a vague idea what routing is all about).

    2 - The bigger the organization, the more it becomes steeped in 'procedures' and 'processes' meant to isolate the user from network operations. If you could convince them that it was an issue that had to be escalated further, it would seem to sit there and go no further. They tend to get lost in a mire of corporate policies, and rarely if ever do issues get routed to the network ops.

    Needless to say, it was very frustrating when an admin from another smaller service wanted to contact our netops. I guess the bigger sysadmins would have established direct contacts, because we never got contacts from the larger ones.
  • You're a sysadmin... imagine the pain of ordinary users trying to report real problems to their ISPs.
  • Pay phones can recive calls, you can usualy find a phone number printed on the thing somewhere.

    You obviously don't live here in the states. Everyone in the states knows that the only people who ever recieved phone calls at pay phones were druggies -- drug users used to use pay phones to page their dealers, and buy drugs. So, most pay phones in most major cities in the states no longer accept incoming calls. As everyone is aware, this policy has had significant effects on the availablity of drugs in the states -- it's nearly impossible to drugs here now.
  • We pay for outgoing calls, but we are unable to receive telephone calls.

    Pay phones can recive calls, you can usualy find a phone number printed on the thing somewhere. And you don't need to put any money to pick it up either.

    Amber Yuan 2k A.D
  • You have 2 mail accounts, joe@earthlink.net and one from work. You need to send mail from the work.com account, but ELN's mail server don't relay for other domains. Normally, you would use smtp.work.com (or something along those lines), but ELN also stops you from doing this. How on earth are you going to get the mail out?
  • If you can find the geek phone number or e-mail, you get to Tier 2 or 3 right away, or maybe connected to a pop quiz (get 10 questions right you go to Tier 2, 15 questions right go directly to Tier 3)

    It would be a real pain in the ass to have to take a quiz to get through to the NOC though. Especially if you need to talk to someone *right now*, eg if someone on their network is DoSing your network.

    ---
    Check in...OK! Check out...OK!
  • Yes, I used to work for a company that AOL couldn't get to, period. Not just email, but everything. We found out later that it was a DNS problem, and a change we made weeks earlier still hadn't updated to all the proxy's. This went on for almost 3 months, when finally AOL's cache was completly flushed. All other ISP's had updated their cache within 24 hours. I was so mad I put up a message on the old IP saying that if you where an AOL customer that you wouldn't be able to access our systems until AOL fixed their problematic architecture... we almost got sued for slander, so I don't suggest that route.

    Bottom line - I too talked to (clueless) AOL tech support for hours at a time to no avail. We just had to wait for this wierd caching problem to go away. Problem is, our customers thought it was our problem... they seem to think that we control how they get to the Internet. You just have to educate your customers that you can't control 80% of the process (computer hardware, OS, browser, ISP, backbone/NAP's).
  • Sounds like you have been bullshit'ed:

    Split horizon is a complicated term for distance vector protocols that does something very simple: Its says that a routing update cannot be sent out on an interface which is was recived from. If I hear that the router on serial 0 can reach 10.10.10.0/24 and that is my on ly way to get there, there is no point in me telling the router on s0 that sent me the update about it.

    RIP / IGRP are classful routing protocols and it is very unlikely that any ISP would use them for there IGP now as they don't support the sending of subnet masks in updates. OSPF / ISIS as an IGP is more likely.
  • by TheFlu (213162) on Saturday January 20, 2001 @09:55AM (#493898) Homepage
    As a Systems Administrator myself, it's painfully obvious that others here have run into the same problem I have when I call technical support. You have to take those extra few minutes of time to convince the fellow geek on the other end that you are indeed a member of "geekdom". I propose that we have a secret geek codeword that can immediately identify each of us as a member of the geek commmunity, kind of like fraternities all have secret handshakes. Hmmm...let me get the ball rolling here, I propose the phrase "I hear there's going to be a packet storm at midnight.". Any other suggestions?

    We're all geeks over here>>> The Linux Pimp [thelinuxpimp.com]

  • Hit 'em with a denial of service attack from one of your boxes. Then they'll be pounding on *your* door. Don't hurry to respond.
  • Well, that sucks. That said Bellsouth.net's addresses, with or without my ISPs, do not seem to be on the list.

    And I have to admit, for the first time, to object to a policy of MAPS. Sending email directly from a machine is not only RFC compliant, but actually "correct" - there is no RFC mandated basis for the practice of relaying, authorised or unauthorised, it's just something that happens to work, and was originally supplied by ISPs to make life easier for customers. Indeed, this is one of the reasons why blacklisting SMTP servers that relay is legitimate.

    I'm not arguing that they don't have the right to incidentally, I'm arguing that it's brain-dead and breaks the spirit of Internet cooperation by intentionally breaking an RFC compliant process. Especially as the majority of Linux machines I've seen tend to use sendmail to deliver email directly by default.

    It looks though like the problems I had were a unilateral action by Netcom - indeed, only the ix.netcom.com addresses were effected, I could email to people with @netcom.com or @mindspring.com without the slightest problem.
    --

  • Am I glad I metamoderate. If the moderator who thought this was a troll could email me at peharri (at) yahoo.com and explain why, I'd be very grateful.

    Now the posting I was replying to, which implied it's reasonable to block email if there is a small chance that it's spam even if the method it was sent was RFC compliant and the alternatives are not, and even if a substantial amount of legitimate email gets blocked at the same time, seems to be to be more deserving of the term. But, hey, let's not let logic get in the way of the War On Spam.
    --

  • It would be infinitely better as while the ISP's actions might not be RFC compliant, they at least don't break actions by their customers who are being RFC compliant - the reason ISPs provide SMTP servers to customers is not because that's the RFC mandated way of handling email, but for convenience reasons as clients doing it the "right" way can potentially suffer all sorts of problems from not being online at the same time as the destination server to having software that doesn't correctly understand the email delivery protocols.

    Me, I use sendmail's default SMTP configuration, which causes sendmail to deliver directly. This means I don't have to reconfigure it each time I use a different ISP. But it does mean braindead spam blocking mechanisms like the MAPS DUL are going to block me in future if the ISPs on the list don't redirect 25. So if ISPs are going to accept the legitimacy of that list, they must in my opinion implement redirecting - if they are going to break the protocols, they must make sure their customers, and those their customers are trying to legitimately get in touch with, do not suffer for following the rules.
    --

  • You'd hire a sysadmin who'd readily chose a blocking mechanism that would block email coming from most default-configured Linux/BSD machines?

    A strange policy.

    Much as I accept that spam blocking is an imperfect art, it's always important to ask the question: Will my method prevent someone legitimately contacting someone on my machine? A check for an email containing a URL and all caps subject line may occasionally block a legit email, but not in any way that can't easily be circumvented by the sender. It's an annoyance.

    By comparison, the DUL is non-negotiable. A user who is blocked has to reconfigure their system to use an ISP's mail server, which my experience showed is STILL NO GUARANTEE that the email will eventually be delivered - if the DUL lags behind local policy, for instance. Configuration may be simple - the information about which server to use is readily to hand, the email delivery client having an obvious place to set these things, or it might be more difficult - and with sendmail, at least, it's not straightfoward.

    A better solution, for ISPs that want to prevent their own customers from spamming, is to redirect port 25 to their local email relay. This may not be perfect, but it doesn't break RFC compliant customers email. But that's for the sender ISPs. If an ISP is having problem with spam from a particular other ISP, it needs to figure out a sane way of blocking that spam. As DUL is going to block at least as many legitimate email as illegitimate, it seems reasonable to suggest that DUL is not sane. It might, in those cases, be more appropriate to temporarily block that ISP altogether, and get it to clean up its act.
    --

  • by squiggleslash (241428) on Saturday January 20, 2001 @09:38AM (#493922) Homepage Journal
    I recall suddenly being unable to email anyone with an ix.netcom.com address, being given a boilerplate message by the SMTP server to the effect that my email system was misconfigured and I should use my ISPs. The MX records for ix.netcom.com clearly pointed at the SMTP servers I was delivering email to, and after a lot of stress, including routing email via my ISP's email server (I normally get sendmail to deliver directly) and finding the same problem, contacted them.

    There was a lot of hassle involved. Netcom, then owned by Mindspring, clearly had a massive wall between their system administrators and their support people, with no direct way of contacting the sysadmins. I'd email the support address, and get an email back from someone clearly too clueless to know what an SMTP server or MX address is, insisting the problem must be my end or with my ISP. In the end I basically had to persist, phoning their 1-800 number in the end, getting names of support staff involved, and following up every inch.

    I found it tough. The more you point at RFCs and stuff, the more you look like, well, the sort of people you get on TV claiming you don't need a drivers licence because the states aren't constitutionally allowed to forbid you from using the roads or that banks are allowed to create money because of some legal loophole. The person you're talking to has no idea what an RFC is, or an MX record, or anything like that. All they can do is accept that you've tried it all different ways and can't send email.

    In the end they put a ticket in with their system administrators, who knew exactly what the problem was and fixed it.

    From what I can figure out, the problem was because my ISP's IP address block is smack in the middle of BellSouth's (BS providing the connectivity), and Mindspring had configured the Netcom servers only to accept email sent directly from BellSouth's email servers, not from BellSouth customer IP addresses - my bellsouth.net account continues to this day to have the same problems but I'm buggered if I'm going through the hassle again. This is stupid anyway, but of course as the complaints were coming from people who deliver their own email, or from people with ISPs in similar positions, of which there are probably relatively few, few enough for it to look like most email is being delivered perfectly and therefore it "obviously" being a problem on the deliverer's end.

    Why they did this is anyone's guess. I think, given the problems I have being let onto any IRC servers these days, that a lot of the hacking being done at the moment is being done from Bellsouth.net addresses, but I haven't read anything anywhere to back that up. Mind you, the problems emailing ix.netcom.com started a year ago, whereas EFNet's clamp down is at most 4-5 months old.

    My advice? To be honest, just keep trying, and keep piling on the pressure until they relent. Send email to the support addresses. If you don't get a response, start calling - preferably calling the @Home customer's 1-800 support line. Keep calling, get names of support people, and don't stop until the situation is resolved.

    If Mindspring hadn't finally relented and put in a ticket to their system administrators, I'd probably have used Usenet or something similar to start embaressing them, a little log of an nslookup, telnet to an SMTP port, and then this posted on an appropriate newsgroup. But as it was, it got fixed.
    --

  • I delete those messages.

    Hint: your boss doesn't.

    --
    All men are great
    before declaring war

  • I'm serious, if we could pull off a system like that, the rest of the ISPs out there would have to take notice when 1/2 of their broadband client's switch over.

    Hah!

    Do you have any idea how frightfully outnumbered we are? If every (seriously) technically competent dsl user dropped off the world completely, I doubt they would even notice.

    --
    All men are great
    before declaring war

  • I was having serious problems (line down for 3 months at a time) with our connectivity, so I took matters into my own hands so to speak.

    After searching their webpage, I found an email addy for joeShmuck in accounting, it was first letter of first name, then last name@isp.com (i.e., Jloser@isp.com). Pretty obvious.

    On a hunch, I went to their Management page, and looked at all the bigwig's name's. From there, I sent an email to each of them based on the pattern of the accounting person's email address. Each describing the problem and the lack of customer service to fix it. Soon as the VP of and the Pres of , ect ect, all the big people, read my problem and contacted the network admins, I've had great service. Anything I need, I get (they couldnt figure out how to fix the physical line, so they bought and gave us their wireless gear for our connection).

    Sometimes unorthadox, roundabout ways are the only way to get something done. Contact the Big Cheese's and let them know whats going on. I'm sure it'll be cleared up in no time.

    --Dave
  • by wysoft (301924) on Saturday January 20, 2001 @12:51PM (#493936) Homepage
    Plus, if they used RIP they'd be caging themself to a 15 hop limit. If RIP was still used on the mainstream internet today, the majority of the sites you visited would do nothing but spit back a "Destination unreachable" message. Hell, @Home is such a large network that they probably would exceed 15 hops within their own net!
  • When my small consulting company tries to send email to our customers on the @Home network we get this message back (edited to exclude our domain):

    Unable to deliver the message due to a communications failure.
    550 5.0.0 Mail originating from that domain is not welcome here.

    We host our own email from our Exchange server. There's no reason our domain should be blocked. How can I check if @home is blocking us?
  • It's good to see that I am not alone in my ongoing battles with Adelphia. For months I've been sending pings and traceroutes displaying how Alter.Net/UUNET, which is one of Adelphias main backbone providers, experiences massive amounts of packet loss and latency during prime time hours.

    Since the problem occurs at the fourth hop it effects just about 90% of the sites we are routed through. Unfortunately about one third of the support engineers I speak to don't even understand what the difference is between 30ms and 600ms. The rest of the time I receive "Unfortunately this problem is outside of our network". This is when I forward them UUNET's SLA's and request that they contact UUNET and ask that they request the NOC to look into the problem. Although they say they will escalate the problem they never do and every time I call back attempting to get status on the situation I get the runaround.

    I've attempted to post on Adelphia's message forums requesting that other people contact Adelphia. I figured that if I could educate other members that subscribe to the service they would call in as well. Unfortunately prior to posting support reviews all of the messages prior to allowing them to be displayed in their forums. Needless to say none of my postings reach the public.

    I've even attempted to contact UUNET's support via email and calling but since I am not a direct subscriber to their services they refuse to escalate my requests to review the problems.

    If anyone has any other information on how I can get through to people that can actually help resolve this issue please feel free to email me.

    Thanks,

    - Kujoe
  • by HitSnooze (307452) on Saturday January 20, 2001 @01:55PM (#493947)
    Following Lisa's posting about "@Home blocking PORT 25" I would like to add that AOL does the same thing.

    Our traveling reps use AOL to connect to the net when they are away from the company LAN. We were having issues with email constantly getting rejected. A quick telnet diag revealed that AOL has a Proxy on port 25 that grabs users requests.

    Numerous calls the AOL Joke Support didn't get us anywhere. Just explaining the issue is grueling because the Script Reading techs just spout off canned answers like "We only support AOL mail and not Outlook, etc". I tell them "If AOL is suppose to be a true ISP they shouldn't hinder people trying to use Port 25." When you try to ask for a Senior Tech they keep on with their canned drivel.

    We've had to setup another mailserver at the company that listens on Port 2525 in order for our users to send mail. AOL won't acknowledge the issue at all.

    Every other port works fine, just Port 25 they capture for some reason.

To thine own self be true. (If not that, at least make some money.)

Working...