Crazy Firewall Log Activity — What Does It Mean?

arkowitz writes "I happened to have access to five days worth of firewall logs from a US state government agency. I wrote a parser to grab unique IPs out, and sent several million of them to a company called Quova, who gave me back full location info on every 40th one. I then used Green Phosphor's Glasshouse visualization tool to have a look at the count of inbound packets, grouped by country of origin and hour. And it's freaking crazy looking. So I made the video of it and I'm asking the Slashdot community: What the heck is going on?"

Re:vertical stripes

[jmauro]

It looks like an active attack probably from one source with a number of controlled bots helping out.

The packets from every country at once are probably spoofs sender IP addresses from one or more sources (probably the spike countries).

The spiked country traffic are probably the controlled bots attacking the host actively.

Without seeing the actual packet data it's just a guess though.

Re:Another Slashdot Ad?

[Jah-Wren Ryel]

Wait, is this just an advertisement for Glasshouse? The voice in the video on Green Phosphor's website is exactly the same.

It is totally the same guy - the background noise sounds identical too - like he recorded it on the same microphone with the same environmental conditions.
Hell, he even starts each narration exactly the same with the pattern of, "Hi <name> here."

Re:Another Slashdot Ad?

[NoTheory]

If you check the other uploaded videos on youtube [youtube.com] by the same guy (who's name appears to be "Ben Lindquist", the CEO of Green Phosphor, found on blogger [blogger.com] and twitter [twitter.com]), there is an introduction to Green Phosphor's Glasshouse [youtube.com]. So yeah, Slashvertisement done in the style of Lost.

Welcome to the future of advertising. /sigh.

Mod parent up

[Anonymous Coward]

Exactly. This guy is advertising his own not-very-creative service.

Sure - he just happens to have access to the US State Deapartment logs, but isn't smart enough to look at the packets?

  Astroturf.

Re:That wasn't complaining. THIS is complaining.

[arkowitz]

Everyone always wants me to have labels on the graphs. I don't put them there unless you roll over the data, because I want you to see the patterns in the data without bias first.

I should not have called this graph "crazy looking". It is actually pretty simple and makes it quite clear what is going on, as you can see from the comments submitted by people talking about botnets.

Finally, I am not interested in producing graphs which show you everything "at a glance". Use a pie chart for that. I am making graphs which facilitate a deeper understanding of larger amounts of data than Tufte dreamed of showing using his 2D paradigms.

Re:Skylab Shreds

[osu-neko]

Computers are used by people. People who wake up, work, play, sleep, have weekends, business holidays, religious holidays, events and a pantheon of other reasons why they might act in seeming semi-concert.

You're suggesting that for the five day period in question, the majority of people work up at the same time GMT? Not 7am local time, but 9pm GMT everywhere in the world? Or did you just not actually look at the video (which shows spikes of data from every country in the world at the same time)? "Timezone effects" should eliminate these sorts of lines, not cause them, by spreading that kind of activity out over 24 hours.

Re:I'm confused

[Gerald]

How does /usr/share/GeoIP/GeoIP.dat ban my IP address?

Re:I'm confused

[sopssa]

Eh what? There's several GeoIP databases that you can install locally. In fact it seems like Quova is the only database you have to query remotely, which is somewhat crazy if you ask me. Or buy a server from them.

MaxMind [maxmind.com] is the best known one. Installing it on Linux server using yum merely takes "yum install GeoIP*"

Re:Another Slashdot Ad?

[sopssa]

To be fair though, he didn't link to the companies in submission [slashdot.org], only the video and merely mentioned what he used. I guess kdawson added the links. While certainly promoting their own software, the bitching about it has been taken to quite irrelevant levels in this story. Instead of bitching about that, we could had have much more interesting discussion about what it actually is or did anyone else see such spikes on the same days. Personally I think it might be some botnet scanning either for exploits or to find each other (this might be extremely relevant if some botnet was taken down on the same day and P2P scanning to find other nodes kicked in). Port numbers and a little more info would had been helpful, though.

Re:Skylab Shreds

[HybridJeff]

The graph is kind of misleading, its not actually to scale and its not showing the 5 days he claims in the youtube description. Go to around the 3:05 mark and watch the time stamp when he mouses over Romania. On the far right you can see an early date of 2009-09-15, as he scrolls to the right we can see a date of 2009-09-28 at the second stripe which is roughly in the middle of the graph, continuing on the far right hand side portion of the graph is dated 2009-09-30. The left hand side of the graph shows results over the span of 13 days and the right hand side taking up the same visual space only shows 2-3 days. Basically I just wasted 15 minutes looking over worthless data on a random youtube video that doesn't actually say anything.