Creating a Security Test Environment? 167
Enderandrew writes "Our IT department has been tasked with creating a list of authorized software, and only allowing software to be added to such a list after it has been thoroughly tested. In theory that sounds like a great idea — but how should we test apps to make sure they are secure? We have tools to scan internal websites, and we use MBSA for our Windows servers. However, I'm turning to Slashdot to ask what are the best methods for creating a test environment where I can analyze apps for security vulnerabilities. We're a multi-platform shop, but my main concern is with Windows apps."
The only way to be sure... (Score:4, Insightful)
Re:The only way to be sure... (Score:5, Interesting)
You can't even be sure when you have the source code. [bell-labs.com]
Tell the folks who want this list that you must trust someone at some point and that will always leave you vulnerable.
Re: (Score:2)
Re: (Score:2)
Even then you can't be sure, because the hole could be built into the hardware. The best you can really do is get a stack of identical processors and chip sets and destructively slice all but one of them up and run them through an electron microscope to verify the circuits - but even then you'd only have a statistical guarantee and you still might miss some clever analogu
Re: (Score:3)
Even then you can't be sure, because the hole could be built into the hardware. The best you can really do is get a stack of identical processors and chip sets and destructively slice all but one of them up and run them through an electron microscope to verify the circuits -
Of course that's assuming you have the code for the microscope controller... After all it's only showing stuff on a screen... Who knows if it's real ?
The only way to be sure...is to get rid of MS (Score:4, Interesting)
You can't even be sure when you have the source code. [bell-labs.com]
The point there is that you have to have the code for the whole stack not just an isolated application. For an application to be secure, you have to be able to do a valid code audit. For the code audit to be worth anything it has to be done all the way down to the core: compiler, libraries, utilities and operating system. So you can be sure when you have the source code, but you do have to have all of the source code.
So without even touching on the quality issues with MS, lack of code access rules out all MS products from the system on up to user applications. "Shared" source might be fine for specific, limited, platform-specific development contexts, but is basically the same as "escrow" And "escrow" is just another name for closed source, namely, as Ken Thompson point out, insecure. Ditching MS products won't automagically make your site secure, but it is a necessary first step.
Now there are some short cuts one can take in Ken Thompson's example. However, they all boil down to having the code for the whole system as he points out, not just parts. Even diverse double compiling [dwheeler.com] requires, at the end of the day, a system that has been vetted top to bottom to use as the baseline.
Now the next step is to deal with smaller, more modular units of code. They're not only good design but also easier to manage. Again, that rules out a certain party ...
FWIW it's interesting that the ACM recently pulled the Thompson article [acm.org]. It had been available for over a decade. One wonders how much longer the ACM will be a useful source of technical information.
Re:The only way to be sure... (Score:5, Funny)
You must be new here.
Re: (Score:2)
He did mention "say goodbye to Word and Excel", which _is_ a tactical nuke as far as most companies are concerned. And it is probably safest to suggest it from some far away place, like a stable orbit...
Completely off-topic, I know, but at work I'm about to start a major project that will use OpenOffice for all its documentation needs. And it will not be a trial or something like that: we will be using it because it is the best tool for the job.
Re: (Score:2)
I say we take off and mod down the post from orbit, it's the only way to be sure.
Well, not really, but I always wanted to post from the ISS, so it's a win-win.
Re:The only way to be sure... (Score:4, Insightful)
And the best way to protect a computer is to remove its network and power cables. In the real world though, security isn't the exclusive goal.
I agree that open source apps give a stronger guarantee of security, but going from "we want things to be more secure" to "we want absolute security" to "closed-source apps must necessarily be removed" seems like a stretch, even if it makes for good open source advocacy.
Re: (Score:3, Insightful)
You can't be certain unless you have the source code, so tell the folks who are requiring this list to be created that they can say goodbye to Word and Excel.
The Principle of 'Never Look': When not empowered to act upon the information gathered, DO NOT GATHER THAT INFORMATION.
Suppose you have to de-certify an app due to something your research discovers, what then?
You may not be in a position to challenge this directive, but if you are, INSIST on the power/backing/support to enforce it. The political collateral involved in this will, at a minimum, delay it for a good long while.
As an app security analyst - (Score:2, Informative)
Re: (Score:2)
And also, use an app that supports paragraphs. They'll do wonders for your TPS reports (security analysts might not tell you that but it's still important).
Re:The only way to be sure... (Score:5, Insightful)
That is the problem right there: "a lot of free time" is not what most people have. Scanning the intarwebtubes for reports of vulnerabilities for various applications running on specific OS platforms may bring some results, but to my knowledge there is no unified security test suite benchmark. No matter what you choose, you *WILL* have vulnerabilities. The best you can do is limit those to vulnerabilities you can live with or do not yet know about.
Better yet is providing a SOP for implementing change/updates to all the 'verified' applications for zero-day exploit patches.
IBM does some extensive style work on vulnerability as do several others. I think I'd spend my time working on how to patch/update every 'in house certified' application rather than trying to ensure they have no vulnerabilities. The former keeps things good going forward, the latter only ensures past sins are fixed.
Spend your time where you will, but makes no sense to do extensive in-house testing unless you are writing your own in-house software. Rely on outside testing groups, work with them even. http://www.google.com/search?hl=en&output=googleabout&btnG=Search+our+site&q=software%20vulnerability%20testing%20results [google.com] results in about 242k hits and 142k for the same with +ibm added http://www.google.com/search?hl=en&q=software+vulnerability+testing+results+%2Bibm&btnG=Search [google.com]
Re: (Score:2, Interesting)
of all security products that i'm aware of, it's the only one people never seem to be let down by.
check them out, give them a call, ask them questions. if it's not for you, the
Re: (Score:2)
You can also take provisions to have the code run in a secured environment. Have a look into the Windows Steady State if you want to maintain an immutable desktop that gets reset to an initial configuration chosen by you every time you restart the PC. That will prevent enterprising users like myself from fiddling with everything from windows themes and UI (I love Royale Noir!) up to installing all manner of unsanctioned programmes like Geany, Python, whathaveyou. Or run the OS on a virtual machine.
Or if tha
Can an idiot use it? (Score:1, Insightful)
Number 1 solution (Score:4, Funny)
Re: (Score:2, Funny)
Re:Number 1 solution (Score:4, Funny)
Re: (Score:2)
Well, what if they said, "So easy, even an analyst can do it."?
Surely this should be marked 'redundant'.
Re: (Score:2)
Would you tell that to an ex-boss of mine who actually ruined a network cable and its socket in his attempt to unplug it without pressing down the little securing pin on it?
And may I be there to watch?
Government... (Score:4, Insightful)
Is your boss ex-military or something to that effect? If so, he may not understand the complexity of the task he is assigning.
Re: (Score:3, Informative)
Re:Government... (Score:5, Interesting)
It's totally arbitrary and has very little to do with security.
Re: (Score:3, Insightful)
Re: (Score:2, Funny)
Where's the +1 (Conspiracy)?
Re: (Score:2)
Where's the +1 (Conspiracy)?
It's right here:
http://en.wikipedia.org/wiki/Magic_Lantern_(software) [wikipedia.org]
It uses common OS vulnerabilities, apparently they didn't get patched.
Also mentioned some of the Anti-virus vendors removed this
from the databases so it could remain in the system.
Also look up Echelon and Carnivore.
Also another tasty bit is right here:
http://en.wikipedia.org/wiki/Room_641A [wikipedia.org]
Enjoy !
Sieg Heil !
What am I bid? (Score:2, Interesting)
My brother is a high up in the military and complains of this 'seal of approval' constantly. Microsoft salespeople and other constantly will send their products to get 'evaluated' and get the seal of approval the next day as if someone can evaluate their product in 24 hours. Whereas other products that are open source or actually supply the source code can take MONTHS!
It's totally arbitrary and has very little to do with security.
I also spoke with someone who does application and system security certification for a national government. Basically companies pay, the team then plays around and tries to guess what's in the blackbox and what it's doing in there, and then after a while rubber stamps the approval. The tools audited that way are not ones selected by the government, for that matter. The vendors present something and then provide the money for the certification. No money == no review == no certification.
The dude was ra
If only... (Score:2, Funny)
If only the Internet had some kind of search engine where one could easily access the experiences of thousands of sys admins and/or developers.
Re: (Score:3, Insightful)
Re: (Score:1)
Ah, ya got me.
I forgot that everything on the Internet is anecdotal, including this post.
Re: (Score:2)
Re: (Score:2)
"I haven't had any problems" proves nothing about the security of the product.
"I have discovered this security flaw
Re: (Score:2)
No, it doesn't make any sense. Reading that someone working in a similar environment has identified a security issue doesn't prove anything. For example, someone may have misidentified the problem and only found a partial solution or a solution that only masks the symptoms. There are a multitude of online forums that attest to this happening.
Perhaps I didn't make myself clear. Solutions are a completely different matter. All I was addressing is the boolean answer to "Is this product secure?".
If it has a security hole, it is not secure. (Disproved).
If nobody has found a security hole, it may just not be an obvious enough hole. (Insufficient evidence to prove)
Now, moving into the realm of workarounds, feasibility, acceptable vulnerability, etc. gets sticky beyond imagination. That's a judgment call, more than anything.
Re: (Score:2)
I love this one... it's like Marco Polo for nerds.
Re: (Score:2)
Yeah, that will happen about the same time that someone takes the idea that Amazon has of selling books, and changes it to "renting" or "borrowing" books. Next thing you know, the government will get in on the game, and subsidize these ideas, and us taxpayers will all have places were we can go and read books for free! Think of the chaos, think of the publishers right to profit!
Didnt we just see another story... (Score:1)
I swear I saw another story like this one, about VMWare being used to test botnets, and being the perfect test environment for debugging etc., etc.
Sometimes, do we really need duplicates of stories only if we change the headlines.
Re: (Score:2)
I swear I saw another story like this one, about VMWare being used to test botnets, and being the perfect test environment for debugging etc., etc.
http://xkcd.com/350/ [xkcd.com]
Thoroughly tested (Score:5, Insightful)
Just how thorough of testing is being asked? Some security flaws escape eyes for years (see DNS flaw). Some flaws are obvious. But, in general, you can never be 100% certain of a program be 'secure' unless you know its has passed some milspec cert.
While optimistic, I think the new policy is a bit misguided in its wording.
Re: (Score:3, Insightful)
Well, I'm asking what is the best way to set up a test environment for testing. I make my best effort while explaining to my bosses that it is near impossible to declare anything void of vulnerabilities.
All I can do is make my best effort.
Re: (Score:2)
All I can do is make my best effort.
Godspeed, brave soldier.
Security at what level? (Score:5, Informative)
VMware is your best friend in this case. When dealing with client/server software, I'd install it in a VM, and then nmap it to see what affect it had against the machine with or without a firewall. Just to see what sort of ports were open, to characterize the software.
You can also use a lot of the great tools from SysInternals to poke around a bit more in the softwares workings, but using only software that is 100% security certified means you're going to have a bunch of people with blank hard disks. If you're using Windows and are paranoid to that point about security, I wouldn't look too far under the hood of that operating system if I were you.
There is the 'Good Enough' line. The point of systems security is not necessarily to maintain a paranoid, logged level of dilligence against every packet (though DPI isn't a -horrible- idea - it's ALL situational, tho
Re:Security at what level? (Score:4, Insightful)
This was to have been implemented today in my organization but in three stages to minimize disruption. We must conform to FDCC dictated by DHS. I received a USB drive with some instructions and files that I can use to download and install VMWare to create a sandbox for testing. The instructions are lengthy so I've just skimmed them but it appears that a software package is installed that when run establishes the baseline security of the virtual machine. Then the software to be tested is run in the virtual machine and if the base changes it fails.
I think that DHS or some Federal Agencies have lists of software that is FDCC compliant and this should ease the burden of testing if the lists can be easily accessed. I'll probably test one of my applications this weekend.
At any rate search on FDCC for more informaton.
Re:Security at what level? (Score:4, Insightful)
I think most answers here (at least the reasonable ones) are going to have more questions than answers. Questions like:
Part of the problem is that there isn't any such thing as completely thorough testing that makes sure everything is completely secure from any kind of attack whatsoever. It doesn't exist. So when you're talking about testing security, you have to know what you're trying to achieve.
Are you just trying to meet some kind of regulation placed on your company from an outside body? Because that outside body should be providing you with information, then, that will tell you how to meet their standards. Are you just trying to satisfy someone's paranoia? Then you just need to come up with a bullshit standard that satisfies that person's paranoia.
For most situations, it's enough to protect against generic hacking over the network, worms, etc. In those cases, as far as the software installed, it's probably enough to buy from major vendors or use respectable FOSS distributors, and keep things patched. If you're using Debian/Redhat/SuSE, then you're probably good. Even Microsoft's stuff (despite its bad reputation on Slashdot), if kept up-to-date with patches, is fine. A decent firewall goes a long way, and you can look into IDS stuff.
Another valid concern is employees, so make sure they're locked down as much as possible, not given access to resources they don't need. Ideally you'll audit what you can, keep backups going back so if some data goes missing (or is mysteriously changed), you can recover it.
Or you might just be asking about making sure a given piece of software isn't malware? So you do some research online and find if anyone is complaining about it. Maybe you install it, scan for opened ports, look at the network traffic coming off for anything anomalous.
But if you're talking about auditing the security of Firefox or MS Office for bugs that allow for privilege escalation or something, and you're unwilling to rely on the analysis of other experts.... well, good luck with that.
Re: (Score:2)
Just because you haven't discovered a vulnerability yet doesn't mean it doesn't exist. I can't really prove that software is secure. Really, I'm testing more for software that I can discover to be a liability.
Again, nothing is perfect here. I just need to come up with the best solution within my means.
Nessus (Score:5, Informative)
Re: (Score:2)
Nessus looks great. Thanks for the link!
testing (Score:1)
I would say, have lab of machines.
* install the new apps on your standard images, and see what security security or config changes they require.
->* also, look for how the app installs and how it appears to work.
->* maybe do some network sniffs and see what the app is doing
* also, check how the server side pieces work and how they talk to other serversa and to the clients
etc.
-Nex6
Veracode (Score:4, Informative)
Veracode [veracode.com]
Based on its breakthrough binary analysis technology, Veracode offers the world's first subscription-based security testing service that provides organizations with the only automated and independent assessment of security risks in applications, whether those applications are built in house, purchased as commercial off-the-shelf software or developed offshore.
Disclaimer: I know the founders but I am not involved in the company at all.
- SR
snakeoil .. (Score:2)
Why don't they put this 'technology' in Windows ? Or how about designing a compiler that don't allow insecure code?
Re: (Score:3, Interesting)
Reinventing the wheel? (Score:2, Interesting)
security test environment .. (Score:5, Insightful)
You can't test for security vulns, especially in a Windows environment, as there is so many interlocking components that behave differently depending on the configuration. Introducing a service pack into a previously 'secure' environment and all bets are off. All you can do is patch, patch patch
You could also intalss a second intrusion detection [wikipedia.org] system that monitors the first for unauthorized access and keeps a full audit of access alterations etc. You could also compartmentalize your system so as a breech in a web service doesn't automatically lead to a total system compromise.
no rootkits (Score:5, Funny)
You should deny the installation of rootkits, they cause maintenance and security problems
Re:no rootkits (Score:5, Funny)
Ban Windows except in Virtual Machines (Score:3, Funny)
Seriously. All you need to do is install a user-friendly Linux distro in the workers' machines, and install Windows using VirtualBox.
That's the only way to be sure.
If you're talking about installing software on the Windows Servers, I can only say this: ARE YOU OUT OF YOUR MIND!?!?
Re: (Score:2)
Penetration testing, for starters (Score:4, Insightful)
Fix the typo in the story title (Score:1)
Re: (Score:2)
Put 20 hackers in a room... (Score:5, Funny)
and refuse to give them hot pockets until they crack the program.
An Accountability Sh!tstorm (Score:5, Insightful)
This is a no-win situation for the persons assigned to certifying an application. I personally have a very hard time communicating with managers who believe, with an unshakable faith, this is a reasonable solution to a perceived problem. When it blows up, MY head rolls, not theirs. The ages-old "Get IT in my office *now*" blame-shifting game.
The right way to handle this would be to push back hard and explain why this is an epic failure in the making and resources quagmire. That isn't typically a good political solution though.
I would love some advice as to how to change such foolish thinking.
Maybe you've got a good thing going there, but if there have been IT organizational changes recently, this may be a harbinger of things to come.
Give it to sales (Score:5, Funny)
If a group from sales can't break an app, it's secure.
You might also use a bunch of chimps. The only difference there is all of the poo flinging, screaming and downright annoyance factor, but it's hard to find good chimps, so it's easier to just put up with it and use folks from sales.
Not as easy as it seems (Score:2, Interesting)
One of the problems is that there are lots of specialized applications. Users are often used to downloading open-source applications (like, say, an ssh client) when they need it -- and they don't want to wait three months to a year for it to undergo testin
Fools Errand. (Score:5, Insightful)
1) There is no secure software. Just software that doesn't have any known vulnerabilities.
2) Define 'thoroughly tested'.
3) White-listing applications is going to cause your company pain as people wait for required tools to become available.
4) You can't afford to do this yourself.
Look at it this way. When PHB is shouting for his new shiny piece of software to work, you can bet that 'thoroughly tested' gets severely watered down. You'll be left with some software on the list that has vulnerabilities whilst other people can't get their jobs done because the software hasn't been listed yet. Even if you avoid those pitfalls and manage to 'thoroughly test' each application, how many applications does your company use? Say you can test an application in 2 weeks - For one fulltime employee you'll white-list 24 applications in a year. The fact that you are asking /. for help suggests that you don't have much knowledge or experience in this field, so you're going to have to pay for training or for an employee with that training.
This model is doomed to failure, and whoever suggested it should be shown the door and replaced with a CISSP qualified guy (or gal). Security is a process, and what you need are tools to help carry out that process. Someone with a CISSP qualification will know where to find those tools.
Some of those tools might include auditing (to find what applications are in use), firewalls, IDS systems (to detect suspicious traffic), patch management (to mitigate vulnerable software), an intelligence provider (to tell you about new vulnerabilities and patches in a timely manner), a Security Event Management system (to manage security data and drive processes) etc...
For the record, I am a QA lead engineer with experience in enterprise and industrial security. I do this kind of stuff for a (good) living.
More about knowledge of infrastructure... (Score:4, Insightful)
The likely goal (from a management perspective) is to establish an authoritative list to allow for efficient management of vulnerabilities. If you know what's out there and "authorized" you can respond to threats far more quickly than otherwise.
Your management would be silly to expect you (unless you are an application analysis company) to thoroughly vet every application that comes through the door. It would be a terrible waste of time.
Do some basic analysis (what ports does it open, does it connect out to other systems, that sort of thing). Beyond that, I believe the value is simply in knowing about the application and being able to track any potential issues.
Well... (Score:2, Informative)
The workplace I work for is a rather large multi-billion (possibly trillion) dollar business. When testing application compatability they use development servers which are a mirror to the production ones but completely disconnected from the external network.
1) Apply new software to test enviroment
2) Distribute access to a test group
3) Gather reports, determine impact
4) Distribute into production if deemed appropriate
It isn't the most cost effective solution but it works when you're trying to roll out an upd
Re: (Score:2, Informative)
I guess my (off-topic) point was that a trillion dollar company is unusual (if not non-existent). The largest publicly held company (by market capitalization) is ExxonMobil and it clocks in at $501.17 billion as of April this year.
Nonetheless, I suspect the OP works for a smaller company if he's the sole one tasked with this project and he's asking Slashdot for help.
Look beyond apps (Score:3, Interesting)
but my main concern is with Windows apps."
First, secure your OS somehow. A Windows install will almost certainly be less secure then a comparable OS X/Linux/BSD install, not only because of the openness of code, but security through obscurity. Your real trouble isn't with skilled hackers, they can get through almost anything if it isn't the nightly build, but rather script kiddies who use 1337hax0rtool.exe to attack.
And there is no such thing as a secure system, only a more secure system and a less secure system.
you're asking slashdot? (Score:4, Funny)
Boss: create me a secure test environment.
guy: OK, my first step is to ask the people of the internet.
types: dear slashdot, how can I create a secure test environment?
slashdot responses:
-do not use any microsoft products. they are the borg.
-the important thing is whether you will use vi or emacs.
-use a ham radio instead
-who's going to "helm" the next LOTR "vehicle"
Decide what do you mean by security? (Score:2)
Start with "Security Engineering" by Ross Anderson [amazon.com]. The first edition is online [cam.ac.uk].
Penetration Testing Tools (Score:5, Informative)
There is no such thing as an app that is "known secure", only apps that are "unknown risk" and "definite risk".
With that in mind, you can mitigate your risk by:
1 - Closing ports down that you don't absolutely need talking to the world. Nmap is your friend here.
2 - Scan for as many known attack vectors as you can. A good start? Metasploit. Get it. Use it. The bad guys are already probing you with it.
3 - Personally, I also like to run a different server OS than desktop (i.e.: you probably have Windows on the desktop, so use Linux in the server room). Exploiting shared vulnerabilities between client and server makes life so much easier for the bad guy that REALLY wants to spoil your week.
4 - Beware of trust. In this case, beware of trust relationships between machines. You don't want one compromised server leading to a bunch more.
5 - Containment. You CANNOT guarantee every system is secure, so design your network to allow for the eventuality that some portion WILL be compromised. Limit the damage before it happens.
Oh, and after you use the black hat tools to test your network, scrub those systems you used to bare metal. Don't trust that those systems are still trustworthy.
Re: (Score:2)
please use different terminology for your subject header.
I feel dirty just reading that..
first image - sex toys demanding answers to calculus questions....
Re: (Score:2)
All good tips, thanks!
Re: (Score:2, Insightful)
Certainly agreed.
You should certainly do all these things. However, some amount of focused, possibly manual, application-specific investigation can also be worthwhile. I *think* this is what the original poster was referring to.
Investigate the tool conceptually. Identify how it works, what it trusts, how it safeguards against problems. Essentially do your own black box security review from what you know about the program.
Consider asking the vendor to comment on steps they take to ensure security. You m
Good luck with that (Score:2, Interesting)
Compatibility testing between different software suites: Good. This is the second reason to have a list because this is what you're going to run into a lot more than security issues. Some apps may require java1.2.3 another java4.5.1. Or having multiple Oracle apps on the same workstation. These are the issues your application packagers should be documenti
waiting to thoroughly test app and system updates (Score:2)
waiting to thoroughly test app and system updates can be as bad for security as not testing apps. Will you wait weeks to mounts to test a windows update just to get hacked by some one uses what the updated fixed to get in.
Bad idea (Score:2)
Our IT department has been tasked with creating a list of authorized software, and only allowing software to be added to such a list after it has been thoroughly tested
That is an incredibly bad idea which will make you the target of user hatred. Your staff is in business to do the business. This is like telling a carpenter that he has to get a new brand of hammer approved by the corporate tool testers instead of just going down the street and buying whatever hammer the hardware store has.
It's also a profoun
Re: (Score:2)
Enumerating badness, I see. Well, good luck with that.
Re: (Score:2)
In this case, it could still be the right approach. You need to estimate the costs of doing nothing (software free-for-all), and doing everything (total lock-down). Those costs should include the risks of evil/buggy software and practices, and the costs to productivity and employee morale. If your business is a minimum-wage bureaucracy (e.g. a call center), it makes sense to lean to the left and stamp out standardized systems with heavily-tested software. If your business is a high-paid software enginee
Nonsense. (Score:2)
You can take measures to get some degree of confidence regarding any applications.
You put a bloody allegory that is frankly pointless, doing what you are suggesting (allow people to put whatever they think they need) is, using another better known allegory, to allow the fools run the asylum.
Useless Red Tape (Score:2, Insightful)
It certainly cannot be secure because all apps have different functions, outputs and installation requirements. You would have to come up with a "lowest common denominator" for the security requirements that apply for all. And if so, it would be of such low common denomination that the term "secure" be
No software (Score:3, Funny)
Still doesn't solve the problem! (Score:2)
I'm pretty sure if you do away with software completely you'll be pretty safe.
Didn't you hear about the 0day abacus exploit?
ISA Server (Score:2)
You should set up a small ISA Server lab, and use its monitoring tools to tell you EXACTLY what the apps you are testing are doing, when they are doing it, and from where.
Another advantage of ISA, is that you can control apps via Active Directory. Combine that with packet inspection, and you have something where you can know the exact behavior of an app on your network. One DC, one ISA box and a couple of work
Secunia (Score:2, Informative)
well if you are using all commercial applications and not homebrew stuff, I highly recommend checking these guys out.
Secunia [secunia.com][secunia.com]
The will run a scan on all software on the system. Tell you what is there, what has vulnerabilites, patches availible, secure and what has reached their end of life. For those with patches missing or vulnerabilites it will then rate them on criticality. Plus you can also scan remote machines as well.
I find this very usefull for tracking down those bugger programs that are
appsec (Score:2)
If you have a kickass appsec team you can put them on the job (few companies do these days). If not, hire out a contracting company like ioactive, leviathan, immunity, xforce, etc.
Pass them a list of software you want looked at, and they should be able to send it back to you with ratings on whatever scales interest you (long/short term threat to a corporate network, etc).
I'm not sure if one or any of the companies I mentioned offer services like that. They mostly do product security for the companies that
2 words (Score:2, Informative)
waste of time (Score:2, Insightful)
It seems a more reasonable approach would be to:
1. pick software that is in wide enough distribution that exploits are published and patched quickly.
2. Set up the production box on a private LAN. Scan the hell out of it. Lock it down. Scan it again.
If you think your team is going to find zero-day exploits... then you are either lost or secretly have the crack team (in which case you wouldn't be posing this qu
Speaking as an appsec guy (Score:2, Informative)
As so many have no doubt said above, you CANNOT determine which apps are secure, and there really is no existing 'test suite' for application security in general. It is far too broad a ground to cover.
What you need to do is identify 'known good' versions of applications. Essentially that is going to involve finding a version of each app, or maybe several versions, which have no known unmitigatable security issues. You can do that by looking for CVE's etc.
Those become your good baseline. From there you'll ju
What you really need is... (Score:2, Interesting)
You can't possibly test everything. (Score:3, Insightful)
Give your employer a cost-benefit analysis comparing the migration to either a Linux or Mac server backbone that won't let users f*ck it up so easily, to the zillions of hours of overtime they'd have to pay a whole crew of code-monkeys and network goons to moonlight scrutinize all possible apps for all possible vulnerabilities, and let them decide
Try This (Score:2)
Try this:
Set up any box, make it OpenBSD, hell, SELinux.
http://www.openbsd.org/ [openbsd.org]
http://www.nsa.gov/selinux/ [nsa.gov]
Lock that bad boy down, complete with Honeypot.
Then go here - piss these guys off (not hard - be patient):
http://www.wolfware.dk/intro/welcome.asp [wolfware.dk]
Did I remind you to watch from a disk-loaded Linux box?
A good time will be had by all.
You'll have to toss the hardware on both machines, but eh, if you grab the Honeypot traffic (if they don't catch it) you could write a book.
PROFIT !!!
Virtual machines and a sniffer / IDS (Score:3, Informative)
Well, for a security test environment, I'd find it quick and easy to set up a bunch of virtual machines on an isolated network (that is similar to your production network, with proxies and firewalls to the big bad internet where appropriate, etc.). This will make your test environment easy to clone & reset to a known configuration.
Then you want to place a sniffer (wireshark) where you can see all traffic between the virtual machines. This gives you some idea of what a piece of software is "doing"... what remote servers it's trying to connect to, whether it bothers to use any type of encryption or at least obfuscation when it sends data around in the network, etc. Might want to run portscans (nmapfe) as well to see what vulnerabilities the software opens up on your host, and whether you can exploit them using commonly available hacker tools.
Finally, it'd also be informative to have an intrusion detection system (such as snort + acidlab) on that network (as well as on you production network) to help catch and interpret suspicious network activity.
So there are some basic things you can do to easily assess what risks your applications pose from the outside. It will help you catch basic hacker attacks such as IRC / VNC backdoors and stuff. More sophisticated attacks (which might conceal traffic to the outside via sneaky TCP-over-DNS or the like, or hide backdoors using port-knocking) would be harder to detect... for those you'd just have to have an accountability trail back to your suppliers of that software, especially if you have no way to inspect the source code for that kind of malicious embedded trojan.
Re: (Score:3, Funny)
Re: (Score:3, Insightful)
Perhaps he should try getting his software from someone who creates it with their brains instead of their muscle.
Re: (Score:2)
> I thought we'd gotten away from the days of "e"-everything.
Right. Now we're into the days of "i"-everything.
Re: (Score:3, Informative)
MBSA is far from perfect, but it is what IS demands of us. See, us lowly SysAdmins know nothing about security. But IS tells us that Microsoft is secure and Linux isn't because (direct quote) "is is programmed by teenage turds in their basement that don't know anything."
I'm glad we have those specialized security experts (spoonfed by Microsoft) to keep me in line.
Re: (Score:2)