June 29, 2011

Problems with Pen-Testing

Posted in Pen-Testing at 23:44 by Alex McGeorge

Everyone loves to have a whinge about what’s wrong with pen-testing. I’m no exception and I do it in less than 500 words.

Word Count: < 500

I loosely define penetration testing as: the practice of lawfully attacking a customer’s computer based assets, at their behest, with the intention of exposing and exploiting security vulnerabilities in one or more assets.

Good penetration tests strive to be as close to real world attacks as possible. I believe that permission to use active exploitation is a key differentiator between a penetration test and a vulnerability assessment.

1) Time on target. Some attackers have the luxury of time on their side, hacking you may be their hobby and they can spend weeks or months doing it. As a consultancy we’re limited to your budget; typically this means two weeks is the most I may spend on a penetration test. Interestingly I think it’s likely that some teams working for hire on the black market are subject to time restraints too, time is money after all.

2) Scope is the enemy of real results. Attackers don’t care how you prefer they attack you. While scope can be important, don’t underestimate its power to skew your results towards the irrelevant. I can’t touch your custom built ASP application running on IIS5 because it’s critical to your accountants? Guess who doesn’t give a shit.

3) “We’re uneasy about exploitation.” Attackers don’t care about this either, if throwing a memory corruption exploit at your Domino servers means they have a higher chance of getting paid, guess what they’re going to do. If you want to make sure you’ve got admins standing over a box I’m about to pop in case it goes down, I can work with that. Don’t be sheepish about pushing admins to make it happen, the longer you wait the less valid my results will be.

4) Don’t patch me out of a vulnerability while I’m doing the test. If I find something simple but really dangerous for you, I’ll let you know I found it immediately. If you have to patch it, again I can understand that. But don’t silently patch it and tell me it didn’t exist (I have screen shots) and at least level the playing field. Did I find SQLI? Give me a DB dump if you plan to patch it immediately. RFI? Put my shell on your server.

5) Reporting. It is my obligation to inform you of my complete results including methodologies, what I tested, and how I got in. If my accounting of this data to you is limited to a collection of automatically generated reports from the commercial tools I used, you should fire me. You can run those tools yourself and get the same results.

So, is pen-testing permanently broken? No, but it is difficult to do right and it is expensive. There may be a follow up post at some point about not hiring morons.

This post brought to you by: Dead Kennedys – Too Drunk to Fuck


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: