What's the point of "penetration testing"?

    Over the last 8 years in IT Security, I've had at least a professional interest in the idea of penetration testing and the opinion of this service has evolved as the IT Security market niche matures and grows.  I wanted to take a minute to discuss it with the readers out there, and maybe solicit some opinions on the topic if you're willing to offer yours.  I'll reserve my personal opinion for the end, but wanted to present some thoughts, rebuttals and commentary on these here.  I'm going to address penetration testing in the context of web applications - but this can be allied virtually to any technology out there.

Let's first look at the arguments for penetration testing:

• Penetration testing provies a hackers-eye view of your web application attack surface
• Penetration testing provides an outsider's view of your web application attack surface 
• Penetration testers will often find ways to manipulate your applications in ways your developers never thought possible
• Penetration testing offers the client an opportunity to get a snapshot picture of your security posture
• A penetration test goes more in-depth than a "security scan" by identifying and exploiting real weaknesses

    Those are some compelling points, to be sure.  Security is a very strange f1sh, it changes so drastically so often it's almost impossible to be entirely up-to-date all the time, unless that is your sole job.  This is precisely what penetration testers are great at - they focus their entire energy on researching, identifying, and exploiting security weaknesses in, in this example, web applications.  There really isn't any amount of "scanning" that an automated tool can do which will match the power and adaptive capability of the human mind - I don't think anyone will argue that - so the value of employing someone who is extremely versed in this sort of thing is akin to having your transmission looked at by a transmission-only specialist... you do it because you want to go to the expert.  There are varying degrees of expertise; of course, and let's not even try and disagree that you get what you pay for.  If you want a top-notch security expert, you're likely going to be hiring someone with a shady past, and it's going to cost a lot - but at least you know you're getting the top talent matching wits with your pro-active security measures.  But what about the other side of the coin?

Let's look at arguments against penetration testing:

• Penetration testing can be argued to be a test of the 'tester' not the target
• Penetration testing isn't an exact science, and rarely standardized
• Penetration testing results are inconsistent
• Penetration testing is too expensive
• Penetration testing is only a snapshot in time

    With those arguments against penetration testing - how can one reasonably conclude it's a good idea?  Well, the fact of the matter is that penetration testing is expensive, inconsistent and rarely an exact, standardized process (unless you pick one of the top firms which have standardized).  Yes, sometimes the results are inconsistent and a mere snapshot in time, not an accurate assessment of your stategy as a whole.  The argument has also been made that a penetration test result is often a test of the "tester's" intelligence and hacking prowess, and not necessarily of the defenses... however I would say think twice about that argument.  Isn't that the point?  You hire the best, they put their mind to the test against your defenses?  So now the pros are weighed against the cons... and the money issue is always on the forefront of the decision to go one way or the other.  I will only offer you these words...Strike a balance in your strategy - but do not fail to test yourself.

    Remember, the right balance when it comes to penetration testing is in moderation.  You can't reasonably have a penetration test done once a week, as it would destroy your budget.  You also shouldn't do it once a year - as that's probably too rare.  The right balance is a combination of automated tools which you and your security team can use to self-assess plus a seasoned expert tester to check your sanity and environment.  Heed my warning... find your vulnerabilities because if you're not testing the security of your web applications - rest-assured someone else is.