“Optimism is an occupational hazard of programming; feedback is the treatment.” Kent Beck, an agile pioneer, wrote this to emphasise the role of meaningful feedback in building robust, reliable software. Penetration and external security testing are two ways of getting feedback on the resilience of systems to attack.
We live in a world where people with bad intentions can launch attacks on an organisation’s systems. In that context, an independent check that your systems are robust enough to counter an attack can be a good idea. Even if your organisation has the world’s best people, they will, from time to time, miss the chance for protection or make an error.
Results from penetration testing should identify areas of a system that have security issues so they can be remediated. External penetration testing should provide actionable feedback about the system’s robustness to attack.
Understand the different types of penetration test and how they relate to different types of threat. Make sure the scope or statement of work agreed focuses on the areas of the system you want feedback on.
Web application or application programming interface (API) testing will focus on the validation, authentication and interaction patterns implemented by the development team.
Infrastructure testing will focus on the configuration of servers, networks and firewalls that support the operation of the system.
Testing from the public internet will give you an indication of the types of weaknesses that any teenager hacking for the “lulz” might find.
Testing from networks in your infrastructure will show you what can be exploited by insiders or attackers who have already gained access via previous attacks.
A code review will allow a security-focused development expert to give you fine-grained feedback, based on a copy of your application code.
Understand the nature of a penetration tester’s toolkit. Testers will use automated scans as well as experience, intuition and expertise to find weaknesses in systems. Automated scanners often throw up false positive results. Do not accept copy/paste results from the output of these tools. Test results should contain validated results, or the supplier is wasting your time.
Do what you can to maximise the amount of time the tester has to dedicate to testing. The penetration tester will have limited time, so this will allow them to focus on more complex attacks.
Check the environment to be used during testing is ready, and that access is granted before testing starts.
Have someone on hand to explain or assist with any required context to complete the test. Penetration testers often need to get up to speed with systems very quickly.
Many scanners are available as open source software for download. You can use them ahead of time to find the obvious stuff and fix that ahead of the test.
Set expectations that results should be presented in terms of consequences and impact. If possible, the tester should provide suggestions for fixing the issue. The quality of the reporting is what makes the feedback actionable. It should be possible to reduce time spent fixing by getting draft feedback at the end of each day, rather than waiting for the final report.
Results should be presented in terms of consequences and impact. If possible, the tester should provide suggestions for fixing the issue
If you are working in an agile environment, consider several smaller engagements rather than a big bang test. By splitting the budget into several smaller efforts, you can get earlier feedback, leaving fewer issues to be discovered at the last moment.
The best security partners will provide insight into more subtle security techniques and the latest attack vectors. If they can provide insight beyond what is available in your organisation, that is incredibly valuable.
As with any service, some providers are better than others, and it is a good idea to review the outcomes after each round of testing to ensure you are seeing the value you expect. Spend time on finding a provider that meets your expectations.
Jim Gumbley is a security expert at global IT consultancy ThoughtWorks.
This was first published in December 2015
Enjoy the benefits of CW+ membership, learn more and join.