Preparing for an Application Penetration Test
Timeline & Cost
When new customers approach us, their most common initial questions are around timeline and cost. Luckily, penetration testing is an increasingly planned for activity and less of a last-minute surprise for development teams than it used to be.
The cost of a penetration test varies greatly depending on the complexity of the work that needs to be completed, and the amount the company charges for that work. Generally, most application penetration tests average around 5 days, although they can vary from two days to six weeks for a single application. Hourly rates vary, $200 to $300 per hour is generally what most penetration testing firms charge. Some firms will also charge for project management, technical leadership, tool fees, retesting, and other items as well, so that can influence the final price. Given an average project of 40 hours that is $250 per hour, that multiplies out to roughly a $10,000 price tag. I would estimate that 90% of application assessments are performed for between $4,000 and $30,000.
Selecting a Penetration Testing Company
Referrals seem to be one of the best ways to find quality companies, especially if you know someone who is a very informed buyer (an expert in the field). I would also recommend speaking with several companies if you are looking for feedback or guidance on what type of assessment you should perform. Evaluate the company based on their blog and any public information you can find out about the company, such as the LinkedIn profiles of the people who would potentially be performing the work. Do not hesitate to ask who would potentially be performing the work and their biographies if you would like to evaluate the skills of the person(s) potentially performing the work.
Customer Mindset
First, you (the customer), need to align your attitude and perspective with the task at hand. A penetration test is a valuable tool for development teams and the company overall. This is the time to identify and fix bugs proactively, which is going to save you time, money, and eliminate headaches down the road. It will also increase your understanding and exposure to security.
Sometimes teams do not want bugs to be identified in their systems. This may be because they either do not want additional work or have a business interest in reducing the number of findings in the report. This is not the best attitude to get value from a penetration test. Performing work for a group of people that really do not want to see accurate results can be very unproductive.
A lot of pentesters spend a significant amount of time hamstrung by technology or access issues. It makes no sense to squander valuable penetration testing hours begging for credentials. Having a responsive customer who is ready to work through any issues is very helpful and rewarding for us and it ensure you get the most value out of every hour. I love being in close communication with my customers. Not surprisingly, it is almost always startups that are most responsive to issues.
I remember one time I sent over a Cross-Site Scripting finding notification to a startup on a weekend evening, and they replied back within about an hour, indicating that they believed the issue was fixed, and it was ready to be retested. It was gratifying to be working with a customer who cared enough about security concerns to take immediate action on it, especially after previously working with a large company who took a minimum of a month to perform a similar action. It is the responsive customers who partner with pentesters who get the most value out of each test.
Pentester Mindset
In addition to the customer, a good attitude is also very important for any pentester to be successful. A successful pentest is one that enables their clients to fix their most threatening bugs. Finding and exploiting the most technically complex bugs is important and exciting, but a pentester’s work does not matter a whole lot if the customer is unwilling or unable to act on the results. We have seen many pentest reports where reproduction steps or evidence is lacking the detail required for a client to reproduce the issue.
It is important that pentesters do their part to help the organization be successful, and that includes advising and guiding the customer where possible, including preparing for the pentest. A penetration tester should not be surprised if their customer has not fixed something if they did not first truly care about helping them. This can be a challenge on a short remote project, although if the pentester truly cares about helping their customer, their assistance and guidance will naturally come through.
Project Approach
Okay, now that the pentester and the customer have their perspectives aligned, what actionable steps can be taken?
Collect, document, and discuss known issues. If you know of a few security issues, or something that has been fixed in production, but is not fixed in the QA configuration, let the consultant know to allow discussion about the best approach. Examples of this would be TLS configuration, platform configuration, which includes returning debug information such as stack traces.
If the application is running on a development server that was setup just for testing and is not internet accessible, it is often the best approach to confirm any potential deployment issues in production. Conversely, if you know of a few specific security bugs, I believe it can be a good idea not to tell the penetration testers of these issues so you can gauge their level of coverage.
The trade-off is that the tester may spend excessive time validating and reporting on issues you already knew existed. It can be a wise to ask the pentester to notify immediately of any potentially significant issues, then you can guide the pentester on whether it is a known issue.
It is also important to guide the pentester on any areas of concern. This sometimes includes file upload areas or new authentication mechanisms. If you just added specific functionality to the application, or an area you are especially concerned about, be sure to identify that as an area of concern.
Whitelist Penetration Testing IPs
Often, a WAF or other protection mechanism will hinder attacker’s traffic. More often, than not, they will slow an attacker’s progress through throttling, IP blacklisting, or blocking common attack strings. This is valuable in real-world attacks since it raises cost/benefit equation. Many attackers will not spend 20 hours discovering a WAF bypass when there are other systems out there that are easier to exploit. Some WAF bypasses are built into tools and an attacker just might get lucky in trying the right technique and bypass the WAF without much effort.
There is some tradeoff between leaving these defenses in place to simulate a real-world attack versus removing protections and identifying as many vulnerabilities as possible. In almost all cases, it is better to find something an attacker may not be able to find than to miss something while trying to simulate a real-world attack. The goal of a penetration test is generally to identify as much as possible and not to simulate a single attacker. That 20-hour WAF bypass could burn valuable pentester hours that could be better spent discovering more bugs. We want to protect against all attackers, which means finding and fixing as much as possible. An attacker may attempt to exploit a certain vulnerability using an attack that includes a WAF bypass, or bypass a blacklisting mechanism by rotating through a large number of IPs.
Fix the Small Stuff First
By reviewing and fixing what you can in advance of the test, it can significantly reduce the number of findings in some cases. This allows the penetration tester to spend more time finding more critical issues and lower the total number of issues contained in the report. By consistently trying to fix issues over time, it can greatly increase the security of the application, and decrease the technical debt that is accumulated. Over several years, this can mean you will have a very manageable 5 finding report instead of an overwhelming 15 finding report.
Here are two great sites that will identify many common low-severity security issues. They can be a bit “noisy” in what they find, however they both offer gradings for the issues they find.
Keep in mind that penetration testing is just a small part of an overall security program for an organization. The recommendations made here fit well into a security program. By taking these steps you will both create a more functional and efficient pentest, as well as improve your overall approach to security by adopting a proactive mindset and actions.
