When testing web applications and APIs protected by a Web Application Firewall (WAF), a common question arises: should we keep the WAF on, or should we turn it off for the test? While this may seem like a straightforward question, the approach you choose can be the difference between a report that drives genuine security improvement, and one that delivers little beyond a false sense of assurance.
In this article, we explore both methods, evaluate the pros and cons, and share our perspective on which approach tends to provide more value based on our experience with clients.
Firstly, what is a WAF?
A WAF is an application layer security control that inspects requests made to an application to determine whether they are legitimate or malicious, blocking anything that looks harmful. Behaviours a WAF looks out for include, but are not limited to, SQL injection, Cross-Site Scripting (XSS), and general misuse. WAFs should be implemented on all web applications and APIs but should not be used to replace secure coding practices or other protection mechanisms such as firewalls or input validation, rather, they should assist these.
When we carry out assessments, WAFs can heavily skew results and impact the effectiveness of the test. We’ll be discussing two approaches, and how beneficial each could be.
WAF On Approach
If your end goal is to understand whether your controls are working as intended, then testing with the WAF on is a valid approach. This would be a more “true to life” assessment, as this is what’s served publicly. That said, it’s not uncommon for WAF bypasses to be found and actively leveraged by threat actors, and if bypassed, the application would rely solely on how securely the developers coded it in the first place.
Further to this, a penetration test is a time-limited activity: the consultant is given a fixed number of days to target the application. Within that time, it may not be possible to bypass the WAF, meaning the full application stays protected, and the report ends up offering a limited perspective on the true security posture of the target. A threat actor, who hasn’t agreed to any scope or time limit, has an unlimited amount of time and may ultimately be more successful in navigating around the defences. Threat actors routinely bypass these protections and often publicise how it was done, meaning any application using the same WAF could be vulnerable to the same technique. For example, if the application’s true IP address has ever been exposed, say, before it sat behind the WAF, an attacker could add it directly to their hosts file to bypass the WAF entirely. An additional example could be WAFs that only check requests up to a certain size. If a threat actor were to send an obtusely large request this could result in it not being checked, bypassing protections, and executing the desired behaviour.
Ultimately, testing with the WAF on will only tell you how effective the protection is at the present time, and whether the currently implemented rule set was functioning as intended. Additionally, the assessment would mean your budget is being spent assessing the security posture of a third-party component, when the best value could come from understanding how your developers are creating products. This understanding could then be used to implement secure coding practices.
WAF Off Approach
Assessments can also be carried out with the WAF disabled, or with its configuration altered to allow us to bypass the protection. We favour the latter, as the defences stay in place for everyone else and the application remains protected in the meantime.
Testing with the WAF disabled will inform you of the application’s true attack surface and security posture. This matters because WAFs can be bypassed if misconfigurations are present. It will also assess how securely the developers have designed the code and highlight whether secure coding practices were being followed. Where they weren’t, the findings and recommendations in the report will flag the issues and inform developers on what needs to improve.
Adopting the WAF off approach ensures your budget is spent assessing what you’re in control of: how your development team is building your assets. In our view, this is the best-value approach to testing, as it leaves you fully able to implement changes that improve the confidentiality, integrity, and availability of your data and service.
This approach paints the most honest picture and forces you to see what would be possible if the WAF were bypassed, or not functioning as intended. This information can then be used, as above, to implement fixes and improve the organisation’s security standing, rather than hiding behind a protection that could fail. The lessons learned will also help your development team build skills, ensuring the next round of assets is developed with a more security-focused mindset.
As security consultants, we want to make sure ultimate value is delivered for the budget being spent, and that you understand what issues are present and how to address them. This is why we always ask for our testing IP addresses to be allow-listed past these protections, and why we seek application source code, these small assists help the allocated consultant find issues more efficiently and identify more impactful issues.
At the end of the day, a penetration test is a time-limited service, a consultant is allocated a fixed number of days to assess the target. A threat actor isn’t working within those confines. Anything you can do to assist the testing should be done, since any time not spent testing or assessing the application directly, is money coming out of your budget.
Conclusion
Having a WAF is great security protection; however, it should be seen as a second line of defence. The first is how securely your code has been implemented, and whether your developers are placing security at the forefront. A WAF can also become a huge blind spot, as many organisations never assess their application’s security beyond it, believing that it’s all that is required for protection.
Don’t fall into the trap of relying solely on the WAF to secure your application or API. Assessments should be carried out beyond it to build a proper understanding of how well the code has been developed. If it’s important for you to understand whether your WAF would have blocked a given attack, a multi-phased approach can be used, with the main assessment carried out past the defence, and a separate check on whether the issue would have been caught with the WAF enabled.
If a multi-phased approach is carried out and the WAF would have caught the issue that was exploited once bypassed, changes should still be introduced. This is because a threat actor, unlike a consultant, has an unrestricted timeframe, and could use it to slowly bypass protections regardless.
No matter how security-mature your organisation is, or how large or restrictive your budget, we can work within your constraints to help you get the most out of your next assessment.
So, if you’re planning an assessment of a web application or API that is protected by a WAF, and want to make sure the results actually drive improvement, not just tick a compliance box, we’d be happy to talk through the right approach for your goals, whether that’s WAF off, WAF on, or a multi-phased test that gives you both. Get in touch to find out how we can help.
