As a CTO, you won’t be surprised to learn that I think technology is great. However, technology isn’t always the answer, and in many cases, it can be counterproductive to the job, or process, it’s designed to improve.
Take the example of self-service checkouts. Undoubtedly, they save time and ultimately make retailers more efficient overall, but have you ever tried to buy alcohol through self-service checkouts? It’s often a frustrating experience, seemingly taking much longer than the traditional, human-operated till system. What was meant to make things quicker has, in some circumstances, slowed things down when the process isn’t standard.
That’s just one trivial example, but there are many more. A CRM system designed to enhance customer relationships could hinder them, and a communications system designed to streamline communications could cause information overload. The list goes on.
The truth is that even the best technology can hinder progress and results. In a rush to implement it, we may end up causing more problems than we fix.
While many technological improvements have been made in how we conduct and deliver penetration testing over the years, one tech solution seems to split opinions: client portals.
When I was younger and a bit of a sci-fi nerd (I still am), portals were a cool concept – connecting distant worlds. But the portals we’re discussing here aren’t so cool. Portals can cover a range of activities but mostly they provide a central online space where test providers can gather test requirements from their clients, communicate during testing and upload test reports in ‘easily’ accessible formats for their clients to access.
On the surface, portals seem beneficial, making things slicker and presenting detailed information in an easy-to-read format. Brilliant. But there are often ‘unconsidered’ disadvantages if you scratch under the surface.
The Disadvantages of Offering a Portal to Our Clients
While there are situations where portals can be beneficial, after evaluating several third-party portal solutions over the years and considering our clients’ needs, we have identified some key issues that have led us to avoid implementing this solution so far.
Service Quality
We take pride in the quality of our testing services, and any portal solution must meet those same high standards. However, we believe that many portals fall short in several important areas:
Scoping
Some potential clients propose that a portal could streamline the scoping process by allowing them to upload scoping questionnaires. They believe this would enable the test provider to quickly generate a quote based on the provided information.
However, scoping is crucial for delivering successful tests, and this approach may only suit standardised or “tick-box” testing requirements.
Many of the projects we undertake are more complex, so a scoping questionnaire alone is insufficient for providing an accurate test proposal. To effectively scope these projects and ensure clients achieve their desired outcomes, a walkthrough of the application is necessary -something a portal cannot facilitate.
Communication
In-test communication can be vital, and while a portal may assist with general communication throughout a test, it is often no more effective than a Slack or Teams channel.
Crucially, when it comes to flagging critical and high-risk issues, the nuances and complexities involved often necessitate real-time communication. From experience, relaying high-risk or critical issues through a text-based system often falls short. It’s generally more effective and quicker to discuss these issues over a phone call, where the matter can be fully explained, demonstrated, and any questions can be addressed.
Reporting
Our reports are key deliverables for our clients, and as such, they must maintain the highest standards. Reports should not only help clients understand the issues uncovered but also facilitate effective remediation and mitigation actions.
However, many portals do not meet these goals, and their reporting capabilities often fall short.
Poorly designed portals can lead to missed vulnerabilities, ineffective prioritisation, or incomplete remediation plans. Additionally, some portals rely heavily on automated systems for data aggregation and reporting, which may lack the contextual understanding provided in a carefully prepared manual report.
Another significant issue is reporting customisation and compatibility. Clients frequently request tailored reports or integration with existing tools like risk management platforms, Security Information and Event Management (SIEM) systems, or ticketing software. Due to the standardised nature of portal reports, customisation and integration may not be possible, potentially resulting in additional efforts to extract and reformat data to meet reporting requirements.
Security/Risk
As a security test provider, our goal is to highlight and mitigate security risks for our clients, not to introduce them.
We can conduct a thorough security audit of any potential third-party portal solution; however, storing our clients’ sensitive information within an external infrastructure poses risks that we are not willing to accept, and any breach of a third-party supplier’s portal could expose critical vulnerabilities to attackers.
Even if we were to host the portal on our own infrastructure, centralising sensitive test data within a single application introduces risks from external threats that we would not be comfortable with.
Additionally, there are compliance and internal policy concerns to consider.
Many clients must comply with regulations such as PCI DSS, ISO 27001, and GDPR. Depending on the hosting location of the portal, it may not meet data residency laws or industry-specific regulations. Clients may also have limited visibility into how their data is stored, managed, and accessed by the supplier, which could potentially violate internal security policies.
Another significant risk is data continuity. If access to the portal were revoked, such as at the end of a contract or if the portal supplier went out of business, clients could lose valuable records of vulnerabilities, findings, and remediation efforts.
Never say never but….
The reasons outlined above are the main reasons we’ve (so far) decided against implementing portals for our clients. But that’s not to say we never will.
If the right solution comes along and solves the issues outlined then yes, we probably will. (If you are a portal provider and think your solution solves the issues mentioned feel free to reach out and I’ll give it a whirl).
But until then we’ll be staying away from client portals, we’ll just stick with the old trusted manual methods.
Something to consider – Portal DIY
Client portals (where clients access a test provider’s portal) as outlined above, are only one side of the portal solution and many of our clients operate their own test portals which we report into.
This route is mainly used by those who conduct regular testing and have the resources to do so. However, they can provide a strategic way to centralise penetration testing reports without sacrificing security or long-term access.
So, if you have the resources and testing frequency warrants it, an internally managed portal may give you the best of both worlds.
