Bugcrowd Q&A | The CISO'S Guide to Crowdsourced Security

Crowdsourced security has its roots in the bug bounty movement, but today it brings unique value to numerous security techniques, including penetration testing and attack surface management. Crowdsourcing plays an important role in proactive risk reduction and is something every CISO should understand.

We spoke to Bugcrowd’s Chief Information and Security Officer, Nick McKenzie, to get his perspective on crowdsourced security testing and why organisations should be racing to get on board.

Let's start with the basics: what role does crowdsourced security testing have to play in modern security programmes?

The ways I see crowdsourced security working with modern security programmes are twofold. It could be used as a different approach to penetration testing, essentially doing pen-testing with a crowd, versus using an incumbent pen-testing provider off the street. As an alternative to the more traditional testing methods, crowdsourced testing comes with its own benefits. Crowdsourced obviously has the advantage in terms of mass when it comes to pen-testers. You also get the ability to select pen-testers based on their skills, abilities, success on various previous programs – you can't do that in a traditional pen-testing world.

Crowdsourced security can also be used as an augmentation to the traditional testing regimes that an organisation might have, be it scanners or other existing security processes. A lot of people view crowdsourced security as a vulnerability disclosure or bug bounty process that acts as a last line of security testing defence. Using it like this, you could have all the typical security controls in your organisation and then on top of those, a crowdsourced vulnerability disclosure to catch anything that your existing security tools might have missed.

What challenges do organisations face driving crowdsourced testing vs traditional approaches, and how can they be managed?

As far as the challenges that I’ve seen, it's really centred around the optics of what crowdsourced security testing means for an organisation and how well it is adopted by the various teams. Fortunately for us, this issue is easily managed by sitting down and talking to the organisation. Their biggest concern is usually around the trust of using a crowdsourced model and the risks that potentially come with it.

Where traditional approaches are pretty straightforward when you're reviewing them from legal and operational risk perspectives, the crowdsourced model relies on adopting the masses. It all comes down to risk appetite, but we find that when we talk it through with our customers and can reassure them that we thoroughly vet our testers, how we imbue trust with the crowd, and how we select the researchers, those challenges are usually overcome.

So what are the biggest surprises being seen by organisations adopting crowdsourced testing?

Funnily enough, before I came to Bugcrowd, I was a customer as well! So, I can give some perspective from both sides of the fence. I think the biggest surprise that you would see as a customer is the volume of findings that get unearthed that simply haven't been picked up by all the tools your organisation has invested in, be it your own pen-test teams or a third-party provider. It really digs up all the nitty gritty issues that your tools and your processes just don't find. We’ve launched programs with multiple customers that within hours have been picking up critical findings. Customers are shocked that they’ve spent so much on security and still have critical issues slipping through the cracks.

Before anyone throws their expensive tools out the window, the success of the crowdsourced model largely boils down to math. If you can throw hundreds or thousands of people at a particular problem or ask them to look for vulnerabilities across a particular system, you’re always going to find more. It also comes down to the diversity of the crowd. Each person thinks differently to the person next to them and after you multiply that again and again, you're going to get so many different perspectives on any given problem. What this allows for is really deep and detailed findings – that’s definitely what surprises people the most.

To wrap up with our final question: what are the lessons learned and key takeaways for organisations starting their crowdsourced testing journey?

Unfortunately, there’s no silver bullet answer to this question. I think it really depends on the risk appetite and maturity of the organisation. A lot of the big corporates that we work with are the more risk averse ones like banks and other financial institutions that want to start slow – which is perfectly fine! We call it a crawl, walk, run approach. They start small both on scope and on the number of researchers that we put on the program, then over time as they grow and build trust with the community, they can expand. But then there's other companies that might be more mature and better equipped to handle the volume and they just say, “stuff it, we're getting hacked left, right, and centre! There's no reason to crawl, let’s start at a run.” Business culture and risk appetite are going to play a huge role in how an organisation wants to move forward. Some want to build up slow, others want to get the report and go with as much as possible.

At the end of the day, we always try to attune ourselves to what the customer wants. If a business wants to start at a run, we encourage that. Our model is all about adopting as much as you possibly can, be that crawling, walking, or running. It's all an assessment from the customer’s side. What volumes can their security teams handle? How much trust do they have in the community? That’s what really determines the speed of adoption, but at the end of the day, the message is simple: take on as much your organisation is prepared to handle and go from there.