Study: Bug bounties ‘often inefficient and expensive’

A study by code analysis firm Veracode has found most security researchers are driven by bug fixes and not payment, bringing into question the bug bounty model. 

While it’s true that a handful of researchers who report security issues through bounty programs like HackerOne have earned $1 million over a few years, the bug bounty model might not necessarily be the most effective or efficient way of getting software makers — which are every large firm these days — to patch security bugs swiftly.   

Some researchers have different motivations, according to the results of a survey by Veracode, a code analysis firm. Veracode was founded by Chris Wysopal, a white hat hacker who made his mark through the L0pht hacking group.   

A Veracode survey of 1,000 techies from the US and Europe involved in software development found that just 18% of respondents expected to be paid for reporting a bug, whereas a much larger 47% expected regular updates on the fix, and 37% expect enough information to validate the fix. 

“The study shows security researchers are generally reasonable and motivated by a desire to improve security for the greater good,” Veracode notes. 

While Google, Microsoft and other large organizations have turned bug bounties into successful bug fixing programs, big crowds of prize hunters don’t always work well for organizations or the bulk of researchers -- since most rewards go to a small population, according to some studies. Another option could be to hire consultants or penetration testers rather to find and fix issues through a competition.

Backing up its argument, Veracode found that over a third of companies received reports from researchers who did it without any financial or repetitional incentive in the past 12 months.

“For those organizations that received an unsolicited vulnerability report, 90% of vulnerabilities were disclosed in a coordinated fashion between security researchers and the organization. This is evidence of a significant shift in mindset that working collaboratively is the most effective approach toward transparency and improved security,” Veracode notes. 

On the other hand, security researchers expectations around the timing of bug fixes wasn’t realistic. 

According to the study, 65% of security researchers expected a fix in less than 60 days, even though most companies don’t fix a bug until well after the 90 days stipulated by Google Project Zero, which usually targets widely-used software. 

Nonetheless, Veracode research has found that just under half of all reported bugs are still vulnerable three months after discovery, suggesting organizations do work to fix them promptly once known about.    

 “The alignment that the study reveals is very positive,” said Veracode's CTO and co-founder, Chris Wysopal. 

“The challenge, however, is that vulnerability disclosure policies are wildly inconsistent. If researchers are unsure how to proceed when they find a vulnerability it leaves organizations exposed to security threats giving criminals a chance to exploit these vulnerabilities," said Wysoplal. 

Of course, Wysopal wants customers to use Veracode tools and processes to eliminate bugs in software during the development process. 

But he also argues that vulnerability disclosure policies matter for organizations and researchers, even though financially-incentivized bug bounty programs are popular. 

"A strong disclosure policy is a necessary part of an organization’s security strategy and allows researchers to work with an organization to reduce its exposure. A good vulnerability disclosure policy will have established procedures to work with outside security researchers, set expectations on fix timelines and outcomes, and test for defects and fix software before it is shipped.”