Google Project Zero defends 90 day deadline: 97.5% of bugs fixed before disclosure

Google Project Zero security analysts have had run-ins with Microsoft and Apple after releasing details about un-patched flaws in each company’s software when they failed to meet its 90-day deadline. 

But newly released figures from the Google group suggest its policy has had an overall positive impact for end-users, even if it threw a spanner in the works for people responsible for shipping security patches. 

According to Google, 95.8% of bugs it's reported and disclosed on its bug trackers were fixed by the affected vendor or open-source project within the 90 deadline since the group’s inception in 2014. 

As of this week, the group has 1,585 vulnerabilities tagged as “fixed” in its issue tracker, while just 66 bugs were disclosed without a patch being available. 

Back in January 2015, one of those 66 instances concerned a flaw in then-current Windows 8.1, which drew the ire of staff at the Microsoft Security Response Center (MSRC) after Google denied its request to hold off releasing details for just two days beyond the 90 day deadline. 

Microsoft said Google’s decision felt “less like principles and more like a ‘gotcha’” that exposed users to danger. Project Zero had only two weeks prior exposed another Microsoft bug before it could release a patch. 

Google’s hard deadline didn’t mesh with Microsoft’s long-running policy for “coordinated vulnerable disclosure” or “responsible disclosure” that requires researchers to withhold disclosure until the vendor has released a patch to the public. 

However, the next month, on February 13, 2015, Project Zero relaxed its stance, adding a new 14-day grace period. That measure has helped vendors release fixes in 174 instances before Google published details. 

Once bugs in this 14-day grace group is included, Project Zero says that 97.5% of all issue it found were fixed under deadline, with 1,224 fixed within 90 days, and just 36 bugs disclosed before a patch was available to users. 

On the other hand, it imposes a 7-day deadline if a flaw it finds is actively being exploited in the wild. 

The new stats were released in a wide-ranging FAQ published on Wednesday that details a dozen questions to do with its approach, such as whether its disclosures, the timing of them, and the level of detail in its reports endanger users and help attackers. 

Project Zero argues its deadline approach is superior to coordinated disclosure and sets up the right “balance of incentives”. Google previously used coordinated disclosure for over a decade, but the group says the results were not “compelling”. 

“Many fixes took over six months to be released, while some of our vulnerability reports went unfixed entirely! We were optimistic that vendors could do better, but we weren't seeing the improvements to internal triage, patch development, testing, and release processes that we knew would provide the most benefit to users,” the team writes. 

The crux of their argument against coordinated disclosure is that it assumes only the bug report and the vendor know of the flaw. However, sometimes others have discovered the same flaw or acquired it.

On the flip-side, it claims one software vendor had a 40 percent faster response time compared to  their seven year average for the same target, while another doubled the regularity of their security updates. 

The only two cases were Project Zero has opted for coordinated disclosure were Spectre and Meltdown CPU flaws, which to some extent affected all chipmakers, and a design issue affecting the kernel used by Apple’s iOS and macOS.

While reports from Project Zero and the threat of disclosure likely holds more weight for the Google team than, say, independent researchers, the company does encourage all researchers to adopt the deadline approach, be it 90 days or not. The idea is that it will eventually raise the bar for patching if it is widely adopted. 

Project Zero also says it doesn’t give product teams within Google access to technical vulnerability reports with the exception of a “small number” of security engineers who work inside Project Zero as a “20% project”. The group says Google wouldn’t benefit anyway since most Google product bugs aren’t discovered by it.