Inside Cybersecurity

May 19, 2024

Daily News

Researcher: DARPA effort will rate cybersecurity of software for public

By Christopher J Castelli / October 5, 2015

The cybersecurity of commercial software and systems will be independently rated in a new Pentagon-funded initiative, creating an unprecedented, publicly available tool for companies and individual consumers to find the most secure products in the marketplace, prominent security researcher Peiter Zatko told Inside Cybersecurity.

Zatko, a.k.a. Mudge, came to fame -- and testified before Congress in 1998 -- as a member of the high-profile hacker group the L0pht. He later went on to spearhead cybersecurity research at the Defense Advanced Research Projects Agency before joining Google in 2013.

In June, however, he announced he would leave his position at Google to stand up a new cybersecurity entity modeled on Underwriters Laboratories, the global independent safety science company known for product safety standards development, testing and certification. He has also compared the effort to Consumer Reports, which was formed as a nonprofit in 1936 to provide unbiased product testing and ratings.

Zatko's new endeavor picked up speed in late September when his Cyber Independent Testing Laboratory LLC in Waltham MA, won a $499,935 contract for “Consumer Security Reports” from the Air Force, which made the award on behalf of DARPA.

Buyers of software and systems badly need a way to discern which products have relatively better cybersecurity, according to Zatko.

“There have been enough stories in the news that your average consumer (corporation or individual) knows that they need better security,” he told Inside Cybersecurity last week in an interview via email. “They know security is important, and that they don't have it, but they don't have any idea on how to get it, which is frustrating and upsetting. They want to make better, more informed decisions, but don't have the tools to do so."

The new initiative will provide unbiased product ratings for the public good, he said.

“Our intention is to provide them with the information and tools they need, in a non-partisan fashion, and without profit incentives getting in the way of providing unbiased and quantified ratings of the software and systems they are purchasing,” Zatko said. “Think of this as a cybersecurity parallel to nutritional facts on food, energy star ratings on appliances, or vehicle information guides in the windows of new cars.”

“Firmware is definitely included” among the products that will be tested and rated, he said. “Hardware is not the main focus, but does come into play in certain situations.”

The endeavor will also tackle cybersecurity issues related to the ever-expanding Internet of Things, or IoT for short.

“Understanding the hygiene of the software/firmware you are purchasing or deploying, and whether the development process included efforts to harden the product and ensure robustness is needed to understand the risk you currently carry in your environment,” he said.

“The IoT is a part of this environment,” he continued. “We will include analysis and comparative ratings, on the robustness and security in software, for IoT devices as well as more traditional operating systems, applications and services.”

The results of this work will help buyers and the cyber insurance business assess cyber risks, Zatko said.

“We will be making the results and methodologies publicly available,” he said. “This will provide consumers, companies, insurance and actuarial teams, with quantifiable measurements of ‘how much risk’ different products or solutions introduce to your environment.”

The scope of the effort, DARPA spokesman Jared Adams told Inside Cybersecurity via email, is to apply and extend state-of-the-art automated program analysis -- static and dynamic -- to produce “an expressive security rating, not dissimilar to how the existing determines safety ratings for safes and vaults by using attacker tactics techniques and procedures . . . to identify the areas with highest implementation risk.”

“The cyber metrics and measurements are designed to provide managers the ability to evaluate and quantify risk in their present and future information technology environments and to quantifiably compare the security and robustness of software packages and components,” Adams said.

The automated evaluations provided by Cyber Independent Testing Laboratory “will also aid in the early identification and reporting of entire classes of vulnerabilities targeted by the existing security exploit market,” he added.

The timing on deliverables under the contract is to be determined, Adams said.

The Cyber Independent Testing Laboratory “is proposed in two tracks that run in parallel,” he said. The first track defines the metrics, ratings and certifications required to provide actionable measurements of software risk. The second track describes the mechanisms and processes needed to assess existing software according to the metrics defined in the first track.

A “key feature” of the second track, Adams said, “is devaluing the current exploit market by automating static and runtime integrity analysis at scale.” -- Christopher J. Castelli (