Cyber-ITL Logo

Our Mission

We work for a fair, just, and safe software marketplace for all consumers, empowering consumers to protect themselves.

As one of the only nonprofit research organizations of our kind, we test software and computing products through expert scientific inquiry into safety and risk. More importantly, we advise, empower and educate consumers in their use of those products and software. With our partners and supporters, we’re making the digital age safer for everyone.

Ford Foundation logo
Consumer Reports logo
The Digital Standard logo

December, 27th

How Risky is the Software You Use?

34C3, Leipzig GER

Three Big Questions

1 What works to improve software security?
2 How do you recognize when it’s done?
3 Who's doing it?
Rating Charts for Tires. We want to do something like this but for software.
Something like this, but for software security.

Our Goals

  1. Remain independent of vendor influence
  2. Automated, comparable, quantitative analysis
  3. Act as a consumer watchdog
  4. Always bring data to the conversation

Not our goals

Our Analytic Pipeline Today

Comparing Results:
The State of IoT

When we compare use of common safety features and software hygiene practices in major brands of smart TVs (Samsung UN55KS9000 and LG 55UH8500) to a reasonably secure Linux install, we can see the sorry state that IoT is currently in.

Smart TV security scores
Alan Turing feels your pain
Alan Turing Feels Your Pain

“Security” can be hard to define.

When asked if an application is secure, a security expert might ask:

  1. Are passwords and keys correctly handled?
  2. Are there any backdoors or hidden functionality?
  3. Are there any bugs that can be exploited to allow code execution?

These sorts of questions can’t be answered in an automated fashion, due to theoretical obstructions ("undecidability") first identified by Alan Turing.

Thus, to measure security in a practical fashion, we employ heuristics.

Predicting Security
Using Heuristics

We don’t need to find any specific vulnerabilities in order to assess how secure software is. Instead, we can observe the software’s safety features, build quality, complexity, and other heuristics.

Some heuristics directly impact software security, while others might just be properties of software that are generally only found in cases where development teams know what they’re doing. As long as they correlate, it doesn’t matter.

Our scientists and engineers are building the machinery to both study the efficacy of these heuristics and also to apply them at-scale.

Claude Shannon
Claude Shannon wants more information

Join Us Next


19 Jan 2019  /  Washington D.C.


CITL Results

For more info about our work and partnerships, watch Sarah Zatko @DEF CON 25

Sign Up for Updates

You'll get our newsletter with the latest on our testing results.