False positives and false negatives happen all the time. What's more, a static analysis tool can misidentify false positives and false negatives. If these errors are not caught, they could have a significant and noticeable impact on the code.
For that reason, we explain what is a false positive, outline the difference between false positives and false negatives, and provide a false positive example as well as a false negative example.
Read along or jump ahead to the section that interests you the most:
[toc]
➡️ reduce false positives with static analysis
What Is a False Positive?
A false positive is an issue that doesn’t actually exist in the code. It doesn’t need to be fixed. This happens when no rule violation exists, but a diagnostic is generated.
Meanwhile, a true positive is an issue that needs to be fixed. It violates a rule and is, in fact, a real problem.
But sifting the true positives from the false ones can be tricky. And false negatives can be even trickier.
What Is a False Negative?
A false negative is an issue that goes undetected. This happens when a rule violation exists, but no diagnostic is created.
Meanwhile, a true negative means you don’t have an issue. There is no rule violation.
So, finding false negatives is really tricky. How will you know if there’s a bug you’ve missed?
What Causes False Positives and False Negatives?
There are two primary causes of false positives and false negatives.
Tools Make Mistakes
Tools aren’t perfect. They make mistakes. And false positives and negatives are inevitable.
That’s why it’s critical to have a human reviewing over your code — and any violations detected by the tool.
For instance, you may have a rule that there can be no Divide By Zero (DBZ) issues. The tool may then flag a section of code with a DBZ issue. So, you take a closer look at it and realize that there isn’t actually an issue here. You just had a false positive.
Undecidable Rules
You might have coding rules that can’t be decided — they’re undecidable. And that means it can’t be enforced with 100% accuracy.
How Does Undecidability Happen?
Undecidability can happen when you lack visibility.
If you had perfect visibility into everything in your program, you’d be able to decide whether a rule was violated or not. You could review diagnostics from a static analyzer and know “That’s a false positive!”
But, you don’t know everything that’s gone into your program. Other programmers wrote code for other parts of the program that you don’t have access to (e.g., firmware). Input came in from elsewhere. So, without clear visibility into everything, you can’t tell if there’s a real problem.
How to Diagnose False Positives and False Negatives?
There are some false positives and negatives that are no-brainers. They’re clearly black or white.
But there’s always a grey area.
Identifying False Positives and False Negatives
Deciding on diagnostics is subjective. It depends on the industry you’re working in. And it depends on the coding rules you’re working with.
False Positives Vary
False positives for one company might not be false positives for another.
Here's a false positive example. You might be developing software that will go in a car. Lives could be at risk if there are issues in the software. So, if you have a rule that there can be no DBZ issues — and you get a diagnostic that there are — you’ll need to carefully evaluate each violation.
But, you might be developing software to go in an entertainment system. So, you’d want to dismiss false positives quickly. You only want to look at true positives.
False Negatives Vary
Likewise, false negatives for one company might not be false positives for another.
Here's a false negative example. You might use CERT or MISRA coding rules if you need to be really defensive about your program. A rule would be a false negative if it didn’t catch the possibility of something happening.
But, for another company, it would only be a false negative if it didn’t catch something that will absolutely happen.
As you expand your visibility, what you would consider a false positive or false negative gets refined.
📕 Related Resource: Apply coding rules with a static code analysis tool.
How to Prove False Positives and False Negatives?
How much work you need to do to prove false positives and negatives varies. If you’re in a high-risk, safety-critical industry, you’ll need to prove it false. If you’re in a lower-risk industry, you might be able to review the diagnostic, dismiss it as false, and move on.
False Positive Example and False Negative Example
Different developers have different interpretations of diagnostics. This has to do with both the industry they are working in — and their experience.
Here's how three types of developers interpret diagnostics.
How to Reduce False Positives and False Negatives?
Unfortunately, false positives and negatives are inevitable.
False positives cost additional review time. And they may cause real issues to be hastily dismissed.
False negatives are a key concern for mission-critical software developers. For these developers, false positives are better than false negatives.
Not All Code Checkers Are the Same…
Not all code checkers — e.g., MISRA checkers — are the same. Some are more accurate than others. And some will give you more false positives and negatives in your diagnostics.
📕 Related Resource: Learn how to compare MISRA checkers.
Choose the Best Code Checker for False Positive Example and False Negative Example
Choosing the right code analyzer gives you better diagnostics and helps you to better identify false positives and negatives.
When you get the right diagnostics, you can reduce false positives and negatives. So, you’ll have safe and secure code, consistent style, and an easier-to-maintain codebase.
See how Perforce static analysis tools Helix QAC and Klocwork can help improve your code quality. Register for a free trial.