In this article, I’ll try to assess the current situation concerning static analysis of C/C++ code. This study has a slightly philosophical character and in no way claims to be absolutely complete and objective. There also won’t be any discussions of which analyzer is better. Such comparisons are usually a pointless action: there will always be people who disagree with the chosen methodology or suspect the researcher in being biased. To do the research, I’ll try to take the list of popular contemporary analyzers of C/C++ code (for example, from Wikipedia) and understand, which tools don’t already fit there and if there is a need to add something there.
The world is rapidly changing, so the products which were previously at the peak of the popularity are now abandoned or are superseded by new more effective and convenient solutions. Despite this fact, a lot of “oldschool” tools still set the tone in the industry, adding the support of new language standards, build systems and IDEs. Lately there is also a tendency towards the search for vulnerabilities in software products. It’s not enough for the users to detect errors in the code: they need to understand that these errors cause potential vulnerabilities in the programs. Fortunately, it turned out that the existing diagnostics of some analyzers already correlate with the vulnerabilities of the methodology CWE, as an example.
As you see, developers of static analyzers have a difficult task: in addition to the fact that the detection of errors in C/C++ code is already a non-trivial thing, users are becoming more and more demanding over the time. It’s no longer enough for them just to get a report about the bugs found (even with convenient functions of search and filtering). Now it’s almost a standard for an analyzer to have systems of code analysis with additional abilities to control various metrics and indicators, advanced means of the result display, including graphs and charts. Often such situations are fired up by the competition between the producers, aiming to get the new and keep the old customers.
At the same time, the analyzer developers aren’t always ready to introduce those new features to their analyzers. The obstacle for this can be a complex infrastructure or limited resources. A possible way out here is the integration with other systems.
It’s clear that not all the vendors are able to meet the competition. A lot of analyzers on the market are becoming obsolete. Others, in the pursuit of new-fangled “features” don’t pay the necessary attention to the main direction of their development – high qualitative search for bugs in the programs. The consequence of all this is the fact that the list of several dozen popular analyzers presented now on the market, can be reduced to less than ten tools, suitable for industrial use in modern realities. Let’s take for example, the list of static analyzers, given on Wikipedia and see what tools are these. The most relevant tools (in my humble opinion) will be marked with .
AdLint. A specialized analyzer, used in avionics in automotive industry. The latest release was in October, 2014, according to the information on the product page.
Bauhaus. A high-quality multidisciplinary static analyzer. Commercial component of this product (Axivion Bauhaus Suite) is presented on the vendor’s website. Up to date.
Code Dx. A commercial analyzer, aimed at comprehensive testing of application security and vulnerability detection.
Cppcheck. Quite a high-quality open source free analyzer. The authors continue working on this tool, improving support for new language standards. Although, perhaps, it’s not that great as it may seem. For example, it’s slightly odd to see in the Release History of the latest version 1.79 the improvement of the support of the auto keyword for the C++11. C++11 should have been supported already.
Cpplint. A utility, which is not a full-fledged tool to search for errors, and is intended to check the compliance to the program style, accepted in Google.
Clang. A compiler, having quite extensive, up to date and useful abilities for the analysis of the source code.
Coccinelle. A utility, that is not really designed to look for errors in the code.
Coverity. A good modern static analyzer. At the same time, quite pricey.
CppDepend. A commercial analyzer, aimed at the analysis of various code metrics and code improvement, rather than the error detection.
ECLAIR. Quite a decent commercial analyzer. However, the support up to the C++11 standard says that it’s not fully relevant for the modern situation. To purchase it, you should fill in quite a detailed questionnaire with a large number of parameters, affecting the final price of the product.
Eclipse. Popular development environment. Still, the built-in Codan analyzer has much room for improvement. At the same time it provides developed means by the creation of custom diagnostics and interaction with the environment. Eclipse supports other static analyzers as extensions.
Fluctuat. A specialized tool, focused on the analysis of floating-point operations.
Frama-C. A good up to date C++ code analyzer.
Goanna. The project was acquired by Synopsys (Coverity).
Klocwork Static Code Analysis. A good code analyzer for C/C++ languages. An up to date, actively developing product.
Lint. Currently, has more of a historical interest. It is a generic name for the tasks of code analysis for portability and syntax errors, detected by the majority of contemporary compilers.
LDRA Testbed. A specialized analysis tool with a rich history of development. In addition to the static analysis, supports dynamic analysis. It is an up-to-date and high-quality product.
Parasoft C/C++test. High-quality commercial product that includes a set of diverse tools for full code testing. Up-to-date.
PC-Lint. A great commercial analyzer. Recently has slowed down the development. There was some information that the authors decided to rewrite the analyzer kernel on Clang.
Polyspace. A commercial static analyzer from MathWorks company. A high-quality up-to-date product.
PVS-Studio. A high-quality commercial analyzer, aimed at detection of various errors in the code. Up-to-date.
QA-C. A solid commercial product, focused on the code quality and compliance with the set standards. Up-to-date.
SLAM project. A Microsoft project, aimed at the assessment of the software security (mostly – Microsoft drivers). Not a full-fledged analyzer.
SonarQube. A set of tools for the metrics analysis and detection of errors in the code. The project is mostly designed to improve the quality of the code. Has advanced tools for visualization and integration.
Sparse. A specialized utility for the detection of errors in the Linux kernel. The latest release dates back to the year 2014.
Splint. The modification of Lint to search for bugs and vulnerabilities in the code. The project is not developing since 2010.
Visual Studio. An integrated development environment by Microsoft. Has a built-in static analyzer, aimed primarily at the code improvement and is not really useful in detection of errors.
As you see, the Wikipedia list gives such tools that cannot be really referred to the tools of full-fledged analysis of C/C++ code. A large number of analyzers are out of date and don’t meet the present-day requirements. Some have a very narrow specialization, aimed at checking only C programs or rather costly. At the same time there is no guarantee that an expensive commercial product will meet your demands and expectations.
Thus, the given list can be shortened to 10 points, acceptable in terms of the price/quality ratio of the product. I would name Coverity, Klocwork, Parasoft C/C++test, PVS-Studio and PC-Lint among leaders. We also shouldn’t underestimate the possibilities of analysis, implemented in modern GCC and Clang. We can also add ReSharper C++, which is not actually a specialized analyzer, but contains some built-in C++ diagnostics with the ability to display the results right in the IDE editor.
You can find other lists of popular static analyzers, for example, this one. But is needs reviewing. In any case, the selection of static analysis tool must be approached very carefully, based on reviews, feedback of other users and personal experience. Still, most likely, you won’t be able to find a 100% suitable tool immediately. Perhaps, you will have to try several analyzers and choose 1-2 for using on a regular basis.
So, as I have mentioned in the beginning of the article, the world is constantly changing. The tools also change, along with the growing demands, but the applicability of static analysis tasks, the improvement of its quality and reliability, vulnerability detection is only increasing. Programs become more and more complex, requiring the use of increasingly sophisticated analysis techniques. In such circumstances, only the strongest players can survive on the market.
Do you have any doubts that static analysis is a necessary thing? Take a look at this remarkable collection of bugs, gathered by PVS-Studio team while checking various open source projects.
Use static code analysis and find best tools for you.
A blogger Bill Diver suggested to make a reblog of his article, which we will gladly do.