Automated Secure Code Review Anyone?

Summary

Imagine you are given the source code for a JAVA web application with about a million lines and were asked to review the code for any vulnerabilities and report back in a week. This can turn into an interesting exercise if you are a team of one. What are your options: crowdsource, ask for mercy, do your best? Thankfully there are other options, such as automated code review software. In this post I'll talk about Checkmarx's cloud based solution that does security code analysis and use OWASP's WebGoat (v5.4) project to see how it measures up.

The Role of Automated Code Review Software

Generally speaking automated source code analysis is used to assess compliance based on a predefined set of rules or best practices. The analysis tool may provide the means for team collaboration and suggestions to fix the issues detected. The detected flaws may be displayed in a developer friendly interface quite similar to popular IDEs. Most tools use OWASP 10 and SANS Top 25 as benchmarks for application security flaws when performing their audits. Using a documentation tool, such as Doxygen to view the general structure and call graphs will provide and better understanding when tracking the issues detected by the analysis tool. As with any automated security solutions, an analyst should expect to have false positives in the results and be ready to eliminate them. Also the analyst should keep in mind that business logic flaws or insecure use of software libraries will not be detected by these automated tools.

Looking at Checkmarx CxCloud On Demand

Checkmarx's web/cloud based secure code analysis solution provides analysis for JAVA, C#, PHP, C, C++, Visual Basic 6.0, VB.NET, Flash, APEX, Ruby, JavaScript, Perl, Objective C, PL/SQL, HTML5 source code so it should be able to handle WebGoat without any issues. CheckMarx also is able to detect OWASP Top 10 vulnerabilities and other high risk flaws, which is great for this test run.

OWASP WebGoat UI

After signing up for a free trial account, which supports the audit of applications up to 50K lines of code, we can easily create a project and upload the zipped war file.

Define Parameters and Upload Code

The upload and analysis of WebGoat took about 15 minutes and can vary depending connectivity and the load on the analysis system though theoretically load shouldn't be an issue since this is a scalable cloud based solution. At first glance it appears that the application is at 100% risk, with 514 high risk flaws, surprise!

Dashboard

The breakdown of these flaws suggests that the application has a problem with input validation since the high risk vulnerabilities are mostly composed of XSS, SQL injection, and other injection type vulnerabilities. Not surprisingly medium risk vulnerabilities include XSRF, parameter tampering, and more XSS.

Scan Summary

Drilling down on these reported issues can be accomplished by clicking on the "Open Code Viewer" button and this takes the analyst to the IDE like drill down view. The UI shows the code with the vulnerability at the bottom pane and the associated chain of code under the "Attack Vector" section, which saves time while eliminating false positives.  To mark a finding as false positive, the analyst only has to check the box of the item in the "Results Table" and change its result state to "not exploitable." One shortcoming of this view is that it can't show the use of a vulnerable class in other classes of the application so an analyst needs to manually scan for the reuse of this code.

Code Viewer

One great feature of this view is that the ‘Flow Chart’ tab will track the use of the vulnerable class and function across all other classes and let see the actual extent of the problem. 

Flow Chart View

So how does it measure up on an easy target as WebGoat? It depends... WebGoat has both real and simulated vulnerabilities and this makes it difficult to evaluate the findings. Also some vulnerabilities, such as 'Forgot Password' are business logic flaws that can't be detected by the analysis tool.

On the other hand, Checkmarx CxCloud definitely shines in detecting XSS, SQL injection, various other injections, parameter tampering, hard coded credentials, application configuration issues, bad crypto usage, and session mismanagement.

Conclusion

Checkmarx CxCloud, with its slick UI, intuitive drilldown views and inclusion of security best practices, is a great help for developers that want to integrate security into their SDLC. It will find the most of the trivial vulnerabilities and sometimes surprise you by tracking down a manually hard to trace rouge code. The free trial account is easy to setup and use. So all you have to keep in mind while using this tool is that business logic flaws could always be lurking behind the scenes and that manual inspection of the code is always necessary.