We take a look at 11 key criteria when choosing a static analysis tool for modern code
Photo by Caleb Jones on Unsplash
For companies writing and maintaining software at scale today, SAST (Static Application Security Testing) has become an essential tool for increasing code security and reducing cyber risk. First-party code remains a frequent target for attacks on all types of organizations, with attackers probing applications constantly to look for weaknesses and vulnerabilities, known and unknown. A wide variety of analyses have found that both the rate and severity of cyberattacks have been on the rise in recent years. For example, ransomware attacks grew by 150% over the course of 2020, according to one research study while another put the attack increase at 485%year-over-year. Organizations are now regularly paying seven and eight-figure ransoms to free their systems and data from ransomware gangs. Attackers are also exploiting the window between when a vulnerability is announced and when a patch is released. This is why the CISA announced that all critical vulnerabilities must be patched within two weeks on government systems, forcing software companies to supply patches within this timeframe. In this hostile environment, finding a SAST solution that can minimize attack surface and simplify the writing and maintenance of secure code is essential.
Definition: What is SAST
Static Application Security Testing (SAST) is a tool or service that scans the source or binary code of applications. Classified as a “white-box” testing solution, SAST searches for known vulnerabilities and security flaws in code structure. Most SAST solutions also prioritizes vulnerabilities based on perceived severity and recommends remediation steps. SAST tools do not analyze applications in runtime; They perform their analysis on static code. SAST scans are usually run by application security (AppSec) teams but are often consumed by developers.
SAST can provide line-of-coding fault and vulnerability analysis, guiding developers to discover vulnerabilities and quickly fix code. In theory, this leads to more secure applications with fewer security flaws. In practice, some SAST tools highlight so many false positives that developers may ignore the output. In addition, some SAST tools require hours to complete scans. This limits the ability of the tool to keep pace with rapid versioning of applications and releases of new code multiple times per day. Most SAST systems to date operate in a siloed fashion alongside other application security tools like software composition analysis (SCA) for identifying security flaws in open source software, libraries, frameworks, and SDKs used in applications alongside proprietary first-party code. This isolation means SAST cannot analyze to see whether vulnerabilities or code flaws it finds actually present real risks to the application.
Key Criteria to Consider When Choosing a SAST
When searching for a SAST solution, there are multiple overlapping characteristics to weigh and probe.
Accuracy
The accuracy of a SAST tool is perhaps the most important consideration. SAST tools that generate false positives at rates in excess of 50% are creating too much noise. This can prove disruptive to development and AppSec teams by forcing them to validate each SAST finding as a real risk. A high rate of false positives can also make severity analysis far less relevant because of the possibility that the most severe bugs may also have the highest false positive rates. On the other hand, SAST tools that fail to detect real vulnerabilities and code flaws create a false sense of security that is perhaps even more troubling than too many false positives. To minimize false negatives and ensure that the latest known exploits and vulnerabilities are incorporated into scans, a good SAST solution must integrate with a comprehensive selection of vulnerability databases and threat intelligence feeds. This integration is becoming more important due to the speed with which malware and ransomware attacks propagate today across tightly networked enterprise environments.
Holistic Application Awareness
In modern applications, first-party code that SAST would scan is only a small piece of the technology pie. As much as 70% or 80% of the code running in apps today is open source from third parties. Most applications also bring in numerous APIs for various services. Open source libraries are bundled into packs to deliver key functionality with a single line of code, saving developers time they might have spent building and integrating a payment module or a GIS-handling system.
To be effective, SAST must cooperate closely with the security tools used to validate and police these other parts of the application. Those other tools include Software Composition Analysis (SCA), linters that look for common formatting mistakes, secrets detection which look for inadvertently shared secrets, and container and cloud-native security posture measurement platforms that test whether these types components are properly secured. The best SAST tools can work in conjunction with these other tools to compose a single view of custom and OSS code components as a single application. The most sophisticated secure coding platforms actually take inputs from multiple tools with multiple viewpoints and create an intermediate abstraction layer. This layer effectively enables holistic analysis and maps all the relevant data into a graph that accommodates dozens of facets and enables more accurate assessments of which alerts are false positives or real problems.
Ease of Use
How easy it is for AppSec teams and developers to use a SAST solution is critical to adoption and realizing the security value of the technology. When examining a SAST, you should run user tests with any potential users to see how difficult it is for them to learn how to navigate the interface. The team’s feedback is essential here. Another good question to ask is how long it takes to configure the SAST tool and how difficult it is to make configuration changes. In addition, a SAST tool should minimize repetitive steps and offer easy-to-understand workflows. Modern SAST solutions offer useful remediation steps to developers in their development workflows rather than forcing them to go to a second interface or to pull insights out of emails, Slack or other mediums.
If a SAST generates false positives, does it initially present them to developers and AppSec teams or is there a way to curate and filter alerts so as not to overwhelm users with noise? Above all, a key question is will the SAST tool effectively shift easy and obvious fixes left or is it too complicated and only useful for the AppSec team? If shifting these early security fixes left in your organization is important, this is a critical question. You can choose to make the entire process reactive, with the AppSec team doing all detection work and shipping required fixes back to the development teams. This tends to slow down code velocity and increase unnecessary technical security debt. Fixing code as early as possible tends to have positive multiplier effects for the entire code production and product development process, reducing infrastructure costs for CI/CD jobs and while increasing overall code quality.
Language Coverage and Versatility
Another critical consideration is whether the SAST tool covers all the languages that your development teams use. Obviously, it is better to have a single SAST tool that can cover all languages. This becomes more of a consideration as the number of languages in use in enterprises with multiple applications continues to increase. Just as important as which languages are covered is the quality of language coverage. Many SAST tools started life focused on older languages and only recently added coverage of newer languages like Go and Python. These older tools may handle the older languages, like Java and .Net, better and generate fewer false positives against those technologies while generating a far higher percentage of both false positives and false negatives against newer languages.
Understanding the language coverage roadmap of a SAST provider is also important. If the company plans to add more languages, how quickly will this happen? And how long might it take the organization to add new languages upon customer request? Language roadmap and agility tell you a lot about the organization, its vision, and the underlying capability and agility of the tool’s underlying technology.
Speed of Scan
How long it takes to run a scan can be extremely important. For example, if a company has a fast-changing code base and is shipping new versions multiple times per day, then a SAST tool that requires 3 to 4 hours to run a scan will not be able to keep up. By the time the alerts come back, then the developers will need to push the code into production; they won’t have time to pursue changes. This problem can be exacerbated if a SAST can only run on one or a handful of code repositories at any given time. Many modern companies have dozens or even thousands of applications, each living in their own repositories. So the ability of a SAST to run parallel scans at scale, and do so quickly, can become critical to getting the most benefit from the tool. In addition, if you want to truly shift code security fixes left and encourage first-class secure development practices, then individual developers and small teams must be able to invoke a SAST scan whenever they need or want. Anything less would discourage those teams from relying on SAST as part of their development process.
Tuning, Set Up and Maintenance
Closely tied to ease of use are the considerations of tuning, set up and maintenance. Longer tuning and configuration periods, sometimes lasting six months or more, can mean a longer ramp to production. Long tuning and config times also tend to indicate greater complexity and a more brittle application that does not handle more dynamic development patterns and high-velocity code landings.
When considering a SAST, maintenance requirements should factor into consideration of total-cost-of-ownership. Some SAST systems require a near full-time employee to maintain and operate. Others are simpler and can be easily handled by existing resources on an AppSec team. There also may be a range of delivery types. A true SaaS has zero infrastructure or data transport and storage costs outside of the core product. Hosted SaaS, which is growing in popularity, may require the enterprise to pay for on-prem or cloud capacity in a dedicated server. Some are offering a hybrid version of the two. The upshot? Make sure you understand exactly what delivery model your SAST implementation will have and all the associated costs and maintenance requirements.
Alongside maintenance requirements, it is important to ask what types of supporting technologies are required. For example, a SAST that covers .NET code and focuses primarily on Windows applications may require SQLServer to store results. This can add both license and storage hardware or software costs to the equation.
Product Evolution and Velocity
Seeing how quickly a SAST solution is adding new features and functionality tells you a lot about the metabolism and responsiveness of the SAST provider and its team. Seeing rapid additions of new features and capabilities is generally a positive sign. Similarly, if a SAST tool has rapidly expanded language coverage, then this indicates that the pace of new language coverage is likely to continue, which provides you more future optionality in language choice. A slower pace of change might not only reflect organizational metabolism but also the age and nature of the underlying SAST technology. Legacy SAST products written primarily as monoliths in older languages may require considerably more time to test changes due to the tight couplings of internal systems and lower tolerance for failure of any single component. While slow feature and language evolution may indicate caution and care in crafting robust code or improving the user experience, more often than not faster is better and more appropriate to modern software development where teams can pick their own languages, tools and development environments.
Required Infrastructure Footprint
The infrastructure requirements of a SAST tool are important both for cost and operational considerations. If a SAST can only run on-premise or on physical hardware, then the additional cost of this hardware — and scaling to N+1 or N+2 coverage — must be taken into account. In addition, if SAST is tied to physical hardware. You should also consider both the required compute capacity of the servers and storage requirements. If the SAST tool can run in a cloud or hybrid environment, then hardware costs may be less of an issue. However, if the SAST requires a very large cloud instance, this can quickly cost more than physical hardware. Another key consideration is whether the SAST can operate in a distributed configuration and environment. This may be useful if you want to provide higher availability and resilience, and support multiple development teams in different geographic locations. Some of the newer SAST options are actually SaaS products, requiring no maintenance by the customers. The extreme simplicity of SaaS can come with risk to security if the SAST vendor must take your source code off of your servers in order to perform a scan.
API Structure and Ease of Integration
To be effective, a SAST solution should make its data and findings broadly accessible to other systems. Ideally, a SAST solution should have a broad set of pre-baked integrations with CI/CD tools, version control and code repositories, and other AppSec, DevOps, or DevEx tools. For auditing and ease of ingestion into other systems of record such as ERP, SIEM, or SOAR, output formats are important to consider. JSON is a common and useful output that many databases and technology systems work with. Legacy SAST tools may still output XML or other dated styles of feeds. This can mean more integration and customization work is required, potential even an intermediate ETL layer to move SAST data into more modern systems of record such as Key Value Stores or indexing systems like Elastic.
Pricing Model
Naturally, you need to understand how much SAST will cost. This can easily range from nothing for open source SAST tools to seven-figure annual contracts for larger proprietary SAST systems. Different models of pricing that a provider might invoke include: cost per scan, cost per seat or user, cost per server or core, or cost per line of code scanned. To ensure that you budget properly and understand the cost/benefit of a SAST tool, create a simple model based on the pricing system and conservatively estimate usage and throughput, including concurrency. Equally important, understand how costs might change as usage or the number of users grow. There might be volume discounts or, conversely, there might be cost triggers or per-feature costs that can add up.
Conclusion: Choose Wisely Because A SAST Is A Commitment
By now you probably realize that selecting a SAST tool is a project that requires considerable research and modeling. It might be hard to recognize which SAST solution is right for your organization but you should be able to use this guide to build a checklist to make your decision process more objective and systematic. SAST regret is a sad thing. Ripping a SAST out of your DevOps stack because it does not work as advertised is disruptive and time-consuming. The good news? Modern SAST tools are rapidly improving and reaching feature parity with legacy systems. Because they are built as cloud-native or API-driven systems, the newer SAST systems are also easier to deploy and integrate, with less risk of failure and less downside should you decide to switch to another tool. SAST is shifting left with the times, and if properly deployed and integrated, a modern SAST tool can elevate your shift left security efforts while enabling you to ship more secure code, faster.
To take a risk-free look at a modern SAST tool, create a free account for ShiftLeft CORE at https://www.shiftleft.io/register with demo apps available for eight languages.
Top comments (0)