For the majority of software products, security is primarily a cost factor. Regular readers of my blog keep asking me how they should afford all these expensive and time-intensive security counter measures like code reviews, code analysis, and pen testing.
It is interesting to see that the majority of security related costs in many products still accumulate at the end of the development lifecycle. The closer a product gets to its scheduled release date, the more investments in security are being made. I frequently still see a “test security in” approach in products that quickly evolve in the beginning without considering security, while there are few product teams that try to get the security related parts of the product right in the first place to avoid the high costs of extensive late-stage security counter measures and, in the worst case, major design changes in a late development stage.
Security vulnerability aggregators like the NVD and the underlying security vulnerability databases list around 60 thousand different publicly known vulnerabilities. There are indications from research performed by IBM that the number of actual security vulnerabilities that are not publicly known could higher by a factor of 20, which indicates well over a million open or latent security vulnerabilities in major products that are tracked in the NVD.
The cost of fixing a defect at various stages of the development lifecycle grows exponentially from the requirements stage over design, coding, testing, and maintenance. This observation lead to the undisputed statement that attempts to test quality into a product are generally futile, at least when cost is a factor to be considered. As security is one aspect of software quality, the statement also holds for the unfeasibility to test security into a product. Yet, the pen testing tool and consulting industry is booming, and while many software vendors use security testing tools and pen testing, few are using security requirements and design analysis early enough in the product lifecycle to prevent security issues in the first place and avoid the horrid costs of fixing security defects after the product has been released.
To change a product team’s Secure Software Development Lifecycle (SSDL) approach and ensure acceptance, it is important to meet a product team where they are, and not try to force new development models on them when they are not yet ready for a major change. When working with product teams, I generally try to establish a more proactive model than what the team is using, presenting various options and recommendations, and then work with the team leads to find out what works best for them.
| Approach | Advantage | Disadvantage | Rating |
| Fix security defects as they are discovered (the “backwards facing” approach – post-release). | The vulnerabilities get fixed eventually… At least some of them. | It is crucial to fix security defects, but waiting until the customer finds them is really a bad idea. Relying on security fixes as the primary means of dealing with security problems is expensive and only covers a fraction of the potential security problems. | Extremely reactive (this is bad!), very costly, super inefficient, and generally nothing anyone should do. |
| Security penetration testing of running code using tools and / or pen testers (“dynamic application security testing” approach, pre-release, end of development). | Some of the security issues that the customer would find are now found before the product is released. Helps reduce public shaming. | Often too close to the planned release date to fix all issues that are found, the product ships with open security defects. Too late in the development lifecycle, this means that fixes are usually very expensive and might even require design changes (i.e. might never get fixed). | Very reactive (still pretty bad!), costly, and generally leads to a “best effort” approach on security. |
| Analyzing source code for potential security issues (“static application security testing” approach, pre-release, during development) | Somewhat improved cost/benefit ratio due to the comparatively early stage of execution. Can be very effective if automated tools are used that force individual developers to analyze their source code before committing to source control. | Expensive and somewhat inefficient if a “manual code review by security experts” approach is used. Does not discover significant design problems, and is isolated to source code view (e.g. cannot consider deployment specific issues / implications) | Somewhat reactive, because it does not prevent vulnerabilities, but merely removes them. |
| Using checklists before and during coding (“how to write better code cookbooks” approach, pre-release, during design and development) | Good approach to improve security in specific / targeted problem areas. Can in particular help to avoid common pitfalls. | Checklists are generic and can never address all of a product’s specific needs. Can lead to a false sense of security, as a fully covered checklist does not allow general conclusions on the security quality of a product, that is, determine the set of still-open security gaps. They are often too generic to draw conclusions on the security status of a specific product. | Somewhat proactive, helps with both educating developers and avoiding common security problems. Still somewhat reactive, because checklists are created to address problems that have been discovered in the past. |
| Systematic analysis of product security requirements and design specific threats | Helps to design security into a product early in the lifecycle, and reduces the need for rework, as small architectural changes can greatly reduce the risk of vulnerabilities. Avoids unpleasant surprises at product release by ensuring that all relevant security standards are covered. | Although modeling tools are available, in-depth analysis still requires a certain level of security expertise to create a correct model and interpret the results correctly. Such security expertise is not cheap, and not always readily available. | Very proactive (which is good), very effective, and probably the most cost efficient way to spend your security dollars. |
Typically the best solution is to use all of these approaches as needed in a closed-loop SSDLC program, investing most of the available resources in the proactive counter measures, then in the automatable counter measures, and least in the manual and most reactive counter measures.