Skip to main content. - Software Security Testing in Quality Assurance and Development

Navigation: Home | Links | About |

Setting the appropriate security defect handling expectations in development and QA

By Robert Auger

If you've worked in information security you've likely had to report a security defect to development in an effort to remediate the issue. Depending on your organization and its culture this can be a rather difficult task. As an information security professional it is your job to detect, communicate, and see to the remediation of such issues in your company as these issues are discovered. Likely development is saying that they're to busy to fix the issue and that if they try fixing it they'll miss the deadline for their release, resulting in their group getting penalized (sometimes bonuses are tied to release cycles) or getting a negative comment on their performance review. In other situations development may just be stubborn requiring full proof of concept code before taking your security defect seriously. Development may even refer to infosec as a group that impedes progress by throwing bugs into the grinding gears of a given software release.

As an infosec professional you may feel at times helpless, or unable to do your job successfully due to the actions and stances of other groups. If you're currently in this situation there are a few things that you can do to get development either on the same page as you, or at least in agreement to the handling of these issues when they inevitably creep up. Before diving into the details let's review a some of the core activities in the world of software development to better understand how they operate with relation to defect management.

Development schedules

While there are various approaches to software development (Waterfall, Agile, Etc) the one thing they do have in common is an expectation for when an update is going to ship. Often the expectation of specific updates are scoped and a release schedule is formalized. In many organizations after a project scope is established the project scope is locked which makes it rather difficult to insert any other changes into a given release. Schedules are used in development for work measurements, as well as by groups such as marketing which may tie a promotion to the newly released product or service.

Defect triage

Before a given release is shipped, a list of existing defects is compiled and reviewed by the product development team (which may or may not include QA) who determine which issues are acceptable to ship and which must be fixed before the production release. Some organizations impose limits on the quantity of known high priority defects that may exist in a given release (example 0 urgent, 1 high, 2 medium, and 10 low) and have a fairly mature process involving stringent change controls. The majority of defects are identified prior to bug triage which can make the introduction of additional defects in an already scoped software release rather difficult. There are of course common exceptions to the rule, such as if core functionality fails, or if stability and performance are in doubt.

Bug severity and prioritization

Generally it is either QA or development that assign priority/severity to a given defect and utilize a well defined baseline for determining how serious an issue is. Below is a high level example of what this may look like in an organization.

Priority 1:

  • Core product functionality fails to operate as expected.
  • The product becomes unreliable or unstable in common use cases.
  • The product fails to install properly.
  • Compliance violation.

Priority 2:

  • Secondary or low use functionality fails to operate as expected.
  • Less critical performance of core functionality.
  • The product fails to install in certain less used use cases.
  • Compliance violation.

Priority 3:

  • Rarely used production functionality fails to operate as expected.
  • Rarely used functionality fails to perform as desired.
  • The product fails to install on less popular operating system variants.

This helps to clarify how a particular defect should be assigned without question by the development or testing groups.

Getting everyone on the same page

To get a given security defect the attention it deserves you must first have a grasp of the processes and expectations of the development group to understand how introduction of an issue is going to impact the involved parties. You will need to remove ambiguity and doubt for security defects by having a firm stance on their priorities, as well as how your organization will address them when they come up. An approach that has been successful for various companies is to create an expectation for a given security defect in the form of a priority matrix, similar to how development has a priority list. Identify and create a list of the attacks and weaknesses likely to affect your application, and identify priorities for how they should be remediated based on security standards, policies, and requirements.

The level of detail and specific priorities that you put into this list will likely vary. For the purpose of keeping things simple here's a short sample list.

Priority 1:

  • Denial of service, or impact to availability
  • Sensitive company or customer data disclosure
  • Unauthorized system access
  • XSS [1] vulnerability
  • An attacker is actively a exploiting a flaw that can result in the compromise of the user, the system, or the data.
  • SQL [3] Injection
  • CSRF [2] protections lacking on component requiring user state
  • Compliance Violation (PCI, HIPPA, SOX, etc..)

Priority 2:

  • Open URL Redirectors (may be higher depending on the organization)
  • Less sensitive data disclosure (build numbers, system paths, etc...)
  • Less severe compliance violation
  • CSRF [3] protections lacking on component not requiring user state
  • Non sensitive backup files in web root

Priority 3:

  • Non sensitive data disclosure
  • Weak CAPTCHA which is not actively being exploited
  • Difficult to achieve race conditions which require local access which do not result in privilege escalation or data compromise
  • Denial of service, or impact to availability for non less important systems
  • Improper file system permissions for non critical data


After clearly documenting the security department's expectations speak with management in development/QA and outline your teams concerns for particular security defects in your company and explain your decision process. Chances are after speaking with development this list may grow or be readjusted based additional information you may not have been aware of. Ensure your teams are on the same page with regards to remediation time lines to ensure that when issues creep (whether they are discovered internally, or in the wild) a clearly documented time line expectation is formulated. The time line for a security defect will vary based on the type of issue, the probability of the issue from being discovered, and the necessary factors that need to align it to be abused.

After ironing out the specifics ensure:

  • That a conclusion is reached on how to communicate this information to both development and QA groups. This may take the form of an update to a company's defect priority matrix, a new policy, as well as employee training.
  • Ensure that development/QA have an appropriate security contact if they have any additional questions or need assistance verifying if an issue is properly addressed. This will prevent issues from hanging around longer than are necessary, or an issue getting dropped.
  • Ensure that when security defects are filed, that development/QA have enough information to verify if an issue has in fact been addressed (will vary depending on the defect type). This may be in the form of reproduction instructions, or specific advice in the ticket on how to address the issue.
  • Ensure security defects have a unique identifier in your bug tracking system to track security defects vs normal defects. This may take the form of a security checkbox, or defect type option. Ensure the security team responsible for application security is notified when issues are filed, as well as periodically check on old issues to ensure things are handled in a smooth, timely manner.


It is important to understand that development's core job is to develop and ship code. Consequently it is security's job to ensure their applications meet a certain level of security based on their standards and policies. Getting information security and development on the same page is a requirement for effective vulnerability remediation.

Note: This article does not reflect the views of my employer.


The Cross-site Scripting FAQ

The Cross-site Request Forgery FAQ

WASC Threat Classification 'SQL Injection'

About the author

 Robert Auger co-founded the Web Application Security Consortium (WASC) - a group dedicated to developing and promoting "security standards of best practice" for the World Wide Web - in 2004 where he currently leads multiple projects. He also runs the oldest application security site on the web and also runs where he addresses security throughout the Software Development Lifecycle. Robert currently is an application security engineer at PayPal where and works on application security, and Secure Development Lifecycle program development.