The Elements of Application Security Testing (With Apologies to Strunk and White)

In dynamic analysis, users often have questions around tested elements.  Although questions vary, users typically want to know:

• What are the tested elements?
• Do they really represent progress?
• When are they generated?
• Why do the numbers sometimes suddenly jump when I start the test phase?

The last one seems to be the strangest thing to see, like it might be indicative of an error or something.  But, if we answer the first three questions it becomes clear. Logical, even. We’ll explain that as much as possible in this blog.

Playing by the Rules

HCL AppScan’s dynamic testing is defined by rules. Strictly speaking, AppScan’s rules are made up of variants that define the work. However, for this article, we’ll use the term more generally:

• Rules are the specific test instructions to detect an issue.
• Each rule is used to identify a specific Issue type. Multiple rules can identify the same issue type.
• Rules test specific HTTP components. For example, an active test can modify HTTP query parameters, or a passive test may examine a response header.
• The scanner maintains a list of rules for each HTTP component type. The term we use is entity type.
• Each list of rules is made up of one or two sublists called rule ranges. The first range is the list of rules that can run in parallel to each other. The second are rules that must run in a serial fashion, one after the other. Some lists are made up entirely of parallelable rules, so they contain a single range. Other lists have both parallel and serial ranges.

​The scanner performs two passes during a single test phase. It first runs all the rules that can run in parallel in multiple threads (defined in the configuration), and then it switches to a single-threaded mode to run all the rules that must run serialized.

The HTTP Component Entity Crisis

You may guess – and correctly — that if there are entity types there must be entities to type. So, what are entities? An HTTP component – or entity type – can be any of the individual parts of an HTTP session (a pair of a request and a response) that we can test. The host, path, directories, parameters, cookies, request, response, and so on are all entity types.

​When the scanner collects a request-response session it extracts the relevant entities. Therefore, an entity is a specific instance of an entity type in a specific session. We also call them security entities because they are the components that are tested for vulnerabilities. We generate the entities during the explore phase as we collect and analyze the HTTP traffic.

You may think to yourself, that’s a lot of entities! And that’s a lot of tests to run, with each having its own list of rules! And you’d be correct. Without any optimizations there would be tens of millions of tests to run — many of which find the same issues over and over. Why? Because while there are many request-response sessions that AppScan identifies, there are large groups that stem from the same server-side code.

So, what are the optimizations? There are several types, many of which you can configure or influence in some way. We use DOM similarity and DOM redundancy to decide if a set of request-response sessions – or even whole pages – stem from the same server-side code. We then pick a representative set to test and ignore the rest. Redundancy tuning configuration can also influence the number of entities. There are other heuristics and comparisons that we do to pick the optimal set of HTTP components to use in testing. That is, create security entities for.

This is the first important thing to understand: not all HTTP components, which you can see in the application data, turn into actual security entities, which you don’t see. Again, this is due to optimizations.

Another important thing to understand is that the optimizations, aside from the configured ones, are performed only once AppScan has collected enough data. How much is enough? Sometimes it’s a minimal number of sessions, sometimes we wait until the end and we have as much as possible. Remember this: some optimization is done at the very end of the explore. This is important, as it will answer some of the questions above.

Elementwise

We now have our security entities, our rule ranges, and our two passes of the test phase. We pick an entity and run all the rules in one range in the first pass, then we pick it again (if needed) and run all the rules in the second range. This pairing of entity and rule range is called a tested element. An entity may contribute a single tested element if it has only one rule range – or two tested elements otherwise. Because of optimizations, types of entities and their distribution in the application it is nearly impossible to calculate in advance, the number of tested elements AppScan will get out of the explore data (HTTP components).

Now that you understand what a tested element represents and means in the DAST scanning we can get into the issue of progress. The scanner counts off the rule ranges that it has completed. That is the representation of progress.

​Although we’ve talked a bit about how AppScan generates tested elements (during the explore), there are more options. Entities are also generated when importing manual explore. As the scanner processes the sequence it generates the entities, and this happens in two steps. AppScan doesn’t optimize some entities, and they can be generated as they are imported. AppScan tries to optimize the rest; they are generated at the end of the import.

Learn More

To learn more about AppScan’s technical capabilities, please request a demo now.