A match is not a finding

A matcher answers one question: did this response contain that. It is a good question, and for a large class of checks it is the whole job.

It is not the same question as “is this target exposed, and can I defend the claim”. The distance between the two is triage time, and it is paid by whoever reads the report.

This page shows one check where the distance is total: two servers whose responses are indistinguishable to any matcher, one of which is broken. Then it says what the declarative model is genuinely better at, and what cxg does not claim.

Two servers, one response shape

Both servers below set a session cookie. Save this as session-server.py.

session-server.py

#!/usr/bin/env python3
"""Issue a session cookie. Two modes, identical cookie shape."""
import http.server
import secrets
import sys

MODE = sys.argv[1] if len(sys.argv) > 1 else "sequential"
PREFIX = "7f3a9c1e04b6d258"
counter = 0x9c4e1a77b3d05e21


class Handler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        global counter
        if MODE == "random":
            token = secrets.token_hex(16)
        else:
            counter += 1
            token = PREFIX + "%016x" % counter
        self.send_response(200)
        self.send_header("Set-Cookie", "SESSIONID=%s; Path=/; HttpOnly" % token)
        self.send_header("Content-Type", "text/html")
        self.end_headers()
        self.wfile.write(b"<html><body>ok</body></html>")

    def log_message(self, *args):
        pass


class Server(http.server.HTTPServer):
    allow_reuse_address = True


Server(("127.0.0.1", 8901), Handler).serve_forever()

sequential issues a fixed sixteen-character prefix followed by a counter. Every identifier it hands out is derived from the last one, so anyone holding a session can compute the next. random issues sixteen bytes from the operating system’s cryptographic source.

Start the broken one and take three cookies:

python3 session-server.py sequential

# in a second terminal
for i in 1 2 3; do curl -s -D - -o /dev/null http://127.0.0.1:8901/ | grep -i '^Set-Cookie'; done

Set-Cookie: SESSIONID=7f3a9c1e04b6d2589c4e1a77b3d05e22; Path=/; HttpOnly
Set-Cookie: SESSIONID=7f3a9c1e04b6d2589c4e1a77b3d05e23; Path=/; HttpOnly
Set-Cookie: SESSIONID=7f3a9c1e04b6d2589c4e1a77b3d05e24; Path=/; HttpOnly

Now the sound one:

Set-Cookie: SESSIONID=e7c706b55788dae548ef8871dacef30c; Path=/; HttpOnly
Set-Cookie: SESSIONID=14b4d20d9e23871115c0520944d9eb3f; Path=/; HttpOnly
Set-Cookie: SESSIONID=70a223dba51e3fd7af6478f70fb8dc4a; Path=/; HttpOnly

Same header, same name, same length, same alphabet. Both match SESSIONID=[0-9a-f]{32}, and both fail every regex that excludes the other. Any single response from the broken server is a perfectly ordinary-looking session cookie, because it is one. The defect is not in a response. It is in the relationship between responses, and no matcher is holding more than one.

The check that can see it

A template that samples the target and measures the sample can. Save this as session-token-predictable.py.

session-token-predictable.py

#!/usr/bin/env python3
#
# @id: session-token-predictable
# @name: Predictable session identifier
# @author: Bugb Docs
# @severity: high
# @description: Samples session identifiers across many requests and confirms predictability by measuring how much of the token actually varies and whether it advances in sequence.
# @tags: session, entropy, web, cwe-330
# @cwe: CWE-330
# @confidence: 90

import json
import math
import os
import re
import sys
import urllib.error
import urllib.request

SAMPLES = 24
TIMEOUT = 5
COOKIE = re.compile(r"SESSIONID=([0-9a-f]+)")


def collect(base):
    """Request the target SAMPLES times and return the session tokens issued."""
    tokens = []
    for _ in range(SAMPLES):
        try:
            with urllib.request.urlopen(base, timeout=TIMEOUT) as response:
                header = response.headers.get("Set-Cookie", "")
        except (urllib.error.URLError, OSError):
            return tokens
        match = COOKIE.search(header)
        if not match:
            return tokens
        tokens.append(match.group(1))
    return tokens


def effective_bits(tokens):
    """Measured Shannon entropy, summed per character position across the sample."""
    total = 0.0
    varying = []
    for i in range(len(tokens[0])):
        column = [t[i] for t in tokens]
        counts = {c: column.count(c) for c in set(column)}
        if len(counts) > 1:
            varying.append(i)
            for n in counts.values():
                p = n / len(column)
                total -= p * math.log2(p)
    return total, varying


def sequential(tokens, varying):
    """True when the varying part of the token advances by a constant step."""
    if not varying:
        return False
    lo, hi = varying[0], varying[-1] + 1
    try:
        values = [int(t[lo:hi], 16) for t in tokens]
    except ValueError:
        return False
    steps = {values[i + 1] - values[i] for i in range(len(values) - 1)}
    return len(steps) == 1 and steps.pop() > 0

The verdict, and the conditions under which it declines to give one:

session-token-predictable.py

def scan(host, port):
    scheme = "https" if port == 443 else "http"
    base = "%s://%s:%d/" % (scheme, host, port)

    tokens = collect(base)

    # The check audits the distribution, so it needs the whole sample. Anything
    # less cannot support a verdict either way, and says so by staying quiet.
    if len(tokens) < SAMPLES or len(set(len(t) for t in tokens)) != 1:
        return []
    if len(set(tokens)) != len(tokens):
        return []

    bits, varying = effective_bits(tokens)
    in_order = sequential(tokens, varying)

    if bits >= 32 and not in_order:
        return []

    width = len(tokens[0])
    return [
        {
            "template_id": "session-token-predictable",
            "template_name": "Predictable session identifier",
            "matched_at": base,
            "severity": "high",
            "confidence": 90,
            "title": "Session identifier is predictable",
            "description": (
                "Across %d issued identifiers, %d of %d characters never changed "
                "and the sample carries %.1f bits of measured entropy. "
                "The varying part advances in sequence: %s."
                % (len(tokens), width - len(varying), width, bits,
                   "yes" if in_order else "no")
            ),
            "evidence": {
                "request": "GET %s (x%d)" % (base, len(tokens)),
                "response": "\n".join(tokens[:4] + ["..."] + tokens[-2:]),
                "matched_patterns": [
                    "constant_positions=%d/%d" % (width - len(varying), width),
                    "effective_bits=%.1f" % bits,
                    "sequential=%s" % str(in_order).lower(),
                ],
                "data": {
                    "samples": len(tokens),
                    "varying_positions": varying,
                    "effective_bits": round(bits, 2),
                    "sequential": in_order,
                },
            },
            "cwe": "CWE-330",
            "remediation": "Issue session identifiers from a cryptographically secure random source, using the full width of the identifier.",
        }
    ]


def main():
    host = os.environ.get("CERT_X_GEN_TARGET_HOST")
    port = int(os.environ.get("CERT_X_GEN_TARGET_PORT", "80"))
    if not host:
        if len(sys.argv) < 2:
            sys.exit("usage: session-token-predictable.py <host> [port]")
        host = sys.argv[1]
        if len(sys.argv) > 2:
            port = int(sys.argv[2])
    findings = scan(host, port)
    print(json.dumps(findings, indent=2))


if __name__ == "__main__":
    main()

Three things here have no expression as a matcher. It issues a number of requests it decides on, it computes a statistic over their results, and it holds the finding back until that statistic clears a threshold. The template follows the same contract as any other: an annotation header, the target in the environment, a JSON array on stdout. Write your first template covers that contract in full.

Run it

1. Validate it.

   cxg template validate session-token-predictable.py

   ════════════════════════════════════════════════════════════════════════════════
   CERT-X-GEN Template Validator
   ════════════════════════════════════════════════════════════════════════════════
   
   Found 1 template(s) to validate
   
   ✓ session-token-predictable.py
   
   ════════════════════════════════════════════════════════════════════════════════
   Validation Summary
   ════════════════════════════════════════════════════════════════════════════════
   
     Total Templates: 1
     ✓ Passed: 1
     ✗ Failed: 0
     Success Rate: 100%

2. Run it through the engine against the broken server.

   mkdir -p my-templates && cp session-token-predictable.py my-templates/
   cxg scan --scope http://127.0.0.1:8901 --template-dir ./my-templates

   Findings by Severity:
     CRITICAL: 0
     HIGH:     1
     MEDIUM:   0
     LOW:      0
     INFO:     0
   
     TOTAL: 1

   Truncated to the severity block.

3. Read what it recorded.

   python3 -c "import json;print(json.load(open('scan-results.json'))['findings'][0]['description'])"

   Across 24 issued identifiers, 30 of 32 characters never changed and the sample carries 5.2 bits of measured entropy. The varying part advances in sequence: yes.

   The evidence in scan-results.json carries the sample itself, the positions that moved, and the measurement:

   "matched_patterns": [
     "constant_positions=30/32",
     "effective_bits=5.2",
     "sequential=true"
   ],
   "data": {
     "effective_bits": 5.15,
     "sequential": true,
     "varying_positions": [
       30,
       31
     ],
     "samples": 24
   }

   The measured entropy moves by a few tenths of a bit between runs, because where the counter happens to roll over changes which positions vary.

4. Stop the server, start the sound one, and run the same template again.

   python3 session-server.py random

   cxg scan --scope http://127.0.0.1:8901 --template-dir ./my-templates

   Findings by Severity:
     CRITICAL: 0
     HIGH:     0
     MEDIUM:   0
     LOW:      0
     INFO:     0
   
     TOTAL: 0

Same template, same target address, same cookie shape, opposite verdict. The verdict tracks the property rather than the text.

The refutation is the load-bearing half

The first run is the one that looks impressive. The second is the one that makes the check worth having.

A check that only ever confirms tells you nothing when it is quiet, because you cannot tell “I looked and the property holds” from “I never got to look”. That distinction is the difference between a clean result and no result, and cxg reports both as TOTAL: 0.

Which is why the template has the early returns it has. Fewer samples than it asked for, identifiers of inconsistent width, a repeated identifier: each of those means the check could not audit what it audits, and each returns an empty array rather than a verdict it has not earned. That is a third outcome hiding inside the silence, and a template that does not draw the line reports “no finding” for a target it never measured.

The template corpus makes this explicit. A contributed check is expected to have a specific oracle, an honest branch for “the surface I audit is not present here”, and a fixture that produces every verdict it can emit, the skip included. Contribute a template covers that standard and where it is enforced.

Why this is about the corpus, not the file format

Declarative formats can be given escape hatches, and the mature ones have been. The durable difference is not what a format could be extended to express. It is where a project puts its default and what it holds its checks to. In cxg a template being a program is the ordinary case, requiring no flag and no opt-in, and the bill for that decision is on the template trust model.

What declarative patterns are better at

A comparison that cannot name the other side’s advantages is a sales page. These are real, and they are why cxg reads YAML too.

• Readable without being run. You can review a pattern file by reading it, in the confidence that reading it is the whole story. A code template is a program and reviewing one is reviewing code.
• Nothing to install. A pattern has no runtime to be missing on the scanning host, and a missing runtime is a silent false negative.
• Filterable without executing. The engine can list, search, and select declarative templates from their own structure.
• Bounded by construction. A pattern cannot do anything its format has no word for. A program can do anything you can.

For a check shaped like “request this path, look for this string, at this status code”, the pattern is the right answer and a program is overhead. Why polyglot templates is the full treatment of that choice, including what the code column costs.

What cxg does not claim

Breadth. The established pattern-template ecosystems have been accumulating contributed checks for years, and their corpora are far larger than this one. If what you need is a wide sweep for known, published issues across a large estate, that is what those corpora are for, and cxg is not a replacement for them.

The two are not exclusive. Running a broad pattern sweep for coverage and cxg for the checks that need a verdict is a reasonable arrangement, and the reason the template catalog is worth reading before you assume either way.

Related

• Why polyglot templates is the authoring decision: YAML or code, for the check in front of you.
• The template corpus covers what ships, under which licence, and how it is organised.
• Template trust model is the cost of executing templates, stated in full.
• Write your first template builds a different check that a matcher cannot express, one that chains requests.