Skip to content
Bugb Docs

Verdicts and the ledger

The ledger is the record of every exposure bugb has seen, each with a verdict and the reason it holds. It is the run’s memory: what has been proven, what has been ruled out, and — the part that gets skipped — what was never tested and why.

If you read one thing about bugb’s output, read this. A confirmed count without its untested count is not a security result.

The eight verdicts

Section titled “The eight verdicts”

From the filter’s own enum:

  --verdict {unverified,confirmed,refuted,mitigation_held,error,untestable,suppressed,uncorrelated}
Verdict Means Settled?
unverified tracked, not yet concluded — the default state of everything no
confirmed a probe proved it exploitable, with evidence yes
refuted probes ran against it and it did not hold up yes
mitigation_held the declared control was attacked and held yes
error probing failed operationally — nothing was learned no
untestable no probe could be built for it no
suppressed removed from the export by a @mitigates or @accepts no
uncorrelated a finding arrived that could not be traced to an exposure no

Only a conclusion may settle a threat

Section titled “Only a conclusion may settle a threat”

This is the rule the ledger is built around: a threat’s verdict moves only on evidence. A probe that confirms, or a scan that ran cleanly against it, may settle it. An operational failure may not.

The reason is that cxg’s exit code cannot carry the difference. Exit 0 covers a clean scan, a scan that skipped every probe for lack of identities, and a scan aborted partway. So bugb classifies from the report body instead, and anything that is not a conclusion leaves the threat in the backlog to be re-tested.

In the ledger that reads as two visibly different things:

  REFUTED (1)
    critical  Data.AllocationsDAO #idor                  app/data/allocations-dao.js:57
        2 requests, nothing confirmed


  ERROR (7)
    critical  App.Allocations    #idor                  app/routes/allocations.js:16
        zero requests were issued; the target was never reached
        not tested — error_no_identities

2 requests, nothing confirmed is a result. zero requests were issued is not. A tool that folded the second into the first would report the same run as cleaner than it was, and nothing downstream could tell.

The reason codes

Section titled “The reason codes”

Every record carries a reason as well as a verdict. Observed across those ledgers, most common first — each reason produced exactly one verdict:

Reason Verdict it belongs to Says
new_exposure unverified first seen this run; nothing has tested it
suppressed_by_mitigates suppressed a @mitigates removed its pair from the export
probe_confirmed confirmed a probe proved it, and the evidence is on the record
annotation_confirmed unverified the source claims @confirmed; bugb has not re-verified it
seeded_from_prior confirmed carried in from an earlier run’s ledger, verdict intact
untestable_no_route untestable no route could be resolved, so no probe could be built
probe_refuted refuted probes ran and it did not hold up
error_generation_failed error no template could be generated
probe_no_evidence unverified probes ran but tested something else
uncorrelated_finding uncorrelated a finding could not be traced to an exposure
error_dead_profile error the identity’s session had expired
error_no_identities error there was no identity to run as
error_unknown error the failure did not classify

annotation_confirmed is the one to notice. A @confirmed annotation in the source is somebody’s claim, and the ledger keeps it as unverified until a probe of bugb’s own proves it. --trust-annotations turns that off and takes the annotation at its word.

probe_no_evidence is the second. Its message is explicit about what happened:

the probe produced no evidence for this threat, so nothing was proven or disproven; the gap stays in the backlog to be re-tested (probe note: no probe ran for this threat: cxg generated no template carrying gl-4dfb3fd2c18d, so nothing in this run tested it; the templates that did run tested other threats (nosql-injection-allocations-threshold-anon) — cxg reported: 4 requests, nothing confirmed)

Four requests were sent, and none of them tested this threat. Under a coarser model that is “4 requests, nothing found” — which is how a false clean result gets manufactured.

Identity, and why it is not the pair

Section titled “Identity, and why it is not the pair”

guardlink identifies an exposure by asset::threat. bugb’s ledger keys go further — asset, threat, file, and a hash of the message:

app.allocations::idor@app/routes/index.js#62ad24636cfe

The difference is measurable: on the reference repository, 118 exposures collapse to 69 unique pairs. Under the pair key, a new weakness in a pair that is already known cannot be distinguished from the one already recorded, and the round reports “nothing added”.

A confirmed record is terminal: it is never re-goaled, so the loop does not spend rounds re-proving what it has proven. --retest <THREAT_ID|KEY> forgets one deliberately.

Reading it

Section titled “Reading it”
Terminal window
bugb ledger --run <run-id>                     # every verdict, grouped
bugb ledger --run <run-id> --verdict error     # only what failed operationally
bugb ledger --run <run-id> --json              # the records

Start with --verdict error and --verdict untestable on any run whose confirmed count looks reassuring.

Section titled “Related”