Skip to content

Proof and coverage outcomes

Zero confirmed findings does not establish complete clean coverage. Cert-X-Gen records what the engine proved, what it could not conclude and which checks never ran. Read those three answers together.

This page concerns the pentest engine, including JS probes replayed by plain cxg scan. Native scanner findings remain template-reported; mixed reports retain that distinction.

A generated probe can return confirmed: true. That is input to triage, not permission to put a finding in confirmed_findings. The engine’s own observed signals decide promotion.

proof_status Read it as
proven the engine observed the required proof or control signal
unverified the signal did not establish proof
unsupported this dispatch had no usable proof oracle for the claim

Each finding also carries triage, triage_reason and reason_kind. model_vuln_class retains a model’s different class separately from the hypothesis class; it does not let the model choose its own proof bar.

A refused attacker request alone does not show an application control held. An entitled identity must successfully make the same request in the same dispatch, with a distinct attacker identity refused. Configuration refusals, transport errors and missing capabilities cannot substitute for that control.

The shipped 1.7.0 pipeline was exercised against a local HTTP fixture:

Fixture Observed destination Proof
owner succeeds on /record, attacker refused mitigation_verifications, triage: "refuted" proven
/missing refuses, with no successful owner control inconclusive_refusal unverified
probe claims confirmation on /public-record after routing one identity ambiguous unsupported

The last case sent a request and still proved no cross-identity claim. The engine counts identities actually routed, not identity names a template wrote. The missing-control finding ends with these fields:

"model_vuln_class": null,
"triage": "inconclusive_refusal",
"triage_reason": "refusal without an engine-observed successful entitled control for the same request in this template; the control was not verified",
"reason_kind": "unknown",
"proof_status": "unverified",
"title": "Documentation fixture"
}

Truncated before model_vuln_class. The path and fixture description are specific to this local check; read your report’s own triage_reason.

A run with a dead or unverified profile is also incomplete: report.json carries "complete": false, profile_liveness names each profile’s result, and auth_skipped names the probes that needed it.

WAF, bot challenges and rate limits are interference, not evidence that an application-level mitigation held. Tauri refusals also do not become verified controls.

not_run entries carry template, reason and cause. Causes include capability_missing, sandbox_gate, generation_refused, generation_failed, auth_shortfall, chain_missing and no_oracle_for_class. Preserve the reason as well as the count. A sandbox_gate value is a scheduling refusal; it does not mean cxg sandboxes ordinary template execution.

auth_skipped separately records missing identity premises. Read caveats, rejected_templates, dead_profiles, execution counts and scope statistics alongside these lists. Requests may have been sent for an unsupported check; requests alone do not establish coverage or proof.

Completed JavaScript runs also carry mutation_outcome, distinguishing resolution from still-ambiguous, missing-rewrite and no-verdict outcomes. An attempted rewrite does not mean the original uncertainty was settled.

The new canonical classes include user_enumeration, ip_header_bypass and business_logic. Their proof needs, respectively, repeated present/absent account comparisons, refused baseline versus permitted spoofed-header requests, and an independent observer’s readback of a server-side invariant.

Class-specific engine oracles also cover response-observable SSTI, command injection, SSRF, XXE, path traversal, redirects, credentialed CORS and mass assignment. Boolean-blind SQL injection adds repeated true/false comparisons. Reflection and a template-authored verdict are insufficient substitutes.

OAuth, OIDC and SAML confirmation remains unavailable. Those registered predicates return unverified; do not treat their presence in the class vocabulary as a promise that an OAuth flow was confirmed safe or vulnerable.

The generated pentest reference covers options. Probe inputs and transports explains the additional identity and transport inputs.

Anonymous access needs a protected resource and control

Section titled “Anonymous access needs a protected resource and control”

In 1.8.0, auth_api, missing_authentication (alias missing-authn) and metrics_info_disclosure can request an engine-owned paired GET plan. A @protected_endpoint: GET /private header declares intended protection; a supported threat link or authenticated/elevated route declaration can do the same. An explicitly public route overrides a protected-GET declaration. Review that policy independently: responses cannot prove whether data should be private.

The finding must name the exact attempted GET, including its query. The engine replays it independently as a credential-free guest and as an entitled captured identity. In the local binary fixture, a probe returned confirmed: false with outcome: unevaluated; matching qualifying content from both engine-owned reads still established the finding. A lone anonymous 200 does not.

Observation Outcome
both reads return 2xx with matching nonempty supported structured content confirmed
guest gets 401/403 or a recognized same-origin/relative login redirect, entitled control gets qualifying 2xx content refuted
failed control, unknown body, 404, interference, unrelated redirect, transport or scope failure inconclusive

JSON requires an application JSON Content-Type and a top-level array or a data, items, records or results envelope. Empty/status/error envelopes, HTML and ordinary text do not qualify. Metrics need actual matching Prometheus sample names and a supported metrics Content-Type; HELP/TYPE lines alone do not qualify. The plan does not follow redirects and only replays GET. It records digests and metadata under engine-owned evidence._cxg_missing_authn, not copied response contents or template assertions.

A list or export that must hold only the caller’s own records is often fixed by filtering rows. The fixed endpoint still answers 200, so there is no refusal to compare and, before 1.9.0, the fix stayed inconclusive. A probe can instead ask the engine’s excluded-records oracle, which compares records:

const v = await cxg.oracle.excludedRecords({
attacker: 0,
subject: {request: {path: '/api/requests/export'},
records: {format: 'csv', key: 'service_request_id', match: 'customer_email'}},
entitled: {request: {path: '/api/requests'},
records: {format: 'json', path: 'service_requests', key: 'id'}},
reference: {as: 1, request: {path: '/api/orders'},
records: {format: 'json', path: 'orders', key: 'user.email'}},
});

The engine sends three read-only GETs, each with that identity’s own credentials: subject, the list under test, as the attacker; entitled, the attacker’s own scoped list; and reference, a list a second identity reads of its own records. A subject record outside the attacker’s list that carries a reference value confirms. A non-empty subject wholly inside the attacker’s list, with a non-empty reference, refutes. Anything else is inconclusive, with the reason.

This probe was replayed with cxg scan --pentest --no-author against a loopback fixture whose export either returned every row or only the attacker’s. The leaking export:

[CONFIRMED] idor GET /api/requests/export — excluded-records oracle: engine observed 2 record(s) outside the attacker's entitled list carrying a 'customer_email' value the reference identity owns

The filtered export, with the same probe:

[REFUTED ] idor GET /api/requests/export — excluded-records oracle: engine observed all 1 subject record(s) inside the attacker's entitled list and none of the 1 value(s) the reference identity owns outside it

Each truncated to the verdict line. The first run exited 2. The second exited 0, and its finding landed in mitigation_verifications with proof_status: "proven" and the comparison under evidence._cxg_excluded_records.

The scope may carry a landing path. On 1.9.2 the filtered export, replayed with --scope http://127.0.0.1:8933/dashboard, refuted the same way:

[REFUTED ] idor GET /api/requests/export — excluded-records oracle: engine observed all 1 subject record(s) inside the attacker's entitled list and none of the 1 value(s) the reference identity owns outside it

Truncated to the verdict line. It exited 0 with proof_status: "proven". On 1.9.1, plain cxg scan without --pentest gave the same verdict. The oracle’s three requests went to /api/requests/export, /api/requests and /api/orders at the origin, without the /dashboard prefix that other request helpers keep. 1.9.0 compared the same records but did not bind the verdict to a finding under a path scope, so the same run came back INCONCLUSIVE_REFUSAL. Use 1.9.1 or later, with a refreshed pipeline, before relying on this verdict with a path in --scope.

The verdict applies to cross-identity classes such as idor and broken_access_control, and settles a finding whose endpoint binds to the subject request. A probe cannot supply its own Authorization or Cookie for the three requests. Report confirmed only on a confirmed verdict. The template guide documents every field.

Authorization refutation compares the same request

Section titled “Authorization refutation compares the same request”

An entitled control succeeding and a distinct attacker being denied can refute only when method, URL, query, body and noncredential headers match within the same dispatch. Authorization/Cookie values may differ. A differing custom, Accept or CSRF header leaves the cause of refusal unresolved. Route both arms through cxg.rawAs or cxg.fetchAs and use an exact method-qualified endpoint.

For known tiers, a successful control ranked below the denied attacker remains inconclusive. Use real identity premises or a clean anonymous role; role names alone establish no privilege relationship. Generic disclosure labels such as sensitive_data_exposure do not acquire a positive proof oracle from an HTTP 200 or SQL text. Use auth_api only when the actual claim is unauthorized access.