Skip to content

Run a pentest in CI

A pentest in CI has a failure mode a scan does not: the saved session expires, the run probes the app logged out, and every real finding comes back refuted. A dead session makes the run incomplete with exit 5. This guide restores a session from a secret, verifies it, and wires that exit into the job.

  • cxg and the pentest orchestrator installed. See the web-application pentest.
  • A captured session exported once, locally, as a Playwright storage_state. That artifact is what CI restores; nobody logs in during the run.

Restore the session into an isolated directory

Section titled “Restore the session into an isolated directory”

A saved session is a credential. Keep it out of the operator’s home store and in a directory the run owns, mode 0700. cxg reads the state from a file, from stdin, or from a base64 environment variable. The last is what a CI secret store gives you.

  1. Create the auth directory with tight permissions.

    Terminal window
    mkdir -p ci-auth && chmod 700 ci-auth
  2. Import the session. In CI, pipe the secret straight in rather than writing it to disk in the clear.

    Terminal window
    printf '%s' "$SESSION_STATE" | cxg pentest auth import \
    --profile live --target https://staging.example.com \
    --storage-state - --auth-dir ./ci-auth --ci

    --storage-state - reads stdin. --ci makes import refuse a world-readable destination rather than write a credential a fellow user could read:

    ✗ auth dir ci-auth is world-accessible (mode 0o755); a saved session is a credential. Run `chmod 700 ci-auth` and retry, or run without CI mode to override.

    That refusal exits 2. It comes from auth import, not from run, so guard it on the import step rather than expecting the run to report it.

auth verify is a cheap liveness gate: exit 0 if the session is alive, non-zero if it is dead. Run it first and skip the expensive pipeline when it fails.

Terminal window
cxg pentest auth verify --profile live --auth-dir ./ci-auth

The check is one authenticated API request, to /api/me unless the profile saved another path. Against a loopback fixture, an expired profile printed:

✗ session for expired expired — re-capture and re-import
· authenticates by cookie: no JWT-shaped value in localStorage or sessionStorage (0 entries searched); this identity is either cookie-authenticated or holds no credentials at all
· dead session: API liveness returned 401

Truncated before a one-line cookie summary. It exits 1. If your app has no /api/me, add --liveness-path to the import step: without it, 1.9.2 falls back to the weaker landing-page test. See tell cxg which endpoint proves a session. The guard is:

Terminal window
cxg pentest auth verify --profile live --auth-dir ./ci-auth \
|| { echo "session expired, re-import"; exit 1; }

Pass --ci to run, or set CXG_CI=1 for pipelines that cannot change the invocation:

Terminal window
cxg pentest run \
--codebase ./app --target https://staging.example.com \
--auth live --auth-dir ./ci-auth \
--ai --ai-provider claude \
--attestation "$CI_ENGAGEMENT_ATTESTATION" \
--ci

Since 1.9.0 a dead profile stops being silent whether or not --ci is set. On 1.9.2, this supplied-probe replay selected one live and one expired profile, without --ci:

Terminal window
cxg scan --pentest --codebase . --scope http://127.0.0.1:8931 --template-dir probes \
--auth mechanic,expired --auth-dir ci-auth --session-dir sess-dead \
--no-author --no-code-graph
[1b] inspecting 2 captured auth profiles…
· mechanic role=mechanic tier=30 auth=cookie
· API liveness returned 200
· expired (DEAD: status=401)
· dead session: API liveness returned 401

The run continued for the ready profile and skipped the probe that needed both:

[service] unavailable: expired at http://127.0.0.1:8931; continuing ready profiles; application verification remains incomplete
· skip export-bola: unavailable profile(s): expired
[5] scan incomplete in 1.1s

Excerpts; cookie summaries and banner lines are omitted between them. It exited 5, and report.json read "complete": false and "status": "incomplete", with the service listed as unavailable. The duration varies by run. The same run with --ci, and with --skip-health-check, also exited 5.

What --ci still changes is the auth directory. The 1.9.2 cxg pentest run --help reads:

--ci
Non-interactive CI mode: refuse a world-accessible `--auth-dir`.

Truncated after the first line. The full text is in the generated reference. 1.9.0’s help still described the 1.8.0 behaviour, that a dead session only warns without --ci and that --skip-health-check force-marks profiles alive; neither held on 1.9.0 either. 1.9.1 corrected it.

The gate is only as good as the endpoint it calls.

--skip-health-check no longer turns the gate off. Since 1.9.0 it skips only periodic health checks and re-authentication; the startup check still runs.

Read the exit code with the report. Both cxg scan --pentest and cxg pentest run use these engine exits:

Exit Meaning CI action
0 No confirmed findings; inspect ambiguity, skips and coverage. Pass only with the coverage your job requires.
1 No templates were available to run. Investigate. Usually a missing findings.sarif.
2 Confirmed findings present. Fail the build, publish report.json.
3 Hard-killed: a 5xx streak, a scope violation, or under --no-restart a desktop target that died mid-scan. Investigate the target.
4 Refused run, missing probe inputs, or unusable supplied probes. Inspect the report; other runnable probes may have executed. Fix the invocation or prerequisites.
5 A selected profile or its service was dead or unverified. Ready profiles still ran; the report is incomplete. Fail loud, re-capture or fix the liveness endpoint.

The full list of what exit 4 refuses is in the cxg pentest run reference. Treat 5 as a failure even when some probes ran: the report is complete: false, and a probe that needed the missing identity never executed.

A full pentest on every pull request is slow. --only-routes <FILE>, or the repeatable --only-route <ENTRY>, holds the run to a list of routes. Every hypothesis whose route or enclosing function is not on the list is set aside before ranking and before the --max-templates budget, with its reason:

Terminal window
cxg pentest run --codebase ./app --target http://127.0.0.1:8902 \
--only-route 'GET /orders/{id}'
[1g] route allowlist (--only-route): 1 entry, 2 hypothesis(es) kept, 2 excluded before ranking (reasons in route_allowlist.excluded in report.json)
− gl-bace41da94395805-unauth POST /orders/export
− bnd-c140f1d187-post-orders-export POST /orders/export

Truncated to the allowlist line. GET /orders/{id} matched the handler declared as /orders/<id>: path templates and concrete paths match each other across router spellings. The same exclusions are in report.json:

{
"id": "gl-bace41da94395805-unauth",
"vuln_class": "unauth",
"method": "POST",
"path": "/orders/export",
"function": "export_orders",
"file": "app.py",
"line": 24,
"source": "guardlink",
"reason": "neither its route POST /orders/export nor its function app.py::export_orders is in the allowlist"
}

One entry of route_allowlist.excluded.

An entry is METHOD /path (ANY or * for every method, a trailing * for a prefix), /path, file::Function, or file:Function, one per line, or JSON. A code graph’s route or attack-surface answer can be passed as written, so a CI step can hand cxg the routes a change reached without reformatting them.

An empty list is a decision, not a mistake. It means nothing is in scope, and the run exits 0 without contacting the target:

[1g] route allowlist (empty-routes.txt) is EMPTY: no route is in scope, so nothing is tested. This is a skip, not a pass — no request was sent to http://127.0.0.1:8902.
report → s-empty/report.json

Treat that 0 as “skipped”, not “clean”, when you report it. A file cxg cannot read is refused with exit 4:

✗ --only-routes 'missing.txt' could not be read: [Errno 2] No such file or directory: 'missing.txt'

Omitting the flag is a full run. The allowlist works by ranking hypotheses, so a run that does not rank them refuses it, also with exit 4:

✗ --only-routes/--only-route cannot be enforced on a --template-dir replay, which runs every template in the directory without ranking hypotheses; drop one or the other.

--template-lang py is refused the same way.

  • Record authorization. --attestation writes your engagement ID and operator into the audit-log header, the dispute-ready record that you had permission. Strongly recommended for any non-local target.
  • Bound the AI cost. --max-templates caps how many probes the ranker may select, and --generation-timeout caps each AI call.

Scanner-only cxg scan does not fail on completed findings by default. --fail-on confirmed,errored,timed-out opts into exit 2 for those execution statuses, after reports are written. Refuted and skipped are also selectable. All-candidate loading failure exits 1 by default; a matching status gate selects 2 instead.

A mixed scan also retains the probe engine’s own nonzero exits, including confirmed findings and refused probes. Read probe_results, executions[], coverage gaps, and each finding’s verdict source before declaring success. --ci guards session health; it does not turn scanner findings into a gate.