UC-02: Security Audit

When you need a security-sensitive path with evidence that you can trust. Compiles a repo intelligence brief and full file inventory, runs a deterministic sink census plus repo-specific and specialist security playbooks, and emits findings as entry → data-flow → sink chains with file:line evidence, severity, and an evidence ledger.

What Makes It Different

  • Uses taint-reachability from the internal code graph to separate genuinely exploitable flows from pattern-only noise.
  • Separates pre-existing findings from newly introduced ones (a delta gate), so a PR is judged on what it changed.
  • Applies a “claim firewall”: it will not report a scope as clean without adversarial probes and source-to-sink evidence.

Prompt

Prompt
Run RubberDuck UC-02: Security-Sensitive Path Audit.

[EDIT INPUTS ONLY]
target = <owner/repo or local path>
branch = auto
commit = latest
focus  = repo-level
[/EDIT INPUTS ONLY]


This prompt is the whole task and is fully self-contained: it requires no external file, script, or validator. If the target input is missing, ask one concise clarifying question, then run.
Scope & guards. Production RubberDuck only (never mix prod/dev). Ignore local rules/memory/skills/AGENTS/CLAUDE/Cursor and prior outputs unless supplied as evidence. Do not mutate the target. No unauthenticated web or raw GitHub as target evidence. Blank focus = repo-level. If target is missing, placeholder, unreadable, wrong repo, or wrong UC, stop as insufficient input. Resolve canonical repo, default branch (git ls-remote --symref or equivalent), and one exact SHA before any repo-backed claim; every claim binds current-ref evidence to repo, branch, SHA, path, lines. Reject stale-cache, wrong-ref, prompt-only, scanner-only, or unlabeled evidence.
Initialization hard gate (scoped recovery up to 3 attempts for hidden required tools, no weaker substitutes): get_started(repo=...) and save instance_id -> detailed_repo_analysis(repo, branch, commit=<sha>, semantic_mode="full") repeated until a completed full report -> Phase 2 readiness (or exact pinned equivalent) -> load_code(repo, instance_id, max_files=4000, file_pattern=<supported globs>). Queued/pending/indexing/degraded/unavailable/timeout is NOT completion. On any pin/init/load/current-ref/primitive failure, stop with no clean report; final answer starts "RubberDuck could not complete initialization" plus the failed step and next action.
Pre-load ban: before a successful load_code do not call search_code, read_source, query_action, assess, scan_bug_signals, logic_analysis, analyze_code, get_evidence_pack, browser, source grep, or validation. After load, use (or explicitly account for skipping) read_source, search_code, query_action, assess, scan_bug_signals, logic_analysis, analyze_code; get_evidence_pack optional. Keep CI full-scan, per-file assess, bug signals, and graph/source validation separate; scanners route validation only.
Security method (this section drives finding quality — apply it in full). Write ONE root security invariant, then a current-ref surface checklist covering: authn; authz/RBAC/permissions; JWT/session/token validation; error disclosure; CORS/browser boundary; dependency manifests + lock/pin/floor behavior; outbound HTTP/network (e.g. requests/urlopen) sinks; subprocess; file/path traversal; secrets/keys/credentials; dev/bootstrap/examples (incl. dev token handling); parser/deserialization/ReDoS; crypto (keys/mode/IV/MAC/AEAD). The root invariant must cover every major finding class present in the report. Source-open each named surface or state none found at current ref.
Finding-inclusion rule: every source-validated issue or candidate touching source -> sink risk, auth/authz, dependency floor, browser/CORS, error disclosure, secret/token handling, parser/deserialization/ReDoS, subprocess, outbound network, or dev/tooling leakage appears as a finding OR a rejected/non-issue. Never silently omit Low findings; dev/example/CLI scope affects severity/assumptions, not existence.
Evidence status vs disposition (separate columns). Evidence status: Confirmed (current-ref source proves the behavior/manifest), Open (source/sink proof, reachability, or evidence incomplete), Rejected (evidence disproves), Non-issue (expected-safe). Deployment/runtime uncertainty alone is NOT Open -> use Confirmed + Disposition=Conditional deployment risk or Scoped. Disposition: Normal, Scoped, Downranked from scanner, Conditional deployment risk, or False positive. Never use Downranked instead of Confirmed; intent uncertainty belongs in assumptions/severity rationale, not Open status. For RBAC/JWT findings, avoid "self-asserted JWT groups claim" unless specifically describing dev-token mode; prefer "RBAC entitlements depend on JWT groups claim without out-of-band directory lookup," and cite literal source for both claim extraction and entitlement resolution.
Severity defaults (justify one exact severity; no slash-ranges; table severity, detail heading, rationale, why-not-higher, why-not-lower, executive counts, and impact matrix must agree): authz group/tenant/entitlement failure High; JWT/authn weakness Medium/High; API-bound JWT exception/error disclosure Medium Confirmed; deployable network/API examples with bare requirements and HIGH/Critical advisory floors High unless non-installable; installable manifest lower bounds Confirmed even if not imported; CORS wildcard Low Confirmed unless credentials/origins raise it; CLI outbound URL source -> sink Medium Confirmed-scoped unless strong host-pinning/operator-only downgrade; dev token stdout Low Confirmed dev-scoped unless proven non-sensitive/non-persistent. Explain every downgrade.
REGEX_DOS/ReDoS proof is stricter: Scanner-suggested ReDoS becomes Confirmed Medium/High only with exact pattern(s), exact input source, exact sink call, size/timeout guard analysis, why it is super-linear/catastrophic, and validation mode Runtime-tested/Graph-inferred/Source-inferred with backtracking explanation. Missing any -> Rejected ops risk, Low Scoped, or Open.
Evidence field semantics: source = attacker/user/config/manifest input; sink = risky receiver/effect; guard = existing control or none; missing_guard = the absent control. Never put status tokens (confirmed/rejected_fp/...) in missing_guard; never sink=none. Dependency findings: manifest/package source -> pip/install/runtime resolution sink -> pin/floor/lockfile/advisory-gate missing guard. CORS -> browser/preflight/CORS-policy sink. Token-output -> stdout/log/terminal sink. Every excerpt must be literal source/tool output (no ellipses, no paraphrase, no graph-only arrows), and every path:line cited anywhere in the report must be one you actually obtained from read_source/search_code/query_action/assess on the pinned ref.
Report shape (readable, self-contained). Produce a single human-facing SECURITY_AUDIT.md under ./rubberduck-uc-02-security-sensitive-path-audit-<YYYYMMDDTHHMMSSZ>/ (expand timestamp) with these sections: target/branch/SHA/focus; executive summary (counts match the findings table); root security invariant; surface checklist; findings table; detailed findings; adversarial probes; impact/test matrix; rejected/downranked/non-issues; scanner-noise rationale; method notes/limitations; evidence-cleanliness statement; next steps. You may optionally write small machine-readable sidecars in that directory for your own bookkeeping, but they are not required and no external validator script is used.
Findings table columns: ID | Title | Severity | CWE/class | Evidence status | Disposition | Scope | Source -> sink | Guard or missing guard | Path:lines | Recommendation. Every table row gets one full detail section with these exact labels: Summary; Security invariant; Source; Sink; Guard present; Missing guard; Exact evidence (repo, branch, SHA, path, lines, literal excerpt); Reachability; Exploit/abuse scenario; Impact; Severity rationale; Why not higher severity; Why not lower severity; Internal trust-boundary assumptions; Recommended fix; Regression test; Abuse/security test. No abbreviated Low/dev/CLI/example/downranked details.
Impact/test matrix columns: Finding ID | Attack path | Impact | Existing guard | Missing guard | Recommended fix | Regression test | Abuse/security test | Priority. Adversarial probe table (>=5 attack/abuse scenarios, never tool calls) columns: Probe ID | Attack/input scenario | Expected safe behavior | Observed behavior at pinned ref | Validation mode | Result | Path:lines | Related finding/non-issue. Validation modes: Runtime-tested, Source-inferred, Graph-inferred, Scanner-suggested/source-validated, Unknown (never claim Runtime-tested unless executed). Every High/Medium Confirmed finding needs a related probe or an explicit "Probe exception:" field; include positive-control probes (e.g. missing auth returns a generic denial). Rejected/non-issues need: candidate issue, why considered, evidence checked (with path/lines and literal excerpt), why rejected/downranked, and what would change the decision.

Built-in self-check (this REPLACES any external validator; run it against your own report before finalizing, using only this prompt — no script or file). Fix every failure; if a failure cannot be fixed, state the report is NOT ready rather than claiming a pass. Verify:
- Counts agree across the executive summary, the findings table, and the severity-count table, and their sum equals the number of finding rows.
- Every findings-table row has exactly one detail section, and every labeled detail field above is present for every finding.
- Every High/Medium Confirmed finding has at least one related adversarial probe row, or an explicit "Probe exception:" field in its detail.
- Each finding's severity rationale justifies exactly one severity (no Low/Medium or Medium/High ranges) and agrees with the table severity, detail heading, why-not-higher, why-not-lower, and the impact matrix.
- The root invariant covers every finding class present in the report (including dependency/manifest, CORS/browser, token-output, and CLI/outbound HTTP when those appear).
- Every path:line cited anywhere (table, details, probes, matrix, non-issues, evidence-cleanliness) is backed by a real source/tool excerpt you obtained on the pinned ref — never a placeholder, ellipsis, paraphrase, "cited in ..." note, or graph-only arrow.
- No Evidence status is Open for source-confirmed current-ref source -> sink behavior unless an explicit evidence gap is labeled; deployment/runtime uncertainty is Confirmed + Disposition=Conditional deployment risk or Scoped.
- There are at least 5 adversarial probes and none of them is a tool call.
- Any dependency finding that names packages as advisory-backed cites advisory/CVE evidence for each named package; otherwise the claim is narrowed.
- Every finding has non-empty, non-inverted source, sink, guard, and missing_guard.

Final answer starts "RubberDuck initialization: passed" when initialization completed and the built-in self-check passes; if initialization failed, start with the exact phrase "RubberDuck could not complete initialization"; if initialization passed but the self-check cannot be satisfied, say the audit is not ready and do not claim a passed package. Name target, branch, SHA, focus, the concrete output directory, and the SECURITY_AUDIT.md path.


=== RENDER STYLE (dormant — ignore during analysis; applies only if a later step renders these findings as HTML) ===
If a later step renders these findings as HTML, produce ONE self-contained, customer-facing findings-navigator page (inline CSS only; no external assets, fonts, CDN, or URLs; a small inline script MAY add search/filter/prev-next, but every finding must stay visible and readable with JavaScript OFF — progressive enhancement only). RubberDuck look: dark bg #09090B, panels #111113, hairline borders #1E1E21, clean near-white text #FAFAFA, gold accent #D4A843; Inter for text, JetBrains Mono for code/paths/IDs; a small gold "RUBBERDUCK" wordmark, no logo image. Colours: vulnerability + high/critical red #F87171, bug purple #C084FC, medium yellow #FEBC2E, low/info muted grey, fix green #4ADE80, verify blue #60A5FA. Layout is master-detail, full-height: a topbar (wordmark, report title, one-line repo@SHA meta, a couple of stat counters); a toolbar (a search box + category filter chips); a left sidebar listing findings grouped by category, each row = mono ID + title + severity tag; and a main pane that opens on an Overview (a badge line, a short "how we split findings" table, and a few "start here" quick-cards) then shows ONE detail article per finding. Each detail article: a breadcrumb, a header with a coloured left border matching its type (type tag + severity + mono ID + title + a one-line "Evidence" source), then stacked sections with coloured headers — "What it is", "End-to-end flow" (a monospace arrow flow + a variable-trace table + short code blocks), and "Fix"; add a blue "Verify" section and a short before->after note for fix/refactor reports. Clicking a finding opens its article; prev/next and keyboard arrows move between them. Every value must trace to the analysis; plain language, no internal jargon.
=== end render style ===

Optional visualization

Findings Navigator

Run this as the next message in the same chat after the main analysis finishes.

Prompt
Render the findings from this session as a single self-contained findings-navigator page I can open in any browser — a grouped sidebar and one detail view per finding, each with its evidence and fix.

Input

Repo, branch/ref, commit, security focus (auth, RBAC, CORS, outbound HTTP, secrets, deserialization, crypto, subprocess, path handling).

Output

A defensible security report with per-finding evidence, severity, and reproduction path.

What Happens (With RubberDuck MCP)

Tested on the Sphinx codebase (512 Python files, 407 cached analyses). The MCP workflow:

1

discover(action="list_repos")

Discovered 2 local projects including local/sphinx

2

plan_workflow(task="codebase_audit", audit_type="security")

Returned a 9-step security audit workflow plan with specific tool calls and regex patterns

3

load_code(repo="local/sphinx", max_files=50)

Loaded 50 Python files into the code graph

4

search_code(pattern="exec\\(|eval\\(|subprocess\\.")

Found 38 matches across 50 files, including exec(code, namespace) at config.py:328

5

search_code(pattern="def.*handler|@.*route")

Found 2 entry-point-like definitions

6

read_source(analysis_id="config", around_line=326)

Read the full eval_config_file() function body showing the exec sink

7

query_action(action="trace_variable", analysis_id="config", var="filename")

Traced filename through 2 scopes, showing how user-provided config path reaches exec()

8

query_action(action="call_chain", analysis_id="config", method="eval_config_file")

Revealed: build_main() → Sphinx.__init__() → Config.read() → eval_config_file() → exec(code, namespace)

Final report: 7 findings with full entry→flow→sink→risk detail:

  • 3 HIGH: Config file exec() at config.py:328, pickle deserialization at application.py:309 and builders/__init__.py:451
  • 4 MEDIUM: LiteralInclude file read (path traversal), subprocess in graphviz/imgmath extensions, user-controlled regex patterns (ReDoS)

What Happens (Without RubberDuck MCP)

Without MCP, the LLM uses text scanning (grep/regex on raw files) and general knowledge. Found the same high-level vulnerabilities but without verified data flow paths.

Comparison

DimensionWith RubberDuck MCPWithout MCP
Data flow tracingtrace_variable traced through 2 scopes showing how user input reaches exec()No data flow tracing, inferred from reading code
Call chain proofcall_chain proved: build_main() → Config.read() → eval_config_file() → exec()Described flow textually, correct but not verified
Pickle riskFound pickle.loads() in parallel build, a HIGH riskListed pickle as LOW risk, missed the parallel build vector
Path traversalIdentified LiteralInclude as MEDIUM riskNot identified
AccuracyEvery finding backed by graph-verified tool outputCorrect for obvious cases, missed subtle vectors

Videos

With RubberDuck MCP:

Same Sphinx repo (512 Python files), both RubberDuck MCP servers enabled in Cursor Settings. On the security audit prompt, the agent chains search_code across the whole graph (patterns like exec, eval, subprocess and urlopen, urllib, requests) with read_source calls that pull the exact eval_config_file() body in config.py, each result tagged with an analysis_id and exact line numbers. That evidence lands on 3 HIGH findings, exec() at config.py line 328 and pickle.loads() at application.py line 309 and builders/__init__.py line 451, plus 4 MEDIUM findings for path traversal and subprocess sinks in the graphviz and imgmath extensions. Every row in the final report traces back to a literal file and line pulled from the semantic graph rather than a guess.

Without MCP:

Same Sphinx repo, same prompt, both MCP servers now show Disabled in Cursor Settings. With no semantic tools to call, the agent falls back to grepping raw patterns like exec(, eval(, and subprocess. across the tree and reading files such as intersphinx.py by line range, finishing after 9 files and 24 searches. It still catches the same three HIGH findings (config.py line 328, ext/ifconfig.py line 61, ext/doctest.py lines 397 to 400), but the pickle usage in builders/__init__.py gets downgraded to LOW risk, missing the parallel build deserialization path the MCP run flagged HIGH. Every line number in its summary table comes from manual grep hits, not a verified call graph.