Diagrams — SDD-07: Secure Code Review Stack

Diagram 1 — The Five-Layer Pipeline (Architecture)

flowchart TB
    SRC([source code repository]) --> L1
    subgraph L1["Layer 1 — Structural (AST)"]
        AST[tree-sitter / native AST<br/>syntax checks · custom structural rules]
    end
    L1 --> L2
    subgraph L2["Layer 2 — Pattern (Semgrep)"]
        SG[Semgrep<br/>pattern matching over AST<br/>known-bad patterns · framework rules<br/>runs in seconds · pre-commit friendly]
    end
    L2 --> L3
    subgraph L3["Layer 3 — Data-Flow (CodeQL)"]
        CQL[CodeQL<br/>compile to QL DB · query taint flows<br/>cross-file/cross-function sinks<br/>deep · slow · noisy]
    end
    L3 --> L4
    subgraph L4["Layer 4 — LLM Triage"]
        LLM[for each finding: TP or FP? why?<br/>SUPPRESS false positives WITH REASON<br/>escalate true positives<br/>never confirms, only suppresses]
    end
    L4 --> L5
    subgraph L5["Layer 5 — Autofix with Approval"]
        FIX[LLM generates patch<br/>→ opens PR<br/>→ human reviews → merges<br/>THE HUMAN IS THE GATE]
    end

    style L1 fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style L2 fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style L3 fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style L4 fill:#1a1015,stroke:#5eead4,color:#e4e4e8
    style L5 fill:#1a1015,stroke:#5eead4,color:#e4e4e8

Reading: The stack is a depth/cost-ordered pipeline. Each layer runs where its cost is justified: AST and Semgrep are fast enough for pre-commit (creation-time guardrails); CodeQL is slow and runs in nightly CI; the LLM triage runs on the findings batch; autofix opens PRs a human reviews. Each layer compensates for the failure mode of the layer before it — Semgrep's shallowness is covered by CodeQL's depth, CodeQL's noise is covered by the LLM triage, the LLM's hallucination is covered by the suppression audit, and autofix's risk is covered by the review gate. The composition is the product.

Diagram 2 — Semgrep vs CodeQL vs LLM Triage (Comparison)

flowchart LR
    subgraph SEMGREP["Semgrep — Pattern"]
        S1[matches known-bad patterns]
        S2[fast — seconds]
        S3[pre-commit friendly]
        S4[SHALLOW — misses deep flows]
    end
    subgraph CODEQL["CodeQL — Data-Flow"]
        C1[compiles to QL DB, queries taint]
        C2[slow — minutes, build-bound]
        C3[nightly CI]
        C4[DEEP but NOISY — 70-90% FP]
    end
    subgraph LLM["LLM Triage — Semantic"]
        L1[suppresses FP with reason]
        L2[seconds per finding]
        L3[on the findings batch]
        L4[HALLUCINATION — bounded by suppress-only]
    end
    SEMGREP -.shallow, covered by.-> CODEQL
    CODEQL -.noisy, filtered by.-> LLM
    LLM -.can hallucinate, audited by.-> HUMAN[human suppression audit]

    style SEMGREP fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style CODEQL fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style LLM fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style HUMAN fill:#1a1015,stroke:#5eead4,color:#e4e4e8

Reading: Each layer has a strength and a failure mode, and each layer's failure mode is the next layer's job. Semgrep is fast but shallow — covered by CodeQL. CodeQL is deep but noisy — filtered by the LLM. The LLM can hallucinate — bounded by the suppress-only constraint and audited by a human. No layer is sufficient alone; the chain of compensations is the design.

Diagram 3 — The False-Positive Economy (Why the LLM Triage Exists)

flowchart TB
    subgraph WITHOUT["Without LLM Triage"]
        W1[CodeQL: 1000 findings] --> W2[~70-90% false positives]
        W2 --> W3[human must review all 1000]
        W3 --> W4[result: reviewer tunes queries aggressively OR abandons tool]
    end
    subgraph WITH["With LLM Triage"]
        T1[CodeQL: 1000 findings] --> T2[LLM suppresses ~700-900 FP WITH REASON]
        T2 --> T3[human reviews ~100-300 true candidates]
        T3 --> T4[result: reviewer sees signal, tool stays usable]
        T2 -.suppressions logged.-> AUDIT[5% sample audit for hallucination]
    end
    WITHOUT -.the economic case.-> WITH

    style WITHOUT fill:#1a1015,stroke:#9494a0,color:#9494a0
    style WITH fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style AUDIT fill:#1a1015,stroke:#5eead4,color:#e4e4e8

Reading: The single highest-leverage addition to the stack is the LLM triage, and the reason is economics. CodeQL's depth produces a false-positive rate that makes the tool unusable in practice — reviewers either tune the queries so aggressively that real bugs pass, or abandon the tool. The LLM suppresses the obvious false positives at cents per finding, leaving the human with the candidates that matter. The constraint that makes this safe is suppress-with-reason-never-confirm: the LLM can only reduce the queue, and every suppression is logged and sampled for hallucination audit.

Diagram 4 — Where the Human Goes (The Double Gate)

flowchart TB
    FINDINGS[(deterministic findings<br/>AST + Semgrep + CodeQL)] --> TRIAGE
    subgraph TRIAGE["LLM Triage"]
        T[per finding: TP or FP?<br/>suppress FP with reason<br/>escalate TP]
    end
    TRIAGE -->|suppressed| SUPPLOG[(suppression log<br/>5% human sample audit)]
    TRIAGE -->|escalated TP| AUTOFIX
    subgraph AUTOFIX["Autofix"]
        A[LLM generates patch<br/>opens PR on isolated branch]
    end
    AUTOFIX --> REVIEW
    subgraph REVIEW["Human Review Gate"]
        R[human reviews diff<br/>tests must pass<br/>human merges or rejects]
    end
    REVIEW -->|merge| FIXED([finding fixed])
    REVIEW -->|reject| BACK([back to triage / manual])

    style TRIAGE fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style AUTOFIX fill:#0d1b2a,stroke:#5eead4,color:#5eead4
    style REVIEW fill:#1a1015,stroke:#5eead4,color:#e4e4e8
    style SUPPLOG fill:#1a1015,stroke:#5eead4,color:#e4e4e8

Reading: The human appears at two gates, and both are non-optional. Gate one: the suppression log is sampled (5% of LLM suppressions reviewed by a human) to catch the hallucinated dismissal of a true positive. Gate two: every autofix PR is human-reviewed before merge — this is the backstop against both LLM error and prompt-injection-driven malicious patches. The human is not removed from the loop; the loop concentrates human attention on the decisions that require judgment (is this suppression sound? is this patch safe?) and automates the rest.

# Diagrams — SDD-07: Secure Code Review Stack --- ## Diagram 1 — The Five-Layer Pipeline (Architecture) ```mermaid flowchart TB SRC([source code repository]) --> L1 subgraph L1["Layer 1 — Structural (AST)"] AST[tree-sitter / native AST syntax checks · custom structural rules] end L1 --> L2 subgraph L2["Layer 2 — Pattern (Semgrep)"] SG[Semgrep pattern matching over AST known-bad patterns · framework rules runs in seconds · pre-commit friendly] end L2 --> L3 subgraph L3["Layer 3 — Data-Flow (CodeQL)"] CQL[CodeQL compile to QL DB · query taint flows cross-file/cross-function sinks deep · slow · noisy] end L3 --> L4 subgraph L4["Layer 4 — LLM Triage"] LLM[for each finding: TP or FP? why? SUPPRESS false positives WITH REASON escalate true positives never confirms, only suppresses] end L4 --> L5 subgraph L5["Layer 5 — Autofix with Approval"] FIX[LLM generates patch → opens PR → human reviews → merges THE HUMAN IS THE GATE] end style L1 fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style L2 fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style L3 fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style L4 fill:#1a1015,stroke:#5eead4,color:#e4e4e8 style L5 fill:#1a1015,stroke:#5eead4,color:#e4e4e8 ``` **Reading**: The stack is a depth/cost-ordered pipeline. Each layer runs where its cost is justified: AST and Semgrep are fast enough for pre-commit (creation-time guardrails); CodeQL is slow and runs in nightly CI; the LLM triage runs on the findings batch; autofix opens PRs a human reviews. Each layer compensates for the failure mode of the layer before it — Semgrep's shallowness is covered by CodeQL's depth, CodeQL's noise is covered by the LLM triage, the LLM's hallucination is covered by the suppression audit, and autofix's risk is covered by the review gate. The composition is the product. --- ## Diagram 2 — Semgrep vs CodeQL vs LLM Triage (Comparison) ```mermaid flowchart LR subgraph SEMGREP["Semgrep — Pattern"] S1[matches known-bad patterns] S2[fast — seconds] S3[pre-commit friendly] S4[SHALLOW — misses deep flows] end subgraph CODEQL["CodeQL — Data-Flow"] C1[compiles to QL DB, queries taint] C2[slow — minutes, build-bound] C3[nightly CI] C4[DEEP but NOISY — 70-90% FP] end subgraph LLM["LLM Triage — Semantic"] L1[suppresses FP with reason] L2[seconds per finding] L3[on the findings batch] L4[HALLUCINATION — bounded by suppress-only] end SEMGREP -.shallow, covered by.-> CODEQL CODEQL -.noisy, filtered by.-> LLM LLM -.can hallucinate, audited by.-> HUMAN[human suppression audit] style SEMGREP fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style CODEQL fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style LLM fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style HUMAN fill:#1a1015,stroke:#5eead4,color:#e4e4e8 ``` **Reading**: Each layer has a strength and a failure mode, and each layer's failure mode is the next layer's job. Semgrep is fast but shallow — covered by CodeQL. CodeQL is deep but noisy — filtered by the LLM. The LLM can hallucinate — bounded by the suppress-only constraint and audited by a human. No layer is sufficient alone; the chain of compensations is the design. --- ## Diagram 3 — The False-Positive Economy (Why the LLM Triage Exists) ```mermaid flowchart TB subgraph WITHOUT["Without LLM Triage"] W1[CodeQL: 1000 findings] --> W2[~70-90% false positives] W2 --> W3[human must review all 1000] W3 --> W4[result: reviewer tunes queries aggressively OR abandons tool] end subgraph WITH["With LLM Triage"] T1[CodeQL: 1000 findings] --> T2[LLM suppresses ~700-900 FP WITH REASON] T2 --> T3[human reviews ~100-300 true candidates] T3 --> T4[result: reviewer sees signal, tool stays usable] T2 -.suppressions logged.-> AUDIT[5% sample audit for hallucination] end WITHOUT -.the economic case.-> WITH style WITHOUT fill:#1a1015,stroke:#9494a0,color:#9494a0 style WITH fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style AUDIT fill:#1a1015,stroke:#5eead4,color:#e4e4e8 ``` **Reading**: The single highest-leverage addition to the stack is the LLM triage, and the reason is economics. CodeQL's depth produces a false-positive rate that makes the tool unusable in practice — reviewers either tune the queries so aggressively that real bugs pass, or abandon the tool. The LLM suppresses the obvious false positives at cents per finding, leaving the human with the candidates that matter. The constraint that makes this safe is suppress-with-reason-never-confirm: the LLM can only reduce the queue, and every suppression is logged and sampled for hallucination audit. --- ## Diagram 4 — Where the Human Goes (The Double Gate) ```mermaid flowchart TB FINDINGS[(deterministic findings AST + Semgrep + CodeQL)] --> TRIAGE subgraph TRIAGE["LLM Triage"] T[per finding: TP or FP? suppress FP with reason escalate TP] end TRIAGE -->|suppressed| SUPPLOG[(suppression log 5% human sample audit)] TRIAGE -->|escalated TP| AUTOFIX subgraph AUTOFIX["Autofix"] A[LLM generates patch opens PR on isolated branch] end AUTOFIX --> REVIEW subgraph REVIEW["Human Review Gate"] R[human reviews diff tests must pass human merges or rejects] end REVIEW -->|merge| FIXED([finding fixed]) REVIEW -->|reject| BACK([back to triage / manual]) style TRIAGE fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style AUTOFIX fill:#0d1b2a,stroke:#5eead4,color:#5eead4 style REVIEW fill:#1a1015,stroke:#5eead4,color:#e4e4e8 style SUPPLOG fill:#1a1015,stroke:#5eead4,color:#e4e4e8 ``` **Reading**: The human appears at two gates, and both are non-optional. Gate one: the suppression log is sampled (5% of LLM suppressions reviewed by a human) to catch the hallucinated dismissal of a true positive. Gate two: every autofix PR is human-reviewed before merge — this is the backstop against both LLM error and prompt-injection-driven malicious patches. The human is not removed from the loop; the loop concentrates human attention on the decisions that require judgment (is this suppression sound? is this patch safe?) and automates the rest.