Skip to content

Commit 95bd475

Browse files
committed
feat(paper): implementation
1 parent 2ed17ea commit 95bd475

68 files changed

Lines changed: 5295 additions & 29 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,3 +244,61 @@ credentials.token = "${GITHUB_TOKEN}"
244244
- **findings.json**: Complete vulnerability data with all 16 fields
245245
- **report.html**: Visual report with severity colors, code snippets, AI summary
246246
- **report.sarif**: SARIF format for CI/CD integration
247+
248+
## Research-Backed Design
249+
250+
BACO's LLM integration is informed by a survey of 30 papers and 7 projects from
251+
[Awesome-LLMs-for-Vulnerability-Detection](https://github.com/huhusmang/Awesome-LLMs-for-Vulnerability-Detection).
252+
The full analysis and integration roadmap live in [`docs/papers-integration-analysis.md`](docs/papers-integration-analysis.md)
253+
and the task list in [`todo.md`](todo.md).
254+
255+
### Integrated Papers
256+
257+
#### Tier 1 — Quick wins (days → 2 weeks)
258+
259+
| Task | Paper | arxiv | baco target |
260+
|------|-------|-------|------------|
261+
| Agentic FP filter | Sifting the Noise (ISSTA 2026) | [2601.22952](https://arxiv.org/abs/2601.22952) | `src/llm_verification.rs` + `src/confidence_refinement.rs` |
262+
| Hierarchical context extraction | Context-Enhanced Vuln Detection | [2504.16877](https://arxiv.org/abs/2504.16877) | new `src/context/` module |
263+
| CWE KB RAG (BM25, no vectors) | VulInstruct (FSE 2026) | [2511.04014](https://arxiv.org/abs/2511.04014) | new `src/retrieval/` module |
264+
| Regression suite | SV-TrustEval-C (SP 2025) | [2505.20630](https://arxiv.org/abs/2505.20630) | `tests/integration/` |
265+
266+
#### Tier 2 — Structural improvements (2 → 6 weeks)
267+
268+
| Task | Paper | arxiv/code | baco target |
269+
|------|-------|-------------|------------|
270+
| MoE per-CWE/language routing | MoEVD (FSE 2025) | [2501.16454](https://arxiv.org/abs/2501.16454) | new `src/router/` module |
271+
| Triple-path context (AST/CFG/DFG) | VulTriage | [2605.09461](https://arxiv.org/abs/2605.09461) | `src/context/` extension |
272+
| LLM→semgrep rule synthesis | MoCQ | [2504.16057](https://arxiv.org/abs/2504.16057) | new `src/rulesynth/` phase |
273+
| Global FP suppression | AutoCVE | [code](https://github.com/larlarua/AutoCVE) | `src/findings.rs` + `src/root_cause_dedup.rs` |
274+
| Six-phase parallel orchestration | Cloudflare security-audit-skill | [code](https://github.com/cloudflare/security-audit-skill) | `src/scanner/pipeline/orchestrator.rs` |
275+
276+
#### Tier 3 — Heavy infrastructure (months)
277+
278+
| Task | Paper | arxiv/code | baco target |
279+
|------|-------|-------------|------------|
280+
| CPG-guided slicing | LLMxCPG (Usenix 2025) | [2507.16585](https://arxiv.org/abs/2507.16585) | new `src/cpg/` + Joern |
281+
| Adversarial exploit synthesis | QRS | [2602.09774](https://arxiv.org/abs/2602.09774) | new `src/exploit/` + sandbox |
282+
| Specialized reasoning LLM | R2Vul + VULPO | [2504.04699](https://arxiv.org/abs/2504.04699) + [2511.11896](https://arxiv.org/abs/2511.11896) | new `LlmProvider::TgiServed` |
283+
284+
#### Cross-cutting insights
285+
286+
| Insight | Paper | baco application |
287+
|---------|-------|-----------------|
288+
| Rationale validation via LLM-as-judge | CORRECT — [2504.13474](https://arxiv.org/abs/2504.13474) | `src/llm_verification.rs` |
289+
| Statement-level localization | SecVulEval — [2505.19828](https://arxiv.org/abs/2505.19828) | `src/findings.rs` |
290+
| Dataset hygiene (chronological splits) | PrimeVul (ICSE 2025) — [2403.18624](https://arxiv.org/abs/2403.18624) | `tests/fixtures/` |
291+
| Confidence calibration per user study | Closing the Gap (ICSE 2025) — [2412.14306](https://arxiv.org/abs/2412.14306) | `src/confidence_refinement.rs` |
292+
| Semantic Trap guard for fine-tuning | Semantic Trap — [2601.22655](https://arxiv.org/abs/2601.22655) | `docs/fine-tuning-guidelines.md` |
293+
294+
### Surveyed but deferred/skipped
295+
296+
- **AgentFlow** (2604.20801) — Python-only, defer until Rust interop stabilizes
297+
- **AgenticSCR** (2601.19138) — niche pre-commit case
298+
- **VulnLLM-R** (2512.07533) — GPU cluster required
299+
- **OpenAnt** — heavy UniFFI build
300+
- **DeepAudit** — too large/divergent
301+
- **FocusVul** (2505.17460) — paper withdrawn
302+
303+
For the full list of surveyed papers and their verdicts, see
304+
[`docs/papers-integration-analysis.md`](docs/papers-integration-analysis.md).

prompts/phases/llm_static_analysis.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ You are an OFFENSIVE SECURITY RESEARCHER and EXPERT SECURITY REVERSE ENGINEER. Y
88
- **File**: %%FILE_PATH%%
99
- **Lines**: %%LINE_RANGE%%
1010
- **Context**: %%CONTEXT_LINES%% lines before/after
11+
- **CWE Specs**: %%CWE_SPECS%%
1112
- **Code**:
1213
```%%LANGUAGE%%
1314
%%CODE_CONTENT%%

prompts/phases/llm_verification.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,19 @@ Return JSON with format:
2121
"mitigating_factors": ["optional mitigation 1", ...],
2222
"related_patterns": ["optional pattern 1", ...]
2323
}
24+
25+
## Triage Step
26+
27+
When a finding is marked as `NeedsReview`, invoke the triage filter for additional analysis:
28+
29+
1. **Triage Prompt**: Send a zero-shot prompt asking "Is this finding a true positive or false positive?"
30+
2. **Expected Output**: JSON with `{"verdict": "true_positive"|"false_positive", "confidence": 0.0-1.0, "reasoning": "..."}`
31+
3. **Integration**:
32+
- If triage returns `false_positive`: Set status to `FalsePositive`, add reasoning to `verification_notes`
33+
- If triage returns `true_positive`: Keep `Confirmed` status, boost confidence by +0.10
34+
- On parse failure: Fall back gracefully to `NeedsReview` status
35+
36+
## Confidence Refinement Factors
37+
38+
- `TriageTruePositive`: +0.10 boost when triage confirms true positive
39+
- `TriageFalsePositive`: -0.25 penalty when triage identifies false positive
File renamed without changes.

src/confidence_refinement.rs

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
//! - Generates confidence explanations
99
//! - Integrates with AnalysisContext (T5)
1010
11-
use crate::context::AnalysisContext;
11+
use crate::analysis_context::AnalysisContext;
1212
use crate::findings::{VerificationStatus, VulnerabilityFinding};
1313
use std::collections::HashMap;
1414

@@ -55,6 +55,10 @@ pub enum ConfidenceFactor {
5555
TestCodeRelated,
5656
/// Code is in dependency/vendor
5757
ThirdPartyCode,
58+
/// Triage confirmed true positive
59+
TriageTruePositive,
60+
/// Triage identified false positive
61+
TriageFalsePositive,
5862
}
5963

6064
/// Historical data for confidence refinement.
@@ -357,6 +361,21 @@ impl ConfidenceRefinementPhase {
357361
}
358362
}
359363

364+
// Factor 9: Tiage-based adjustments
365+
if let Some(ref notes) = finding.verification_notes {
366+
if notes.contains("triage") || notes.contains("Triage") {
367+
if notes.contains("false_positive") || notes.contains("False positive") {
368+
refined_score = (refined_score - 0.25).max(0.0);
369+
factors.push(ConfidenceFactor::TriageFalsePositive);
370+
explanations.push("Triage identified as false positive".to_string());
371+
} else if notes.contains("true_positive") || notes.contains("True positive") {
372+
refined_score = (refined_score + 0.10).min(1.0);
373+
factors.push(ConfidenceFactor::TriageTruePositive);
374+
explanations.push("Triage confirmed as true positive".to_string());
375+
}
376+
}
377+
}
378+
360379
// Clamp final score
361380
refined_score = refined_score.clamp(0.0, 1.0);
362381

src/config.rs

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ pub struct ScannerConfig {
1919
pub tickets: TicketConfig,
2020
#[serde(default)]
2121
pub agent: AgentConfig,
22+
#[serde(default)]
23+
pub router: RouterConfig,
24+
#[serde(default)]
25+
pub aggregation: AggregationConfig,
2226
}
2327

2428
/// Config error with field path and TOML location information
@@ -417,6 +421,45 @@ fn default_trusted_paths() -> Vec<String> {
417421
vec![".".to_string()]
418422
}
419423

424+
/// Router configuration for MoE per-CWE / per-language routing
425+
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
426+
pub struct RouterConfig {
427+
/// Whether the router is enabled
428+
#[serde(default)]
429+
pub enabled: bool,
430+
/// Default prompt template name
431+
#[serde(default = "default_llm_static_analysis")]
432+
pub default_prompt: String,
433+
/// CWE ID -> PromptSpec overrides
434+
#[serde(default)]
435+
pub cwe_overrides: HashMap<String, PromptSpec>,
436+
/// Language -> PromptSpec overrides
437+
#[serde(default)]
438+
pub language_overrides: HashMap<String, PromptSpec>,
439+
}
440+
441+
/// Prompt specification for router overrides
442+
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
443+
pub struct PromptSpec {
444+
/// The prompt template name to use
445+
#[serde(default = "default_llm_static_analysis")]
446+
pub prompt_template: String,
447+
/// Optional model override for this prompt
448+
pub model_override: Option<String>,
449+
}
450+
451+
fn default_llm_static_analysis() -> String {
452+
"llm_static_analysis".to_string()
453+
}
454+
455+
/// Aggregation configuration including false positive store settings
456+
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
457+
pub struct AggregationConfig {
458+
/// Path to the false positive store JSON file
459+
#[serde(default)]
460+
pub fp_store_path: Option<PathBuf>,
461+
}
462+
420463
fn default_max_parallel_tasks() -> usize {
421464
4
422465
}

src/context/mod.rs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
//! Hierarchical context extraction for LLM analysis.
2+
//!
3+
//! Extracts function/module/project-level summaries to provide
4+
//! structured context to LLM prompts.
5+
6+
mod summary;
7+
8+
pub use summary::{ContextExtractor, ContextSummary, FunctionSummary};

0 commit comments

Comments
 (0)