answer-engine is a RAG and tool-calling backend. Its main security boundary is
not "the model follows instructions"; retrieved text, user input, and tool
outputs are all untrusted data.
Documents added to the store may contain malicious instructions such as "ignore previous directions", requests to reveal secrets, or attempts to trigger tools. Treat retrieved chunks as evidence only. They must not be allowed to override system prompts, application policy, tool authorization rules, or caller permissions.
Callers embedding answer-engine should keep secrets out of indexed documents
and should apply normal ingestion controls for private corpora. The engine
returns citations and grounding metadata so downstream applications can inspect
which chunks influenced an answer.
Tools are registered explicitly through ToolRegistry. Do not expose privileged
actions just because a model or retrieved document asks for them. Tool handlers
should validate their own inputs, enforce caller authorization, and avoid using
retrieved text as executable instructions.
The built-in calculator is intentionally limited to arithmetic via an AST walk; it rejects names, calls, attributes, imports, and other Python constructs. New tools should follow the same pattern: small surface area, explicit parameters, and no ambient authority.
Citation markers are not proof. A model can attach a citation to a claim that is
missing from, or contradicted by, the cited chunk. answer-engine runs grounding
verification after retrieval answers and reports unsupported, uncited, and
dangling-citation claims.
An answer that is not supported by its citations must be flagged as lower trust,
not returned as confident. The trustworthy field and grounding report are
part of the safety contract for callers; production integrations should use
them to reject, retry, review, or visually de-emphasize weakly grounded answers.
The current support check is lexical overlap, not entailment. It is designed to catch wrong or missing citations, while making no claim that it proves truth.
This repository is pre-1.0. Security fixes are expected on the active main
branch unless a release branch is explicitly announced.
Please report vulnerabilities privately through GitHub Security Advisories for this repository. Do not open a public issue for a suspected security problem.
Include:
- A concise description of the issue and impact.
- Steps to reproduce, including documents, prompts, or tool definitions.
- Whether the issue involves prompt injection, tool authorization, grounding verification, credential handling, or an API/MCP surface.
We will acknowledge the report, investigate privately, and coordinate disclosure once a fix or mitigation is available.