Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
- **Lifecycle:** Active
- **Authority:** Governing instructions for work in this repository, subordinate
to current explicit user direction and higher-level safety rules
- **Last reviewed:** 25 July 2026
- **Last reviewed:** 27 July 2026
- **Review trigger:** A material change to Von's product focus, authority model,
security posture, or acceptance doctrine

Expand Down Expand Up @@ -114,6 +114,13 @@ never be forced onto newer evidence.
warrant an LLM. Mechanically exact operations may remain deterministic and
observable when that is still the simplest adequate path; their current
encoding is not evidence that the surrounding policy or stage is necessary.
Treat model choice as an evidenced design variable: when a failure plausibly
reflects inadequate model capability, compare a stronger suitable model
before adding durable semantic code unless existing evidence already
distinguishes the cause; when adequate behaviour is too slow or costly,
compare a faster or cheaper model. Diagnose tool, transport, authority, and
context failures before blaming the model, keep comparisons bounded, and do
not turn one successful replay into a permanent routing rule.
7. **Vontology is first-class, not all-consuming.** It is the live authority for
represented concepts, relations, prompts, workflows, and policies. Raw
documents, traces, operational events, caches, and transactional data may
Expand Down Expand Up @@ -266,6 +273,10 @@ hypothetical danger the same veto.
- Low-risk reversible work should normally proceed with provenance and
canonical read-back. A URL or identifier may be strong evidence about the
intended object, but it does not create authority for unrelated effects.
- Read-only is a semantic-effect boundary, not a zero-write storage guarantee.
Bounded derived maintenance is compatible with a read only when it leaves
represented meaning, visibility, and authority unchanged, and any failure is
observable and recoverable.
- If required authority or risk evidence is unavailable, deny only the effect
whose residual risk cannot be justified. Preserve safe reads, bounded
alternatives, partial progress, and a typed explanation instead of failing
Expand Down
18 changes: 14 additions & 4 deletions docs/engineering/prompt_programs_and_model_routing_playbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
- **Kind:** Prompt and model-routing playbook
- **Lifecycle:** Active
- **Authority:** Normative within its stated prompt/model scope
- **Last reviewed:** 25 July 2026
- **Last reviewed:** 27 July 2026

## 1. When to read this

Expand Down Expand Up @@ -149,12 +149,22 @@ not collapsed into an unexplained fatal model badge.
A reasonable default pattern is:

1. try the cheapest adequate model for routine bounded work
2. escalate when confidence, validation, or complexity signals require it
3. use stronger models for difficult synthesis, ambiguous planning, or hard reasoning
4. use narrow fine-tunes when the task is stable and frequent enough to justify them
2. if a miss plausibly reflects model capability, run a bounded comparison
with a stronger suitable model before encoding semantic recovery in code
3. if an adequate route is too slow or expensive, compare a faster or cheaper
model and retain the least costly model that preserves the required outcome
4. use stronger models for difficult synthesis, ambiguous planning, or hard
reasoning, and narrow fine-tunes when a stable frequent task justifies them

Do not hard-code this into many helpers. Centralise it.

Model escalation is not a generic retry. Use available evidence to distinguish
model weakness from tool, authority, transport, context, or harness failure,
and collect only the comparison evidence needed under §7A. A stronger model
that finds the right path but cannot finish inside the latency or cost envelope
is evidence, not a production success; a faster model that produces an adequate
answer may be the better route.

## 7A. Proportionate replay evidence

Use replay evidence before materially changing a shared model policy. Match the
Expand Down
16 changes: 9 additions & 7 deletions docs/engineering/real_path_server_replay_and_telemetry_loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,9 +268,11 @@ Practical rules:
4. Before running a sampled prompt used as acceptance evidence, record the
expectation-first preflight when Tier 2/3 or subjective judgement warrants
it.
5. Treat the harness verdict such as `should_user_be_happy` as supporting
evidence, not as the whole judgement. Compare the real answer and telemetry
against your recorded expectation.
5. The sampler does not issue a semantic verdict. Its collection status says
whether the run evidence was captured, not whether the answer was good.
Compare the real answer and, where relevant, effects, model, timing, and
telemetry against the user job, or use an explicit evaluator when the
experiment genuinely needs one.
6. For operational prompts that are really asking Von to mutate or inspect its
own task/message state, record whether actual task/message tool use happened.
A generic "I can help with that" answer is not a pass for a create/update
Expand Down Expand Up @@ -383,12 +385,12 @@ sampler harness when it fits the task:
- `scripts/run_live_kb_tool_prompt_sampler.py`

It already creates a fresh authenticated conversation, captures the response,
resolves history location, fetches persisted debug telemetry, and emits a
conservative verdict. It defaults to the `JVNAUTOSCI-2070` isolated agent-test
resolves history location, fetches persisted debug telemetry, and emits an
unscored collection record. It defaults to the `JVNAUTOSCI-2070` isolated agent-test
backend and will reject a non-agent-test server unless you pass
`--allow-non-agent-test-server` for an explicitly interactive-server check.
Keep in mind that its verdict complements rather than replaces the
expectation-first human judgement.
Collection success means the evidence was captured; assess the user outcome
separately.

The sampler also supports controlled multi-arm model comparison. Use repeated
`--compare-model` flags, and optionally `--include-active-model-arm`, when you
Expand Down
24 changes: 17 additions & 7 deletions docs/engineering/security_considerations.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
- **Kind:** Security guidance with dated deployment-posture observations
- **Lifecycle:** Active
- **Authority:** Canonical security guidance routed by [`AGENTS.md`](../../AGENTS.md)
- **Last reviewed:** 25 July 2026
- **Last reviewed:** 27 July 2026
- **Evidence boundary:** Statements about current users, deployments, and
implemented controls are dated observations and must be revalidated; the
security requirements do not expire merely because implementation evidence
Expand Down Expand Up @@ -233,14 +233,19 @@ Current implementation details:

### 4. MCP Tool Access Control and Agentic-AI Threats

**Current**: Ordinary adaptive turns receive a standing read-only projection
of internal MCP capabilities. The gateway binds trusted actor and namespace
context outside model arguments. Gmail reads are projected only when a
represented actor-to-profile relation authorises a configured profile, which
the entry point injects without letting the model choose it.
**Current as of 27 July 2026**: When internal MCP is enabled, authenticated
ordinary turns receive a bounded additive representation aperture. Trusted
actor and namespace values are server-bound; creation scope is fixed; effects
on existing subjects require actor- or organisation-scoped authority; and
ordinary turns cannot change visibility scope. Gmail reads are projected only
when a represented actor-to-profile relation authorises a configured profile,
which the entry point injects without letting the model choose it. Revalidate
the catalogue and adaptive-turn service before relying on this dated
implementation claim.

**Protection**:
- The ordinary-turn projection contains no write-category capability
- The ordinary-turn projection excludes undelegated effects; an operation's
catalogue category alone does not establish authority
- Gateway handlers distinguish trusted ambient actor context from raw
tool-payload identity claims
- Actor-private RAG, chat, turn, experiment, and critique-memory reads validate
Expand Down Expand Up @@ -276,6 +281,8 @@ the entry point injects without letting the model choose it.
relevant deployment limits of tools when those facts help the model and
runtime choose proportionate assurance. Avoid a universal verb-based risk
taxonomy.
- Apply the semantic read-only boundary in [`AGENTS.md`](../../AGENTS.md) to
derived maintenance.
- Log enough tool input/output metadata to investigate suspicious tool use while
redacting secrets and private content where required.

Expand Down Expand Up @@ -584,6 +591,9 @@ be added before broader external contribution or partner deployment.

## Change Log

- **2026-07-27**: Distinguished logical read semantics from physical write
purity and documented the bounded ordinary-turn representation-effect
aperture
- **2026-07-25**: Replaced categorical write/destructive guardrails with
delegated-capability and residual-risk guidance
- Distinguished the actor's maximum capability from semantic action choice
Expand Down
83 changes: 0 additions & 83 deletions scripts/classify_replay_action_outcome.py

This file was deleted.

Loading