Skip to content

test: establish Phase 0 context migration baselines#3442

Open
JasonW404 wants to merge 1 commit into
developfrom
test/context-phase-0-baseline
Open

test: establish Phase 0 context migration baselines#3442
JasonW404 wants to merge 1 commit into
developfrom
test/context-phase-0-baseline

Conversation

@JasonW404

Copy link
Copy Markdown
Member

Summary

  • freeze legacy Jinja2 and managed ContextManager prompt/message snapshots for zh/en, empty/full sources, tools, builtin tools, skills, memory, KB, managed/external agents, and special characters
  • capture speed/full identity, same-tenant multi-user, cross-tenant, database filter, and ContextManager cache isolation baselines
  • freeze current context budget defaults and compression trigger behavior
  • production behavior is unchanged

Behavior and risk

This PR intentionally records current behavior, including known gaps: conversation and KB lookup lack an explicit tenant parameter, and ContextManager caching is keyed only by conversation ID. These assertions are migration guards for later phases, not endorsements of those behaviors.

Risk is limited to test maintenance. Rollback is removal of the four added baseline files.

Local UT

  • backend/.venv/bin/pytest -q test/backend/context_baseline test/sdk/core/agents/test_agent_context/unit/test_phase0_budget_baseline.py — 22 passed
  • standard runner on the three added test files — 22 passed, 100%
  • test/backend/utils/test_context_utils.py test/backend/utils/test_context_component_types.py — 179 passed
  • test/sdk/core/agents/test_agent_context — 243 passed
  • relevant agent creation, conversation, and run manager suites — 150 passed
  • Ruff and git diff --check passed

The repository-wide runner was also attempted with its default 20 file workers. The added Phase 0 files passed, while unrelated backend app files hit the runner's 600-second per-file timeout under resource contention; no Phase 0 assertion failed.

Functional verification

SDK / real model / Langfuse

Model: qwen3.6-plus through the configured DashScope-compatible endpoint.

  • speed: output SPEED_OK; Langfuse trace 012f22b87747bb0bf91e638f2d3f1881
  • full: final context included system + memory user + KB user messages; output FULL_OK; Langfuse trace 238080b9272f63f17c55e45ef582ecb4

Traces were inspected for model input/output, final message ordering, usage, latency, and errors.

Backend API / logs

  • speed /api/agent/run: SSE completed with final_answer=API_SPEED_OK
  • full /api/agent/run with signed local JWT: identity resolved to user_id/tenant_id; SSE completed with final_answer=API_FULL_OK
  • logs confirmed conversation creation, identity/tenant resolution, memory lookup, context component assembly, ContextManager creation, real model input, token accounting, and cleanup

No frontend production code is changed, so browser validation is not applicable.

Follow-up

Phase 1 will use these guards to tighten the backend-to-SDK authorized snapshot boundary and fix tenant/user/cache isolation without silently changing prompt content or ordering.

Copilot AI review requested due to automatic review settings July 16, 2026 10:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants