Skip to content

Python: Add a HarnessAgent with available features and sample#6041

Open
westey-m wants to merge 12 commits into
microsoft:mainfrom
westey-m:harness-agent-python
Open

Python: Add a HarnessAgent with available features and sample#6041
westey-m wants to merge 12 commits into
microsoft:mainfrom
westey-m:harness-agent-python

Conversation

@westey-m
Copy link
Copy Markdown
Contributor

Motivation and Context

#6034

Description

  • Add a HarnessAgent class with available features
  • Add a basic sample to demonstrate usage (better console to follow)
  • Add a compaction strategy based on context window size

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings May 22, 2026 15:55
@moonbox3 moonbox3 added documentation Improvements or additions to documentation python labels May 22, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Python HarnessAgent convenience wrapper (with default context providers and compaction), introduces a context-window–based compaction strategy, and provides a sample demonstrating the bundled agent pipeline.

Changes:

  • Introduce HarnessAgent (experimental) that assembles history + compaction + todo/mode/memory/skills providers and optionally telemetry.
  • Add ContextWindowCompactionStrategy plus tests covering threshold-triggered tool eviction and truncation.
  • Add a HarnessAgent sample (README + interactive research loop) and minor formatting/import tweaks elsewhere.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/samples/02-agents/harness/README.md Adds documentation for the HarnessAgent sample and feature overview.
python/samples/02-agents/harness/harness_research.py Adds an interactive research assistant sample using HarnessAgent + Foundry web search.
python/packages/foundry_hosting/tests/test_responses.py Minor whitespace adjustment in tests.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py Import cleanup/reordering.
python/packages/core/tests/core/test_harness_agent.py Adds unit tests validating HarnessAgent assembly and options.
python/packages/core/tests/core/test_compaction.py Adds unit tests for ContextWindowCompactionStrategy.
python/packages/core/agent_framework/_harness/_agent.py Implements HarnessAgent plus assembly helpers and defaults.
python/packages/core/agent_framework/_compaction.py Adds ContextWindowCompactionStrategy and exports it.
python/packages/core/agent_framework/init.py Exports HarnessAgent, DEFAULT_HARNESS_INSTRUCTIONS, and ContextWindowCompactionStrategy.
.devcontainer/devcontainer.json Updates the Python devcontainer image/features and adds VS Code extensions.

Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/samples/02-agents/harness/harness_research.py Outdated
Comment thread python/packages/core/tests/core/test_harness_agent.py Outdated
Comment thread python/packages/core/tests/core/test_compaction.py
@github-actions github-actions Bot changed the title Add a HarnessAgent with available features and sample Python: Add a HarnessAgent with available features and sample May 22, 2026
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 88%

✓ Correctness

The core logic of ContextWindowCompactionStrategy and HarnessAgent is correct. The TokenBudgetComposedStrategy properly re-annotates tokens between phases, and the two-phase compaction pipeline works as designed. The main new issue is a duplicate from pathlib import Path import introduced in _responses.py, which is clearly a merge/editing artifact. The previously-flagged issues (_assemble_instructions empty-string fallback, sample docstring placement, test coverage for disable_memory, and test docstring/assertion mismatch) remain valid and unresolved.

✓ Security Reliability

No significant security or reliability issues found. The PR introduces proper input validation in both ContextWindowCompactionStrategy and HarnessAgent constructors, correctly bounds threshold parameters, and delegates to well-tested existing compaction infrastructure. The only concrete code issue is a duplicate import introduced in _responses.py, which is harmless at runtime but indicates an incomplete merge.

✓ Test Coverage

The PR adds a HarnessAgent class and ContextWindowCompactionStrategy with good assembly/construction test coverage, but the primary runtime API method HarnessAgent.run() has zero test coverage. The existing tests verify assembly and configuration but never execute the agent's main behavior. Additionally, get_session() is untested and there is no test verifying that both compaction phases (tool eviction + truncation) fire together in a single call.

✓ Design Approach

The main design issue I found is in the new HarnessAgent sample: it instructs the model to save its final report to file memory, but the sample never configures a memory store, so the HarnessAgent assembly path never adds MemoryContextProvider. That leaves the sample advertising a capability it does not actually expose.


Automated review by westey-m's agents

Comment thread python/packages/core/tests/core/test_harness_agent.py
@moonbox3
Copy link
Copy Markdown
Contributor

moonbox3 commented May 22, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework
   _compaction.py6645591%104, 158–159, 177–178, 193–196, 203–205, 229, 243, 250, 265–266, 320, 327, 343, 393, 417, 447, 494, 500, 502, 521, 565–570, 582, 654, 656, 671, 715, 777, 899, 960, 962, 978, 985, 990, 1005, 1013, 1107, 1110, 1128, 1144, 1240, 1267, 1272, 1351
packages/core/agent_framework/_harness
   _agent.py70494%129, 323–324, 326
packages/foundry_hosting/agent_framework_foundry_hosting
   _responses.py68410584%183–186, 251, 328–329, 339, 376, 431, 445, 497, 500–504, 523, 526, 532, 534, 555–557, 586–588, 593, 595, 602, 604–605, 607, 609, 615–618, 620–622, 626, 629, 634–640, 643–644, 646–647, 655–660, 939, 952, 1418–1420, 1422, 1469–1470, 1472–1473, 1475–1476, 1478–1479, 1484, 1493, 1496–1498, 1500, 1514, 1527, 1572–1573, 1575, 1579–1583, 1585, 1592–1593, 1595–1596, 1602, 1604–1608, 1615, 1621, 1643, 1649
TOTAL36397431288% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
7273 34 💤 0 ❌ 0 🔥 1m 53s ⏱️

Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
westey-m and others added 4 commits May 26, 2026 09:49
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_agent.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants