Skip to content

test: guard ToolInvoker imports so chat-generator tests run under Haystack 3.0#3535

Merged
julian-risch merged 2 commits into
mainfrom
test/guard-toolinvoker-imports
Jul 3, 2026
Merged

test: guard ToolInvoker imports so chat-generator tests run under Haystack 3.0#3535
julian-risch merged 2 commits into
mainfrom
test/guard-toolinvoker-imports

Conversation

@julian-risch

@julian-risch julian-risch commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Related Issues

  • Part of deepset-ai/haystack-private#446

Haystack 3.0 removes haystack.components.tools (ToolInvoker has no drop-in replacement; tool execution moved into the Agent). In ten chat-generator test suites, the module-level from haystack.components.tools import ToolInvoker fails. As a consequence, none of the tests are run.

Proposed Changes:

For aimlapi, amazon_bedrock, cohere, cometapi, meta_llama, mistral, ollama, openrouter, orcarouter, togetherai:

  • Wrap the import in try/except ImportError with a ToolInvoker = None fallback (as prototyped in try lifecycle Haystack branch #3467).
  • Add @pytest.mark.skipif(ToolInvoker is None, ...) to the tests that actually use ToolInvoker (one pipeline/live test per suite; two in ollama).

How did you test it?

  • Haystack 2.x: unit tests pass.
  • Haystack v3 branch (installed git+https://github.com/deepset-ai/haystack.git@v3 into the test envs): all ten suites now collect and run. The failures this unmasks belong to the other known v3-incompatibility categories (lazy OpenAI client init and serialization changes).

The failing tests are predominantly test_init_*/test_from_dict_fail_wo_env_var (v3's lazy OpenAI client init — no client, and no API-key error, at __init__ anymore), test_serde_in_pipeline/test_to_dict (v3 serialization format changes), and the async test_run_async* suites in mistral/openrouter (same lazy-client root cause). Those are tracked separately in the compatibility effort.

cohere's chat_generator.py also mentions haystack.components.tools in its docstring usage example — that is not a runtime import and is left for a follow-up that reworks the example for 3.0.

Notes for the reviewer

The skipped-on-3.0 tests are integration/live pipeline tests that chain generator → ToolInvoker. Once the 3.0 migration path for tool invocation in these examples is decided (e.g. Agent), they can be rewritten instead of skipped.

Checklist

🤖 Generated with Claude Code

@julian-risch @claude
test: guard ToolInvoker imports for Haystack 3.0 compatibility
ed78fad 
Haystack 3.0 removes haystack.components.tools, so the module-level
ToolInvoker import fails at test collection and hides every other test
result in these chat-generator suites. Fall back to None on ImportError
and skip the pipeline tests that need ToolInvoker.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (togetherai)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (orcarouter)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (aimlapi)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (openrouter)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (cometapi)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (mistral)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (meta_llama)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Coverage report (cohere)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report (amazon_bedrock)

This PR does not seem to contain any modification to coverable code.

@github-actions

github-actions Bot commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Coverage report (ollama)

This PR does not seem to contain any modification to coverable code.

@julian-risch julian-risch mentioned this pull request Jul 2, 2026
Open
julian-risch added a commit that referenced this pull request Jul 2, 2026
@julian-risch @claude
test: trust test modules in seven more chat-generator suites
7d34f94 
Their serde-in-pipeline tests round-trip tools whose functions are
defined in the test modules, so Pipeline.loads hits the Haystack 3.0
trusted-module allowlist once #3533 fixes the dict comparison and #3535
lets the suites collect.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 2, 2026
Merged
Merged
@julian-risch julian-risch changed the title test: guard ToolInvoker imports so chat-generator suites collect under Haystack 3.0 test: guard ToolInvoker imports so chat-generator tests run under Haystack 3.0 Jul 2, 2026
@julian-risch julian-risch marked this pull request as ready for review July 2, 2026 15:10
@julian-risch julian-risch requested a review from a team as a code owner July 2, 2026 15:10
@julian-risch julian-risch requested review from bogdankostic and removed request for a team July 2, 2026 15:10
bogdankostic
bogdankostic approved these changes Jul 3, 2026
View reviewed changes

@bogdankostic bogdankostic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, failing orcarouter tests are unrelated.

@julian-risch julian-risch merged commit 7f469ff into main Jul 3, 2026
78 of 84 checks passed
@julian-risch julian-risch deleted the test/guard-toolinvoker-imports branch July 3, 2026 08:40
sjrl
sjrl reviewed Jul 3, 2026
View reviewed changes
Comment thread integrations/aimlapi/tests/test_aimlapi_chat_generator.py
Comment on lines +422 to 423
@pytest.mark.skipif(ToolInvoker is None, reason="ToolInvoker is not available in the installed haystack-ai version")
def test_pipeline_with_aimlapi_chat_generator(self, tools):

@sjrl sjrl Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@julian-risch I think it would have been better to replace this pipeline with one that uses an Agent such that it would run in both Haystack 2 and 3

@julian-risch julian-risch Jul 3, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a follow up issue: https://github.com/deepset-ai/haystack-private/issues/452

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sjrl sjrl sjrl left review comments
@bogdankostic bogdankostic bogdankostic approved these changes

Assignees

No one assigned

Labels

integration:aimlapi integration:amazon-bedrock integration:cohere integration:cometapi integration:meta-llama integration:mistral integration:ollama integration:openrouter integration:orcarouter integration:togetherai

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@julian-risch @sjrl @bogdankostic