Skip to content

Add CrewAI instrumentation: LLM inference event listener#224

Draft
venkata-pavani wants to merge 1 commit into
open-telemetry:mainfrom
venkata-pavani:crewai-instrumentation
Draft

Add CrewAI instrumentation: LLM inference event listener#224
venkata-pavani wants to merge 1 commit into
open-telemetry:mainfrom
venkata-pavani:crewai-instrumentation

Conversation

@venkata-pavani

Copy link
Copy Markdown

Ports the LLM-call slice of the donated OpenInference CrewAI instrumentation onto opentelemetry-util-genai: an event-bus listener converts LLMCallStarted/Completed/Failed events into chat inference invocations (usage tokens, response id, finish reasons, content capture).

Tests: unit tests driving the listener with real CrewAI events, a VCR-backed Agent.kickoff() integration test, and a weaver conformance scenario - both replaying the genuinely-recorded inference cassette ported unchanged from
open-telemetry/donation-openinference (commit 6cdd644d).

The conformance tox env overrides crewai's opentelemetry-api pins via UV_OVERRIDE (tests/overrides-conformance.txt) so it can resolve against the git-pinned conformance stack.

Known gap: CrewAI's event bus dispatches handlers on a thread pool, so util-genai's context attach/detach logs a ValueError per LLM call; fix requires an attach_context opt-out in util-genai (separate discussion).

Assisted-by: Claude Fable 5

Description

Please include a summary of the change and which issue, if any, is fixed.
Include the motivation and context for the change.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How has this been tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. List any relevant details for your test
configuration.

  • Test A

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

  • Followed the style guidelines of this project
  • Changelog updated if the change requires an entry
  • Unit tests added
  • Documentation updated

Ports the LLM-call slice of the donated OpenInference CrewAI
instrumentation onto opentelemetry-util-genai: an event-bus listener
converts LLMCallStarted/Completed/Failed events into chat inference
invocations (usage tokens, response id, finish reasons, content
capture).

Tests: unit tests driving the listener with real CrewAI events,
a VCR-backed Agent.kickoff() integration test, and a weaver
conformance scenario - both replaying the genuinely-recorded
inference cassette ported unchanged from
open-telemetry/donation-openinference (commit 6cdd644d).

The conformance tox env overrides crewai's opentelemetry-api pins
via UV_OVERRIDE (tests/overrides-conformance.txt) so it can resolve
against the git-pinned conformance stack.

Known gap: CrewAI's event bus dispatches handlers on a thread pool,
so util-genai's context attach/detach logs a ValueError per LLM call;
fix requires an attach_context opt-out in util-genai (separate
discussion).

Assisted-by: Claude Fable 5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@linux-foundation-easycla

Copy link
Copy Markdown

CLA Missing ID

One or more co-authors of this pull request were not found. You must specify co-authors in commit message trailer via:

Co-authored-by: name <email>

Supported Co-authored-by: formats include:

  1. Anything <id+login@users.noreply.github.com> - it will locate your GitHub user by id part.
  2. Anything <login@users.noreply.github.com> - it will locate your GitHub user by login part.
  3. Anything <public-email> - it will locate your GitHub user by public-email part. Note that this email must be made public on Github.
  4. Anything <other-email> - it will locate your GitHub user by other-email part but only if that email was used before for any other CLA as a main commit author.
  5. login <any-valid-email> - it will locate your GitHub user by login part, note that login part must be at least 3 characters long.

Alternatively, if the co-author should not be included, remove the Co-authored-by: line from the commit message.

Please update your commit message(s) by doing git commit --amend and then git push [--force] and then request re-running CLA check via commenting on this pull request:

/easycla

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant