Skip to content

Feat: Add MLflow Analytics provider#6266

Open
WeichenXu123 wants to merge 10 commits intoFlowiseAI:mainfrom
WeichenXu123:mlflow-tracing
Open

Feat: Add MLflow Analytics provider#6266
WeichenXu123 wants to merge 10 commits intoFlowiseAI:mainfrom
WeichenXu123:mlflow-tracing

Conversation

@WeichenXu123
Copy link
Copy Markdown

Summary

Adds MLflow as an analytics/tracing provider in Flowise, allowing users to send LLM traces to an MLflow tracking server. Traces are logged as MLflow spans with proper parent-child hierarchy (CHAIN → LLM/TOOL), token usage, and model metadata.

Uses the official https://www.npmjs.com/package/@mlflow/core TypeScript SDK (^0.2.0).

Motivation

Flowise already supports analytics providers like LangSmith, Langfuse, LunaryAI, and Opik, but lacked support for the popular MLflow tracing provider.

Provider setup

130f0577-363c-4592-b31f-c9b84dadcecd

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Copilot AI review requested due to automatic review settings April 22, 2026 11:46
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates MLflow as an analytic provider, adding credential definitions, UI assets, and tracing logic within the AnalyticHandler for chains, LLMs, and tools. The review feedback highlights a critical concurrency issue where the global initialization of MLflow could lead to configuration leaks between requests. Additionally, several bugs were identified where span maps are overwritten rather than merged, potentially causing data loss in complex execution flows, and a recommendation was made to improve type safety for configuration options.

Comment thread packages/components/src/handler.ts Outdated
if (mlflowUsername) mlflowOptions.trackingServerUsername = mlflowUsername
if (mlflowPassword) mlflowOptions.trackingServerPassword = mlflowPassword

mlflow.init(mlflowOptions)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The mlflow.init() function configures a global singleton in the MLflow SDK. Since AnalyticHandler instances are created per chatId, concurrent requests with different MLflow configurations (e.g., different tracking URIs or credentials) will overwrite each other's settings. This can lead to traces being sent to the wrong tracking server in a multi-user environment. If the MLflow SDK supports it, consider using a client-based approach to create independent instances instead of global initialization.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the MLflow SDK supports it, consider using a client-based approach to create independent instances instead of global initialization.

Not supported for now.

I added check in code to prevent this case concurrent requests with different MLflow configurations (e.g., different tracking URIs or credentials) will overwrite each other's settings .

Comment thread packages/components/src/handler.ts Outdated
Comment thread packages/components/src/handler.ts Outdated
Comment thread packages/components/src/handler.ts Outdated
Comment thread packages/components/src/handler.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds MLflow as a new analytics/tracing provider alongside existing providers (LangSmith/Langfuse/Lunary/LangWatch/Arize/Phoenix/Opik), enabling Flowise to emit chain/LLM/tool traces to an MLflow tracking server via the @mlflow/core SDK.

Changes:

  • Add @mlflow/core dependency and lockfile updates for MLflow + transitive OpenTelemetry/Databricks SDK packages.
  • Add MLflow provider configuration to the UI analytics settings (including optional experiment ID).
  • Implement MLflow span lifecycle wiring in AnalyticHandler (CHAIN/LLM/TOOL spans + flush on shutdown) and add an MLflow credential definition.

Reviewed changes

Copilot reviewed 4 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pnpm-lock.yaml Adds @mlflow/core and transitive dependencies needed for MLflow tracing
packages/components/package.json Declares @mlflow/core dependency for components package
packages/components/src/handler.ts Implements MLflow provider init + CHAIN/LLM/TOOL span start/end/error handling
packages/components/credentials/MLflowApi.credential.ts Adds credential schema for MLflow tracking server connection/auth
packages/ui/src/ui-component/extended/AnalyseFlow.jsx Adds MLflow as an analytics provider option in the UI
packages/ui/src/assets/images/mlflow.svg Adds MLflow icon asset
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pnpm-lock.yaml Outdated
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants