Summary
When OpenAIBackend is pointed at a non-OpenAI endpoint (vLLM, Ollama, LiteLLM, OpenRouter with non-OpenAI models, etc.) and format= is passed, it logs this message on every inference call:
"Mellea assumes you are NOT using the OpenAI platform, and that other model providers have less strict requirements on supporting JSON schemas passed into format=. If you encounter a server-side error following this message, then you found an exception to this assumption. Please open an issue..."
With @generative functions the schema is passed on every call, so every call generates this line. In a loop of 100 calls against a vLLM endpoint the log fills up with 100 identical lines.
Location
mellea/backends/openai.py ~line 936 — inside the generate() hot path, inside the _format is not None / else branch.
Expected behaviour
The message should appear once per backend instance (at construction time, or on first use) or be dropped to DEBUG level. It is an internal assumption note, not an actionable warning for the user. The only time it is actionable is when a server error actually follows it — at which point the user already has a stack trace.
Proposed fix
Either:
- Move the log call into
__init__ when _server_type != _ServerType.OPENAI, so it fires once, or
- Change
MelleaLogger.get_logger().info(...) → .debug(...) so it does not appear at the default INFO level.
Option 2 is a one-line fix. Option 1 is slightly more correct since it ties the message to backend setup rather than the generate loop.
Related
Summary
When
OpenAIBackendis pointed at a non-OpenAI endpoint (vLLM, Ollama, LiteLLM, OpenRouter with non-OpenAI models, etc.) andformat=is passed, it logs this message on every inference call:With
@generativefunctions the schema is passed on every call, so every call generates this line. In a loop of 100 calls against a vLLM endpoint the log fills up with 100 identical lines.Location
mellea/backends/openai.py~line 936 — inside thegenerate()hot path, inside the_format is not None/elsebranch.Expected behaviour
The message should appear once per backend instance (at construction time, or on first use) or be dropped to
DEBUGlevel. It is an internal assumption note, not an actionable warning for the user. The only time it is actionable is when a server error actually follows it — at which point the user already has a stack trace.Proposed fix
Either:
__init__when_server_type != _ServerType.OPENAI, so it fires once, orMelleaLogger.get_logger().info(...)→.debug(...)so it does not appear at the defaultINFOlevel.Option 2 is a one-line fix. Option 1 is slightly more correct since it ties the message to backend setup rather than the generate loop.
Related
@generativewith OpenAI-compatible proxies (e.g. OpenRouter) fails with 400: response_format schema missingadditionalProperties: false#1491 covers the case where a proxy does terminate on the OpenAI platform and the assumption turns out to be wrong — that is the error case this log is trying to flag. Fixing the chattiness here does not affect that issue.