Skip to content

docs(generators): document OpenAI token limits#11796

Open
camgrimsec wants to merge 2 commits into
deepset-ai:mainfrom
camgrimsec:docs/openai-token-limit-guidance
Open

docs(generators): document OpenAI token limits#11796
camgrimsec wants to merge 2 commits into
deepset-ai:mainfrom
camgrimsec:docs/openai-token-limit-guidance

Conversation

@camgrimsec

Copy link
Copy Markdown
Contributor

Related Issues

  • None

Proposed Changes:

Adds production cost-control guidance to the OpenAI and Azure OpenAI generator docs.

The new notes recommend setting max_completion_tokens through generation_kwargs to bound response length, latency, and API spend. This is docs-only and does not change runtime behavior.

Pages updated:

  • OpenAIChatGenerator
  • OpenAIGenerator
  • AzureOpenAIChatGenerator
  • AzureOpenAIGenerator

How did you test it?

  • git diff --check
    • passed

I also attempted a local docs dependency install with npm ci under docs-website/. It did not complete locally:

  • with Node 26/npm 11: failed because package.json and package-lock.json were out of sync for @emnapi/*
  • retried with Node 22/npm 10: timed out before node_modules was created

No code paths changed.

Notes for the reviewer

This intentionally documents the operational setting rather than changing generator defaults or adding runtime warnings. Users who need larger completions remain unaffected.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@camgrimsec camgrimsec requested a review from a team as a code owner June 27, 2026 21:23
@camgrimsec camgrimsec requested review from julian-risch and removed request for a team June 27, 2026 21:23
@vercel

vercel Bot commented Jun 27, 2026

Copy link
Copy Markdown

@camgrimsec is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant