From 7ee0bc6a8baa37c5d3c8476dcfb2b5fd57dd04c2 Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Tue, 14 Jul 2026 19:01:19 +0800 Subject: [PATCH 1/8] Add self-hosted model guide for the common.ai provider MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Self-hosted setup (Ollama, vLLM) was only documented as scattered fragments across the connection, hook, and retry-policy pages, with no single place explaining the base_url connection pattern, when to use the ollama: vs openai: model-id prefix, or the base_url/api_base naming difference across hooks — leaving Dag authors to piece it together from source. --- providers/common/ai/docs/index.rst | 1 + providers/common/ai/docs/retry_policies.rst | 3 +- .../common/ai/docs/self_hosted_models.rst | 135 ++++++++++++++++++ 3 files changed, 138 insertions(+), 1 deletion(-) create mode 100644 providers/common/ai/docs/self_hosted_models.rst diff --git a/providers/common/ai/docs/index.rst b/providers/common/ai/docs/index.rst index 7de5f5a8ac431..c342b58d55486 100644 --- a/providers/common/ai/docs/index.rst +++ b/providers/common/ai/docs/index.rst @@ -134,6 +134,7 @@ See the Optional dependencies table below for the exact package each extra insta Operators Examples Retry Policies + Self-hosted models HITL Review Observability diff --git a/providers/common/ai/docs/retry_policies.rst b/providers/common/ai/docs/retry_policies.rst index 8456b1262aa3f..a24e5df448143 100644 --- a/providers/common/ai/docs/retry_policies.rst +++ b/providers/common/ai/docs/retry_policies.rst @@ -161,7 +161,8 @@ Local LLM support ----------------- For environments where exception data must not leave the infrastructure, point -to a local model via Ollama or vLLM: +to a local model via Ollama or vLLM -- see :ref:`howto/self_hosted_models` for +general self-hosted connection setup: .. code-block:: python diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst new file mode 100644 index 0000000000000..eccf0b8c6d8a5 --- /dev/null +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -0,0 +1,135 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. _howto/self_hosted_models: + +Self-hosted models +=================== + +This guide covers pointing :class:`~airflow.providers.common.ai.hooks.pydantic_ai.PydanticAIHook` +at a self-hosted, OpenAI-compatible endpoint (Ollama, vLLM, or similar) instead +of a cloud provider. + +Why self-hosted +---------------- + +Running a model locally keeps prompt and response data inside your own +infrastructure, avoids per-call cost, and works offline -- the same motivation +already covered for ``LLMRetryPolicy`` in :doc:`retry_policies`. + +Connection setup +----------------- + +A self-hosted endpoint uses the same ``pydanticai`` connection type as any +other provider -- there is no dedicated connection type for Ollama or vLLM. +Set ``host`` to the base URL of the endpoint and set the model in the +``extra`` JSON. Leave ``password`` empty if the endpoint doesn't check for a +key: pydantic-ai fills in a placeholder API key itself whenever ``host`` is +set, no key was passed, and no provider-specific API-key environment variable +is set -- this applies to both the ``openai`` and ``ollama`` provider classes. +See :ref:`howto/connection:pydanticai` for the full field reference; only the +fields that differ for a self-hosted setup are called out below. + +Model identifier format +------------------------- + +The ``extra`` JSON's ``model`` value keeps the ``provider:model`` format the +hook always expects, but which ``provider`` prefix to use depends on the +endpoint (checked against the pinned ``pydantic-ai-slim`` 2.5.0): + +- **vLLM** has no dedicated provider class in pydantic-ai -- ``openai:`` + is the only option. pydantic-ai's ``openai`` provider class talks to + whatever ``host`` points at, so it works against vLLM's OpenAI-compatible + API even though the weights being served aren't OpenAI's. +- **Ollama** has two valid options: the dedicated ``ollama:`` prefix + (recommended) applies Ollama-specific model-profile tuning based on the + model name (``llama``, ``gemma``, ``qwen``, ``deepseek``, ``mistral``, + ``command``, and ``gpt-oss`` prefixes get tuned defaults); the generic + ``openai:`` prefix also works, since Ollama exposes an + OpenAI-compatible API too, but skips that tuning. + +Both prefixes need the same ``openai`` extra +(``pip install "apache-airflow-providers-common-ai[openai]"``) -- pydantic-ai's +Ollama provider is built on the same underlying ``openai`` Python package as +its OpenAI provider, and there's no separate ``ollama`` extra. + +Examples +-------- + +Ollama +^^^^^^ + +.. code-block:: json + + { + "conn_type": "pydanticai", + "host": "http://localhost:11434/v1", + "extra": "{\"model\": \"ollama:llama3.2\"}" + } + +The generic ``openai:llama3.2`` form (see :doc:`connections/pydantic_ai`) +also works against the same endpoint; use ``ollama:`` unless you have a +reason to skip the model-profile tuning. + +vLLM +^^^^ + +.. code-block:: json + + { + "conn_type": "pydanticai", + "host": "http://localhost:8000/v1", + "extra": "{\"model\": \"openai:llama3\"}" + } + +vLLM has no dedicated pydantic-ai provider, so ``openai:`` is the +only format to use. + +Cross-hook naming differences +------------------------------- + +:class:`~airflow.providers.common.ai.hooks.langchain.LangChainHook` and +:class:`~airflow.providers.common.ai.hooks.llamaindex.LlamaIndexHook` also +support self-hosted endpoints through the same ``host`` connection field, but +the underlying constructor keyword each hook passes it to is not uniform: + +.. list-table:: + :header-rows: 1 + + * - Hook + - Connection field + - Constructor keyword + * - ``PydanticAIHook`` + - ``host`` + - ``base_url`` + * - ``LangChainHook`` + - ``host`` + - ``base_url`` + * - ``LlamaIndexHook`` + - ``host`` + - ``api_base`` + +Where to go next +------------------ + +- :ref:`howto/connection:pydanticai` -- the full connection field reference. +- :doc:`retry_policies` -- the "Local LLM support" section covers pointing + ``LLMRetryPolicy`` at a self-hosted endpoint. +- pydantic-ai isn't limited to Ollama and vLLM -- any OpenAI-compatible + endpoint works the same way. See the + `pydantic-ai install docs `__ + for the full list of providers it supports. From 099e684ba8b226ba63b1aff93dc3ddefb955ed1b Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Tue, 14 Jul 2026 21:58:19 +0800 Subject: [PATCH 2/8] fixup! Add self-hosted model guide for the common.ai provider --- .../common/ai/docs/self_hosted_models.rst | 230 ++++++++++++++---- 1 file changed, 176 insertions(+), 54 deletions(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index eccf0b8c6d8a5..76d2140b7f459 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -20,32 +20,161 @@ Self-hosted models =================== -This guide covers pointing :class:`~airflow.providers.common.ai.hooks.pydantic_ai.PydanticAIHook` -at a self-hosted, OpenAI-compatible endpoint (Ollama, vLLM, or similar) instead -of a cloud provider. - -Why self-hosted ----------------- - -Running a model locally keeps prompt and response data inside your own -infrastructure, avoids per-call cost, and works offline -- the same motivation -already covered for ``LLMRetryPolicy`` in :doc:`retry_policies`. - -Connection setup ------------------ - -A self-hosted endpoint uses the same ``pydanticai`` connection type as any -other provider -- there is no dedicated connection type for Ollama or vLLM. -Set ``host`` to the base URL of the endpoint and set the model in the -``extra`` JSON. Leave ``password`` empty if the endpoint doesn't check for a -key: pydantic-ai fills in a placeholder API key itself whenever ``host`` is -set, no key was passed, and no provider-specific API-key environment variable -is set -- this applies to both the ``openai`` and ``ollama`` provider classes. -See :ref:`howto/connection:pydanticai` for the full field reference; only the -fields that differ for a self-hosted setup are called out below. +This guide serves an open model on your own infrastructure, points +:class:`~airflow.providers.common.ai.hooks.pydantic_ai.PydanticAIHook` at it +instead of a cloud provider, runs a ``@task.llm`` task and an +``AgentOperator`` with tool calling against it, and then routes the same +setup through a bearer-token AI gateway instead of talking to the server +directly. Running a model locally keeps prompt and response data inside your +own infrastructure, avoids per-call cost, and works offline. + +Before you start +------------------ + +This guide assumes a working :doc:`apache-airflow:installation/index` +(Airflow 3.0+) already exists. Its job stops at wiring Airflow to a server +that's already running -- it doesn't cover installing or operating the +model-serving stack itself. + +Prerequisites +^^^^^^^^^^^^^ + +You need an OpenAI-compatible model server reachable from where Airflow +runs, and hardware able to run the model you pick (check the server's own +docs for GPU/CPU requirements): + +- `Ollama `__ -- the server used throughout this guide. +- `vLLM `__ -- a higher-throughput alternative, + commonly GPU-hosted. +- `LM Studio `__ -- a desktop app with a built-in + OpenAI-compatible server. + +1. Start an OpenAI-compatible server +-------------------------------------- + +Install Ollama and pull a model: + +.. code-block:: bash + + ollama pull gemma3 + +Ollama serves an OpenAI-compatible API at ``http://localhost:11434/v1`` once +running. vLLM and LM Studio plug into the steps below the same way -- only +the base URL and default port differ; see each project's own docs for how to +start it. + +2. Create the connection +-------------------------- + +Every LLM call goes through a Pydantic AI connection (``conn_type`` +``pydanticai``, default connection id ``pydanticai_default``). For a local, +keyless server: + +.. code-block:: bash + + export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma3"}}' + +- ``host`` -- the server's base URL. +- ``password`` -- left empty for a keyless local server. That triggers two + separate things: the Airflow hook forwards no key at all when ``password`` + is empty (only ``base_url`` goes to the provider constructor); pydantic-ai's + provider class then fills in its own placeholder API key and does send + that placeholder in the actual HTTP request. Ollama doesn't check it, so + the request still succeeds either way. +- ``extra["model"]`` -- ``ollama:`` picks up Ollama-specific tuning; + see :ref:`Model identifier format ` below for + when to use ``openai:`` instead. + +See :ref:`howto/connection:pydanticai` for the full connection field +reference. + +3. Run your first Dag with ``@task.llm`` +------------------------------------------- + +The ``@task.llm`` decorator turns a function that returns a prompt string +into a task that sends that prompt to the connection above and returns the +response: + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_quickstart.py + :language: python + :start-after: [START howto_quickstart_llm] + :end-before: [END howto_quickstart_llm] + +Run it like any other Dag (``airflow dags test quickstart_llm``); the +``summarize`` task completes using the local model, not a cloud provider. + +4. Run an agent with tool calling +------------------------------------ + +Tool calling needs a model that supports pydantic-ai's ``tools`` capability +-- not every small local model does, and ``gemma3`` from the previous steps +doesn't. Pull a tools-capable model and point the same connection at it +instead: + +.. code-block:: bash + + ollama pull gemma4 + +.. code-block:: bash + + export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma4"}}' + +``AgentOperator`` can wrap an existing Airflow hook as a tool via +``HookToolset``. This example wraps an HTTP connection named ``my_api`` -- +point it at any reachable HTTP endpoint that returns JSON: + +.. code-block:: bash + + export AIRFLOW_CONN_MY_API='{"conn_type": "http", "host": ""}' + +.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_agent.py + :language: python + :start-after: [START howto_operator_agent_hook] + :end-before: [END howto_operator_agent_hook] + +``HttpHook`` defaults to ``method="POST"`` +(:class:`~airflow.providers.http.hooks.http.HttpHook`), fixed when the hook +is constructed. If ``my_api`` points at a GET-only endpoint, pass +``HttpHook(http_conn_id="my_api", method="GET")`` instead of the bare +constructor shown above. + +Routing through an AI gateway +-------------------------------- + +The same wiring routes through a bearer-token AI gateway instead of talking +to the server directly -- swap three connection fields and nothing else +changes: + +- ``host`` -- the gateway's base URL, with no ``/v1`` suffix; pydantic-ai's + ``openai`` provider appends the API path itself. +- ``password`` -- the bearer token the gateway expects. +- ``extra["model"]`` -- ``openai:``, where ```` is the route + name the gateway exposes for the model you want. + +This is a real constraint, not a gap to be filled later: the hook forwards +only ``api_key`` and ``base_url`` to the provider class +(``_get_provider_kwargs``, `hooks/pydantic_ai.py:101-128`) -- there is no +extension point for custom headers. A gateway that authenticates with a +bearer token works through this connection shape; one that requires +additional custom headers does not. + +You can validate this pattern locally with any OpenAI-compatible proxy that +speaks bearer-token auth -- for example, LiteLLM's proxy mode running in +front of a model server. That's a different role than the ``litellm:`` model +prefix covered in the reference section below: here LiteLLM stands in for +the gateway itself, reached through the generic ``openai:`` prefix, +not addressed via its own model-prefix. + +Reference +---------- + +The rest of this page is reference material for the connection shapes +above. + +.. _self_hosted_models_model_id: Model identifier format -------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ The ``extra`` JSON's ``model`` value keeps the ``provider:model`` format the hook always expects, but which ``provider`` prefix to use depends on the @@ -67,40 +196,28 @@ Both prefixes need the same ``openai`` extra Ollama provider is built on the same underlying ``openai`` Python package as its OpenAI provider, and there's no separate ``ollama`` extra. -Examples --------- - -Ollama -^^^^^^ +.. note:: -.. code-block:: json + ``LiteLLMProvider`` is an exception: its constructor takes ``api_base`` + instead of ``base_url``, so passing ``host`` raises a ``TypeError`` that + the hook catches and silently falls back to environment-variable auth, + ignoring ``host`` entirely. Point a LiteLLM proxy's OpenAI-compatible + endpoint via ``openai:`` instead -- this is the same proxy role + used in the gateway section above, just addressed through the working + prefix rather than the rejected ``litellm:`` one. - { - "conn_type": "pydanticai", - "host": "http://localhost:11434/v1", - "extra": "{\"model\": \"ollama:llama3.2\"}" - } +Other OpenAI-compatible servers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The generic ``openai:llama3.2`` form (see :doc:`connections/pydantic_ai`) -also works against the same endpoint; use ``ollama:`` unless you have a -reason to skip the model-profile tuning. - -vLLM -^^^^ - -.. code-block:: json - - { - "conn_type": "pydanticai", - "host": "http://localhost:8000/v1", - "extra": "{\"model\": \"openai:llama3\"}" - } - -vLLM has no dedicated pydantic-ai provider, so ``openai:`` is the -only format to use. +The same ``openai:`` plus ``host`` setup works against any other +server that exposes an OpenAI-compatible API -- for example +`LM Studio `__, +`llama.cpp `__'s ``llama-server``, or +`LocalAI `__. Point ``host`` at that server's +OpenAI-compatible endpoint the same way as the vLLM example above. Cross-hook naming differences -------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :class:`~airflow.providers.common.ai.hooks.langchain.LangChainHook` and :class:`~airflow.providers.common.ai.hooks.llamaindex.LlamaIndexHook` also @@ -124,12 +241,17 @@ the underlying constructor keyword each hook passes it to is not uniform: - ``api_base`` Where to go next ------------------- +------------------- - :ref:`howto/connection:pydanticai` -- the full connection field reference. - :doc:`retry_policies` -- the "Local LLM support" section covers pointing ``LLMRetryPolicy`` at a self-hosted endpoint. +- :doc:`examples` -- more runnable Dags against the ``pydanticai`` + connection type; point one at any of the connections on this page. - pydantic-ai isn't limited to Ollama and vLLM -- any OpenAI-compatible endpoint works the same way. See the `pydantic-ai install docs `__ for the full list of providers it supports. +- `pydantic-ai's OpenAI-compatible models docs + `__ -- background on the + ``openai`` provider class this guide's ``openai:`` examples use. From a94d579a83145e27ef7c0e147dff326791f69b59 Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Tue, 14 Jul 2026 22:41:14 +0800 Subject: [PATCH 3/8] fixup! fixup! Add self-hosted model guide for the common.ai provider --- .../common/ai/docs/self_hosted_models.rst | 26 ++++++++++++++----- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index 76d2140b7f459..f3cd03cfd7474 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -40,19 +40,29 @@ Prerequisites ^^^^^^^^^^^^^ You need an OpenAI-compatible model server reachable from where Airflow -runs, and hardware able to run the model you pick (check the server's own -docs for GPU/CPU requirements): +runs, and hardware able to run the model you pick: -- `Ollama `__ -- the server used throughout this guide. +- `Ollama `__ -- the server used throughout this guide; + see its `hardware support docs `__ for + GPU/CPU requirements. - `vLLM `__ -- a higher-throughput alternative, - commonly GPU-hosted. + commonly GPU-hosted; see its + `GPU installation requirements `__. - `LM Studio `__ -- a desktop app with a built-in - OpenAI-compatible server. + OpenAI-compatible server; see its + `system requirements `__. + +Any other OpenAI-compatible server works the same way -- see pydantic-ai's +`OpenAI-compatible models docs `__ +for the full list of what qualifies. This page deliberately keeps only the +servers exercised by this guide; for how to point Airflow at any of the +others, see :ref:`self_hosted_models_other_servers` below. 1. Start an OpenAI-compatible server -------------------------------------- -Install Ollama and pull a model: +This guide uses Ollama as its reference server. Install it and pull a +model: .. code-block:: bash @@ -153,7 +163,7 @@ changes: This is a real constraint, not a gap to be filled later: the hook forwards only ``api_key`` and ``base_url`` to the provider class -(``_get_provider_kwargs``, `hooks/pydantic_ai.py:101-128`) -- there is no +(``_get_provider_kwargs``, ``hooks/pydantic_ai.py:101-128``) -- there is no extension point for custom headers. A gateway that authenticates with a bearer token works through this connection shape; one that requires additional custom headers does not. @@ -206,6 +216,8 @@ its OpenAI provider, and there's no separate ``ollama`` extra. used in the gateway section above, just addressed through the working prefix rather than the rejected ``litellm:`` one. +.. _self_hosted_models_other_servers: + Other OpenAI-compatible servers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From 41efcb50a9dd3a238bd6cb421290d49228ed454d Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Wed, 15 Jul 2026 11:04:01 +0800 Subject: [PATCH 4/8] fixup! fixup! fixup! Add self-hosted model guide for the common.ai provider --- providers/common/ai/docs/self_hosted_models.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index f3cd03cfd7474..81467b934bebd 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -53,7 +53,7 @@ runs, and hardware able to run the model you pick: `system requirements `__. Any other OpenAI-compatible server works the same way -- see pydantic-ai's -`OpenAI-compatible models docs `__ +`OpenAI-compatible models docs `__ for the full list of what qualifies. This page deliberately keeps only the servers exercised by this guide; for how to point Airflow at any of the others, see :ref:`self_hosted_models_other_servers` below. From 7ef4b9f8263dd41154e305db923426a2938216c6 Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Wed, 15 Jul 2026 14:42:02 +0800 Subject: [PATCH 5/8] fixup! fixup! fixup! fixup! Add self-hosted model guide for the common.ai provider --- .../common/ai/docs/self_hosted_models.rst | 22 ++++++------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index 81467b934bebd..427c95b63545d 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -61,12 +61,12 @@ others, see :ref:`self_hosted_models_other_servers` below. 1. Start an OpenAI-compatible server -------------------------------------- -This guide uses Ollama as its reference server. Install it and pull a -model: +This guide uses Ollama as its reference server. +`Install it `__ and pull a model: .. code-block:: bash - ollama pull gemma3 + ollama pull gemma4 Ollama serves an OpenAI-compatible API at ``http://localhost:11434/v1`` once running. vLLM and LM Studio plug into the steps below the same way -- only @@ -82,7 +82,7 @@ keyless server: .. code-block:: bash - export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma3"}}' + export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma4"}}' - ``host`` -- the server's base URL. - ``password`` -- left empty for a keyless local server. That triggers two @@ -117,17 +117,9 @@ Run it like any other Dag (``airflow dags test quickstart_llm``); the ------------------------------------ Tool calling needs a model that supports pydantic-ai's ``tools`` capability --- not every small local model does, and ``gemma3`` from the previous steps -doesn't. Pull a tools-capable model and point the same connection at it -instead: - -.. code-block:: bash - - ollama pull gemma4 - -.. code-block:: bash - - export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma4"}}' +-- not every small local model does. The connection from the previous steps +already points at a model that supports it, so this step runs against the +same connection with no changes. ``AgentOperator`` can wrap an existing Airflow hook as a tool via ``HookToolset``. This example wraps an HTTP connection named ``my_api`` -- From 1075aa2233f04cc8517a2b8ff270d303eefe95cc Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Wed, 15 Jul 2026 15:58:14 +0800 Subject: [PATCH 6/8] fixup! fixup! fixup! fixup! fixup! Add self-hosted model guide for the common.ai provider --- .../common/ai/docs/self_hosted_models.rst | 21 +++++++----- .../common/ai/example_dags/example_agent.py | 33 +++++++++++++++++++ 2 files changed, 46 insertions(+), 8 deletions(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index 427c95b63545d..e49d1b50659e7 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -131,14 +131,19 @@ point it at any reachable HTTP endpoint that returns JSON: .. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_agent.py :language: python - :start-after: [START howto_operator_agent_hook] - :end-before: [END howto_operator_agent_hook] - -``HttpHook`` defaults to ``method="POST"`` -(:class:`~airflow.providers.http.hooks.http.HttpHook`), fixed when the hook -is constructed. If ``my_api`` points at a GET-only endpoint, pass -``HttpHook(http_conn_id="my_api", method="GET")`` instead of the bare -constructor shown above. + :start-after: [START howto_agent_self_hosted] + :end-before: [END howto_agent_self_hosted] + +The hook above is constructed with ``method="GET"`` because the target +endpoint (Ollama's own ``/api/tags``) is GET-only, while ``HttpHook`` +defaults to ``method="POST"`` +(:class:`~airflow.providers.http.hooks.http.HttpHook`) -- fixed once the +hook is constructed, not something the model can change per call. + +The prompt also names the endpoint directly instead of asking the model to +discover one. ``HttpHook`` raises on any response outside the 2xx/3xx range, +so if the model guesses a path that doesn't exist, the task fails outright +rather than the agent recovering gracefully. Routing through an AI gateway -------------------------------- diff --git a/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent.py b/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent.py index 79a0c5fe333b2..53cf7e3a4aed7 100644 --- a/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent.py +++ b/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent.py @@ -110,6 +110,39 @@ def example_agent_operator_hook(): example_agent_operator_hook() +# --------------------------------------------------------------------------- +# 2b. Hook-based tools against a GET-only endpoint (self-hosted models tutorial) +# --------------------------------------------------------------------------- + + +# [START howto_agent_self_hosted] +@dag(tags=["example"]) +def example_agent_self_hosted(): + from airflow.providers.http.hooks.http import HttpHook + + # GET-only endpoint (e.g. Ollama's /api/tags); HttpHook defaults to POST. + http_hook = HttpHook(http_conn_id="my_api", method="GET") + + AgentOperator( + task_id="list_models", + prompt="Call /api/tags and summarize which models are available.", + llm_conn_id="pydanticai_default", + system_prompt="You are an API assistant. Use the tools to call the endpoint and report what you find.", + toolsets=[ + HookToolset( + http_hook, + allowed_methods=["run"], + tool_name_prefix="http_", + ) + ], + ) + + +# [END howto_agent_self_hosted] + +example_agent_self_hosted() + + # --------------------------------------------------------------------------- # 3. @task.agent decorator with dynamic prompt # --------------------------------------------------------------------------- From a7de048dbba06cfddd482cd2165b0d85528792b3 Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Wed, 15 Jul 2026 17:47:55 +0800 Subject: [PATCH 7/8] fixup! fixup! fixup! fixup! fixup! fixup! Add self-hosted model guide for the common.ai provider --- .../common/ai/docs/self_hosted_models.rst | 105 ++++++++++++++---- 1 file changed, 82 insertions(+), 23 deletions(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index e49d1b50659e7..e63f04a3b5b47 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -40,11 +40,13 @@ Prerequisites ^^^^^^^^^^^^^ You need an OpenAI-compatible model server reachable from where Airflow -runs, and hardware able to run the model you pick: +runs, and hardware able to run the model you pick. Ollama, vLLM, and LM +Studio are three common choices, used as the examples throughout this +guide: -- `Ollama `__ -- the server used throughout this guide; - see its `hardware support docs `__ for - GPU/CPU requirements. +- `Ollama `__ -- see its + `hardware support docs `__ for GPU/CPU + requirements. - `vLLM `__ -- a higher-throughput alternative, commonly GPU-hosted; see its `GPU installation requirements `__. @@ -54,46 +56,103 @@ runs, and hardware able to run the model you pick: Any other OpenAI-compatible server works the same way -- see pydantic-ai's `OpenAI-compatible models docs `__ -for the full list of what qualifies. This page deliberately keeps only the -servers exercised by this guide; for how to point Airflow at any of the -others, see :ref:`self_hosted_models_other_servers` below. +for the full list of what qualifies. This page uses these three as running +examples rather than trying to cover every option; for how to point Airflow +at any of the others, see :ref:`self_hosted_models_other_servers` below. 1. Start an OpenAI-compatible server -------------------------------------- -This guide uses Ollama as its reference server. -`Install it `__ and pull a model: +Ollama is the reference server this guide proves end to end. The vLLM and LM +Studio tabs show equivalent wiring for readers already running one of those +instead, but this guide hasn't exercised them itself. -.. code-block:: bash +.. tab-set:: + + .. tab-item:: Ollama + :sync: ollama + + `Install it `__ and pull a model: + + .. code-block:: bash + + ollama pull gemma4 - ollama pull gemma4 + Ollama serves an OpenAI-compatible API at ``http://localhost:11434/v1`` + once running. -Ollama serves an OpenAI-compatible API at ``http://localhost:11434/v1`` once -running. vLLM and LM Studio plug into the steps below the same way -- only -the base URL and default port differ; see each project's own docs for how to -start it. + .. tab-item:: vLLM + :sync: vllm + + See `vLLM's installation docs `__ + for how to install it and serve a model. + + .. tab-item:: LM Studio + :sync: lmstudio + + See how to `run LM Studio as a local server + `__, which covers + starting its built-in server. 2. Create the connection -------------------------- Every LLM call goes through a Pydantic AI connection (``conn_type`` -``pydanticai``, default connection id ``pydanticai_default``). For a local, -keyless server: +``pydanticai``, default connection id ``pydanticai_default``). -.. code-block:: bash +.. tab-set:: + + .. tab-item:: Ollama + :sync: ollama + + For a local, keyless server: + + .. code-block:: bash + + export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma4"}}' + + .. tab-item:: vLLM + :sync: vllm + + .. code-block:: bash + + export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:8000/v1", "extra": {"model": "openai:gemma4"}}' + + .. tab-item:: LM Studio + :sync: lmstudio + + .. code-block:: bash + + export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:1234/v1", "extra": {"model": "openai:gemma4"}}' - export AIRFLOW_CONN_PYDANTICAI_DEFAULT='{"conn_type": "pydanticai", "host": "http://localhost:11434/v1", "extra": {"model": "ollama:gemma4"}}' + ``http://localhost:1234`` is the default server port shown in `LM + Studio's OpenAI-compatibility docs + `__. - ``host`` -- the server's base URL. - ``password`` -- left empty for a keyless local server. That triggers two separate things: the Airflow hook forwards no key at all when ``password`` is empty (only ``base_url`` goes to the provider constructor); pydantic-ai's provider class then fills in its own placeholder API key and does send - that placeholder in the actual HTTP request. Ollama doesn't check it, so - the request still succeeds either way. + that placeholder in the actual HTTP request. A server with no auth check + doesn't care, so the request still succeeds either way. - ``extra["model"]`` -- ``ollama:`` picks up Ollama-specific tuning; - see :ref:`Model identifier format ` below for - when to use ``openai:`` instead. + vLLM and LM Studio have no dedicated pydantic-ai provider class, so they + use ``openai:`` instead. See + :ref:`Model identifier format ` below for + details. + +The examples above all use ``gemma4`` to illustrate the format -- the actual +value must match a model your server is really serving: + +- Ollama -- ``ollama list`` shows the tags you've pulled. +- vLLM -- defaults to its ``--model`` startup argument; the + ``--served-model-name`` flag can set an alias explicitly (e.g. to + ``gemma4``) if you want the served name to differ from ``--model``. +- LM Studio -- the model identifier shown for whatever you loaded in its UI. +- Any of the three (and any other OpenAI-compatible server) -- + ``curl /v1/models`` lists what the server actually reports as + available. See :ref:`howto/connection:pydanticai` for the full connection field reference. From 83dfefd6667f88e03d1d8db2d81e22ad6f0aa744 Mon Sep 17 00:00:00 2001 From: Wei Lee Date: Wed, 15 Jul 2026 18:31:27 +0800 Subject: [PATCH 8/8] fixup! fixup! fixup! fixup! fixup! fixup! fixup! Add self-hosted model guide for the common.ai provider --- .../common/ai/docs/self_hosted_models.rst | 54 ++++++++++++++----- 1 file changed, 42 insertions(+), 12 deletions(-) diff --git a/providers/common/ai/docs/self_hosted_models.rst b/providers/common/ai/docs/self_hosted_models.rst index e63f04a3b5b47..fcb7c6f711796 100644 --- a/providers/common/ai/docs/self_hosted_models.rst +++ b/providers/common/ai/docs/self_hosted_models.rst @@ -40,9 +40,14 @@ Prerequisites ^^^^^^^^^^^^^ You need an OpenAI-compatible model server reachable from where Airflow -runs, and hardware able to run the model you pick. Ollama, vLLM, and LM -Studio are three common choices, used as the examples throughout this -guide: +runs, and hardware able to run the model you pick. The provider is built on +pydantic-ai, which reaches these servers by speaking the OpenAI-compatible +wire protocol -- that's why the server needs to expose an OpenAI-compatible +API. Any server that does works the same way; see +pydantic-ai's `OpenAI-compatible models docs +`__ for the full list of what +qualifies. Ollama, vLLM, and LM Studio are three common choices, used as the +examples throughout this guide: - `Ollama `__ -- see its `hardware support docs `__ for GPU/CPU @@ -54,11 +59,9 @@ guide: OpenAI-compatible server; see its `system requirements `__. -Any other OpenAI-compatible server works the same way -- see pydantic-ai's -`OpenAI-compatible models docs `__ -for the full list of what qualifies. This page uses these three as running -examples rather than trying to cover every option; for how to point Airflow -at any of the others, see :ref:`self_hosted_models_other_servers` below. +This page uses these three as running examples rather than trying to cover +every option; for how to point Airflow at any of the others, see +:ref:`self_hosted_models_other_servers` below. 1. Start an OpenAI-compatible server -------------------------------------- @@ -176,9 +179,34 @@ Run it like any other Dag (``airflow dags test quickstart_llm``); the ------------------------------------ Tool calling needs a model that supports pydantic-ai's ``tools`` capability --- not every small local model does. The connection from the previous steps -already points at a model that supports it, so this step runs against the -same connection with no changes. +-- not every small local model does. + +.. tab-set:: + + .. tab-item:: Ollama + :sync: ollama + + The connection from the previous steps already points at a model + that supports it, so this step runs against the same connection + with no changes. + + .. tab-item:: vLLM + :sync: vllm + + vLLM needs extra server startup flags to enable tool calling for a + given model (``--enable-auto-tool-choice`` plus a matching + ``--tool-call-parser``) -- see its + `Tool Calling docs `__ + for how to pick the right ones for your model. + + .. tab-item:: LM Studio + :sync: lmstudio + + LM Studio's built-in server supports tool calling out of the box + through ``/v1/chat/completions`` -- see its + `Tool Use docs `__. + Its docs note that smaller models, or ones not trained for tool use, + may return improperly formatted tool calls. ``AgentOperator`` can wrap an existing Airflow hook as a tool via ``HookToolset``. This example wraps an HTTP connection named ``my_api`` -- @@ -244,7 +272,9 @@ Model identifier format The ``extra`` JSON's ``model`` value keeps the ``provider:model`` format the hook always expects, but which ``provider`` prefix to use depends on the -endpoint (checked against the pinned ``pydantic-ai-slim`` 2.5.0): +endpoint (verified against ``pydantic-ai-slim`` 2.5.0, the version this +guide was developed against, and 2.10.0, the latest at the time of +writing): - **vLLM** has no dedicated provider class in pydantic-ai -- ``openai:`` is the only option. pydantic-ai's ``openai`` provider class talks to