Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,9 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the
* - ``task_startup_timeout``
- ``10.0``
- Seconds to wait for the bundle subprocess to connect after launch. Increase this if your
bundle startup is slow (e.g. on constrained hardware).
bundle startup is slow (e.g. on constrained hardware). The task is already ``running``
while the coordinator waits, so exceeding this fails the task rather than leaving it
``queued``, and whatever the bundle printed goes to the task log.

.. _go-sdk/edge-worker:

Expand All @@ -457,6 +459,10 @@ Limitations
languages, so task names and dependencies are declared in Python with
:func:`@task.stub <airflow.sdk.task.stub>`. This applies to both deployment modes and is a documented
known limitation.
* **In coordinator mode the pid recorded on the task instance is the Airflow supervisor's, not the bundle's.**
The supervisor reports the task as running before it launches the bundle, and the server ties the run to the
pid it was given at that point, so that is the pid the API and UI show. The bundle's own pid goes to the task
log.

The following are a non-exhaustive list of features the **Edge Worker** path has yet to implement. They are
the main reason the coordinator path is recommended: in coordinator mode the Python supervisor handles these
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -681,7 +681,9 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the
* - ``task_startup_timeout``
- ``10.0``
- Seconds to wait for the JVM subprocess to connect after launch. Increase this if your
JVM startup is slow (e.g. on constrained hardware or with a large classpath).
JVM startup is slow (e.g. on constrained hardware or with a large classpath). The task
is already ``running`` while the coordinator waits, so exceeding this fails the task
rather than leaving it ``queued``, and whatever the JVM printed goes to the task log.

.. note::

Expand Down Expand Up @@ -724,5 +726,8 @@ Limitations

* **One JVM subprocess per task instance.** Each task instance spawns a fresh JVM. Tasks that need to share
in-process state between instances should use XCom or an external store instead.
* **The pid recorded on the task instance is the Airflow supervisor's, not the JVM's.** The supervisor reports
the task as running before it launches the JVM, and the server ties the run to the pid it was given at that
point, so that is the pid the API and UI show. The JVM's own pid is logged to the task log when it starts.
* **Limited support for assets, deferral, and other Airflow features.** They may be implemented in the future
based on user feedback and demand.
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,9 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the
* - ``task_startup_timeout``
- ``10.0``
- Seconds to wait for the Node.js subprocess to connect after launch. Increase this if your bundle
startup is slow (e.g. on constrained hardware).
startup is slow (e.g. on constrained hardware). The task is already ``running`` while the
coordinator waits, so exceeding this fails the task rather than leaving it ``queued``, and
whatever the bundle printed goes to the task log.

Limitations
-----------
Expand All @@ -292,3 +294,6 @@ Limitations
bundles. To serve multiple bundles, register multiple coordinators on separate queues.
* **One Node.js subprocess per task instance.** Tasks that need to share in-process state between instances
should use XCom or an external store instead.
* **The pid recorded on the task instance is the Airflow supervisor's, not the Node.js process's.** The
supervisor reports the task as running before it launches the bundle, and the server ties the run to the pid
it was given at that point, so that is the pid the API and UI show. The bundle's own pid goes to the task log.
Loading
Loading