Restore pluggable email backend for task failure/retry alerts#69877
Draft
potiuk wants to merge 1 commit into
Draft
Restore pluggable email backend for task failure/retry alerts#69877potiuk wants to merge 1 commit into
potiuk wants to merge 1 commit into
Conversation
Since apache#57354, task email_on_failure / email_on_retry alerts were routed unconditionally through SmtpNotifier, silently ignoring the [email] email_backend configuration. Custom backends (SES, SendGrid, org-internal) stopped delivering failure/retry emails even though the deprecated email_on_* parameters still worked. This restores pluggability while keeping the single-notifier design that apache#57354 introduced: - New [email] email_notifier option selects the BaseNotifier used for failure/retry alerts (default: SmtpNotifier). Any BaseNotifier whose constructor accepts to / from_email / subject / html_content can be used. - A legacy [email] email_backend (non-default) is transparently wrapped in a new LegacyEmailBackendNotifier adapter in common.compat, so existing SES/SendGrid/custom email_backend setups keep working without change. - Both the worker task-runner path and the DAG-processor callback path resolve the notifier through the same _send_error_email_notification function, so the selected backend is used consistently regardless of how the task failed. The adapter loads the backend lazily from config at notify time, keeping the Task SDK free of a static dependency on airflow.utils.email. Adds LegacyEmailBackendNotifier to apache-airflow-providers-common-compat.
eladkal
reviewed
Jul 14, 2026
Comment on lines
+2397
to
+2409
| email_notifier: | ||
| description: | | ||
| Dotted path to the notifier class used to deliver ``email_on_failure`` and | ||
| ``email_on_retry`` task alerts. The class must be a ``BaseNotifier`` subclass whose | ||
| constructor accepts ``to``, ``from_email``, ``subject`` and ``html_content`` keyword | ||
| arguments. Override it to route failure/retry alerts through a backend other than SMTP | ||
| (for example an SES- or SendGrid-based notifier). Both the worker task-runner path and | ||
| the DAG-processor callback path load the notifier from this single option, so the | ||
| selected backend is used consistently regardless of how the task failed. | ||
| version_added: 3.3.1 | ||
| type: string | ||
| example: ~ | ||
| default: "airflow.providers.smtp.notifications.smtp.SmtpNotifier" |
Contributor
There was a problem hiding this comment.
We want all email related settings out of core
#30531
[DISCUSS] Drop email integration from Airflow Core
the community disagreed with me about removing all of that as part of Airflow 3. That ship has sailed but I think we should avoid adding more oil to this fire. We need to complete the Core->SMTP migration and deprecate all email related stuff from core.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Restore the ability to deliver
email_on_failure/email_on_retrytask alerts through a pluggable backend (SES, SendGrid, or any custom notifier), which regressed in #57354.Why
email_on_failure/email_on_retryare deprecated (slated for Airflow 4, migrate toSmtpNotifier) but still functional. Since #57354, task failure/retry emails are routed unconditionally throughSmtpNotifier, silently ignoring the[email] email_backendconfiguration. Deployments using a customemail_backend(Amazon SES, SendGrid, org-internal) stopped receiving failure/retry alerts even though the config option still exists and the deprecated params still work. This restores that behaviour while keeping the single-notifier design #57354 introduced.How
_send_error_email_notification(shared by both the worker task-runner path and the DAG-processor callback path) now resolves the notifier by precedence:[email] email_notifier(new option, defaultSmtpNotifier) — anyBaseNotifierwhose constructor acceptsto/from_email/subject/html_content.[email] email_backend(non-default) — transparently wrapped in a newLegacyEmailBackendNotifier(added toapache-airflow-providers-common-compat) so existing SES/SendGrid setups keep working unchanged. The backend is loaded lazily from config at notify time, so the Task SDK keeps no static dependency onairflow.utils.email.SmtpNotifier.Because both failure-email entry points funnel through the same function, the selected backend is used consistently regardless of how the task failed (normal failure vs. zombie/killed).
Tests
TestEmailNotifications(task-sdk): default still usesSmtpNotifier(unchanged regression tests), customemail_notifierselected, legacyemail_backendwrapped and invoked, explicitemail_notifieroverrides legacy backend, unresolvable notifier logged (no raise).TestExecuteEmailCallbacks(DAG-processor path): unchanged and green — confirms both entry points stay consistent.test_notifier.py(common.compat): unit tests forLegacyEmailBackendNotifierdispatch.Notes
email_on_*params are not un-deprecated; this only keeps their existing behaviour pluggable until removal in Airflow 4.{PR_NUMBER}.improvement.rstonce this PR number is known.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines