Do not merge: add CDK deadlock diagnostics for pinned prerelease#1043
Draft
devin-ai-integration[bot] wants to merge 2 commits into
Draft
Do not merge: add CDK deadlock diagnostics for pinned prerelease#1043devin-ai-integration[bot] wants to merge 2 commits into
devin-ai-integration[bot] wants to merge 2 commits into
Conversation
Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1780315213-deadlock-diagnostics#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1780315213-deadlock-diagnosticsPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Do not merge. This is a diagnostic-only branch requested by gl_anatolii.yatsuk for the Zendesk Support heartbeat/deadlock investigation in https://github.com/airbytehq/oncall/issues/12725. The intended workflow is to publish/pin a prerelease image from this branch to affected users, collect stderr heartbeat/thread-dump evidence, then discard the branch.
The connector entrypoint now always starts a daemon diagnostic heartbeat in this branch. Every 30s it writes directly to stderr:
If the emitted message count is frozen for 3 heartbeat intervals, it appends a full Python thread dump, then repeats the dump every 10 stalled intervals. This adapts the diagnostic behavior from #1027 for prerelease pinning, where no additional environment variable can be set on the affected connections.
Also registers the concurrent source queue while active so the heartbeat can distinguish likely stdout backpressure/full-queue stalls from other blocked-thread patterns.
Validation:
poetry run pytest unit_tests/test_entrypoint.py -k 'deadlock_diagnostics or launch_starts_deadlock'poetry run pytest unit_tests/test_entrypoint.pypoetry run ruff check airbyte_cdk/entrypoint.py airbyte_cdk/sources/concurrent_source/concurrent_source.py airbyte_cdk/sources/concurrent_source/queue_registry.py unit_tests/test_entrypoint.pypoetry run ruff format --check airbyte_cdk/entrypoint.py airbyte_cdk/sources/concurrent_source/concurrent_source.py airbyte_cdk/sources/concurrent_source/queue_registry.py unit_tests/test_entrypoint.pypoetry run mypy --config-file mypy.ini airbyte_cdk/entrypoint.py airbyte_cdk/sources/concurrent_source/concurrent_source.py airbyte_cdk/sources/concurrent_source/queue_registry.pyLink to Devin session: https://app.devin.ai/sessions/d5860b3efd2d4aaeb00803ff700663b5