Skip to content

Do not merge: add CDK deadlock diagnostics for pinned prerelease#1043

Draft
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1780315213-deadlock-diagnostics
Draft

Do not merge: add CDK deadlock diagnostics for pinned prerelease#1043
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1780315213-deadlock-diagnostics

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented Jun 1, 2026

Summary

Do not merge. This is a diagnostic-only branch requested by gl_anatolii.yatsuk for the Zendesk Support heartbeat/deadlock investigation in https://github.com/airbytehq/oncall/issues/12725. The intended workflow is to publish/pin a prerelease image from this branch to affected users, collect stderr heartbeat/thread-dump evidence, then discard the branch.

The connector entrypoint now always starts a daemon diagnostic heartbeat in this branch. Every 30s it writes directly to stderr:

STDOUT_HEARTBEAT: t=... msgs=... bytes=... print_blocked=... queue_size=... queue_full=...

If the emitted message count is frozen for 3 heartbeat intervals, it appends a full Python thread dump, then repeats the dump every 10 stalled intervals. This adapts the diagnostic behavior from #1027 for prerelease pinning, where no additional environment variable can be set on the affected connections.

Also registers the concurrent source queue while active so the heartbeat can distinguish likely stdout backpressure/full-queue stalls from other blocked-thread patterns.

Validation:

  • poetry run pytest unit_tests/test_entrypoint.py -k 'deadlock_diagnostics or launch_starts_deadlock'
  • poetry run pytest unit_tests/test_entrypoint.py
  • poetry run ruff check airbyte_cdk/entrypoint.py airbyte_cdk/sources/concurrent_source/concurrent_source.py airbyte_cdk/sources/concurrent_source/queue_registry.py unit_tests/test_entrypoint.py
  • poetry run ruff format --check airbyte_cdk/entrypoint.py airbyte_cdk/sources/concurrent_source/concurrent_source.py airbyte_cdk/sources/concurrent_source/queue_registry.py unit_tests/test_entrypoint.py
  • poetry run mypy --config-file mypy.ini airbyte_cdk/entrypoint.py airbyte_cdk/sources/concurrent_source/concurrent_source.py airbyte_cdk/sources/concurrent_source/queue_registry.py

Link to Devin session: https://app.devin.ai/sessions/d5860b3efd2d4aaeb00803ff700663b5

Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1780315213-deadlock-diagnostics#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1780315213-deadlock-diagnostics

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

PyTest Results (Fast)

4 097 tests  +3   4 086 ✅ +3   8m 13s ⏱️ -13s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit ac5d348. ± Comparison against base commit a7b0929.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

PyTest Results (Full)

4 100 tests  +3   4 088 ✅ +3   12m 4s ⏱️ +27s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit ac5d348. ± Comparison against base commit a7b0929.

♻️ This comment has been updated with latest results.

Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
@devin-ai-integration devin-ai-integration Bot changed the title feat: add opt-in CDK deadlock diagnostics Do not merge: add CDK deadlock diagnostics for pinned prerelease Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant