Skip to content

fix: do not crash on unparseable cursor state value during concurrent cursor construction#1070

Draft
devin-ai-integration[bot] wants to merge 3 commits into
mainfrom
devin/1783684033-fix-datetime-state-converter-bad-cursor-value
Draft

fix: do not crash on unparseable cursor state value during concurrent cursor construction#1070
devin-ai-integration[bot] wants to merge 3 commits into
mainfrom
devin/1783684033-fix-datetime-state-converter-bad-cursor-value

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes a hard source startup crash for oncall issue airbytehq/oncall#13084 (HubSpot), where the source dies before any stream is read with ValueError: No format in [...] matching True. Because the source dies at startup, no new state is ever committed — which is why manual state edits appeared to "revert" after each failed sync.

Root cause

HubSpot's associations_* streams (e.g. associations_contacts_companies) complete without a cursor value, so they persist the no-cursor sentinel state {"__ab_no_cursor_state_message": true}. On the next run the parent has incremental_dependency: true, so the CDK reconstructs parent state from the child state in ModelToComponentFactory._instantiate_parent_stream_state_manager. When no parent/global state is extracted, a fallback re-keyed the single child state value as the parent cursor:

cursor_values = child_state.values()          # -> [True]  (the sentinel value)
if cursor_values and len(cursor_values) == 1:
    ...
    stream_state=AirbyteStateBlob({cursor_field: list(cursor_values)[0]})  # -> {"lastmodifieddate": True}

That boolean then reached the datetime cursor parser during ConcurrentCursor construction and crashed the whole source.

Change (root fix)

In _instantiate_parent_stream_state_manager, do not treat the __ab_no_cursor_state_message sentinel (via the exported NO_CURSOR_STATE_KEY constant) as a cursor value. When the child state is only that sentinel, skip the re-keying and yield empty parent state so the parent starts cleanly from its configured start date:

if set(child_state.keys()) == {NO_CURSOR_STATE_KEY}:
    return ConnectorStateManager([])

Defense-in-depth (kept)

The earlier guard in DateTimeStreamStateConverter._get_sync_start is retained as a backstop: a boolean saved cursor value is never a valid datetime, so it is ignored (with a warning) and falls back to the start date rather than crashing. This guard is intentionally scoped to bool — a genuinely malformed datetime string must still raise, because StateDelegatingStream's api_retention_period cursor-age validation (#890) deliberately fails fast on unparseable cursor strings (test_cursor_age_validation_raises_error_for_unparseable_cursor).

No public interface, schema, spec, or state-format changes; non-breaking.

Declarative-First Evaluation

This is a CDK library fix in the shared declarative machinery (ModelToComponentFactory), not a connector-level change, so declarative-component alternatives do not apply.

Test Coverage

  • Factory-level regression (test_instantiate_parent_stream_state_manager_ignores_no_cursor_sentinel): a parent ParentStreamConfig with incremental_dependency: true and a child state of {"__ab_no_cursor_state_message": True} now produces empty parent state and no longer raises ValueError: No format ... matching True.
  • Converter-level (unit_tests/sources/streams/concurrent/test_datetime_state_converter.py): parametrized _get_sync_start boolean cases plus test_get_sync_start_with_boolean_state_value_does_not_raise and test_convert_from_sequential_state_with_boolean_state_value_does_not_raise.

All new tests fail without the corresponding fix and pass with it. test_model_to_component_factory.py (143 tests incl. test_state_delegating_stream.py) and the converter suite pass. ruff check, ruff format, and mypy are clean on the changed files.

Related to https://github.com/airbytehq/oncall/issues/13084:

Note: this addresses the hard startup crash. The customer's original, softer complaint (slow syncs / repeated *_property_history re-emissions) is a separate performance concern tracked in the same issue.

Link to Devin session: https://app.devin.ai/sessions/ee9a064393c44676a341378e32c2a746

… cursor construction

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1783684033-fix-datetime-state-converter-bad-cursor-value#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1783684033-fix-datetime-state-converter-bad-cursor-value

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

PyTest Results (Fast)

4 132 tests  +5   4 120 ✅ +5   7m 39s ⏱️ -2s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit e6ff882. ± Comparison against base commit dd9558c.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

PyTest Results (Full)

4 135 tests  +5   4 123 ✅ +5   11m 54s ⏱️ + 4m 39s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit e6ff882. ± Comparison against base commit dd9558c.

♻️ This comment has been updated with latest results.

devin-ai-integration Bot and others added 2 commits July 10, 2026 12:12
…il-fast

Co-Authored-By: bot_apk <apk@cognition.ai>
…rsor value

Co-Authored-By: bot_apk <apk@cognition.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants