fix: do not crash on unparseable cursor state value during concurrent cursor construction#1070
Draft
devin-ai-integration[bot] wants to merge 3 commits into
Draft
Conversation
… cursor construction Co-Authored-By: bot_apk <apk@cognition.ai>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1783684033-fix-datetime-state-converter-bad-cursor-value#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1783684033-fix-datetime-state-converter-bad-cursor-valuePR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
…il-fast Co-Authored-By: bot_apk <apk@cognition.ai>
…rsor value Co-Authored-By: bot_apk <apk@cognition.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a hard source startup crash for oncall issue
airbytehq/oncall#13084(HubSpot), where the source dies before any stream is read withValueError: No format in [...] matching True. Because the source dies at startup, no new state is ever committed — which is why manual state edits appeared to "revert" after each failed sync.Root cause
HubSpot's
associations_*streams (e.g.associations_contacts_companies) complete without a cursor value, so they persist the no-cursor sentinel state{"__ab_no_cursor_state_message": true}. On the next run the parent hasincremental_dependency: true, so the CDK reconstructs parent state from the child state inModelToComponentFactory._instantiate_parent_stream_state_manager. When no parent/global state is extracted, a fallback re-keyed the single child state value as the parent cursor:That boolean then reached the datetime cursor parser during
ConcurrentCursorconstruction and crashed the whole source.Change (root fix)
In
_instantiate_parent_stream_state_manager, do not treat the__ab_no_cursor_state_messagesentinel (via the exportedNO_CURSOR_STATE_KEYconstant) as a cursor value. When the child state is only that sentinel, skip the re-keying and yield empty parent state so the parent starts cleanly from its configured start date:Defense-in-depth (kept)
The earlier guard in
DateTimeStreamStateConverter._get_sync_startis retained as a backstop: a boolean saved cursor value is never a valid datetime, so it is ignored (with a warning) and falls back to the start date rather than crashing. This guard is intentionally scoped tobool— a genuinely malformed datetime string must still raise, becauseStateDelegatingStream'sapi_retention_periodcursor-age validation (#890) deliberately fails fast on unparseable cursor strings (test_cursor_age_validation_raises_error_for_unparseable_cursor).No public interface, schema, spec, or state-format changes; non-breaking.
Declarative-First Evaluation
This is a CDK library fix in the shared declarative machinery (
ModelToComponentFactory), not a connector-level change, so declarative-component alternatives do not apply.Test Coverage
test_instantiate_parent_stream_state_manager_ignores_no_cursor_sentinel): a parentParentStreamConfigwithincremental_dependency: trueand a child state of{"__ab_no_cursor_state_message": True}now produces empty parent state and no longer raisesValueError: No format ... matching True.unit_tests/sources/streams/concurrent/test_datetime_state_converter.py): parametrized_get_sync_startboolean cases plustest_get_sync_start_with_boolean_state_value_does_not_raiseandtest_convert_from_sequential_state_with_boolean_state_value_does_not_raise.All new tests fail without the corresponding fix and pass with it.
test_model_to_component_factory.py(143 tests incl.test_state_delegating_stream.py) and the converter suite pass.ruff check,ruff format, andmypyare clean on the changed files.Related to https://github.com/airbytehq/oncall/issues/13084:
Link to Devin session: https://app.devin.ai/sessions/ee9a064393c44676a341378e32c2a746