Skip to content

fix: stagger collection-bucket copy pipe schedules (CM-XXXX) #4420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gaspergrom merged 2 commits into from
Aug 3, 2026
Merged

fix: stagger collection-bucket copy pipe schedules (CM-XXXX) #4420

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0e0febb
fix: stagger collection-bucket copy pipe schedules to avoid leaderboa…
gaspergrom Jul 31, 2026
c56eadc
fix: move collection-bucket copy pipes to a separate hour span from l…
gaspergrom Jul 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_0.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_0_ds
COPY_MODE replace
COPY_SCHEDULE 0 1 * * *
COPY_SCHEDULE 0 4 * * *

@cursor cursor Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schedules shifted hours, not minutes

Medium Severity

COPY_SCHEDULE values move the hour by +3 (1→4, 2→5, 3→6) while keeping the same minutes. The intended fix was a minute-level stagger inside the 1–3 AM window (+2 / −2/+3 minutes) so all 26 pipes keep unique start minutes there. Tinybird cron is minute hour ..., so these edits change the wrong field relative to that plan, delay collection refreshes by three hours, and leave the stated 1–3 AM uniqueness check unverified for the new 4–6 AM window. Pipe DESCRIPTION text still documents the old 01:00 / 2:05 / 3:00 times.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c56eadc. Configure here.

2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_1.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_1_ds
COPY_MODE replace
COPY_SCHEDULE 5 1 * * *
COPY_SCHEDULE 5 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_2.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_2_ds
COPY_MODE replace
COPY_SCHEDULE 10 1 * * *
COPY_SCHEDULE 10 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_3.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_3_ds
COPY_MODE replace
COPY_SCHEDULE 15 1 * * *
COPY_SCHEDULE 15 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_4.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_4_ds
COPY_MODE replace
COPY_SCHEDULE 20 1 * * *
COPY_SCHEDULE 20 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_5.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_5_ds
COPY_MODE replace
COPY_SCHEDULE 25 1 * * *
COPY_SCHEDULE 25 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_6.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_6_ds
COPY_MODE replace
COPY_SCHEDULE 30 1 * * *
COPY_SCHEDULE 30 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_7.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_7_ds
COPY_MODE replace
COPY_SCHEDULE 35 1 * * *
COPY_SCHEDULE 35 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_8.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_8_ds
COPY_MODE replace
COPY_SCHEDULE 40 1 * * *
COPY_SCHEDULE 40 4 * * *
2 changes: 1 addition & 1 deletion ...ces/libs/tinybird/pipes/activityRelations_collection_bucket_clean_enrich_copy_pipe_9.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE activityRelations_collection_deduplicated_cleaned_bucket_9_ds
COPY_MODE replace
COPY_SCHEDULE 45 1 * * *
COPY_SCHEDULE 45 4 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_0.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 5 2 * * *
COPY_SCHEDULE 5 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_1.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 10 2 * * *
COPY_SCHEDULE 10 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_2.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 15 2 * * *
COPY_SCHEDULE 15 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_3.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 20 2 * * *
COPY_SCHEDULE 20 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_4.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 25 2 * * *
COPY_SCHEDULE 25 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_5.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 30 2 * * *
COPY_SCHEDULE 30 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_6.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 35 2 * * *
COPY_SCHEDULE 35 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_7.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 40 2 * * *
COPY_SCHEDULE 40 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_8.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 45 2 * * *
COPY_SCHEDULE 45 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_copy_pipe_9.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_consolidated_ds
COPY_MODE append
COPY_SCHEDULE 50 2 * * *
COPY_SCHEDULE 50 5 * * *
2 changes: 1 addition & 1 deletion services/libs/tinybird/pipes/collection_insights_aggregate_projects_copy_pipe.pipe
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@ SQL >
TYPE COPY
TARGET_DATASOURCE collection_insights_aggregate_projects_ds
COPY_MODE replace
COPY_SCHEDULE 0 3 * * *
COPY_SCHEDULE 0 6 * * *
Loading