Skip to content

fix: accept arbitrary JSON userData in ApifyRequestList#966

Open
vdusek wants to merge 1 commit into
masterfrom
fix/request-list-arbitrary-json-user-data
Open

fix: accept arbitrary JSON userData in ApifyRequestList#966
vdusek wants to merge 1 commit into
masterfrom
fix/request-list-arbitrary-json-user-data

Conversation

@vdusek

@vdusek vdusek commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Description

  • _RequestDetails.user_data was typed dict[str, str], but the platform requestListSources input allows arbitrary JSON values in userData, so e.g. {"depth": 1} raised a ValidationError. Retyped to dict[str, JsonSerializable] (the same alias crawlee's Request.from_url declares).
  • Fixed a related pre-existing crash: crawlee's Request.from_url mutates a non-empty user_data dict passed to it (writes __crawlee back into it), and _process_remote_url reused one dict for every URL extracted from a requestsFromUrl source. Any such source with non-empty userData and 2+ extracted URLs crashed with a TypeError on the second request. Each call now gets its own copy of the dict.
  • Added regression tests for both cases.

@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Jun 11, 2026
@vdusek vdusek self-assigned this Jun 11, 2026
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.88%. Comparing base (2cc5602) to head (1800d5d).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #966      +/-   ##
==========================================
- Coverage   89.91%   89.88%   -0.04%     
==========================================
  Files          49       49              
  Lines        3085     3085              
==========================================
- Hits         2774     2773       -1     
- Misses        311      312       +1     
Flag Coverage Δ
e2e ?
integration 56.95% <0.00%> (-0.04%) ⬇️
unit 78.70% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek requested a review from Pijukatel June 11, 2026 13:21
@vdusek vdusek marked this pull request as ready for review June 11, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants