feat: v2 SDK rewrite with Pydantic + httpx

[FrancescoSaverioZuppichini]

Summary

Complete SDK rewrite matching the JS SDK 1:1:

• Pydantic v2 for all request/response models with automatic camelCase serialization
• httpx for sync and async HTTP clients
• ApiResult[T] wrapper pattern - no exceptions, just status: "success" | "error"
• Nested resources - sgai.crawl.start(), sgai.monitor.create(), sgai.history.list()
• uv as package manager with modern src/ layout

Changes

• Restructured from nested scrapegraph-py/ to root-level uv library
• All Pydantic models in single schemas.py with CamelModel base class
• Sync client (ScrapeGraphAI) and async client (AsyncScrapeGraphAI)
• 32 examples (16 sync + 16 async) for all endpoints
• 28 unit tests with mocked httpx
• Simplified CI workflows for uv

API Surface

from scrapegraph_py import ScrapeGraphAI, ScrapeRequest

sgai = ScrapeGraphAI()  # reads SGAI_API_KEY from env
result = sgai.scrape(ScrapeRequest(url="https://example.com"))

if result.status == "success":
    print(result.data["results"]["markdown"]["data"])

Test plan

• All 28 unit tests pass (uv run pytest tests/test_client.py -v)
• Integration tests pass with real API key
• Lint passes (uv run ruff check .)
• CI workflow runs successfully

🤖 Generated with Claude Code

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 3 package(s) with unknown licenses.

See the Details below.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 8c59f38.

Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

License Issues

uv.lock

Package	Version	License	Issue Type
pydantic	2.13.0	Null	Unknown License
pytest	9.0.3	Null	Unknown License
ruff	0.15.10	Null	Unknown License

OpenSSF Scorecard

Scorecard details

Package	Version	Score	Details
pip/annotated-types	0.7.0	Unknown	Unknown
pip/anyio	4.13.0	Unknown	Unknown
pip/certifi	2026.2.25	🟢 6.6	Details Check Score Reason Code-Review 🟢 5 Found 1/2 approved changesets -- score normalized to 5 Maintained 🟢 10 10 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10 Binary-Artifacts 🟢 10 no binaries found in the repo Security-Policy 🟢 10 security policy file detected Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Token-Permissions 🟢 10 GitHub workflow tokens follow principle of least privilege Pinned-Dependencies 🟢 5 dependency not pinned by hash detected -- score normalized to 5 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Fuzzing ⚠️ 0 project is not fuzzed License 🟢 9 license file detected Signed-Releases ⚠️ -1 no releases found Packaging 🟢 10 packaging workflow detected Branch-Protection ⚠️ 0 branch protection not enabled on development/release branches SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/colorama	0.4.6	Unknown	Unknown
pip/h11	0.16.0	🟢 4.4	Details Check Score Reason Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Code-Review 🟢 5 Found 9/18 approved changesets -- score normalized to 5 Maintained ⚠️ 0 0 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 0 Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Binary-Artifacts 🟢 10 no binaries found in the repo CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Security-Policy ⚠️ 0 security policy file not detected Fuzzing 🟢 10 project is fuzzed Signed-Releases ⚠️ -1 no releases found License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/httpcore	1.0.9	Unknown	Unknown
pip/httpx	0.28.1	Unknown	Unknown
pip/idna	3.11	Unknown	Unknown
pip/iniconfig	2.3.0	Unknown	Unknown
pip/packaging	26.0	Unknown	Unknown
pip/pluggy	1.6.0	Unknown	Unknown
pip/pydantic	2.13.0	Unknown	Unknown
pip/pydantic-core	2.46.0	🟢 7.1	Details Check Score Reason Code-Review 🟢 10 all changesets reviewed Maintained 🟢 10 30 commit(s) and 18 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Pinned-Dependencies 🟢 8 dependency not pinned by hash detected -- score normalized to 8 Fuzzing 🟢 10 project is fuzzed License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Security-Policy 🟢 10 security policy file detected Branch-Protection ⚠️ 1 branch protection is not maximal on development and all release branches Packaging 🟢 10 packaging workflow detected SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/pygments	2.20.0	Unknown	Unknown
pip/pytest	9.0.3	Unknown	Unknown
pip/pytest-asyncio	1.3.0	Unknown	Unknown
pip/python-dotenv	1.2.2	Unknown	Unknown
pip/ruff	0.15.10	Unknown	Unknown
pip/typing-extensions	4.15.0	Unknown	Unknown
pip/typing-inspection	0.4.2	Unknown	Unknown

Scanned Files

• .github/workflows/test.yml
• scrapegraph-py/requirements-test.txt
• scrapegraph-py/uv.lock
• uv.lock