feat: add ScrapeGraphAI client pattern#14
Closed
FrancescoSaverioZuppichini wants to merge 18 commits intomainfrom
Closed
feat: add ScrapeGraphAI client pattern#14FrancescoSaverioZuppichini wants to merge 18 commits intomainfrom
FrancescoSaverioZuppichini wants to merge 18 commits intomainfrom
Conversation
- Add all API request/response schemas matching v2 API exactly - Remove llmConfig from schemas (not exposed in SDK) - Add comprehensive types for all endpoints Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add scrape, extract, search, generateSchema endpoints - Add crawl namespace: start, get, stop, resume, delete - Add monitor namespace: create, list, get, update, delete, pause, resume - Add getCredits, checkHealth, getHistory, getHistoryEntry - Export schemas for client-side validation - Add zod dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace cancelled with deleted in ApiCrawlStatus - Add deleted to ApiHistoryStatus - Move types from src/types/index.ts to src/types.ts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update unit tests for new SDK structure - Add integration tests for live API - Fix schemas to use deleted instead of cancelled - Move types.ts out of folder Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…types - Add unit tests for all scrape formats (markdown, html, json, screenshot, summary, branding, links, images) - Add tests for fetchConfig options (mode, stealth, timeout, headers, cookies, country, scrolls) - Add tests for PDF/DOCX/image document scraping with OCR - Add extract tests for URL, HTML, and markdown inputs with schema - Add search tests with filters (location, timeRange, numResults) - Add crawl/monitor tests with full config options - Fix types to use z.input for request types (allows omitting fields with defaults) - Remove obsolete v1 integration_test.ts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Follow namespace pattern consistent with crawl.* and monitor.* Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename integration tests to *.spec.ts (excluded from CI) - `bun run test` runs only *.test.ts (unit tests for CI) - `bun run test:integration` runs *.spec.ts (live API tests) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove old v1 examples (smartscraper, markdownify, searchscraper, sitemap, agenticscraper) - Add scrape examples (basic, multi-format, pdf, fetchConfig) - Add extract examples (basic, with-schema) - Add search examples (basic, with-extraction) - Add monitor examples (basic, with-webhook) - Update crawl examples for namespace API - Update schema examples for camelCase fields - Update utilities for v2 response shapes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update all API documentation for v2 endpoints - Add examples table with path and description - Add scrape_json_extraction example - Enhance scrape_pdf and scrape_multi_format examples - Update environment variables section Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove process.exit() from crawl example, use if/else instead - Fix non-null assertion in crawl example - Fix undefined variable references in README crawl section - Use consistent example.com URLs across all examples Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Require SGAI_API_KEY env var instead of fallback Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ScrapeGraphAI({ apiKey? }) factory that reads SGAI_API_KEY from env
- Rename client methods: getCredits → credits, checkHealth → healthy
- Remove generateSchema (no longer in API)
- Update all examples to use new client pattern
- Update README with client usage
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ScrapeGraphAI({ apiKey? })factory that readsSGAI_API_KEYfrom env by defaultgetCredits→credits,checkHealth→healthygenerateSchema(no longer in API)Usage
Test plan
bun run testpassesbunx tsc --noEmitpassesbun run buildsucceeds🤖 Generated with Claude Code