Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions skills/firecrawl-scrape/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,17 @@ firecrawl scrape "https://example.com/pricing" --query "What is the enterprise p
| `--include-tags <tags>` | Only include these HTML tags |
| `--exclude-tags <tags>` | Exclude these HTML tags |
| `--redact-pii` | Redact personally identifiable information from output |
| `--max-age <milliseconds>` | Maximum age of cached content. Use `0` to request fresh retrieval |
| `--json` | Emit JSON so metadata such as `metadata.cache` can be inspected |
| `-o, --output <path>` | Output file path |

## Freshness and cache provenance

- Use `--max-age 0` to request a fresh retrieval path before an action that depends on current state. It skips Firecrawl's index cache and can be slower or more failure-prone than allowing cache.
- Use `--json` when you need to inspect metadata. An attested Firecrawl-index cache hit includes `metadata.cache.source = "firecrawl-index"` and `metadata.cache.cachedAt`.
- If `metadata.cache` is absent, do not infer miss, fresh scrape, bypass, or source-page liveness. It only means cache provenance was not attested.
- For job postings, event pages, inventory, and other stateful records, verify the business object itself before acting: prefer an ATS/source API when available, or check the returned content for active apply buttons, closed labels, canonical IDs, and redirects. HTTP 200 and rendered HTML are not enough.

## Tips

- **Prefer plain scrape over `--query`.** Scrape to a file, then use `grep`, `head`, or read the markdown directly — you can search and reason over the full content yourself. Use `--query` only when you want a single targeted answer without saving the page (costs 5 extra credits).
Expand Down
Loading