Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions docs/batch-client-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -1983,7 +1983,7 @@ Derived server-side from `zenrows_params` at submit time.
Precedence:
- `response_type: markdown|plaintext|pdf` → matching format.
- `autoparse: true`, `json_response: true`, or non-empty
`css_extractor` → `json`.
`css_extractor`/`extract` → `json`.
- otherwise → `html`.
Stamped on every successful task result and used to set the
right `Content-Type` when you fetch the content.
Expand Down Expand Up @@ -2699,8 +2699,8 @@ re-fetch the URL right before you download.
#### error

Non-empty only when `status = failed`. Stable strings —
e.g. `"results are larger then 1 gb"` when the combined
results exceed the 1 GiB cap.
e.g. `"results are larger than N gb"` when the combined
results exceed the export byte cap (20 GiB).

<a id="Export.download_url"></a>

Expand Down Expand Up @@ -2838,6 +2838,23 @@ source run's stored result. On chained retries,
`source_run_id` chases back to the run that actually
owns the result. Empty for normally-executed rows.

<a id="TaskResult.created_at"></a>

#### created\_at

When the task was created.

<a id="TaskResult.updated_at"></a>

#### updated\_at

The task's last update. Once a task is terminal
(`successful` / `failed`) this is when it finished — use
it to verify how fresh a result is on recurring jobs.
On a row inherited by a partial rerun (`source_run_id`
set) it is the time of the copy, not of the original
scrape.

<a id="TaskHistoryEvent"></a>

## TaskHistoryEvent Objects
Expand Down
51 changes: 35 additions & 16 deletions docs/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -423,11 +423,9 @@ paths:
- Honors `Idempotency-Key`: a repeat call with the same
key returns the original new-run id.

Large reruns are accepted asynchronously: past a server-side
row-count threshold (the same one that upgrades large
submissions) the call returns `202` and the new run's task
rows stream into storage off the request path. Treat `201`
and `202` both as success.
Reruns are accepted asynchronously: the call returns `202`
and the new run's task rows stream into storage off the
request path. Treat `201` and `202` both as success.
parameters:
- in: query
name: status
Expand All @@ -443,13 +441,13 @@ paths:
- $ref: '#/components/parameters/IdempotencyKey'
responses:
'201':
description: New run created (sync path). All task rows are written.
description: New run created (legacy sync fallback). All task rows are written.
content:
application/json:
schema: { $ref: '#/components/schemas/RerunJobResponse' }
'202':
description: |
New run accepted (large rerun). `retried_tasks` /
New run accepted. `retried_tasks` /
`inherited_tasks` are the final counts, but individual
task rows stream into storage off the request path.
`latest_run.ingest_status` is `pending` in this response
Expand Down Expand Up @@ -1001,8 +999,8 @@ paths:
404s. There is no delete / update — expiry is the cleanup.

The export fails with
`error: "results are larger then 1 gb"` if the combined size
of the results exceeds the 1 GiB cap.
`error: "results are larger than N gb"` if the combined size
of the results exceeds the export byte cap (20 GiB).
responses:
'202':
description: Export accepted; the zip is produced asynchronously.
Expand Down Expand Up @@ -1231,7 +1229,7 @@ components:
Precedence:
- `response_type: markdown|plaintext|pdf` → matching format.
- `autoparse: true`, `json_response: true`, or non-empty
`css_extractor` → `json`.
`css_extractor`/`extract` → `json`.
- otherwise → `html`.
Stamped on every successful task result and used to set the
right `Content-Type` when you fetch the content.
Expand Down Expand Up @@ -1264,16 +1262,23 @@ components:
`custom_headers`, `premium_proxy`, `proxy_country`,
`session_id`, `original_status`, `allowed_status_codes`,
`wait_for`, `wait`, `block_resources`, `json_response`,
`css_extractor`, `autoparse`, `response_type`, `outputs`.
`css_extractor`, `autoparse`, `extract`, `extract_fields`,
`response_type`, `outputs`.

`extract` selects the structured-extraction contract family
(`auto`, `native`, or `standard`); `extract_fields` optionally
narrows the returned fields. Extract is in private beta on the
scraper API — tasks against domains not enrolled fail with the
gateway's upgrade-required error.

`custom_headers` additionally accepts an object of header
name → value pairs (or that object serialised as a JSON
string), e.g. `{"custom_headers": {"Referer":
"https://www.google.com/"}}`. The headers are forwarded to
the scraping target, subject to Fetch's usual
the scraping target, subject to the scraper API's usual
header sanitisation. At most 32 headers / 8 KB serialised;
names must be valid HTTP tokens. Header filtering and
browser-header sanitisation are owned by Fetch,
browser-header sanitisation are owned by the scraper API,
not Conveyor. A task-level
`custom_headers` replaces the job-level object wholesale —
no per-name merging.
Expand Down Expand Up @@ -1770,7 +1775,7 @@ components:

TaskResult:
type: object
required: [task_id, run_id, url, status]
required: [task_id, run_id, url, status, created_at, updated_at]
properties:
task_id: { type: string }
external_id:
Expand Down Expand Up @@ -1813,6 +1818,20 @@ components:
`source_run_id` chases back to the run that actually
owns the result. Empty for normally-executed rows.
spend: { $ref: '#/components/schemas/TaskSpend' }
created_at:
type: string
format: date-time
description: When the task was created.
updated_at:
type: string
format: date-time
description: |
The task's last update. Once a task is terminal
(`successful` / `failed`) this is when it finished — use
it to verify how fresh a result is on recurring jobs.
On a row inherited by a partial rerun (`source_run_id`
set) it is the time of the copy, not of the original
scrape.

ListResultsResponse:
type: object
Expand Down Expand Up @@ -2159,8 +2178,8 @@ components:
nullable: true
description: |
Non-empty only when `status = failed`. Stable strings —
e.g. `"results are larger then 1 gb"` when the combined
results exceed the 1 GiB cap.
e.g. `"results are larger than N gb"` when the combined
results exceed the export byte cap (20 GiB).
download_url:
type: string
format: uri
Expand Down
22 changes: 18 additions & 4 deletions src/zenrows/batch/models.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# generated by datamodel-codegen:
# filename: openapi.yaml
# timestamp: 2026-08-21T10:17:51+00:00
# timestamp: 2026-08-25T14:35:19+00:00

from __future__ import annotations

Expand Down Expand Up @@ -130,7 +130,7 @@ class Format(Enum):
Precedence:
- `response_type: markdown|plaintext|pdf` → matching format.
- `autoparse: true`, `json_response: true`, or non-empty
`css_extractor` → `json`.
`css_extractor`/`extract` → `json`.
- otherwise → `html`.
Stamped on every successful task result and used to set the
right `Content-Type` when you fetch the content.
Expand Down Expand Up @@ -813,8 +813,8 @@ class Export(BaseModel):
error: str | None = None
"""
Non-empty only when `status = failed`. Stable strings —
e.g. `"results are larger then 1 gb"` when the combined
results exceed the 1 GiB cap.
e.g. `"results are larger than N gb"` when the combined
results exceed the export byte cap (20 GiB).

"""
download_url: AnyUrl | None = None
Expand Down Expand Up @@ -945,6 +945,20 @@ class TaskResult(BaseModel):

"""
spend: TaskSpend | None = None
created_at: AwareDatetime
"""
When the task was created.
"""
updated_at: AwareDatetime
"""
The task's last update. Once a task is terminal
(`successful` / `failed`) this is when it finished — use
it to verify how fresh a result is on recurring jobs.
On a row inherited by a partial rerun (`source_run_id`
set) it is the time of the copy, not of the original
scrape.

"""


class ListResultsResponse(BaseModel):
Expand Down
13 changes: 12 additions & 1 deletion tests/test_batch_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ def test_iter_results_auto_paginates(client: ZenRowsBatchClient):
"run_id": "01R000000000000000000A",
"url": "https://example.com/a",
"status": "successful",
"created_at": "2026-08-25T12:00:00Z",
"updated_at": "2026-08-25T12:00:05Z",
}
],
"next_cursor": "abc",
Expand All @@ -110,6 +112,8 @@ def test_iter_results_auto_paginates(client: ZenRowsBatchClient):
"run_id": "01R000000000000000000A",
"url": "https://example.com/b",
"status": "successful",
"created_at": "2026-08-25T12:00:00Z",
"updated_at": "2026-08-25T12:00:05Z",
}
],
"next_cursor": None,
Expand Down Expand Up @@ -999,7 +1003,14 @@ def _task_result(
result_url: str | None = None,
status: str = "successful",
) -> TaskResult:
data = {"task_id": task_id, "run_id": "R", "url": "https://example.com", "status": status}
data = {
"task_id": task_id,
"run_id": "R",
"url": "https://example.com",
"status": status,
"created_at": "2026-08-25T12:00:00Z",
"updated_at": "2026-08-25T12:00:05Z",
}
if external_id:
data["external_id"] = external_id
if result_type:
Expand Down
Loading