Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .github/workflows/context7-refresh.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
set -euo pipefail

if [ -z "$CONTEXT7_API_KEY" ]; then
echo "::warning ::CONTEXT7_API_KEY secret is not configured. Configure it to enable automatic Context7 refreshes."
echo "::warning ::CONTEXT7_API_KEY is missing. Add the secret to refresh Context7 automatically."
exit 0
fi

Expand All @@ -51,30 +51,30 @@ jobs:
;;
400)
if grep -Eiq "active-task|being processed|already.*process" "$body_file"; then
echo "::warning ::Context7 is already processing another library. Retry on the next docs change."
echo "::warning ::Context7 is processing another library. Retry after the next docs change."
cat "$body_file"
elif grep -Eiq "too-early|minimum .* days|required between updates" "$body_file"; then
echo "::warning ::Context7 refresh is not due yet."
cat "$body_file"
else
echo "::error ::Context7 refresh returned HTTP $status."
echo "::error ::Context7 refresh returned HTTP $status. Review the response."
cat "$body_file"
exit 1
fi
;;
404)
echo "::warning ::Context7 library /xquik-dev/x-twitter-scraper-python is not available yet. Add processing may still be pending."
echo "::warning ::Context7 library /xquik-dev/x-twitter-scraper-python is unavailable. Wait for initial indexing."
cat "$body_file"
;;
401|403)
echo "::error ::Context7 authentication failed."
echo "::error ::Context7 authentication failed. Check CONTEXT7_API_KEY."
exit 1
;;
429)
echo "::warning ::Context7 refresh is rate limited. Retry on the next docs change."
echo "::warning ::Context7 rate-limited this refresh. Retry after the next docs change."
;;
*)
echo "::error ::Context7 refresh returned HTTP $status."
echo "::error ::Context7 refresh returned HTTP $status. Review the response."
cat "$body_file"
exit 1
;;
Expand Down
16 changes: 8 additions & 8 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
cff-version: 1.2.0
message: If you use this SDK, cite it using this metadata.
title: Xquik Python SDK for the X (Twitter) Scraper API
title: Xquik Python SDK for Twitter Search & X Automation
type: software
authors:
- name: Xquik
version: 0.4.3
date-released: 2026-07-21
version: 0.11.0
date-released: 2026-08-12
url: https://github.com/Xquik-dev/x-twitter-scraper-python
repository-code: https://github.com/Xquik-dev/x-twitter-scraper-python
license: Apache-2.0
keywords:
- Xquik
- Python
- SDK
- REST API
- Python SDK
- Twitter search
- Twitter API
- X automation
- webhooks
- X
- Twitter scraper
- follower export
2 changes: 0 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Contributing

Thank you for improving the Xquik Python SDK.

Read [GOVERNANCE.md](GOVERNANCE.md) before proposing major changes.

Follow the shared [Xquik contribution policy][contribution-policy].
Expand Down
10 changes: 5 additions & 5 deletions OPENSSF.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ No OpenSSF-defined ineligibility applies.
| CI | Pull requests and pushes run pinned, least-privilege workflows |
| Two-factor authentication | The Xquik-dev organization requires 2FA |

The current matrix covers 10,305 of 10,941 statements, or 94.19%.
The current matrix covers 12,010 of 12,654 statements, or 94.91%.

It covers 1,139 of 1,408 branches, or 80.89%.
It covers 1,149 of 1,418 branches, or 81.03%.

The matrix runs 2,989 Pydantic 2 tests.
The matrix runs 3,029 Pydantic 2 tests.

It runs 2,976 Pydantic 1 tests.
It runs 3,016 Pydantic 1 tests.

## Outstanding Gold Blockers

Expand All @@ -54,7 +54,7 @@ Do not claim Gold while any mandatory criterion remains unmet.
| Independent review | History does not prove 50% qualifying review coverage | Require and record independent reviews |
| Human security review | No completed review exists within 5 years | Commission and publish a scoped review |

This remediation pull request needs a different human reviewer.
Gold eligibility still requires review by a different human.

## Maintenance

Expand Down
93 changes: 23 additions & 70 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,82 +1,49 @@
# X (Twitter) Scraper Python SDK: Tweet Search, Profile Tweets, Followers & Posting

> **Xquik is an independent third-party service.** Not affiliated with X Corp.
> "Twitter" and "X" are trademarks of X Corp.
# Xquik Python SDK: Twitter Search, Followers & X Automation

[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/13738/badge)](https://www.bestpractices.dev/projects/13738)
[![Ask DeepWiki](https://deepwiki.com/badge.svg?url=https%3A%2F%2Fgithub.com%2FXquik-dev%2Fx-twitter-scraper-python)](https://deepwiki.com/Xquik-dev/x-twitter-scraper-python)
[![Skills.sh x-twitter-scraper Skill](https://skills.sh/b/xquik-dev/x-twitter-scraper)](https://skills.sh/xquik-dev/x-twitter-scraper)

<!-- prettier-ignore -->
[![PyPI version](https://img.shields.io/pypi/v/x_twitter_scraper.svg?label=pypi%20(stable))](https://pypi.org/project/x_twitter_scraper/)

Xquik Python SDK for the X (Twitter) Scraper API, a Twitter API SDK and X API alternative for typed tweet search, advanced Twitter search queries, profile tweets, user lookup, follower export, media download, media upload, monitoring, webhooks, and posting automation.
Use the Xquik Python SDK for Twitter search, timelines, profiles & followers.

Manage media, webhooks & X automation with sync or async clients.

Use it to get tweets from profiles, search tweets by keyword or operator query, send tweets, post replies, like, repost, follow, DM, run giveaway draws, and connect AI agents to X data without maintaining scraping infrastructure.
It provides a Twitter API alternative through documented Xquik REST routes.

[SDK API](api.md) | [REST API Docs](https://docs.xquik.com/api-reference/overview) | [OpenAPI Spec](https://xquik.com/openapi.json) | [Webhooks](https://docs.xquik.com/api-reference/webhooks/create) | [OAuth-First MCP Guide](https://docs.xquik.com/mcp/overview) | [TypeScript SDK](https://github.com/Xquik-dev/x-twitter-scraper-typescript)
[Python SDK Guide](https://docs.xquik.com/sdks/python) | [API Map](api.md) | [REST API](https://docs.xquik.com/api-reference/overview) | [Webhooks](https://docs.xquik.com/api-reference/webhooks/create) | [MCP Guide](https://docs.xquik.com/mcp/overview)

It is generated with [Stainless](https://www.stainless.com/).

## Choose the Python SDK

Choose this client for scripts, notebooks, workers, and Python services.
Use synchronous or asynchronous clients through the same resource structure.
Use Pydantic response models for validation and serialization.
Use sync or async resources with typed Pydantic response models.

## Common X Data Tasks
## Common Twitter & X Tasks

Use the linked Python API map for typed method names.
Map each task to its REST route.

| Customer Question | REST Route | Workflow Note |
| Task | REST Route | Usage |
| --- | --- | --- |
| How do I search tweets? | `GET /x/tweets/search` | Use keyword or advanced operator queries. |
| How do I read a profile timeline? | `GET /x/users/{id}/tweets` | Paginate bounded results. |
| How do I scrape followers? | `GET /x/users/{id}/followers` | Use an extraction for complete datasets. |
| How do I scrape following accounts? | `GET /x/users/{id}/following` | Use an extraction for complete datasets. |
| How do I read my home timeline? | `GET /x/timeline` | Approve this private read. |
| How do I monitor an account? | `POST /monitors` | Deliver events through HMAC webhooks. |
| How do I post or reply? | `POST /x/tweets` | Confirm the account and payload. |

## Tweet Search, Profile Tweets & User Lookup

Use `XTwitterScraper` in scripts, notebooks, and synchronous workers.
Use `AsyncXTwitterScraper` in concurrent services.
Both clients return typed Pydantic response models.

## Real-Time Monitoring & Webhooks

Verify HMAC signatures before handling monitor events in Python.
Query event history with the same synchronous or asynchronous client.
| Search tweets without the X API | `GET /x/tweets/search` | Use keyword or advanced operator queries. |
| Read an X profile timeline | `GET /x/users/{id}/tweets` | Paginate bounded results. |
| Scrape Twitter followers | `GET /x/users/{id}/followers` | Use an extraction for complete datasets. |
| Scrape following accounts | `GET /x/users/{id}/following` | Use an extraction for complete datasets. |
| Read a home timeline | `GET /x/timeline` | Approve this private read. |
| Export large X datasets | `POST /extractions` | Poll status, then download results. |
| Download or upload media | `/x/media/*` | Use typed file helpers. |
| Monitor an account | `POST /monitors` | Deliver events through HMAC webhooks. |
| Post or reply | `POST /x/tweets` | Confirm the account and payload. |

## AI Agent Workflows With MCP

Keep application code on the typed REST SDK. For MCP clients, add
`https://xquik.com/mcp`, then follow the [current client compatibility
path](https://docs.xquik.com/mcp/overview#client-compatibility). OAuth-capable
clients complete OAuth 2.1 in the browser. API-key fallback is client-specific.
ChatGPT custom apps require OAuth.

> **Codex OAuth compatibility:** Affected Codex releases discard the RFC 9207
> `iss` callback value even though Xquik returns it. If Codex reports
> `Authorization server response missing required issuer: expected https://xquik.com`,
> use `XQUIK_API_KEY` through the Codex `bearer_token_env_var` setting. Follow the
> [Codex OAuth troubleshooting guide](https://docs.xquik.com/guides/troubleshooting#codex-oauth-issuer-validation-error)
> and track [openai/codex#31573](https://github.com/openai/codex/issues/31573).
Use the typed REST SDK in application code. Add `https://xquik.com/mcp` to MCP clients.
Follow the [MCP guide](https://docs.xquik.com/mcp/overview) for current authentication support.

## Giveaway Draws & Extractions

Launch draws and extractions from Python data pipelines.
Export completed results for downstream analysis.

## Xquik SDK vs Building From Scratch

| Need | Xquik Python SDK | Building From Scratch |
| --- | --- | --- |
| Typed REST calls | Pydantic models and typed request params | Hand-maintained schemas |
| Sync and async apps | Synchronous and asynchronous clients | Separate clients and retries |
| Real-time events | Monitors plus HMAC webhooks | Separate queue and signing work |
| AI agent access | REST SDK plus MCP server | Custom bridge layer |
Run giveaway draws and export extraction results from Python pipelines.

## Package & Registry Trust

Expand All @@ -86,23 +53,9 @@ Export completed results for downstream analysis.
- Citation metadata: [CITATION.cff](CITATION.cff)
- Security policy: [SECURITY.md](SECURITY.md)

## FAQ

### Does this SDK support async Python?

Yes. Import `AsyncXTwitterScraper` and await the same API methods.

### Where are the REST endpoints documented?

Start with the [REST API overview](https://docs.xquik.com/api-reference/overview), then use [api.md](api.md) for generated method names.

### Can this work with AI agents?

Yes. Use the SDK in your app code and follow the [MCP guide](https://docs.xquik.com/mcp/overview) for agent clients.

## Documentation

The REST API documentation can be found on [docs.xquik.com](https://docs.xquik.com/api-reference/overview). The full API of this library can be found in [api.md](api.md).
Read the [REST API guide](https://docs.xquik.com/api-reference/overview) for contracts and [API map](api.md) for Python methods.

## Installation

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ CI also verifies dependencies, licenses, coverage, and reproducible archives.

Maintainers triage reports privately.

Confirmed fixes use private coordination when an embargo is needed.
Confirmed fixes use private coordination during an embargo.

Maintainers add regression tests and run every required gate.

Expand Down
7 changes: 3 additions & 4 deletions context7.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://context7.com/schema/context7.json",
"projectTitle": "Xquik Python SDK",
"description": "Xquik Python REST SDK for X/Twitter workflows: typed sync and async clients for tweet search, user lookup, webhooks, monitors, and extractions. Not affiliated with X Corp.",
"description": "Python SDK for Twitter search, followers, webhooks & X automation through Xquik. Not affiliated with X Corp.",
"branch": "main",
"excludeFolders": [
".devcontainer",
Expand Down Expand Up @@ -30,13 +30,12 @@
"Install from PyPI package x_twitter_scraper, then import x_twitter_scraper.",
"Authenticate with XTwitterScraper api_key or the X_TWITTER_SCRAPER_API_KEY environment variable.",
"Use README.md first for install, authentication, first request, retries, raw responses, uploads, pagination, webhooks, and common workflows.",
"For remote agent access, link to https://docs.xquik.com/mcp/overview. The MCP server is separate from this REST SDK and prefers OAuth 2.1. API-key fallback is client-specific; ChatGPT custom apps require OAuth.",
"If Codex reports Authorization server response missing required issuer: expected https://xquik.com, do not repeat OAuth. Affected Codex releases discard the RFC 9207 iss value even though Xquik returns it. Set XQUIK_API_KEY and bearer_token_env_var = \"XQUIK_API_KEY\" in Codex TOML, then use https://docs.xquik.com/guides/troubleshooting#codex-oauth-issuer-validation-error and track https://github.com/openai/codex/issues/31573.",
"Use https://docs.xquik.com/mcp/overview for current MCP authentication and client compatibility. MCP access is separate from this REST SDK.",
"Use api.md for the generated resource map, method names, request parameters, response types, and endpoint coverage.",
"Use pyproject.toml for package name, license, project URLs, Python version, dependency metadata, and package file includes.",
"Use src/x_twitter_scraper/resources and src/x_twitter_scraper/types for generated Python method names and typed request parameters.",
"Use src/x_twitter_scraper/resources/x/tweets/tweets.py for tweet search, posting, replies, retweets, likes, and tweet retrieval workflows.",
"Use src/x_twitter_scraper/resources/x/media.py, src/x_twitter_scraper/resources/monitors.py, src/x_twitter_scraper/resources/webhooks.py, and src/x_twitter_scraper/resources/extractions.py for media, monitoring, webhook, and bulk extraction workflows.",
"Use src/x_twitter_scraper/resources/x/media.py, src/x_twitter_scraper/resources/monitors/monitors.py, src/x_twitter_scraper/resources/webhooks.py, and src/x_twitter_scraper/resources/extractions.py for media, monitoring, webhook, and bulk extraction workflows.",
"For Python code, show XTwitterScraper for sync usage and AsyncXTwitterScraper for async usage.",
"Never expose API keys, bearer tokens, webhook signing values, user credentials, or local runtime secrets in examples, logs, or commits."
],
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
name = "x_twitter_scraper"
version = "0.11.0"
description = "Python library for the Xquik API. Not affiliated with X Corp."
description = "Python SDK for Twitter search, followers, webhooks & X automation through Xquik. Not affiliated with X Corp."
dynamic = ["readme"]
license = "Apache-2.0"
authors = [
Expand Down
8 changes: 4 additions & 4 deletions scripts/check-reproducible
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@ SOURCE_DATE_EPOCH=946684800 uv build --out-dir "$reproducibility_workspace/secon

first_archives=("$reproducibility_workspace"/first/*)
if [[ "${#first_archives[@]}" -ne 2 ]]; then
echo "Expected one wheel and one source archive." >&2
echo "Distribution count is wrong. Build one wheel and one source archive." >&2
exit 1
fi

for first_archive in "${first_archives[@]}"; do
archive_name="${first_archive##*/}"
second_archive="$reproducibility_workspace/second/$archive_name"
if [[ ! -f "$second_archive" ]]; then
echo "Second build omitted $archive_name." >&2
echo "Second build omitted $archive_name. Check the build inputs." >&2
exit 1
fi
cmp "$first_archive" "$second_archive"
Expand All @@ -39,12 +39,12 @@ for first_archive in "${first_archives[@]}"; do
archive_listing="$(tar -tzf "$first_archive")"
expected_license="/LICENSE"
else
echo "Unexpected distribution type: $archive_name." >&2
echo "Distribution type is unsupported: $archive_name. Build a wheel or source archive." >&2
exit 1
fi

if [[ "$archive_listing" != *"$expected_license"* ]]; then
echo "$archive_name omits its Apache-2.0 license." >&2
echo "$archive_name omits its Apache-2.0 license. Include LICENSE in the package." >&2
exit 1
fi
done
Expand Down
2 changes: 1 addition & 1 deletion scripts/format
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ cd "$(dirname "$0")/.."
echo "==> Running ruff"
uv run ruff format
uv run ruff check --fix .
# run formatting again to fix any inconsistencies when imports are stripped
# Format again after Ruff removes imports.
uv run ruff format

echo "==> Formatting docs"
Expand Down
2 changes: 1 addition & 1 deletion scripts/lint
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@ uv run pyright
echo "==> Running mypy"
uv run mypy .

echo "==> Making sure it imports"
echo "==> Checking package import"
uv run python -c 'import x_twitter_scraper'
13 changes: 4 additions & 9 deletions scripts/test
Original file line number Diff line number Diff line change
Expand Up @@ -7,33 +7,28 @@
set -e

cd "$(dirname "$0")/.."



export DEFER_PYDANTIC_BUILD=false

# Note that we need to specify the patch version here so that uv
# won't use unstable (alpha, beta, rc) releases for the tests
# Pin patch floors to exclude prereleases.
PY_VERSION_MIN=">=3.10.0"
PY_VERSION_MAX=">=3.14.0"

function run_tests() {
echo "==> Running tests with Pydantic v2"
uv run --isolated --all-extras pytest "$@"

# Skip Pydantic v1 tests on latest Python (not supported)
# Pydantic v1 does not support the latest Python.
if [[ "$UV_PYTHON" != "$PY_VERSION_MAX" ]]; then
echo "==> Running tests with Pydantic v1"
uv run --isolated --all-extras --group=pydantic-v1 pytest "$@"
fi
}

# If UV_PYTHON is already set in the environment, just run the command once
# Run one Python version when UV_PYTHON is set.
if [[ -n "$UV_PYTHON" ]]; then
run_tests "$@"
else
# If UV_PYTHON is not set, run the command for min and max versions

# Otherwise, test minimum and maximum versions.
echo "==> Running tests for Python $PY_VERSION_MIN"
UV_PYTHON="$PY_VERSION_MIN" run_tests "$@"

Expand Down
Loading