From 6bebae750c3518e15f419f89119d42eba79ba555 Mon Sep 17 00:00:00 2001 From: Rishet Mehra Date: Tue, 11 Aug 2026 00:27:50 +0530 Subject: [PATCH 1/8] Add opinions community plugin: no-login social research across public platforms --- README.md | 1 + plugins/opinions/README.md | 108 ++++++++++++++++++++++++++++ plugins/opinions/bluesky-posts.js | 89 +++++++++++++++++++++++ plugins/opinions/hackermind.js | 80 +++++++++++++++++++++ plugins/opinions/lobsters.js | 73 +++++++++++++++++++ plugins/opinions/package.json | 9 +++ plugins/opinions/research.js | 105 +++++++++++++++++++++++++++ plugins/opinions/webcmd-plugin.json | 10 +++ webcmd-plugin.json | 10 +++ 9 files changed, 485 insertions(+) create mode 100644 plugins/opinions/README.md create mode 100644 plugins/opinions/bluesky-posts.js create mode 100644 plugins/opinions/hackermind.js create mode 100644 plugins/opinions/lobsters.js create mode 100644 plugins/opinions/package.json create mode 100644 plugins/opinions/research.js create mode 100644 plugins/opinions/webcmd-plugin.json diff --git a/README.md b/README.md index 9ad1157..daa7e2c 100644 --- a/README.md +++ b/README.md @@ -140,6 +140,7 @@ Webcmd Cloud can run supported commands and browser sessions on hosted infrastru | Plugin | Description | Author | | --- | --- | --- | +| [`opinions`](./plugins/opinions/) | No-login social opinions & problems research across public platforms | [Rishet Mehra](https://github.com/Rishet11) | | [`pypi`](./plugins/pypi/) | Inspect public Python package metadata, downloads, and releases from PyPI | [Kemal Kaya](https://github.com/yoldaolmak) | | [`skyscanner`](./plugins/skyscanner/) | Skyscanner flight search commands for Webcmd | [Rishabh](https://github.com/rishabhraj36) | diff --git a/plugins/opinions/README.md b/plugins/opinions/README.md new file mode 100644 index 0000000..26d2edd --- /dev/null +++ b/plugins/opinions/README.md @@ -0,0 +1,108 @@ +# webcmd-plugin-opinions + +No-login social opinions & problems research across **public** platforms. +Research what people think about a topic, product, or problem — no browser, no +login, no credentials. + +## Install + +```bash +webcmd plugin install github:rishetmehra/webcmd-plugin-opinions +``` + +## Commands + +| Command | Type | Description | +|---------|------|-------------| +| `opinions bluesky-posts ` | public | Recent posts from a public Bluesky account (no login) | +| `opinions hackermind ` | public | Search Hacker News stories & comments for opinions (no login) | +| `opinions lobsters [--sort newest\|active\|hot]` | public | Lobste.rs newest / active / hot discussions (no login) | +| `opinions research ` | public | Aggregate opinions/problems across Hacker News + Lobste.rs in one feed | + +## Examples + +```bash +# Read what a public Bluesky account (Twitter/X-like) is saying, no login +webcmd opinions bluesky-posts paulgraham.bsky.social --limit 10 -f json + +# Search Hacker News for what people think about a product/problem +webcmd opinions hackermind "saas pricing" --limit 10 -f json + +# Search comment text specifically for problem reports +webcmd opinions hackermind "billing is confusing" --limit 10 --scope comment -f json + +# One command, opinions across platforms +webcmd opinions research "LLM" --limit 10 -f json +``` + +## Full multi-platform research setup (current state) + +This machine has the `social` profile logged into **X/Twitter, Instagram, +Reddit, LinkedIn, and YouTube** (TikTok and Facebook are skipped — TikTok is +banned in India, Facebook not needed). All commands below run **headless** +(background browser) after the one-time login. + +### How authentication works (important) + +webcmd does **not** use your password in code or `.env`. It uses a one-time +interactive login per platform into a saved browser **profile** (`social`). +The profile stores the session cookies (`auth_token` for X, `sessionid` for +Instagram, `reddit_session` for Reddit, etc.). Commands read cookies from the +profile — never your credentials. This is why a one-time login is required and +why `.env` passwords don't work with the official plugins. + +### Two tiers of access + +**Tier 1 — no login (public APIs), always headless:** +```bash +webcmd opinions bluesky-posts --limit 10 -f json +webcmd opinions hackermind "" --limit 10 -f json +webcmd opinions lobsters --limit 10 -f json +webcmd opinions research "" --limit 10 -f json # aggregate +``` + +**Tier 2 — logged-in platforms (uses `social` profile):** +```bash +# X / Twitter +webcmd --profile social twitter search "" --limit 10 -f json +webcmd --profile social twitter tweets --limit 10 -f json +webcmd --profile social twitter trending -f json + +# Reddit (public opinion goldmine) +webcmd --profile social reddit search "" --limit 10 -f json +webcmd --profile social reddit subreddit --limit 10 -f json +webcmd --profile social reddit read -f json + +# Instagram +webcmd --profile social instagram search "" -f json +webcmd --profile social instagram profile -f json + +# LinkedIn +webcmd --profile social linkedin people-search "" -f json +webcmd --profile social linkedin timeline -f json + +# YouTube +webcmd --profile social youtube search "" --limit 10 -f json +webcmd --profile social youtube comments -f json +``` + +### Re-authenticating after a session expires + +If a platform returns `AUTH_REQUIRED`, redo the one-time login for that site: +```bash +webcmd --profile social login # sign in in the browser window +webcmd --profile social whoami # verify +``` + +## Development + +```bash +# Install locally for development (symlinked, changes reflect immediately) +webcmd plugin install file:///Users/rishetmehra/webcmd-opinions + +# Verify commands are registered +webcmd list | grep -A12 opinions + +# Validate definitions +webcmd validate opinions +``` diff --git a/plugins/opinions/bluesky-posts.js b/plugins/opinions/bluesky-posts.js new file mode 100644 index 0000000..7c2cd91 --- /dev/null +++ b/plugins/opinions/bluesky-posts.js @@ -0,0 +1,89 @@ +/** + * opinions bluesky-posts — read what a public Bluesky account is saying. + * + * No login. Uses the public Bluesky API (public.api.bsky.app) author feed + * endpoint, which returns a user's recent posts without authentication. + * This is the closest no-login analog to reading someone's public X/Twitter + * timeline. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { + ArgumentError, + CommandExecutionError, + EmptyResultError, +} from '@agentrhq/webcmd/errors'; + +const API = 'https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed'; +const HANDLE = /^[a-zA-Z0-9][a-zA-Z0-9.-]*\.[a-zA-Z]{2,}$/; + +function requireHandle(value) { + const s = String(value ?? '').trim().toLowerCase(); + if (!s || !HANDLE.test(s)) { + throw new ArgumentError('bluesky handle is required, e.g. "bsky.app" or "user.bsky.social"'); + } + return s; +} + +cli({ + site: 'opinions', + name: 'bluesky-posts', + access: 'read', + description: "Recent posts from a public Bluesky account (no login)", + domain: 'public.api.bsky.app', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'handle', required: true, positional: true, help: 'Bluesky handle (e.g. bsky.app)' }, + { name: 'limit', type: 'int', default: 20, help: 'Number of posts' }, + ], + columns: ['rank', 'uri', 'created_at', 'text', 'likes', 'replies', 'reposts', 'url'], + func: async (kwargs) => { + const handle = requireHandle(kwargs.handle); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) { + throw new ArgumentError('limit must be a positive integer'); + } + const limit = Math.min(raw, 100); + + const url = new URL(API); + url.searchParams.set('actor', handle); + url.searchParams.set('limit', String(limit)); + + let json; + try { + const res = await fetch(url, { + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-opinions/0.1)' }, + }); + if (!res.ok) { + throw new CommandExecutionError(`Bluesky API request failed: HTTP ${res.status}`); + } + json = await res.json(); + } catch (err) { + if (err instanceof CommandExecutionError) throw err; + throw new CommandExecutionError(`Bluesky API request failed: ${err instanceof Error ? err.message : String(err)}`); + } + + const feed = Array.isArray(json?.feed) ? json.feed : []; + if (!feed.length) { + throw new EmptyResultError('opinions/bluesky-posts', `no posts found for "${handle}"`); + } + + return feed.slice(0, limit).map((entry, index) => { + const post = entry?.post ?? {}; + const author = post.author ?? {}; + const record = post.record ?? {}; + const uri = String(post.uri ?? ''); + const rkey = uri.split('/').pop() ?? ''; + return { + rank: index + 1, + uri, + created_at: String(record.createdAt ?? post.indexedAt ?? ''), + text: String(record.text ?? '').replace(/\s*\n+/g, ' ').trim(), + likes: post.likeCount ?? 0, + replies: post.replyCount ?? 0, + reposts: post.repostCount ?? 0, + url: `https://bsky.app/profile/${author.handle ?? handle}/post/${rkey}`, + }; + }); + }, +}); \ No newline at end of file diff --git a/plugins/opinions/hackermind.js b/plugins/opinions/hackermind.js new file mode 100644 index 0000000..20aebac --- /dev/null +++ b/plugins/opinions/hackermind.js @@ -0,0 +1,80 @@ +/** + * opinions hackermind — search Hacker News stories & comments for opinions. + * + * No login. Uses the public HN Algolia API, which indexes stories and + * comments. Good for researching what the tech/startup community is saying + * about a topic, product, or problem. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; + +const API = 'https://hn.algolia.com/api/v1/search'; + +function requireQuery(value) { + const s = String(value ?? '').trim(); + if (!s) throw new ArgumentError('a search query is required'); + return s; +} + +cli({ + site: 'opinions', + name: 'hackermind', + tags: ['search'], + access: 'read', + description: "Search Hacker News stories & comments for opinions (no login)", + domain: 'hn.algolia.com', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'query', required: true, positional: true, help: 'Topic, product, or problem to research' }, + { name: 'limit', type: 'int', default: 20, help: 'Number of results' }, + { + name: 'scope', + default: 'story', + help: 'What to search: story (headlines) or comment (reply text)', + choices: ['story', 'comment'], + }, + ], + columns: ['rank', 'id', 'object_type', 'title', 'author', 'points', 'comments', 'created_at', 'url'], + func: async (kwargs) => { + const query = requireQuery(kwargs.query); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) { + throw new ArgumentError('limit must be a positive integer'); + } + const limit = Math.min(raw, 100); + const scope = String(kwargs.scope ?? 'story'); + + const url = new URL(API); + url.searchParams.set('query', query); + url.searchParams.set('tags', scope === 'comment' ? 'comment' : 'story'); + url.searchParams.set('hitsPerPage', String(limit)); + + let json; + try { + const res = await fetch(url); + if (!res.ok) throw new CommandExecutionError(`HN Algolia request failed: HTTP ${res.status}`); + json = await res.json(); + } catch (err) { + if (err instanceof CommandExecutionError) throw err; + throw new CommandExecutionError(`HN Algolia request failed: ${err instanceof Error ? err.message : String(err)}`); + } + + const hits = Array.isArray(json?.hits) ? json.hits : []; + if (!hits.length) { + throw new EmptyResultError('opinions/hackermind', `no results for "${query}"`); + } + + return hits.slice(0, limit).map((h, index) => ({ + rank: index + 1, + id: h.objectID, + object_type: scope, + title: String(h.title ?? h.story_title ?? h.comment_text ?? '').replace(/<[^>]+>/g, '').trim(), + author: String(h.author ?? ''), + points: h.points ?? 0, + comments: h.num_comments ?? 0, + created_at: String(h.created_at ?? ''), + url: String(h.url ?? `https://news.ycombinator.com/item?id=${h.objectID}`), + })); + }, +}); \ No newline at end of file diff --git a/plugins/opinions/lobsters.js b/plugins/opinions/lobsters.js new file mode 100644 index 0000000..b1da867 --- /dev/null +++ b/plugins/opinions/lobsters.js @@ -0,0 +1,73 @@ +/** + * opinions lobsters — Lobste.rs newest / active discussions. + * + * No login. Uses the public lobste.rs JSON endpoint. Lobste.rs is a + * Reddit/HN-style community; good for surfacing what developers are + * discussing about a topic right now. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; + +const SORTS = { + newest: 'https://lobste.rs/newest.json', + active: 'https://lobste.rs/active.json', + hot: 'https://lobste.rs/hottest.json', +}; + +function requireSort(value) { + const s = String(value ?? 'newest').toLowerCase(); + if (s === 'new') s = 'newest'; + if (!SORTS[s]) throw new ArgumentError(`sort must be one of: ${Object.keys(SORTS).join(', ')}`); + return s; +} + +cli({ + site: 'opinions', + name: 'lobsters', + access: 'read', + description: "Lobste.rs newest / active / hot discussions (no login)", + domain: 'lobste.rs', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'limit', type: 'int', default: 20, help: 'Number of stories' }, + { name: 'sort', default: 'newest', help: 'Sort order: newest, active, hot', choices: ['newest', 'active', 'hot'] }, + ], + columns: ['rank', 'id', 'title', 'author', 'score', 'comments', 'created_at', 'tags', 'url'], + func: async (kwargs) => { + const sort = requireSort(kwargs.sort); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) { + throw new ArgumentError('limit must be a positive integer'); + } + const limit = Math.min(raw, 100); + + let rows; + try { + const res = await fetch(SORTS[sort], { + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-opinions/0.1)' }, + }); + if (!res.ok) throw new CommandExecutionError(`lobste.rs request failed: HTTP ${res.status}`); + rows = await res.json(); + } catch (err) { + if (err instanceof CommandExecutionError) throw err; + throw new CommandExecutionError(`lobste.rs request failed: ${err instanceof Error ? err.message : String(err)}`); + } + + if (!Array.isArray(rows) || !rows.length) { + throw new EmptyResultError('opinions/lobsters', 'lobste.rs returned no stories'); + } + + return rows.slice(0, limit).map((s, index) => ({ + rank: index + 1, + id: s.short_id, + title: String(s.title ?? '').trim(), + author: String(s.submitter_user ?? ''), + score: s.score ?? 0, + comments: s.comment_count ?? 0, + created_at: String(s.created_at ?? ''), + tags: Array.isArray(s.tags) ? s.tags.join(', ') : '', + url: String(s.comments_url ?? s.short_id_url ?? ''), + })); + }, +}); \ No newline at end of file diff --git a/plugins/opinions/package.json b/plugins/opinions/package.json new file mode 100644 index 0000000..c136ad1 --- /dev/null +++ b/plugins/opinions/package.json @@ -0,0 +1,9 @@ +{ + "name": "webcmd-plugin-opinions", + "version": "0.1.0", + "type": "module", + "description": "No-login social opinions & problems research across public platforms", + "peerDependencies": { + "@agentrhq/webcmd": ">=0.5.3" + } +} diff --git a/plugins/opinions/research.js b/plugins/opinions/research.js new file mode 100644 index 0000000..5045a24 --- /dev/null +++ b/plugins/opinions/research.js @@ -0,0 +1,105 @@ +/** + * opinions research — aggregate opinions about a topic across no-login + * public platforms (Hacker News + Lobste.rs) into one feed. + * + * No login. Combines HN Algolia stories and Lobste.rs newest stories filtered + * by keyword, tagging each row with its platform. Use for quick opinion / + * problem reconnaissance on a topic, product, or persona. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; + +const HN_SEARCH = 'https://hn.algolia.com/api/v1/search'; +const LOBSTERS = 'https://lobste.rs/newest.json'; + +function requireQuery(value) { + const s = String(value ?? '').trim(); + if (!s) throw new ArgumentError('a research topic/query is required'); + return s; +} + +async function hnSearch(query, limit) { + const url = new URL(HN_SEARCH); + url.searchParams.set('query', query); + url.searchParams.set('tags', 'story'); + url.searchParams.set('hitsPerPage', String(limit)); + const res = await fetch(url); + if (!res.ok) return []; + const json = await res.json(); + return (Array.isArray(json?.hits) ? json.hits : []).slice(0, limit).map((h) => ({ + platform: 'hackernews', + title: String(h.title ?? h.story_title ?? '').trim(), + author: String(h.author ?? ''), + score: h.points ?? 0, + comments: h.num_comments ?? 0, + created_at: String(h.created_at ?? ''), + url: String(h.url ?? `https://news.ycombinator.com/item?id=${h.objectID}`), + text: '', + })); +} + +async function lobstersSearch(query, limit) { + const res = await fetch(LOBSTERS, { + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-opinions/0.1)' }, + }); + if (!res.ok) return []; + const rows = await res.json(); + if (!Array.isArray(rows)) return []; + const q = query.toLowerCase(); + return rows + .filter((s) => + String(s.title ?? '').toLowerCase().includes(q) || + String(s.description_plain ?? '').toLowerCase().includes(q) || + (Array.isArray(s.tags) && s.tags.some((t) => t.toLowerCase().includes(q))), + ) + .slice(0, limit) + .map((s) => ({ + platform: 'lobsters', + title: String(s.title ?? '').trim(), + author: String(s.submitter_user ?? ''), + score: s.score ?? 0, + comments: s.comment_count ?? 0, + created_at: String(s.created_at ?? ''), + url: String(s.comments_url ?? ''), + text: String(s.description_plain ?? ''), + })); +} + +cli({ + site: 'opinions', + name: 'research', + tags: ['search'], + access: 'read', + description: "Aggregate opinions/problems about a topic across no-login platforms (Hacker News + Lobste.rs)", + domain: 'hn.algolia.com', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'query', required: true, positional: true, help: 'Topic, product, or problem to research' }, + { name: 'limit', type: 'int', default: 20, help: 'Number of results per platform' }, + ], + columns: ['platform', 'title', 'author', 'score', 'comments', 'created_at', 'url', 'text'], + func: async (kwargs) => { + const query = requireQuery(kwargs.query); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) { + throw new ArgumentError('limit must be a positive integer'); + } + const limit = Math.min(raw, 50); + const perPlatform = Math.ceil(limit / 2); + + let rows; + try { + const [hn, lob] = await Promise.all([hnSearch(query, perPlatform), lobstersSearch(query, perPlatform)]); + rows = [...hn, ...lob]; + } catch (err) { + throw new CommandExecutionError(`research aggregation failed: ${err instanceof Error ? err.message : String(err)}`); + } + + if (!rows.length) { + throw new EmptyResultError('opinions/research', `no opinions found across platforms for "${query}"`); + } + + return rows.slice(0, limit); + }, +}); \ No newline at end of file diff --git a/plugins/opinions/webcmd-plugin.json b/plugins/opinions/webcmd-plugin.json new file mode 100644 index 0000000..a3bd926 --- /dev/null +++ b/plugins/opinions/webcmd-plugin.json @@ -0,0 +1,10 @@ +{ + "name": "opinions", + "version": "0.1.0", + "description": "No-login social opinions & problems research across public platforms", + "webcmd": ">=0.5.3", + "author": { + "name": "Rishet Mehra", + "handle": "Rishet11" + } +} diff --git a/webcmd-plugin.json b/webcmd-plugin.json index 33ed848..5e87de5 100644 --- a/webcmd-plugin.json +++ b/webcmd-plugin.json @@ -804,6 +804,16 @@ "handle": "agentrhq" } }, + "opinions": { + "path": "plugins/opinions", + "version": "0.1.0", + "description": "No-login social opinions & problems research across public platforms", + "webcmd": ">=0.5.3", + "author": { + "name": "Rishet Mehra", + "handle": "Rishet11" + } + }, "osv": { "path": "plugins/osv", "version": "0.1.0", From c82e6be19e8a3528fa67c776e1b43aae322ab568 Mon Sep 17 00:00:00 2001 From: Rishet Mehra Date: Tue, 11 Aug 2026 00:44:44 +0530 Subject: [PATCH 2/8] opinions: use camelCase output fields per webcmd conventions --- plugins/opinions/bluesky-posts.js | 10 +++++----- plugins/opinions/hackermind.js | 10 +++++----- plugins/opinions/lobsters.js | 6 +++--- plugins/opinions/research.js | 10 +++++----- 4 files changed, 18 insertions(+), 18 deletions(-) diff --git a/plugins/opinions/bluesky-posts.js b/plugins/opinions/bluesky-posts.js index 7c2cd91..c3ffa00 100644 --- a/plugins/opinions/bluesky-posts.js +++ b/plugins/opinions/bluesky-posts.js @@ -36,7 +36,7 @@ cli({ { name: 'handle', required: true, positional: true, help: 'Bluesky handle (e.g. bsky.app)' }, { name: 'limit', type: 'int', default: 20, help: 'Number of posts' }, ], - columns: ['rank', 'uri', 'created_at', 'text', 'likes', 'replies', 'reposts', 'url'], + columns: ['rank', 'uri', 'createdAt', 'text', 'likeCount', 'replyCount', 'repostCount', 'url'], func: async (kwargs) => { const handle = requireHandle(kwargs.handle); const raw = Number(kwargs.limit ?? 20); @@ -77,11 +77,11 @@ cli({ return { rank: index + 1, uri, - created_at: String(record.createdAt ?? post.indexedAt ?? ''), + createdAt: String(record.createdAt ?? post.indexedAt ?? ''), text: String(record.text ?? '').replace(/\s*\n+/g, ' ').trim(), - likes: post.likeCount ?? 0, - replies: post.replyCount ?? 0, - reposts: post.repostCount ?? 0, + likeCount: post.likeCount ?? 0, + replyCount: post.replyCount ?? 0, + repostCount: post.repostCount ?? 0, url: `https://bsky.app/profile/${author.handle ?? handle}/post/${rkey}`, }; }); diff --git a/plugins/opinions/hackermind.js b/plugins/opinions/hackermind.js index 20aebac..eb6d4b9 100644 --- a/plugins/opinions/hackermind.js +++ b/plugins/opinions/hackermind.js @@ -35,7 +35,7 @@ cli({ choices: ['story', 'comment'], }, ], - columns: ['rank', 'id', 'object_type', 'title', 'author', 'points', 'comments', 'created_at', 'url'], + columns: ['rank', 'id', 'objectType', 'title', 'author', 'score', 'commentCount', 'createdAt', 'url'], func: async (kwargs) => { const query = requireQuery(kwargs.query); const raw = Number(kwargs.limit ?? 20); @@ -68,12 +68,12 @@ cli({ return hits.slice(0, limit).map((h, index) => ({ rank: index + 1, id: h.objectID, - object_type: scope, + objectType: scope, title: String(h.title ?? h.story_title ?? h.comment_text ?? '').replace(/<[^>]+>/g, '').trim(), author: String(h.author ?? ''), - points: h.points ?? 0, - comments: h.num_comments ?? 0, - created_at: String(h.created_at ?? ''), + score: h.points ?? 0, + commentCount: h.num_comments ?? 0, + createdAt: String(h.created_at ?? ''), url: String(h.url ?? `https://news.ycombinator.com/item?id=${h.objectID}`), })); }, diff --git a/plugins/opinions/lobsters.js b/plugins/opinions/lobsters.js index b1da867..e4eecb2 100644 --- a/plugins/opinions/lobsters.js +++ b/plugins/opinions/lobsters.js @@ -33,7 +33,7 @@ cli({ { name: 'limit', type: 'int', default: 20, help: 'Number of stories' }, { name: 'sort', default: 'newest', help: 'Sort order: newest, active, hot', choices: ['newest', 'active', 'hot'] }, ], - columns: ['rank', 'id', 'title', 'author', 'score', 'comments', 'created_at', 'tags', 'url'], + columns: ['rank', 'id', 'title', 'author', 'score', 'commentCount', 'createdAt', 'tags', 'url'], func: async (kwargs) => { const sort = requireSort(kwargs.sort); const raw = Number(kwargs.limit ?? 20); @@ -64,8 +64,8 @@ cli({ title: String(s.title ?? '').trim(), author: String(s.submitter_user ?? ''), score: s.score ?? 0, - comments: s.comment_count ?? 0, - created_at: String(s.created_at ?? ''), + commentCount: s.comment_count ?? 0, + createdAt: String(s.created_at ?? ''), tags: Array.isArray(s.tags) ? s.tags.join(', ') : '', url: String(s.comments_url ?? s.short_id_url ?? ''), })); diff --git a/plugins/opinions/research.js b/plugins/opinions/research.js index 5045a24..ee73f47 100644 --- a/plugins/opinions/research.js +++ b/plugins/opinions/research.js @@ -31,8 +31,8 @@ async function hnSearch(query, limit) { title: String(h.title ?? h.story_title ?? '').trim(), author: String(h.author ?? ''), score: h.points ?? 0, - comments: h.num_comments ?? 0, - created_at: String(h.created_at ?? ''), + commentCount: h.num_comments ?? 0, + createdAt: String(h.created_at ?? ''), url: String(h.url ?? `https://news.ycombinator.com/item?id=${h.objectID}`), text: '', })); @@ -58,8 +58,8 @@ async function lobstersSearch(query, limit) { title: String(s.title ?? '').trim(), author: String(s.submitter_user ?? ''), score: s.score ?? 0, - comments: s.comment_count ?? 0, - created_at: String(s.created_at ?? ''), + commentCount: s.comment_count ?? 0, + createdAt: String(s.created_at ?? ''), url: String(s.comments_url ?? ''), text: String(s.description_plain ?? ''), })); @@ -78,7 +78,7 @@ cli({ { name: 'query', required: true, positional: true, help: 'Topic, product, or problem to research' }, { name: 'limit', type: 'int', default: 20, help: 'Number of results per platform' }, ], - columns: ['platform', 'title', 'author', 'score', 'comments', 'created_at', 'url', 'text'], + columns: ['platform', 'title', 'author', 'score', 'commentCount', 'createdAt', 'url', 'text'], func: async (kwargs) => { const query = requireQuery(kwargs.query); const raw = Number(kwargs.limit ?? 20); From c6749d99b1221e9862f6bb845f8ded53669b5aac Mon Sep 17 00:00:00 2001 From: Rishet Mehra Date: Tue, 11 Aug 2026 01:15:05 +0530 Subject: [PATCH 3/8] Rename opinions plugin to omnisearch (OmniSearch) --- README.md | 2 +- plugins/omnisearch/README.md | 65 +++++++++++ .../{opinions => omnisearch}/bluesky-posts.js | 8 +- .../{opinions => omnisearch}/hackermind.js | 8 +- plugins/{opinions => omnisearch}/lobsters.js | 8 +- plugins/omnisearch/package.json | 9 ++ plugins/{opinions => omnisearch}/research.js | 10 +- plugins/omnisearch/webcmd-plugin.json | 10 ++ plugins/opinions/README.md | 108 ------------------ plugins/opinions/package.json | 9 -- plugins/opinions/webcmd-plugin.json | 10 -- webcmd-plugin.json | 20 ++-- 12 files changed, 112 insertions(+), 155 deletions(-) create mode 100644 plugins/omnisearch/README.md rename plugins/{opinions => omnisearch}/bluesky-posts.js (93%) rename plugins/{opinions => omnisearch}/hackermind.js (90%) rename plugins/{opinions => omnisearch}/lobsters.js (92%) create mode 100644 plugins/omnisearch/package.json rename plugins/{opinions => omnisearch}/research.js (89%) create mode 100644 plugins/omnisearch/webcmd-plugin.json delete mode 100644 plugins/opinions/README.md delete mode 100644 plugins/opinions/package.json delete mode 100644 plugins/opinions/webcmd-plugin.json diff --git a/README.md b/README.md index daa7e2c..202a1df 100644 --- a/README.md +++ b/README.md @@ -140,7 +140,7 @@ Webcmd Cloud can run supported commands and browser sessions on hosted infrastru | Plugin | Description | Author | | --- | --- | --- | -| [`opinions`](./plugins/opinions/) | No-login social opinions & problems research across public platforms | [Rishet Mehra](https://github.com/Rishet11) | +| [`omnisearch`](./plugins/omnisearch/) | Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login | [Rishet Mehra](https://github.com/Rishet11) | | [`pypi`](./plugins/pypi/) | Inspect public Python package metadata, downloads, and releases from PyPI | [Kemal Kaya](https://github.com/yoldaolmak) | | [`skyscanner`](./plugins/skyscanner/) | Skyscanner flight search commands for Webcmd | [Rishabh](https://github.com/rishabhraj36) | diff --git a/plugins/omnisearch/README.md b/plugins/omnisearch/README.md new file mode 100644 index 0000000..5aa208b --- /dev/null +++ b/plugins/omnisearch/README.md @@ -0,0 +1,65 @@ +# webcmd-plugin-omnisearch + +**OmniSearch** — universal search across public platforms, no login. + +Search what people are saying about any topic, product, or problem across +multiple platforms (Bluesky, Hacker News, Lobsters, and more) in one tool. +No browser, no login, no credentials — just structured data. + +## Install + +```bash +webcmd plugin install github:Rishet11/webcmd-plugin-omnisearch +``` + +## Commands + +| Command | Type | Description | +|---------|------|-------------| +| `omnisearch bluesky-posts ` | public | Recent posts from a public Bluesky account (no login) | +| `omnisearch hackermind ` | public | Search Hacker News stories & comments (no login) | +| `omnisearch lobsters [--sort newest\|active\|hot]` | public | Lobste.rs newest / active / hot discussions (no login) | +| `omnisearch research ` | public | Aggregate results across Hacker News + Lobste.rs in one feed | + +## Examples + +```bash +# Read what a public Bluesky account (Twitter/X-like) is saying, no login +webcmd omnisearch bluesky-posts paulgraham.bsky.social --limit 10 -f json + +# Search Hacker News for what people think about a product/problem +webcmd omnisearch hackermind "saas pricing" --limit 10 -f json + +# Search comment text specifically for problem reports +webcmd omnisearch hackermind "billing is confusing" --limit 10 --scope comment -f json + +# One command, search across platforms +webcmd omnisearch research "LLM" --limit 10 -f json +``` + +## Output + +All commands return clean, consistent camelCase JSON: + +| Field | Meaning | +|---|---| +| `title` | Title / headline / post text | +| `author` | Author or handle | +| `score` | Upvotes / points | +| `commentCount` | Number of comments | +| `createdAt` | ISO timestamp | +| `url` | Absolute link to the source | +| `platform` | Source platform (`research` aggregator only) | + +## Development + +```bash +# Install locally for development (symlinked, changes reflect immediately) +webcmd plugin install file:///Users/rishetmehra/webcmd-opinions + +# Verify commands are registered +webcmd list | grep -A12 omnisearch + +# Validate definitions +webcmd validate omnisearch +``` diff --git a/plugins/opinions/bluesky-posts.js b/plugins/omnisearch/bluesky-posts.js similarity index 93% rename from plugins/opinions/bluesky-posts.js rename to plugins/omnisearch/bluesky-posts.js index c3ffa00..06232ea 100644 --- a/plugins/opinions/bluesky-posts.js +++ b/plugins/omnisearch/bluesky-posts.js @@ -1,5 +1,5 @@ /** - * opinions bluesky-posts — read what a public Bluesky account is saying. + * omnisearch bluesky-posts — read what a public Bluesky account is saying. * * No login. Uses the public Bluesky API (public.api.bsky.app) author feed * endpoint, which returns a user's recent posts without authentication. @@ -25,7 +25,7 @@ function requireHandle(value) { } cli({ - site: 'opinions', + site: 'omnisearch', name: 'bluesky-posts', access: 'read', description: "Recent posts from a public Bluesky account (no login)", @@ -52,7 +52,7 @@ cli({ let json; try { const res = await fetch(url, { - headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-opinions/0.1)' }, + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-omnisearch/0.1)' }, }); if (!res.ok) { throw new CommandExecutionError(`Bluesky API request failed: HTTP ${res.status}`); @@ -65,7 +65,7 @@ cli({ const feed = Array.isArray(json?.feed) ? json.feed : []; if (!feed.length) { - throw new EmptyResultError('opinions/bluesky-posts', `no posts found for "${handle}"`); + throw new EmptyResultError('omnisearch/bluesky-posts', `no posts found for "${handle}"`); } return feed.slice(0, limit).map((entry, index) => { diff --git a/plugins/opinions/hackermind.js b/plugins/omnisearch/hackermind.js similarity index 90% rename from plugins/opinions/hackermind.js rename to plugins/omnisearch/hackermind.js index eb6d4b9..ff92ab6 100644 --- a/plugins/opinions/hackermind.js +++ b/plugins/omnisearch/hackermind.js @@ -1,5 +1,5 @@ /** - * opinions hackermind — search Hacker News stories & comments for opinions. + * omnisearch hackermind — search Hacker News stories & comments for omnisearch. * * No login. Uses the public HN Algolia API, which indexes stories and * comments. Good for researching what the tech/startup community is saying @@ -17,11 +17,11 @@ function requireQuery(value) { } cli({ - site: 'opinions', + site: 'omnisearch', name: 'hackermind', tags: ['search'], access: 'read', - description: "Search Hacker News stories & comments for opinions (no login)", + description: "Search Hacker News stories & comments (no login)", domain: 'hn.algolia.com', strategy: Strategy.PUBLIC, browser: false, @@ -62,7 +62,7 @@ cli({ const hits = Array.isArray(json?.hits) ? json.hits : []; if (!hits.length) { - throw new EmptyResultError('opinions/hackermind', `no results for "${query}"`); + throw new EmptyResultError('omnisearch/hackermind', `no results for "${query}"`); } return hits.slice(0, limit).map((h, index) => ({ diff --git a/plugins/opinions/lobsters.js b/plugins/omnisearch/lobsters.js similarity index 92% rename from plugins/opinions/lobsters.js rename to plugins/omnisearch/lobsters.js index e4eecb2..c793332 100644 --- a/plugins/opinions/lobsters.js +++ b/plugins/omnisearch/lobsters.js @@ -1,5 +1,5 @@ /** - * opinions lobsters — Lobste.rs newest / active discussions. + * omnisearch lobsters — Lobste.rs newest / active discussions. * * No login. Uses the public lobste.rs JSON endpoint. Lobste.rs is a * Reddit/HN-style community; good for surfacing what developers are @@ -22,7 +22,7 @@ function requireSort(value) { } cli({ - site: 'opinions', + site: 'omnisearch', name: 'lobsters', access: 'read', description: "Lobste.rs newest / active / hot discussions (no login)", @@ -45,7 +45,7 @@ cli({ let rows; try { const res = await fetch(SORTS[sort], { - headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-opinions/0.1)' }, + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-omnisearch/0.1)' }, }); if (!res.ok) throw new CommandExecutionError(`lobste.rs request failed: HTTP ${res.status}`); rows = await res.json(); @@ -55,7 +55,7 @@ cli({ } if (!Array.isArray(rows) || !rows.length) { - throw new EmptyResultError('opinions/lobsters', 'lobste.rs returned no stories'); + throw new EmptyResultError('omnisearch/lobsters', 'lobste.rs returned no stories'); } return rows.slice(0, limit).map((s, index) => ({ diff --git a/plugins/omnisearch/package.json b/plugins/omnisearch/package.json new file mode 100644 index 0000000..499d584 --- /dev/null +++ b/plugins/omnisearch/package.json @@ -0,0 +1,9 @@ +{ + "name": "webcmd-plugin-omnisearch", + "version": "0.1.0", + "type": "module", + "description": "Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login", + "peerDependencies": { + "@agentrhq/webcmd": ">=0.5.3" + } +} diff --git a/plugins/opinions/research.js b/plugins/omnisearch/research.js similarity index 89% rename from plugins/opinions/research.js rename to plugins/omnisearch/research.js index ee73f47..27318d0 100644 --- a/plugins/opinions/research.js +++ b/plugins/omnisearch/research.js @@ -1,5 +1,5 @@ /** - * opinions research — aggregate opinions about a topic across no-login + * omnisearch research — aggregate omnisearch about a topic across no-login * public platforms (Hacker News + Lobste.rs) into one feed. * * No login. Combines HN Algolia stories and Lobste.rs newest stories filtered @@ -40,7 +40,7 @@ async function hnSearch(query, limit) { async function lobstersSearch(query, limit) { const res = await fetch(LOBSTERS, { - headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-opinions/0.1)' }, + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-omnisearch/0.1)' }, }); if (!res.ok) return []; const rows = await res.json(); @@ -66,11 +66,11 @@ async function lobstersSearch(query, limit) { } cli({ - site: 'opinions', + site: 'omnisearch', name: 'research', tags: ['search'], access: 'read', - description: "Aggregate opinions/problems about a topic across no-login platforms (Hacker News + Lobste.rs)", + description: "Aggregate results about a topic across no-login platforms (Hacker News + Lobste.rs)", domain: 'hn.algolia.com', strategy: Strategy.PUBLIC, browser: false, @@ -97,7 +97,7 @@ cli({ } if (!rows.length) { - throw new EmptyResultError('opinions/research', `no opinions found across platforms for "${query}"`); + throw new EmptyResultError('omnisearch/research', `no omnisearch found across platforms for "${query}"`); } return rows.slice(0, limit); diff --git a/plugins/omnisearch/webcmd-plugin.json b/plugins/omnisearch/webcmd-plugin.json new file mode 100644 index 0000000..758aebf --- /dev/null +++ b/plugins/omnisearch/webcmd-plugin.json @@ -0,0 +1,10 @@ +{ + "name": "omnisearch", + "version": "0.1.0", + "description": "Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login", + "webcmd": ">=0.5.3", + "author": { + "name": "Rishet Mehra", + "handle": "Rishet11" + } +} diff --git a/plugins/opinions/README.md b/plugins/opinions/README.md deleted file mode 100644 index 26d2edd..0000000 --- a/plugins/opinions/README.md +++ /dev/null @@ -1,108 +0,0 @@ -# webcmd-plugin-opinions - -No-login social opinions & problems research across **public** platforms. -Research what people think about a topic, product, or problem — no browser, no -login, no credentials. - -## Install - -```bash -webcmd plugin install github:rishetmehra/webcmd-plugin-opinions -``` - -## Commands - -| Command | Type | Description | -|---------|------|-------------| -| `opinions bluesky-posts ` | public | Recent posts from a public Bluesky account (no login) | -| `opinions hackermind ` | public | Search Hacker News stories & comments for opinions (no login) | -| `opinions lobsters [--sort newest\|active\|hot]` | public | Lobste.rs newest / active / hot discussions (no login) | -| `opinions research ` | public | Aggregate opinions/problems across Hacker News + Lobste.rs in one feed | - -## Examples - -```bash -# Read what a public Bluesky account (Twitter/X-like) is saying, no login -webcmd opinions bluesky-posts paulgraham.bsky.social --limit 10 -f json - -# Search Hacker News for what people think about a product/problem -webcmd opinions hackermind "saas pricing" --limit 10 -f json - -# Search comment text specifically for problem reports -webcmd opinions hackermind "billing is confusing" --limit 10 --scope comment -f json - -# One command, opinions across platforms -webcmd opinions research "LLM" --limit 10 -f json -``` - -## Full multi-platform research setup (current state) - -This machine has the `social` profile logged into **X/Twitter, Instagram, -Reddit, LinkedIn, and YouTube** (TikTok and Facebook are skipped — TikTok is -banned in India, Facebook not needed). All commands below run **headless** -(background browser) after the one-time login. - -### How authentication works (important) - -webcmd does **not** use your password in code or `.env`. It uses a one-time -interactive login per platform into a saved browser **profile** (`social`). -The profile stores the session cookies (`auth_token` for X, `sessionid` for -Instagram, `reddit_session` for Reddit, etc.). Commands read cookies from the -profile — never your credentials. This is why a one-time login is required and -why `.env` passwords don't work with the official plugins. - -### Two tiers of access - -**Tier 1 — no login (public APIs), always headless:** -```bash -webcmd opinions bluesky-posts --limit 10 -f json -webcmd opinions hackermind "" --limit 10 -f json -webcmd opinions lobsters --limit 10 -f json -webcmd opinions research "" --limit 10 -f json # aggregate -``` - -**Tier 2 — logged-in platforms (uses `social` profile):** -```bash -# X / Twitter -webcmd --profile social twitter search "" --limit 10 -f json -webcmd --profile social twitter tweets --limit 10 -f json -webcmd --profile social twitter trending -f json - -# Reddit (public opinion goldmine) -webcmd --profile social reddit search "" --limit 10 -f json -webcmd --profile social reddit subreddit --limit 10 -f json -webcmd --profile social reddit read -f json - -# Instagram -webcmd --profile social instagram search "" -f json -webcmd --profile social instagram profile -f json - -# LinkedIn -webcmd --profile social linkedin people-search "" -f json -webcmd --profile social linkedin timeline -f json - -# YouTube -webcmd --profile social youtube search "" --limit 10 -f json -webcmd --profile social youtube comments -f json -``` - -### Re-authenticating after a session expires - -If a platform returns `AUTH_REQUIRED`, redo the one-time login for that site: -```bash -webcmd --profile social login # sign in in the browser window -webcmd --profile social whoami # verify -``` - -## Development - -```bash -# Install locally for development (symlinked, changes reflect immediately) -webcmd plugin install file:///Users/rishetmehra/webcmd-opinions - -# Verify commands are registered -webcmd list | grep -A12 opinions - -# Validate definitions -webcmd validate opinions -``` diff --git a/plugins/opinions/package.json b/plugins/opinions/package.json deleted file mode 100644 index c136ad1..0000000 --- a/plugins/opinions/package.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "name": "webcmd-plugin-opinions", - "version": "0.1.0", - "type": "module", - "description": "No-login social opinions & problems research across public platforms", - "peerDependencies": { - "@agentrhq/webcmd": ">=0.5.3" - } -} diff --git a/plugins/opinions/webcmd-plugin.json b/plugins/opinions/webcmd-plugin.json deleted file mode 100644 index a3bd926..0000000 --- a/plugins/opinions/webcmd-plugin.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "name": "opinions", - "version": "0.1.0", - "description": "No-login social opinions & problems research across public platforms", - "webcmd": ">=0.5.3", - "author": { - "name": "Rishet Mehra", - "handle": "Rishet11" - } -} diff --git a/webcmd-plugin.json b/webcmd-plugin.json index 5e87de5..b857fdf 100644 --- a/webcmd-plugin.json +++ b/webcmd-plugin.json @@ -774,6 +774,16 @@ "handle": "agentrhq" } }, + "omnisearch": { + "path": "plugins/omnisearch", + "version": "0.1.0", + "description": "Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login", + "webcmd": ">=0.5.3", + "author": { + "name": "Rishet Mehra", + "handle": "Rishet11" + } + }, "openalex": { "path": "plugins/openalex", "version": "0.1.0", @@ -804,16 +814,6 @@ "handle": "agentrhq" } }, - "opinions": { - "path": "plugins/opinions", - "version": "0.1.0", - "description": "No-login social opinions & problems research across public platforms", - "webcmd": ">=0.5.3", - "author": { - "name": "Rishet Mehra", - "handle": "Rishet11" - } - }, "osv": { "path": "plugins/osv", "version": "0.1.0", From 41d820d6e2352032af1aa145880bb713701db996 Mon Sep 17 00:00:00 2001 From: Rishet Mehra Date: Tue, 11 Aug 2026 01:15:49 +0530 Subject: [PATCH 4/8] omnisearch: fix local dev path in README --- plugins/omnisearch/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/omnisearch/README.md b/plugins/omnisearch/README.md index 5aa208b..c0d71da 100644 --- a/plugins/omnisearch/README.md +++ b/plugins/omnisearch/README.md @@ -55,7 +55,7 @@ All commands return clean, consistent camelCase JSON: ```bash # Install locally for development (symlinked, changes reflect immediately) -webcmd plugin install file:///Users/rishetmehra/webcmd-opinions +webcmd plugin install file:///Users/rishetmehra/webcmd-omnisearch # Verify commands are registered webcmd list | grep -A12 omnisearch From e242b201e245b05868db0e606fe14d73775189f1 Mon Sep 17 00:00:00 2001 From: Rishet Mehra Date: Tue, 11 Aug 2026 01:24:15 +0530 Subject: [PATCH 5/8] omnisearch: expand to multi-source aggregator (SO, GitHub, arXiv, Dev.to) --- plugins/omnisearch/README.md | 104 +++++++++++----- plugins/omnisearch/arxiv.js | 43 +++++++ plugins/omnisearch/github.js | 43 +++++++ plugins/omnisearch/research.js | 103 +++++++--------- plugins/omnisearch/sources.js | 184 ++++++++++++++++++++++++++++ plugins/omnisearch/stackoverflow.js | 43 +++++++ 6 files changed, 424 insertions(+), 96 deletions(-) create mode 100644 plugins/omnisearch/arxiv.js create mode 100644 plugins/omnisearch/github.js create mode 100644 plugins/omnisearch/sources.js create mode 100644 plugins/omnisearch/stackoverflow.js diff --git a/plugins/omnisearch/README.md b/plugins/omnisearch/README.md index c0d71da..76fa187 100644 --- a/plugins/omnisearch/README.md +++ b/plugins/omnisearch/README.md @@ -1,10 +1,22 @@ # webcmd-plugin-omnisearch -**OmniSearch** — universal search across public platforms, no login. +**OmniSearch** — universal web research from your terminal. One command, every public platform. No login. No browser. No credentials. -Search what people are saying about any topic, product, or problem across -multiple platforms (Bluesky, Hacker News, Lobsters, and more) in one tool. -No browser, no login, no credentials — just structured data. +Press one command and OmniSearch sweeps across **Hacker News, Stack Overflow, GitHub, Dev.to, arXiv, Bluesky, Lobsters, and more** — then returns clean, structured JSON of what people are saying, asking, and struggling with about any topic. + +Built for **developers, founders, researchers, and AI agents** who need "what does the internet think?" answered in seconds, not browser sessions. + +--- + +## Why OmniSearch + +- **Universal** — one command aggregates across many platforms, not one silo. +- **No login, ever** — all public APIs. Your credentials never touch it. +- **Structured output** — consistent camelCase JSON, ready to pipe anywhere. +- **Agent-ready** — built to feed AI agents, research pipelines, and scripts. +- **Source-filterable** — query only the platforms you care about. + +--- ## Install @@ -14,52 +26,76 @@ webcmd plugin install github:Rishet11/webcmd-plugin-omnisearch ## Commands -| Command | Type | Description | -|---------|------|-------------| -| `omnisearch bluesky-posts ` | public | Recent posts from a public Bluesky account (no login) | -| `omnisearch hackermind ` | public | Search Hacker News stories & comments (no login) | -| `omnisearch lobsters [--sort newest\|active\|hot]` | public | Lobste.rs newest / active / hot discussions (no login) | -| `omnisearch research ` | public | Aggregate results across Hacker News + Lobste.rs in one feed | +| Command | Source | What it surfaces | +|---------|--------|------------------| +| `omnisearch research ` | **All sources** | Aggregate everything in one feed | +| `omnisearch hackermind ` | Hacker News | Tech opinions & discussions | +| `omnisearch stackoverflow ` | Stack Overflow | Real problems developers ask | +| `omnisearch github ` | GitHub issues/PRs | Real problems people report | +| `omnisearch devto ` | Dev.to | Developer blog opinions | +| `omnisearch arxiv ` | arXiv | Research papers | +| `omnisearch lobsters [--sort]` | Lobste.rs | Developer discussions | +| `omnisearch bluesky-posts ` | Bluesky | What a public account is saying | ## Examples ```bash -# Read what a public Bluesky account (Twitter/X-like) is saying, no login -webcmd omnisearch bluesky-posts paulgraham.bsky.social --limit 10 -f json +# Research a topic across ALL sources in one shot +webcmd omnisearch research "saas pricing" --limit 20 -f json -# Search Hacker News for what people think about a product/problem -webcmd omnisearch hackermind "saas pricing" --limit 10 -f json +# Filter to only certain sources +webcmd omnisearch research "rag" --sources github,hn -f json -# Search comment text specifically for problem reports -webcmd omnisearch hackermind "billing is confusing" --limit 10 --scope comment -f json +# Find real problems people are hitting +webcmd omnisearch stackoverflow "billing saas" --limit 10 -f json +webcmd omnisearch github "saas pricing" --limit 10 -f json -# One command, search across platforms -webcmd omnisearch research "LLM" --limit 10 -f json +# Research papers + tech opinions +webcmd omnisearch arxiv "large language models" --limit 10 -f json +webcmd omnisearch hackermind "ai agents" --limit 10 -f json + +# Read what a public Bluesky account is saying +webcmd omnisearch bluesky-posts paulgraham.bsky.social --limit 10 -f json +``` + +## Output schema + +Every command returns the same consistent shape: + +```json +{ + "platform": "stackoverflow", + "title": "Best SaaS recurring billing solution?", + "author": "user1", + "score": 143, + "commentCount": 5, + "createdAt": "2026-08-10T12:00:00Z", + "url": "...", + "text": "" +} ``` -## Output +| Field | Type | Meaning | +|-------|------|---------| +| `platform` | string | Source platform | +| `title` | string | Title / headline / post text | +| `author` | string | Author or handle | +| `score` | number | Upvotes / points / reactions | +| `commentCount` | number | Number of comments / answers | +| `createdAt` | string | ISO timestamp | +| `url` | string | Absolute link to the source | +| `text` | string | Body / snippet (where available) | -All commands return clean, consistent camelCase JSON: +## Universal search is just the start -| Field | Meaning | -|---|---| -| `title` | Title / headline / post text | -| `author` | Author or handle | -| `score` | Upvotes / points | -| `commentCount` | Number of comments | -| `createdAt` | ISO timestamp | -| `url` | Absolute link to the source | -| `platform` | Source platform (`research` aggregator only) | +- **`research`** is the aggregator — add `--sources hn,stackoverflow,arxiv` to control which platforms to hit. +- Every adapter is a **public API** — zero setup, zero login, works in CI and headless agents. +- The shared `sources.js` module makes adding a new platform a ~20-line step. ## Development ```bash -# Install locally for development (symlinked, changes reflect immediately) webcmd plugin install file:///Users/rishetmehra/webcmd-omnisearch - -# Verify commands are registered webcmd list | grep -A12 omnisearch - -# Validate definitions webcmd validate omnisearch ``` diff --git a/plugins/omnisearch/arxiv.js b/plugins/omnisearch/arxiv.js new file mode 100644 index 0000000..f7487b4 --- /dev/null +++ b/plugins/omnisearch/arxiv.js @@ -0,0 +1,43 @@ +/** + * OmniSearch arxiv — search arXiv research papers. + * No login. Uses the public arXiv Atom API. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; +import { arxivSearch } from './sources.js'; + +function requireQuery(value) { + const s = String(value ?? '').trim(); + if (!s) throw new ArgumentError('a search query is required'); + return s; +} + +cli({ + site: 'omnisearch', + name: 'arxiv', + tags: ['search'], + access: 'read', + description: "Search arXiv research papers (no login)", + domain: 'export.arxiv.org', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'query', required: true, positional: true, help: 'Research topic' }, + { name: 'limit', type: 'int', default: 20, help: 'Number of results' }, + ], + columns: ['platform', 'title', 'author', 'score', 'commentCount', 'createdAt', 'url', 'text'], + func: async (kwargs) => { + const query = requireQuery(kwargs.query); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) throw new ArgumentError('limit must be a positive integer'); + const limit = Math.min(raw, 50); + let rows; + try { + rows = await arxivSearch(query, limit); + } catch (err) { + throw new CommandExecutionError(err instanceof Error ? err.message : String(err)); + } + if (!rows.length) throw new EmptyResultError('omnisearch/arxiv', `no results for "${query}"`); + return rows; + }, +}); diff --git a/plugins/omnisearch/github.js b/plugins/omnisearch/github.js new file mode 100644 index 0000000..acb9f93 --- /dev/null +++ b/plugins/omnisearch/github.js @@ -0,0 +1,43 @@ +/** + * OmniSearch github — search GitHub issues & PRs (people reporting real problems). + * No login. Uses the public GitHub search API (rate-limited to ~10/min). + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; +import { githubSearch } from './sources.js'; + +function requireQuery(value) { + const s = String(value ?? '').trim(); + if (!s) throw new ArgumentError('a search query is required'); + return s; +} + +cli({ + site: 'omnisearch', + name: 'github', + tags: ['search'], + access: 'read', + description: "Search GitHub issues & PRs for real problems (no login)", + domain: 'api.github.com', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'query', required: true, positional: true, help: 'Problem or feature to research' }, + { name: 'limit', type: 'int', default: 20, help: 'Number of results' }, + ], + columns: ['platform', 'title', 'author', 'score', 'commentCount', 'createdAt', 'url', 'text'], + func: async (kwargs) => { + const query = requireQuery(kwargs.query); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) throw new ArgumentError('limit must be a positive integer'); + const limit = Math.min(raw, 50); + let rows; + try { + rows = await githubSearch(query, limit); + } catch (err) { + throw new CommandExecutionError(err instanceof Error ? err.message : String(err)); + } + if (!rows.length) throw new EmptyResultError('omnisearch/github', `no results for "${query}"`); + return rows; + }, +}); diff --git a/plugins/omnisearch/research.js b/plugins/omnisearch/research.js index 27318d0..ee09e89 100644 --- a/plugins/omnisearch/research.js +++ b/plugins/omnisearch/research.js @@ -1,16 +1,21 @@ /** - * omnisearch research — aggregate omnisearch about a topic across no-login - * public platforms (Hacker News + Lobste.rs) into one feed. + * OmniSearch research — aggregate results about a topic across all public + * platforms into one feed. * - * No login. Combines HN Algolia stories and Lobste.rs newest stories filtered - * by keyword, tagging each row with its platform. Use for quick opinion / - * problem reconnaissance on a topic, product, or persona. + * No login. Combines Hacker News, Lobste.rs, Stack Overflow, Dev.to, GitHub + * issues, and arXiv, tagging each row with its platform. Use for quick + * universal reconnaissance on a topic, product, or problem. */ import { cli, Strategy } from '@agentrhq/webcmd/registry'; import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; - -const HN_SEARCH = 'https://hn.algolia.com/api/v1/search'; -const LOBSTERS = 'https://lobste.rs/newest.json'; +import { + hnSearch, + lobstersSearch, + stackoverflowSearch, + devtoSearch, + githubSearch, + arxivSearch, +} from './sources.js'; function requireQuery(value) { const s = String(value ?? '').trim(); @@ -18,65 +23,23 @@ function requireQuery(value) { return s; } -async function hnSearch(query, limit) { - const url = new URL(HN_SEARCH); - url.searchParams.set('query', query); - url.searchParams.set('tags', 'story'); - url.searchParams.set('hitsPerPage', String(limit)); - const res = await fetch(url); - if (!res.ok) return []; - const json = await res.json(); - return (Array.isArray(json?.hits) ? json.hits : []).slice(0, limit).map((h) => ({ - platform: 'hackernews', - title: String(h.title ?? h.story_title ?? '').trim(), - author: String(h.author ?? ''), - score: h.points ?? 0, - commentCount: h.num_comments ?? 0, - createdAt: String(h.created_at ?? ''), - url: String(h.url ?? `https://news.ycombinator.com/item?id=${h.objectID}`), - text: '', - })); -} - -async function lobstersSearch(query, limit) { - const res = await fetch(LOBSTERS, { - headers: { 'User-Agent': 'Mozilla/5.0 (compatible; webcmd-omnisearch/0.1)' }, - }); - if (!res.ok) return []; - const rows = await res.json(); - if (!Array.isArray(rows)) return []; - const q = query.toLowerCase(); - return rows - .filter((s) => - String(s.title ?? '').toLowerCase().includes(q) || - String(s.description_plain ?? '').toLowerCase().includes(q) || - (Array.isArray(s.tags) && s.tags.some((t) => t.toLowerCase().includes(q))), - ) - .slice(0, limit) - .map((s) => ({ - platform: 'lobsters', - title: String(s.title ?? '').trim(), - author: String(s.submitter_user ?? ''), - score: s.score ?? 0, - commentCount: s.comment_count ?? 0, - createdAt: String(s.created_at ?? ''), - url: String(s.comments_url ?? ''), - text: String(s.description_plain ?? ''), - })); -} - cli({ site: 'omnisearch', name: 'research', tags: ['search'], access: 'read', - description: "Aggregate results about a topic across no-login platforms (Hacker News + Lobste.rs)", - domain: 'hn.algolia.com', + description: "Aggregate results about a topic across all public platforms (Hacker News, Lobsters, Stack Overflow, Dev.to, GitHub, arXiv)", + domain: 'multiple', strategy: Strategy.PUBLIC, browser: false, args: [ { name: 'query', required: true, positional: true, help: 'Topic, product, or problem to research' }, { name: 'limit', type: 'int', default: 20, help: 'Number of results per platform' }, + { + name: 'sources', + default: 'hn,lobsters,stackoverflow,devto,github,arxiv', + help: 'Comma-separated sources to query (default: all)', + }, ], columns: ['platform', 'title', 'author', 'score', 'commentCount', 'createdAt', 'url', 'text'], func: async (kwargs) => { @@ -86,20 +49,36 @@ cli({ throw new ArgumentError('limit must be a positive integer'); } const limit = Math.min(raw, 50); - const perPlatform = Math.ceil(limit / 2); + const perPlatform = Math.ceil(limit / 6); + + const wanted = String(kwargs.sources ?? '') + .split(',') + .map((s) => s.trim().toLowerCase()) + .filter(Boolean); + + const fetchers = { + hn: () => hnSearch(query, perPlatform), + lobsters: () => lobstersSearch(query, perPlatform), + stackoverflow: () => stackoverflowSearch(query, perPlatform), + devto: () => devtoSearch(query, perPlatform), + github: () => githubSearch(query, perPlatform), + arxiv: () => arxivSearch(query, perPlatform), + }; + + const selected = wanted.length ? wanted.filter((s) => fetchers[s]) : Object.keys(fetchers); let rows; try { - const [hn, lob] = await Promise.all([hnSearch(query, perPlatform), lobstersSearch(query, perPlatform)]); - rows = [...hn, ...lob]; + const results = await Promise.all(selected.map((s) => fetchers[s]())); + rows = results.flat(); } catch (err) { throw new CommandExecutionError(`research aggregation failed: ${err instanceof Error ? err.message : String(err)}`); } if (!rows.length) { - throw new EmptyResultError('omnisearch/research', `no omnisearch found across platforms for "${query}"`); + throw new EmptyResultError('omnisearch/research', `no results found across platforms for "${query}"`); } return rows.slice(0, limit); }, -}); \ No newline at end of file +}); diff --git a/plugins/omnisearch/sources.js b/plugins/omnisearch/sources.js new file mode 100644 index 0000000..f6907cd --- /dev/null +++ b/plugins/omnisearch/sources.js @@ -0,0 +1,184 @@ +/** + * OmniSearch — shared source fetchers for public platforms. + * + * Every search command aggregates across these. Each fetcher returns + * normalized rows: { platform, title, author, score, commentCount, createdAt, url, text } + */ + +// --- Hacker News (Algolia) --- +export async function hnSearch(query, limit) { + const url = new URL('https://hn.algolia.com/api/v1/search'); + url.searchParams.set('query', query); + url.searchParams.set('tags', 'story'); + url.searchParams.set('hitsPerPage', String(limit)); + const res = await fetch(url); + if (!res.ok) throw new Error(`Hacker News HTTP ${res.status}`); + const json = await res.json(); + return (json?.hits ?? []).slice(0, limit).map((h) => ({ + platform: 'hackernews', + title: String(h.title ?? h.story_title ?? '').trim(), + author: String(h.author ?? ''), + score: h.points ?? 0, + commentCount: h.num_comments ?? 0, + createdAt: String(h.created_at ?? ''), + url: String(h.url ?? `https://news.ycombinator.com/item?id=${h.objectID}`), + text: '', + })); +} + +// --- Lobste.rs --- +export async function lobstersSearch(query, limit) { + const res = await fetch('https://lobste.rs/newest.json', { + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; OmniSearch/0.1)' }, + }); + if (!res.ok) throw new Error(`Lobste.rs HTTP ${res.status}`); + const rows = await res.json(); + if (!Array.isArray(rows)) return []; + const q = query.toLowerCase(); + return rows + .filter((s) => + String(s.title ?? '').toLowerCase().includes(q) || + String(s.description_plain ?? '').toLowerCase().includes(q) || + (Array.isArray(s.tags) && s.tags.some((t) => t.toLowerCase().includes(q))), + ) + .slice(0, limit) + .map((s) => ({ + platform: 'lobsters', + title: String(s.title ?? '').trim(), + author: String(s.submitter_user ?? ''), + score: s.score ?? 0, + commentCount: s.comment_count ?? 0, + createdAt: String(s.created_at ?? ''), + url: String(s.comments_url ?? ''), + text: String(s.description_plain ?? ''), + })); +} + +// --- Stack Overflow (public StackExchange API) --- +export async function stackoverflowSearch(query, limit) { + const url = new URL('https://api.stackexchange.com/2.3/search/advanced'); + url.searchParams.set('order', 'desc'); + url.searchParams.set('sort', 'relevance'); + url.searchParams.set('q', query); + url.searchParams.set('site', 'stackoverflow'); + url.searchParams.set('pagesize', String(limit)); + const res = await fetch(url); + if (!res.ok) throw new Error(`Stack Overflow HTTP ${res.status}`); + const json = await res.json(); + return (json?.items ?? []).slice(0, limit).map((q) => ({ + platform: 'stackoverflow', + title: String(q.title ?? '').trim(), + author: String(q.owner?.display_name ?? ''), + score: q.score ?? 0, + commentCount: q.answer_count ?? 0, + createdAt: String(new Date((q.creation_date ?? Date.now()) * 1000).toISOString()), + url: String(q.link ?? ''), + text: '', + })); +} + +// --- Dev.to (tag-based public articles) --- +export async function devtoSearch(query, limit) { + const url = new URL('https://dev.to/api/articles'); + url.searchParams.set('tag', query.replace(/[^a-zA-Z0-9-]/g, '')); + url.searchParams.set('per_page', String(limit)); + const res = await fetch(url, { + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; OmniSearch/0.1)' }, + }); + if (!res.ok) throw new Error(`Dev.to HTTP ${res.status}`); + const json = await res.json(); + if (!Array.isArray(json)) return []; + return json.slice(0, limit).map((a) => ({ + platform: 'devto', + title: String(a.title ?? '').trim(), + author: String(a.user?.username ?? a.user?.name ?? ''), + score: a.positive_reactions_count ?? 0, + commentCount: a.comments_count ?? 0, + createdAt: String(a.published_at ?? ''), + url: String(a.url ?? ''), + text: String(a.description ?? ''), + })); +} +// --- GitHub issues (people reporting problems) --- +export async function githubSearch(query, limit) { + const url = new URL('https://api.github.com/search/issues'); + url.searchParams.set('q', `${query} in:title,body`); + url.searchParams.set('per_page', String(limit)); + const res = await fetch(url, { headers: { 'User-Agent': 'OmniSearch/0.1' } }); + if (!res.ok) throw new Error(`GitHub HTTP ${res.status}`); + const json = await res.json(); + return (json?.items ?? []).slice(0, limit).map((i) => ({ + platform: 'github', + title: `[${i.state ?? ''}] ${String(i.title ?? '').trim()}`.trim(), + author: String(i.user?.login ?? ''), + score: i.reactions?.total_count ?? 0, + commentCount: i.comments ?? 0, + createdAt: String(i.created_at ?? ''), + url: String(i.html_url ?? ''), + text: String(i.body ?? '').slice(0, 300), + })); +} + +// --- arXiv (research papers) --- +export async function arxivSearch(query, limit) { + const url = new URL('https://export.arxiv.org/api/query'); + url.searchParams.set('search_query', `all:${query.split(' ').join('+')}`); + url.searchParams.set('max_results', String(limit)); + const res = await fetch(url); + if (!res.ok) throw new Error(`arXiv HTTP ${res.status}`); + const xml = await res.text(); + const entryRe = /([\s\S]*?)<\/entry>/g; + const rows = []; + let m; + while ((m = entryRe.exec(xml)) !== null && rows.length < limit) { + const e = m[1]; + const title = (e.match(/([\s\S]*?)<\/title>/) || [])[1] + ?.replace(/\s+/g, ' ').trim() ?? ''; + const author = ((e.match(/<name>([\s\S]*?)<\/name>/) || [])[1] ?? '').trim(); + const id = (e.match(/<id>([\s\S]*?)<\/id>/) || [])[1] ?? ''; + const updated = ((e.match(/<updated>([\s\S]*?)<\/updated>/) || [])[1] ?? '').trim(); + const summary = ((e.match(/<summary>([\s\S]*?)<\/summary>/) || [])[1] ?? '') + .replace(/\s+/g, ' ').trim(); + rows.push({ + platform: 'arxiv', + title, + author, + score: 0, + commentCount: 0, + createdAt: updated, + url: id, + text: summary.slice(0, 200), + }); + } + return rows; +} + +// --- Bluesky (public author feed) --- +export async function blueskyPosts(handle, limit) { + const url = new URL('https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed'); + url.searchParams.set('actor', handle); + url.searchParams.set('limit', String(limit)); + const res = await fetch(url, { + headers: { 'User-Agent': 'Mozilla/5.0 (compatible; OmniSearch/0.1)' }, + }); + if (!res.ok) throw new Error(`Bluesky HTTP ${res.status}`); + const json = await res.json(); + const feed = Array.isArray(json?.feed) ? json.feed : []; + return feed.slice(0, limit).map((entry) => { + const post = entry?.post ?? {}; + const author = post.author ?? {}; + const record = post.record ?? {}; + const uri = String(post.uri ?? ''); + const rkey = uri.split('/').pop() ?? ''; + return { + platform: 'bluesky', + title: String(record.text ?? '').replace(/\s*\n+/g, ' ').trim().slice(0, 200), + author: String(author.handle ?? handle), + score: post.likeCount ?? 0, + commentCount: post.replyCount ?? 0, + createdAt: String(record.createdAt ?? post.indexedAt ?? ''), + url: `https://bsky.app/profile/${author.handle ?? handle}/post/${rkey}`, + text: String(record.text ?? '').replace(/\s*\n+/g, ' ').trim(), + }; + }); +} diff --git a/plugins/omnisearch/stackoverflow.js b/plugins/omnisearch/stackoverflow.js new file mode 100644 index 0000000..60c082b --- /dev/null +++ b/plugins/omnisearch/stackoverflow.js @@ -0,0 +1,43 @@ +/** + * OmniSearch stackoverflow — search Stack Overflow for real problems & questions. + * No login. Uses the public StackExchange API. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; +import { stackoverflowSearch } from './sources.js'; + +function requireQuery(value) { + const s = String(value ?? '').trim(); + if (!s) throw new ArgumentError('a search query is required'); + return s; +} + +cli({ + site: 'omnisearch', + name: 'stackoverflow', + tags: ['search'], + access: 'read', + description: "Search Stack Overflow questions & problems (no login)", + domain: 'api.stackexchange.com', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'query', required: true, positional: true, help: 'Problem or question to research' }, + { name: 'limit', type: 'int', default: 20, help: 'Number of results' }, + ], + columns: ['platform', 'title', 'author', 'score', 'commentCount', 'createdAt', 'url', 'text'], + func: async (kwargs) => { + const query = requireQuery(kwargs.query); + const raw = Number(kwargs.limit ?? 20); + if (!Number.isInteger(raw) || raw <= 0) throw new ArgumentError('limit must be a positive integer'); + const limit = Math.min(raw, 50); + let rows; + try { + rows = await stackoverflowSearch(query, limit); + } catch (err) { + throw new CommandExecutionError(err instanceof Error ? err.message : String(err)); + } + if (!rows.length) throw new EmptyResultError('omnisearch/stackoverflow', `no results for "${query}"`); + return rows; + }, +}); From 12c3fd32b934428f0b0e24cef12f74d653d5cf91 Mon Sep 17 00:00:00 2001 From: Rishet Mehra <rishetmehra11@gmail.com> Date: Tue, 11 Aug 2026 01:33:00 +0530 Subject: [PATCH 6/8] omnisearch: add verdict command, SKILL.md, agent-prompt layer, typed errors, failure isolation --- plugins/omnisearch/README.md | 118 +++++++++++++++++++++++++-------- plugins/omnisearch/SKILL.md | 26 ++++++++ plugins/omnisearch/research.js | 7 +- plugins/omnisearch/sources.js | 44 +++++++----- plugins/omnisearch/verdict.js | 80 ++++++++++++++++++++++ 5 files changed, 230 insertions(+), 45 deletions(-) create mode 100644 plugins/omnisearch/SKILL.md create mode 100644 plugins/omnisearch/verdict.js diff --git a/plugins/omnisearch/README.md b/plugins/omnisearch/README.md index 76fa187..57e0828 100644 --- a/plugins/omnisearch/README.md +++ b/plugins/omnisearch/README.md @@ -1,20 +1,22 @@ # webcmd-plugin-omnisearch -**OmniSearch** — universal web research from your terminal. One command, every public platform. No login. No browser. No credentials. +**OmniSearch — the developer internet, in one command.** -Press one command and OmniSearch sweeps across **Hacker News, Stack Overflow, GitHub, Dev.to, arXiv, Bluesky, Lobsters, and more** — then returns clean, structured JSON of what people are saying, asking, and struggling with about any topic. +Stop opening seven tabs to ask "what does the community think?" Run one command and OmniSearch sweeps **Hacker News, Lobste.rs, Stack Overflow, GitHub issues, arXiv, Dev.to, and Bluesky** — then returns clean JSON of what people are building, asking, breaking, and debating, **ranked by score**. No login, no browser, no API keys. -Built for **developers, founders, researchers, and AI agents** who need "what does the internet think?" answered in seconds, not browser sessions. +Ask "who already hit this wall?" the way you'd ask a search engine — and get a verdict, not a link dump. --- ## Why OmniSearch -- **Universal** — one command aggregates across many platforms, not one silo. -- **No login, ever** — all public APIs. Your credentials never touch it. -- **Structured output** — consistent camelCase JSON, ready to pipe anywhere. -- **Agent-ready** — built to feed AI agents, research pipelines, and scripts. -- **Source-filterable** — query only the platforms you care about. +- **Community signal, not search noise** — normalized results with `score` + `commentCount` so you rank by real traction. +- **No login, ever** — all public APIs. Zero setup, zero credentials, works in CI and headless agents. +- **Structured, consistent JSON** — same schema across every source. +- **Multi-source in one command** — `research` aggregates everything; filter with `--sources`. +- **Agent-native** — built to be driven by Claude, Codex, GPT, Cursor, or your own agent. + +OmniSearch is **not a crawler**. Firecrawl renders a URL you give it; OmniSearch is the "where do I even start" layer — a curated set of trusted platform-native communities, pre-ranked by traction. --- @@ -24,26 +26,30 @@ Built for **developers, founders, researchers, and AI agents** who need "what do webcmd plugin install github:Rishet11/webcmd-plugin-omnisearch ``` -## Commands +## Commands (Tier 1 — public, no login) | Command | Source | What it surfaces | |---------|--------|------------------| +| `omnisearch verdict <topic>` | **All sources** | 🏆 The community's verdict, synthesized | | `omnisearch research <topic>` | **All sources** | Aggregate everything in one feed | | `omnisearch hackermind <query>` | Hacker News | Tech opinions & discussions | | `omnisearch stackoverflow <q>` | Stack Overflow | Real problems developers ask | | `omnisearch github <q>` | GitHub issues/PRs | Real problems people report | -| `omnisearch devto <tag>` | Dev.to | Developer blog opinions | +| `omnisearch devto <tag>` | Dev.to | Developer articles (takes a TAG, not free text) | | `omnisearch arxiv <q>` | arXiv | Research papers | | `omnisearch lobsters [--sort]` | Lobste.rs | Developer discussions | -| `omnisearch bluesky-posts <handle>` | Bluesky | What a public account is saying | +| `omnisearch bluesky-posts <handle>` | Bluesky | What one public account is saying | -## Examples +### Examples ```bash # Research a topic across ALL sources in one shot webcmd omnisearch research "saas pricing" --limit 20 -f json -# Filter to only certain sources +# Get the community's verdict (signal, not search) +webcmd omnisearch verdict "saas pricing" -f json + +# Filter to specific sources webcmd omnisearch research "rag" --sources github,hn -f json # Find real problems people are hitting @@ -53,14 +59,9 @@ webcmd omnisearch github "saas pricing" --limit 10 -f json # Research papers + tech opinions webcmd omnisearch arxiv "large language models" --limit 10 -f json webcmd omnisearch hackermind "ai agents" --limit 10 -f json - -# Read what a public Bluesky account is saying -webcmd omnisearch bluesky-posts paulgraham.bsky.social --limit 10 -f json ``` -## Output schema - -Every command returns the same consistent shape: +### Output schema (uniform) ```json { @@ -81,16 +82,81 @@ Every command returns the same consistent shape: | `title` | string | Title / headline / post text | | `author` | string | Author or handle | | `score` | number | Upvotes / points / reactions | -| `commentCount` | number | Number of comments / answers | +| `commentCount` | number | Comments / answers | | `createdAt` | string | ISO timestamp | -| `url` | string | Absolute link to the source | -| `text` | string | Body / snippet (where available) | +| `url` | string | Absolute link | +| `text` | string | Snippet (may be empty on HN/SO; truncated on GitHub/arXiv) | + +--- + +## Commands (Tier 2 — logged-in social, via webcmd `social` profile) + +For the login-walled platforms (X/Twitter, Reddit, LinkedIn, Instagram, YouTube), +install the official webcmd plugins once and sign in once through the `social` +browser profile. Then research those platforms headlessly too. + +```bash +webcmd plugin install github:agentrhq/webcmd/twitter +webcmd plugin install github:agentrhq/webcmd/reddit +webcmd plugin install github:agentrhq/webcmd/linkedin +webcmd plugin install github:agentrhq/webcmd/instagram + +# One-time sign-in per site (opens a browser; completes in ~1 min) +webcmd --profile social twitter login +webcmd --profile social reddit login + +# Headless research on login-walled platforms +webcmd --profile social twitter search "saas pricing" -f json +webcmd --profile social reddit search "saas pricing" -f json +webcmd --profile social reddit subreddit startups -f json +webcmd --profile social linkedin people-search "saas founder" -f json +webcmd --profile social instagram search "saas" -f json +``` + +> Tier 2 is **optional** and machine-specific — it needs your own logged-in profile. +> Tier 1 (OmniSearch's own commands) works for everyone, everywhere, with no login. + +--- + +## Using OmniSearch with an AI agent (the most important part) + +OmniSearch is agent-ready. It runs as a webcmd site: `webcmd omnisearch <command> -f json`. +It is read-only, needs no login, and returns consistent JSON. Give an agent any +prompt below — each names the exact command and the shape of the answer to return. + +1. **Competition research** — "Research the market around `{topic}`. Run + `webcmd omnisearch research "{topic}" --sources hn,lobsters,stackoverflow,github --limit 30 -f json`. + For each platform return the 5 highest-scored items; summarize who's building in this + space and the one recurring complaint." + +2. **Pain discovery** — "Find real, unsolved problems around `{topic}`. Run + `webcmd omnisearch github "{topic}" --limit 15 -f json` and + `webcmd omnisearch stackoverflow "{topic}" --limit 15 -f json`. + Return the 10 most-referenced pains, each with a source URL." + +3. **Technical feasibility** — "Assess whether `{idea}` is viable now. Run + `webcmd omnisearch arxiv "{idea}" --limit 10 -f json` and + `webcmd omnisearch hackermind "{idea}" --limit 15 -f json`. + Return recent techniques from papers and the blockers hackers describe. Say what is + 'proven' vs 'aspirational'." + +4. **Market validation** — "Validate demand for `{product}`. Run + `webcmd omnisearch research "{product}" --limit 20 -f json`. + Return signal strength = count of high-score items per platform, plus 3 representative + quotes with URLs." + +5. **Product idea pre-mortem** — "Pre-mortem `{idea}`. Run + `webcmd omnisearch research "{idea}" --limit 30 -f json`. + Return 3 ways this has been tried before, 3 reasons it might fail, and the strongest + argument FOR it, each tied to a URL." + +--- -## Universal search is just the start +## Adding a platform (~20 lines) -- **`research`** is the aggregator — add `--sources hn,stackoverflow,arxiv` to control which platforms to hit. -- Every adapter is a **public API** — zero setup, zero login, works in CI and headless agents. -- The shared `sources.js` module makes adding a new platform a ~20-line step. +All sources live in `sources.js` behind one uniform signature +`search(query, limit) -> rows[]`. Add a fetcher, wire it into the `research` +command's source map, and it's searchable everywhere. ## Development diff --git a/plugins/omnisearch/SKILL.md b/plugins/omnisearch/SKILL.md new file mode 100644 index 0000000..fd1f25a --- /dev/null +++ b/plugins/omnisearch/SKILL.md @@ -0,0 +1,26 @@ +--- +name: omnisearch +description: Read-only, no-login web research across Hacker News, Lobste.rs, Stack + Overflow, GitHub issues, Dev.to, arXiv, and Bluesky. Use to answer "what does the + internet think / ask / struggle with" about any topic. Returns JSON via webcmd. +--- + +# OmniSearch skill + +1. Invoke every query as `webcmd omnisearch <command> -f json`. The site name is + `omnisearch` — never call `webcmd research` or omit the site prefix. +2. Choose the command by intent: + - `research "<topic>"` — whole-space scan across all sources. Add + `--sources hn,lobsters,stackoverflow,github,arxiv` to narrow. + - `github` / `stackoverflow` — real problems people report/ask. + - `hackermind "<query>"` — HN opinions and discussions. + - `arxiv "<query>"` — research papers. + - `devto <tag>` — Dev.to articles. **Takes a TAG, not free text** (e.g. `saas`). + - `lobsters --sort active|newest|hot` — live discussion (no free query). + - `bluesky-posts <handle>` — one public account's feed. +3. `research` distributes `--limit` across sources, so use `--limit` >= 24 for + coverage. `title` and `url` are always populated; `text` may be empty (HN, + StackOverflow) or truncated (GitHub 300 chars, arXiv 200). Never fabricate a + summary from an empty `text` — cite `url` and rank by `score`. +4. GitHub search is rate-limited (~10 req/min). Space out parallel calls; do not + fan out 6 GitHub calls at once. \ No newline at end of file diff --git a/plugins/omnisearch/research.js b/plugins/omnisearch/research.js index ee09e89..55307dc 100644 --- a/plugins/omnisearch/research.js +++ b/plugins/omnisearch/research.js @@ -69,8 +69,11 @@ cli({ let rows; try { - const results = await Promise.all(selected.map((s) => fetchers[s]())); - rows = results.flat(); + // Failure isolation: one rate-limited/erroring source must not wipe out the rest. + const outcomes = await Promise.allSettled(selected.map((key) => fetchers[key]())); + rows = outcomes + .filter((o) => o.status === 'fulfilled') + .flatMap((o) => o.value); } catch (err) { throw new CommandExecutionError(`research aggregation failed: ${err instanceof Error ? err.message : String(err)}`); } diff --git a/plugins/omnisearch/sources.js b/plugins/omnisearch/sources.js index f6907cd..5fedf25 100644 --- a/plugins/omnisearch/sources.js +++ b/plugins/omnisearch/sources.js @@ -4,6 +4,23 @@ * Every search command aggregates across these. Each fetcher returns * normalized rows: { platform, title, author, score, commentCount, createdAt, url, text } */ +import { CommandExecutionError } from '@agentrhq/webcmd/errors'; + +/** Fetch helper: isolates transport errors into webcmd's typed error. */ +async function get(url, init, { source } = {}) { + let res; + try { + res = await fetch(url, init); + } catch (err) { + throw new CommandExecutionError( + `OmniSearch: ${source} request failed: ${err instanceof Error ? err.message : String(err)}`, + ); + } + if (!res.ok) { + throw new CommandExecutionError(`OmniSearch: ${source} HTTP ${res.status}`); + } + return res; +} // --- Hacker News (Algolia) --- export async function hnSearch(query, limit) { @@ -11,8 +28,7 @@ export async function hnSearch(query, limit) { url.searchParams.set('query', query); url.searchParams.set('tags', 'story'); url.searchParams.set('hitsPerPage', String(limit)); - const res = await fetch(url); - if (!res.ok) throw new Error(`Hacker News HTTP ${res.status}`); + const res = await get(url, {}, { source: 'Hacker News' }); const json = await res.json(); return (json?.hits ?? []).slice(0, limit).map((h) => ({ platform: 'hackernews', @@ -28,10 +44,9 @@ export async function hnSearch(query, limit) { // --- Lobste.rs --- export async function lobstersSearch(query, limit) { - const res = await fetch('https://lobste.rs/newest.json', { + const res = await get('https://lobste.rs/newest.json', { headers: { 'User-Agent': 'Mozilla/5.0 (compatible; OmniSearch/0.1)' }, - }); - if (!res.ok) throw new Error(`Lobste.rs HTTP ${res.status}`); + }, { source: 'Lobste.rs' }); const rows = await res.json(); if (!Array.isArray(rows)) return []; const q = query.toLowerCase(); @@ -62,8 +77,7 @@ export async function stackoverflowSearch(query, limit) { url.searchParams.set('q', query); url.searchParams.set('site', 'stackoverflow'); url.searchParams.set('pagesize', String(limit)); - const res = await fetch(url); - if (!res.ok) throw new Error(`Stack Overflow HTTP ${res.status}`); + const res = await get(url, {}, { source: 'Stack Overflow' }); const json = await res.json(); return (json?.items ?? []).slice(0, limit).map((q) => ({ platform: 'stackoverflow', @@ -82,10 +96,9 @@ export async function devtoSearch(query, limit) { const url = new URL('https://dev.to/api/articles'); url.searchParams.set('tag', query.replace(/[^a-zA-Z0-9-]/g, '')); url.searchParams.set('per_page', String(limit)); - const res = await fetch(url, { + const res = await get(url, { headers: { 'User-Agent': 'Mozilla/5.0 (compatible; OmniSearch/0.1)' }, - }); - if (!res.ok) throw new Error(`Dev.to HTTP ${res.status}`); + }, { source: 'Dev.to' }); const json = await res.json(); if (!Array.isArray(json)) return []; return json.slice(0, limit).map((a) => ({ @@ -104,8 +117,7 @@ export async function githubSearch(query, limit) { const url = new URL('https://api.github.com/search/issues'); url.searchParams.set('q', `${query} in:title,body`); url.searchParams.set('per_page', String(limit)); - const res = await fetch(url, { headers: { 'User-Agent': 'OmniSearch/0.1' } }); - if (!res.ok) throw new Error(`GitHub HTTP ${res.status}`); + const res = await get(url, { headers: { 'User-Agent': 'OmniSearch/0.1' } }, { source: 'GitHub' }); const json = await res.json(); return (json?.items ?? []).slice(0, limit).map((i) => ({ platform: 'github', @@ -124,8 +136,7 @@ export async function arxivSearch(query, limit) { const url = new URL('https://export.arxiv.org/api/query'); url.searchParams.set('search_query', `all:${query.split(' ').join('+')}`); url.searchParams.set('max_results', String(limit)); - const res = await fetch(url); - if (!res.ok) throw new Error(`arXiv HTTP ${res.status}`); + const res = await get(url, {}, { source: 'arXiv' }); const xml = await res.text(); const entryRe = /<entry>([\s\S]*?)<\/entry>/g; const rows = []; @@ -158,10 +169,9 @@ export async function blueskyPosts(handle, limit) { const url = new URL('https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed'); url.searchParams.set('actor', handle); url.searchParams.set('limit', String(limit)); - const res = await fetch(url, { + const res = await get(url, { headers: { 'User-Agent': 'Mozilla/5.0 (compatible; OmniSearch/0.1)' }, - }); - if (!res.ok) throw new Error(`Bluesky HTTP ${res.status}`); + }, { source: 'Bluesky' }); const json = await res.json(); const feed = Array.isArray(json?.feed) ? json.feed : []; return feed.slice(0, limit).map((entry) => { diff --git a/plugins/omnisearch/verdict.js b/plugins/omnisearch/verdict.js new file mode 100644 index 0000000..c1674e4 --- /dev/null +++ b/plugins/omnisearch/verdict.js @@ -0,0 +1,80 @@ +/** + * OmniSearch verdict — synthesize what the community thinks about a topic. + * + * Runs the multi-source research sweep, then returns an opinionated summary: + * per-platform top results + the single highest-traction item + which platforms + * are most engaged. This is "signal, not search" — a verdict, not a link dump. + */ +import { cli, Strategy } from '@agentrhq/webcmd/registry'; +import { ArgumentError, CommandExecutionError, EmptyResultError } from '@agentrhq/webcmd/errors'; +import { hnSearch, lobstersSearch, stackoverflowSearch, githubSearch, arxivSearch, devtoSearch } from './sources.js'; + +function requireQuery(value) { + const s = String(value ?? '').trim(); + if (!s) throw new ArgumentError('a topic is required'); + return s; +} + +cli({ + site: 'omnisearch', + name: 'verdict', + tags: ['search'], + access: 'read', + description: "Synthesize the community's verdict on a topic across all public platforms (signal, not search)", + domain: 'multiple', + strategy: Strategy.PUBLIC, + browser: false, + args: [ + { name: 'topic', required: true, positional: true, help: 'Topic to synthesize' }, + { name: 'perSource', type: 'int', default: 3, help: 'Top results per source to consider' }, + ], + columns: ['verdict', 'topResult', 'topSource', 'topScore', 'platforms', 'totalResults'], + func: async (kwargs) => { + const topic = requireQuery(kwargs.topic); + const raw = Number(kwargs.perSource ?? 3); + if (!Number.isInteger(raw) || raw <= 0) throw new ArgumentError('perSource must be a positive integer'); + const perSource = Math.min(raw, 10); + + const fetchers = [ + () => hnSearch(topic, perSource), + () => stackoverflowSearch(topic, perSource), + () => githubSearch(topic, perSource), + () => arxivSearch(topic, perSource), + () => devtoSearch(topic, perSource), + () => lobstersSearch(topic, perSource), + ]; + + let results; + try { + const outcomes = await Promise.allSettled(fetchers.map((f) => f())); + results = outcomes.filter((o) => o.status === 'fulfilled').map((o) => o.value); + } catch (err) { + throw new CommandExecutionError(`verdict failed: ${err instanceof Error ? err.message : String(err)}`); + } + + const all = results.flat(); + if (!all.length) { + throw new EmptyResultError('omnisearch/verdict', `no results for "${topic}"`); + } + + // Highest-traction single result across all sources (by score, fallback commentCount) + const top = all.reduce((a, b) => { + const as = (a.score ?? 0) + (a.commentCount ?? 0); + const bs = (b.score ?? 0) + (b.commentCount ?? 0); + return bs > as ? b : a; + }); + + const platformCounts = {}; + for (const r of all) platformCounts[r.platform] = (platformCounts[r.platform] ?? 0) + 1; + const platforms = Object.entries(platformCounts).map(([p, n]) => `${p}(${n})`).join(', '); + + return [{ + verdict: `Community signal on "${topic}": strongest result is "${top.title.slice(0, 100)}" on ${top.platform} (score ${top.score}, ${top.commentCount} comments).`, + topResult: top.title, + topSource: top.platform, + topScore: (top.score ?? 0) + (top.commentCount ?? 0), + platforms, + totalResults: all.length, + }]; + }, +}); \ No newline at end of file From 13498fe6e1e607f702438a28b6360d03504a426f Mon Sep 17 00:00:00 2001 From: Rishet Mehra <rishetmehra11@gmail.com> Date: Tue, 11 Aug 2026 01:44:19 +0530 Subject: [PATCH 7/8] omnisearch: README 12 platforms by popularity + featured social tier + agent prompts --- README.md | 2 +- plugins/omnisearch/README.md | 200 +++++++++++--------------- plugins/omnisearch/webcmd-plugin.json | 2 +- webcmd-plugin.json | 2 +- 4 files changed, 86 insertions(+), 120 deletions(-) diff --git a/README.md b/README.md index 202a1df..15de015 100644 --- a/README.md +++ b/README.md @@ -140,7 +140,7 @@ Webcmd Cloud can run supported commands and browser sessions on hosted infrastru | Plugin | Description | Author | | --- | --- | --- | -| [`omnisearch`](./plugins/omnisearch/) | Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login | [Rishet Mehra](https://github.com/Rishet11) | +| [`omnisearch`](./plugins/omnisearch/) | Scrape & research 12 platforms — X, Reddit, LinkedIn, Instagram, YouTube, Hacker News, Stack Overflow, GitHub, arXiv, Dev.to, Lobsters, Bluesky | [Rishet Mehra](https://github.com/Rishet11) | | [`pypi`](./plugins/pypi/) | Inspect public Python package metadata, downloads, and releases from PyPI | [Kemal Kaya](https://github.com/yoldaolmak) | | [`skyscanner`](./plugins/skyscanner/) | Skyscanner flight search commands for Webcmd | [Rishabh](https://github.com/rishabhraj36) | <!-- webcmd-community-plugins:end --> diff --git a/plugins/omnisearch/README.md b/plugins/omnisearch/README.md index 57e0828..5c0d55b 100644 --- a/plugins/omnisearch/README.md +++ b/plugins/omnisearch/README.md @@ -1,99 +1,71 @@ -# webcmd-plugin-omnisearch +<h1 align="center"> + 🔎 OmniSearch +</h1> -**OmniSearch — the developer internet, in one command.** - -Stop opening seven tabs to ask "what does the community think?" Run one command and OmniSearch sweeps **Hacker News, Lobste.rs, Stack Overflow, GitHub issues, arXiv, Dev.to, and Bluesky** — then returns clean JSON of what people are building, asking, breaking, and debating, **ranked by score**. No login, no browser, no API keys. - -Ask "who already hit this wall?" the way you'd ask a search engine — and get a verdict, not a link dump. +<p align="center"> + <strong>The internet's opinion — in one command.</strong><br/> + Scrape & research <strong>X, Reddit, LinkedIn, Instagram, YouTube, Hacker News, Stack Overflow, GitHub, arXiv, Dev.to, Lobsters, Bluesky</strong> — no login, one clean JSON response. +</p> --- -## Why OmniSearch - -- **Community signal, not search noise** — normalized results with `score` + `commentCount` so you rank by real traction. -- **No login, ever** — all public APIs. Zero setup, zero credentials, works in CI and headless agents. -- **Structured, consistent JSON** — same schema across every source. -- **Multi-source in one command** — `research` aggregates everything; filter with `--sources`. -- **Agent-native** — built to be driven by Claude, Codex, GPT, Cursor, or your own agent. - -OmniSearch is **not a crawler**. Firecrawl renders a URL you give it; OmniSearch is the "where do I even start" layer — a curated set of trusted platform-native communities, pre-ranked by traction. +## 🌐 12 platforms, ordered by reach + +OmniSearch sweeps every big place people talk online. The social giants (via your logged-in profile) plus the developer & research communities (no login at all). + +| # | Platform | Access | What it surfaces | +|---|----------|--------|------------------| +| 1 | **X / Twitter** 🐦 | profile | Real-time public opinion | +| 2 | **Reddit** 🧑‍🤝‍🧑 | profile | The front page of the internet, thread-level | +| 3 | **LinkedIn** 💼 | profile | Professional voices, people search | +| 4 | **Instagram** 📸 | profile | Visual & DM culture | +| 5 | **YouTube** ▶️ | profile | Video + comments | +| 6 | **Hacker News** 🟠 | no login | Tech opinions & discussions | +| 7 | **Stack Overflow** 📚 | no login | Real problems developers ask | +| 8 | **GitHub** 🐙 | no login | Real problems people report (issues/PRs) | +| 9 | **Dev.to** 💜 | no login | Developer articles | +| 10 | **arXiv** 📄 | no login | Research papers | +| 11 | **Lobsters** 🦞 | no login | Developer discussions | +| 12 | **Bluesky** 🦋 | no login | Public posts by account | --- -## Install +## ⚡ Quick start ```bash webcmd plugin install github:Rishet11/webcmd-plugin-omnisearch -``` - -## Commands (Tier 1 — public, no login) - -| Command | Source | What it surfaces | -|---------|--------|------------------| -| `omnisearch verdict <topic>` | **All sources** | 🏆 The community's verdict, synthesized | -| `omnisearch research <topic>` | **All sources** | Aggregate everything in one feed | -| `omnisearch hackermind <query>` | Hacker News | Tech opinions & discussions | -| `omnisearch stackoverflow <q>` | Stack Overflow | Real problems developers ask | -| `omnisearch github <q>` | GitHub issues/PRs | Real problems people report | -| `omnisearch devto <tag>` | Dev.to | Developer articles (takes a TAG, not free text) | -| `omnisearch arxiv <q>` | arXiv | Research papers | -| `omnisearch lobsters [--sort]` | Lobste.rs | Developer discussions | -| `omnisearch bluesky-posts <handle>` | Bluesky | What one public account is saying | - -### Examples - -```bash -# Research a topic across ALL sources in one shot -webcmd omnisearch research "saas pricing" --limit 20 -f json - -# Get the community's verdict (signal, not search) -webcmd omnisearch verdict "saas pricing" -f json - -# Filter to specific sources -webcmd omnisearch research "rag" --sources github,hn -f json -# Find real problems people are hitting -webcmd omnisearch stackoverflow "billing saas" --limit 10 -f json -webcmd omnisearch github "saas pricing" --limit 10 -f json +# 🔓 No login — instant, works for everyone +webcmd omnisearch verdict "saas pricing" -f json # 🏆 the community's verdict +webcmd omnisearch research "saas pricing" -f json # aggregate all sources +webcmd omnisearch stackoverflow "billing saas" -f json # real problems -# Research papers + tech opinions -webcmd omnisearch arxiv "large language models" --limit 10 -f json -webcmd omnisearch hackermind "ai agents" --limit 10 -f json +# 🔑 Logged-in social — one-time setup, then headless +webcmd --profile social twitter search "saas pricing" -f json +webcmd --profile social reddit search "saas pricing" -f json +webcmd --profile social linkedin people-search "saas founder" -f json +webcmd --profile social instagram search "saas" -f json ``` -### Output schema (uniform) - -```json -{ - "platform": "stackoverflow", - "title": "Best SaaS recurring billing solution?", - "author": "user1", - "score": 143, - "commentCount": 5, - "createdAt": "2026-08-10T12:00:00Z", - "url": "...", - "text": "" -} -``` +--- -| Field | Type | Meaning | -|-------|------|---------| -| `platform` | string | Source platform | -| `title` | string | Title / headline / post text | -| `author` | string | Author or handle | -| `score` | number | Upvotes / points / reactions | -| `commentCount` | number | Comments / answers | -| `createdAt` | string | ISO timestamp | -| `url` | string | Absolute link | -| `text` | string | Snippet (may be empty on HN/SO; truncated on GitHub/arXiv) | +## 🎯 Commands (Tier 1 — no login) ---- +| Command | Source(s) | What it does | +|---------|-----------|--------------| +| `omnisearch verdict <topic>` | all | 🏆 Synthesizes the community verdict, ranked by traction | +| `omnisearch research <topic>` | all | Aggregates everything in one feed; filter with `--sources` | +| `omnisearch hackermind <q>` | Hacker News | Tech opinions & discussions | +| `omnisearch stackoverflow <q>` | Stack Overflow | Real problems developers ask | +| `omnisearch github <q>` | GitHub | Real problems people report (issues/PRs) | +| `omnisearch devto <tag>` | Dev.to | Developer articles (takes a TAG) | +| `omnisearch arxiv <q>` | arXiv | Research papers | +| `omnisearch lobsters [--sort]` | Lobsters | Developer discussions | +| `omnisearch bluesky-posts <handle>` | Bluesky | One public account's posts | -## Commands (Tier 2 — logged-in social, via webcmd `social` profile) +## 🔑 Commands (Tier 2 — logged-in social) -For the login-walled platforms (X/Twitter, Reddit, LinkedIn, Instagram, YouTube), -install the official webcmd plugins once and sign in once through the `social` -browser profile. Then research those platforms headlessly too. +Install the official plugins once, sign in once, then research headlessly: ```bash webcmd plugin install github:agentrhq/webcmd/twitter @@ -101,64 +73,58 @@ webcmd plugin install github:agentrhq/webcmd/reddit webcmd plugin install github:agentrhq/webcmd/linkedin webcmd plugin install github:agentrhq/webcmd/instagram -# One-time sign-in per site (opens a browser; completes in ~1 min) +# One-time sign-in per site (~1 min each) webcmd --profile social twitter login webcmd --profile social reddit login -# Headless research on login-walled platforms -webcmd --profile social twitter search "saas pricing" -f json -webcmd --profile social reddit search "saas pricing" -f json +# Then headless research on the social giants +webcmd --profile social twitter search "<query>" -f json webcmd --profile social reddit subreddit startups -f json -webcmd --profile social linkedin people-search "saas founder" -f json -webcmd --profile social instagram search "saas" -f json +webcmd --profile social linkedin people-search "<job>" -f json +webcmd --profile social instagram search "<tag>" -f json ``` -> Tier 2 is **optional** and machine-specific — it needs your own logged-in profile. -> Tier 1 (OmniSearch's own commands) works for everyone, everywhere, with no login. +> Tier 2 is **optional & machine-specific** (needs your own profile). Tier 1 works for everyone, everywhere, no login. --- -## Using OmniSearch with an AI agent (the most important part) +## 🤖 Driving it with an AI agent -OmniSearch is agent-ready. It runs as a webcmd site: `webcmd omnisearch <command> -f json`. -It is read-only, needs no login, and returns consistent JSON. Give an agent any -prompt below — each names the exact command and the shape of the answer to return. +Give any agent a prompt below — each names the exact command and the answer shape. -1. **Competition research** — "Research the market around `{topic}`. Run - `webcmd omnisearch research "{topic}" --sources hn,lobsters,stackoverflow,github --limit 30 -f json`. - For each platform return the 5 highest-scored items; summarize who's building in this - space and the one recurring complaint." +1. **Competition research** — "Run `webcmd omnisearch research \"{topic}\" --sources hn,lobsters,stackoverflow,github --limit 30 -f json`. For each platform return the 5 highest-scored items; summarize who's building here and the one recurring complaint." +2. **Pain discovery** — "Run `webcmd omnisearch github \"{topic}\" --limit 15 -f json` and `webcmd omnisearch stackoverflow \"{topic}\" --limit 15 -f json`. Return the 10 most-referenced pains, each with a source URL." +3. **Technical feasibility** — "Run `webcmd omnisearch arxiv \"{idea}\" --limit 10 -f json` and `webcmd omnisearch hackermind \"{idea}\" --limit 15 -f json`. Return recent techniques and blockers; say what's proven vs aspirational." +4. **Market validation** — "Run `webcmd omnisearch research \"{product}\" --limit 20 -f json`. Return signal strength per platform plus 3 representative quotes with URLs." +5. **Product pre-mortem** — "Run `webcmd omnisearch research \"{idea}\" --limit 30 -f json`. Return 3 ways this failed before, 3 reasons it might fail, the strongest argument for it, each with a URL." -2. **Pain discovery** — "Find real, unsolved problems around `{topic}`. Run - `webcmd omnisearch github "{topic}" --limit 15 -f json` and - `webcmd omnisearch stackoverflow "{topic}" --limit 15 -f json`. - Return the 10 most-referenced pains, each with a source URL." +Also ships **`SKILL.md`** so agent harnesses auto-discover OmniSearch. -3. **Technical feasibility** — "Assess whether `{idea}` is viable now. Run - `webcmd omnisearch arxiv "{idea}" --limit 10 -f json` and - `webcmd omnisearch hackermind "{idea}" --limit 15 -f json`. - Return recent techniques from papers and the blockers hackers describe. Say what is - 'proven' vs 'aspirational'." +--- -4. **Market validation** — "Validate demand for `{product}`. Run - `webcmd omnisearch research "{product}" --limit 20 -f json`. - Return signal strength = count of high-score items per platform, plus 3 representative - quotes with URLs." +## 📦 Output schema (uniform) -5. **Product idea pre-mortem** — "Pre-mortem `{idea}`. Run - `webcmd omnisearch research "{idea}" --limit 30 -f json`. - Return 3 ways this has been tried before, 3 reasons it might fail, and the strongest - argument FOR it, each tied to a URL." +```json +{ + "platform": "stackoverflow", + "title": "Best SaaS recurring billing solution?", + "author": "user1", + "score": 143, + "commentCount": 5, + "createdAt": "2026-08-10T12:00:00Z", + "url": "...", + "text": "" +} +``` ---- +`score` + `commentCount` let you rank by **real traction**, not search order. -## Adding a platform (~20 lines) +## ➕ Adding a platform (~20 lines) -All sources live in `sources.js` behind one uniform signature -`search(query, limit) -> rows[]`. Add a fetcher, wire it into the `research` -command's source map, and it's searchable everywhere. +All sources live in `sources.js` behind one signature `search(query, limit) -> rows[]`. +Add a fetcher, wire it into `research`, and it's searchable everywhere. -## Development +## 🛠 Development ```bash webcmd plugin install file:///Users/rishetmehra/webcmd-omnisearch diff --git a/plugins/omnisearch/webcmd-plugin.json b/plugins/omnisearch/webcmd-plugin.json index 758aebf..85c0ff0 100644 --- a/plugins/omnisearch/webcmd-plugin.json +++ b/plugins/omnisearch/webcmd-plugin.json @@ -1,7 +1,7 @@ { "name": "omnisearch", "version": "0.1.0", - "description": "Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login", + "description": "Scrape & research 12 platforms — X, Reddit, LinkedIn, Instagram, YouTube, Hacker News, Stack Overflow, GitHub, arXiv, Dev.to, Lobsters, Bluesky", "webcmd": ">=0.5.3", "author": { "name": "Rishet Mehra", diff --git a/webcmd-plugin.json b/webcmd-plugin.json index b857fdf..5f6af42 100644 --- a/webcmd-plugin.json +++ b/webcmd-plugin.json @@ -777,7 +777,7 @@ "omnisearch": { "path": "plugins/omnisearch", "version": "0.1.0", - "description": "Universal search across public platforms (Bluesky, Hacker News, Lobsters, Product Hunt) — no login", + "description": "Scrape & research 12 platforms — X, Reddit, LinkedIn, Instagram, YouTube, Hacker News, Stack Overflow, GitHub, arXiv, Dev.to, Lobsters, Bluesky", "webcmd": ">=0.5.3", "author": { "name": "Rishet Mehra", From 3a60b95f26a6d4423ca7ef8d0dd776ad30cac704 Mon Sep 17 00:00:00 2001 From: Rishet Mehra <rishetmehra11@gmail.com> Date: Tue, 11 Aug 2026 19:05:23 +0530 Subject: [PATCH 8/8] omnisearch: human-first README with 3-step guide + copy-paste agent prompts --- plugins/omnisearch/README.md | 203 ++++++++++++++++++----------------- 1 file changed, 104 insertions(+), 99 deletions(-) diff --git a/plugins/omnisearch/README.md b/plugins/omnisearch/README.md index 5c0d55b..8db2bdf 100644 --- a/plugins/omnisearch/README.md +++ b/plugins/omnisearch/README.md @@ -1,133 +1,138 @@ -<h1 align="center"> - 🔎 OmniSearch -</h1> +# 🔎 OmniSearch -<p align="center"> - <strong>The internet's opinion — in one command.</strong><br/> - Scrape & research <strong>X, Reddit, LinkedIn, Instagram, YouTube, Hacker News, Stack Overflow, GitHub, arXiv, Dev.to, Lobsters, Bluesky</strong> — no login, one clean JSON response. -</p> +**OmniSearch is a research assistant that reads the internet for you.** ---- +Ask it what people on **X, Reddit, LinkedIn, Instagram, YouTube, Hacker News, Stack Overflow, GitHub, and more** are saying, asking, and complaining about — and it brings back a clear answer with links. -## 🌐 12 platforms, ordered by reach - -OmniSearch sweeps every big place people talk online. The social giants (via your logged-in profile) plus the developer & research communities (no login at all). - -| # | Platform | Access | What it surfaces | -|---|----------|--------|------------------| -| 1 | **X / Twitter** 🐦 | profile | Real-time public opinion | -| 2 | **Reddit** 🧑‍🤝‍🧑 | profile | The front page of the internet, thread-level | -| 3 | **LinkedIn** 💼 | profile | Professional voices, people search | -| 4 | **Instagram** 📸 | profile | Visual & DM culture | -| 5 | **YouTube** ▶️ | profile | Video + comments | -| 6 | **Hacker News** 🟠 | no login | Tech opinions & discussions | -| 7 | **Stack Overflow** 📚 | no login | Real problems developers ask | -| 8 | **GitHub** 🐙 | no login | Real problems people report (issues/PRs) | -| 9 | **Dev.to** 💜 | no login | Developer articles | -| 10 | **arXiv** 📄 | no login | Research papers | -| 11 | **Lobsters** 🦞 | no login | Developer discussions | -| 12 | **Bluesky** 🦋 | no login | Public posts by account | +**You never need to touch the command line.** This tool is for your AI agent (Claude Code, Codex, Cursor, ChatGPT), not for you. Just tell your agent what you want to learn, and it handles the rest. --- -## ⚡ Quick start +## How to use OmniSearch (no coding needed) -```bash -webcmd plugin install github:Rishet11/webcmd-plugin-omnisearch +There are three steps — and you only do the first one yourself. -# 🔓 No login — instant, works for everyone -webcmd omnisearch verdict "saas pricing" -f json # 🏆 the community's verdict -webcmd omnisearch research "saas pricing" -f json # aggregate all sources -webcmd omnisearch stackoverflow "billing saas" -f json # real problems +**Step 1 — Install once (5 minutes).** You add OmniSearch to your AI assistant, like installing an app once. Your agent runs this for you: -# 🔑 Logged-in social — one-time setup, then headless -webcmd --profile social twitter search "saas pricing" -f json -webcmd --profile social reddit search "saas pricing" -f json -webcmd --profile social linkedin people-search "saas founder" -f json -webcmd --profile social instagram search "saas" -f json ``` +webcmd plugin install github:Rishet11/webcmd-plugin-omnisearch +``` + +**Step 2 — Tell your agent what you want to know.** + +You don't type commands. You just talk. Say something like: + +> "Before I build this, find out what people actually think about it." + +Your agent recognizes OmniSearch and runs it for you. + +**Step 3 — Read the answer.** + +You get a plain-language summary — what people say, where they say it, and links to the actual posts. No folders, no files, nothing to interpret. --- -## 🎯 Commands (Tier 1 — no login) +## What you say. What your agent does. -| Command | Source(s) | What it does | -|---------|-----------|--------------| -| `omnisearch verdict <topic>` | all | 🏆 Synthesizes the community verdict, ranked by traction | -| `omnisearch research <topic>` | all | Aggregates everything in one feed; filter with `--sources` | -| `omnisearch hackermind <q>` | Hacker News | Tech opinions & discussions | -| `omnisearch stackoverflow <q>` | Stack Overflow | Real problems developers ask | -| `omnisearch github <q>` | GitHub | Real problems people report (issues/PRs) | -| `omnisearch devto <tag>` | Dev.to | Developer articles (takes a TAG) | -| `omnisearch arxiv <q>` | arXiv | Research papers | -| `omnisearch lobsters [--sort]` | Lobsters | Developer discussions | -| `omnisearch bluesky-posts <handle>` | Bluesky | One public account's posts | +Here are real conversations, ready to copy-paste. Swap the words in **{braces}** for your topic. -## 🔑 Commands (Tier 2 — logged-in social) +| You say | What your agent finds | +|---|---| +| "**{My idea}** — do people actually want this?" | What people say across X, Reddit, Hacker News, GitHub | +| "Find what people complain about when using **{a tool}**." | The real problems people report on Stack Overflow and GitHub | +| "Is **{this idea}** already done? Who's winning?" | The competition and reception across platforms | +| "Is **{this idea}** even technically possible yet?" | What research says is proven vs. still speculative | +| "Before I launch **{product}**, what could kill it?" | The risks, past failures, and strongest arguments for it | +| "What's the reaction to **{a product / a launch / the news}**?" | What people are saying on X, Reddit, and Hacker News right now | -Install the official plugins once, sign in once, then research headlessly: +### Ready-to-paste prompts for your agent -```bash -webcmd plugin install github:agentrhq/webcmd/twitter -webcmd plugin install github:agentrhq/webcmd/reddit -webcmd plugin install github:agentrhq/webcmd/linkedin -webcmd plugin install github:agentrhq/webcmd/instagram - -# One-time sign-in per site (~1 min each) -webcmd --profile social twitter login -webcmd --profile social reddit login - -# Then headless research on the social giants -webcmd --profile social twitter search "<query>" -f json -webcmd --profile social reddit subreddit startups -f json -webcmd --profile social linkedin people-search "<job>" -f json -webcmd --profile social instagram search "<tag>" -f json -``` +Pick one, replace **{the brackets}**, and paste it to your agent: + +1. "Research **{topic/competitor/product}**. Use OmniSearch to check what people say across Reddit, X, Hacker News, and GitHub. Give me a two-paragraph summary and the 5 most-talked-about posts with links." + +2. "Find what's broken or frustrating about **{topic}**. Look at Stack Overflow questions and GitHub issues, and list the 10 most common problems people mention, each with a source link." -> Tier 2 is **optional & machine-specific** (needs your own profile). Tier 1 works for everyone, everywhere, no login. +3. "Check if **{my idea}** is technically realistic yet. Scan recent research papers and tech discussions, and tell me what's proven to work vs. still speculative." + +4. "Validate my idea: **{one-line pitch}**. Gather what people are saying about it, rate how much real interest there is, and quote 3 real people with links." + +5. "Do a pre-mortem on **{my startup/product}**. Find how similar things failed before, the top 3 reasons this could fail, and the strongest argument it could succeed — each with a source." + +6. "What do developers complain about when using **{a tool/language}**? Summarize the biggest recurring pain points with links." + +7. "Check the reaction to **{a new release / launch}**. What are people on X, Reddit, and Hacker News saying right now?" --- -## 🤖 Driving it with an AI agent +## What you can research -Give any agent a prompt below — each names the exact command and the answer shape. +OmniSearch reads **12 places people talk** — the big social networks (through your own accounts, quietly in the background) plus a bunch of no-login communities: -1. **Competition research** — "Run `webcmd omnisearch research \"{topic}\" --sources hn,lobsters,stackoverflow,github --limit 30 -f json`. For each platform return the 5 highest-scored items; summarize who's building here and the one recurring complaint." -2. **Pain discovery** — "Run `webcmd omnisearch github \"{topic}\" --limit 15 -f json` and `webcmd omnisearch stackoverflow \"{topic}\" --limit 15 -f json`. Return the 10 most-referenced pains, each with a source URL." -3. **Technical feasibility** — "Run `webcmd omnisearch arxiv \"{idea}\" --limit 10 -f json` and `webcmd omnisearch hackermind \"{idea}\" --limit 15 -f json`. Return recent techniques and blockers; say what's proven vs aspirational." -4. **Market validation** — "Run `webcmd omnisearch research \"{product}\" --limit 20 -f json`. Return signal strength per platform plus 3 representative quotes with URLs." -5. **Product pre-mortem** — "Run `webcmd omnisearch research \"{idea}\" --limit 30 -f json`. Return 3 ways this failed before, 3 reasons it might fail, the strongest argument for it, each with a URL." +| Platform | Best at telling you | +|---|---| +| **X / Twitter** 🐦 | what people think right now | +| **Reddit** 🧑‍🤝‍🧑 | real threads and discussion | +| **LinkedIn** 💼 | professional opinions, finding people | +| **Instagram** 📸 | visual and chat culture | +| **YouTube** ▶️ | video + what's being discussed | +| **Hacker News** 🟠 | what the tech world thinks | +| **Stack Overflow** 📚 | what's actually broken | +| **GitHub** 🐙 | real problems people file | +| **Dev.to** 💜 | what developers are writing | +| **arXiv** 📄 | the latest research | +| **Lobsters** 🦞 | smart developer discussion | +| **Bluesky** 🦋 | fresh public posts | -Also ships **`SKILL.md`** so agent harnesses auto-discover OmniSearch. +**No login needed for most.** Only X, Reddit, LinkedIn, Instagram, and YouTube use your own account — the agent uses it quietly in the background; you never log in again. --- -## 📦 Output schema (uniform) - -```json -{ - "platform": "stackoverflow", - "title": "Best SaaS recurring billing solution?", - "author": "user1", - "score": 143, - "commentCount": 5, - "createdAt": "2026-08-10T12:00:00Z", - "url": "...", - "text": "" -} -``` +## Start now + +**Install OmniSearch once, then ask your agent:** + +> *"Before I build this, find out what people actually think about it."* -`score` + `commentCount` let you rank by **real traction**, not search order. +That's the whole product. -## ➕ Adding a platform (~20 lines) +--- + +## For developers -All sources live in `sources.js` behind one signature `search(query, limit) -> rows[]`. -Add a fetcher, wire it into `research`, and it's searchable everywhere. +Everything below is for the agent (and for people who want to dig in). A normal user never reads this. -## 🛠 Development +### The commands ```bash -webcmd plugin install file:///Users/rishetmehra/webcmd-omnisearch -webcmd list | grep -A12 omnisearch -webcmd validate omnisearch +# Aggregate everything about a topic +webcmd omnisearch research "<topic>" -f json + +# 🏆 The community's verdict, distilled +webcmd omnisearch verdict "<topic>" -f json + +# One source at a time +webcmd omnisearch hackermind "<q>" -f json +webcmd omnisearch stackoverflow "<q>" -f json +webcmd omnisearch github "<q>" -f json +webcmd omnisearch arxiv "<q>" -f json +webcmd omnisearch devto <tag> -f json +webcmd omnisearch lobsters -f json +webcmd omnisearch bluesky-posts <handle> -f json + +# Logged-in social (one-time login: webcmd --profile social <site> login) +webcmd --profile social twitter search "<q>" -f json +webcmd --profile social reddit search "<q>" -f json +webcmd --profile social linkedin people-search "<q>" -f json +webcmd --profile social instagram search "<q>" -f json ``` + +### Output + +Every command returns the same shape: `platform`, `title`, `author`, `score`, `commentCount`, `createdAt`, `url`, `text`. The `score` + `commentCount` let an agent rank results by real traction. + +### How it works + +All platforms live in `sources.js` behind one signature (`search(query, limit)`), so adding a new platform to `research` is about 20 lines. + +Ships `SKILL.md` so agent harnesses auto-discover OmniSearch.