diff --git a/skills/openbrowser/SKILL.md b/skills/openbrowser/SKILL.md new file mode 100644 index 0000000..8167f50 --- /dev/null +++ b/skills/openbrowser/SKILL.md @@ -0,0 +1,318 @@ +--- +name: openbrowser +description: "Use when: open-browser, openbrowser, headless browser testing, semantic tree, navigate and test, headless browse without playwright, CDP testing, site mapping, knowledge graph crawl. Do not trigger for: screenshot testing, visual regression testing, cross-browser testing, Playwright specific features, CSS rendering verification, JavaScript-heavy SPA testing requiring real browser engine." +author: Chris Engelhard +triggers: + - open-browser + - openbrowser + - headless browser testing + - semantic tree + - navigate and test + - headless browse without playwright + - CDP testing + - site mapping + - knowledge graph crawl + - test page content + - check page structure + - form testing without browser + - scrape semantic state +context: + domains: + - testing + - browser-automation + - web-scraping + concerns: + - headless-testing + - semantic-analysis + - form-interaction + - site-mapping + technologies: + - rust + - deno-core + - v8 + patterns: + - http-only-browsing + - semantic-tree-parsing + - element-id-interaction + priority: 8 + tokenBudget: 4000 + +negativeTriggers: + - screenshot testing + - visual regression testing + - cross-browser testing + - Playwright specific features + - CSS rendering verification + - JavaScript-heavy SPA testing requiring real browser engine + +allowed-tools: Bash Read Write Edit + +--- + +# open-browser + +A headless browser built for AI agents. No pixels, no screenshots — just structured semantic state. HTTP + HTML parsing only, no Chromium binary, no Docker, no GPU. + +## Overview + +open-browser fetches URLs, parses HTML, and outputs a clean semantic tree — landmarks, headings, links, buttons, forms, and their actions — in milliseconds. Interactive elements get unique IDs (`[#1]`, `[#2]`) that AI agents use to reference them without CSS selectors. + +**Core principle:** AI agents don't need screenshots. They need to know what's on a page, what they can interact with, and where they can go. + +## When to Use + +Use open-browser for: +- Checking page has specific heading/text +- Verifying link/button exists with correct label +- Testing form submission (login, search, signup) +- Verifying navigation links and routes +- Mapping site structure for test coverage +- Extracting semantic content from pages +- Fast smoke tests (under 200ms per page) +- Checking network requests and subresources +- PDF content extraction +- AI agent workflows that need structured state, not pixels +- Running in CI without browser dependencies + +**When open-browser is the better choice:** +- Speed matters (HTTP GET + HTML parse vs full browser launch) +- You only need semantic/content assertions +- Running in CI without browser dependencies +- Mapping site structure for test coverage planning +- AI agent workflows that need structured state, not pixels + +## When NOT to Use + +Do not use open-browser for: +- Screenshot comparison / visual regression testing +- Testing CSS layouts and rendering +- Testing JavaScript-heavy SPA interactions +- Cross-browser testing (Chrome/Firefox/Safari) +- Testing real WebSocket connections +- Verifying pixel-perfect design +- Drag-and-drop interactions +- File upload handling +- Hover states and CSS-dependent behavior + +**When Playwright is required:** +- You need real JavaScript execution (React/Vue reactivity, async data loading) +- Visual regression testing +- Cross-browser compatibility verification +- CSS-dependent behavior testing +- Complex user interactions (drag-drop, file upload, hover states) +## Command Decision Matrix + +| I want to... | Command | Key flags | +|---|---|---| +| Check what's on a page | `open-browser navigate ` | `--format tree` or `--format json` | +| Test only interactive elements | `open-browser navigate ` | `--interactive-only` | +| Click a link by element ID | `open-browser interact click-id ` | `--format json` | +| Click using CSS selector | `open-browser interact click ` | | +| Type into a form field | `open-browser interact type-id ` | | +| Submit a form | `open-browser interact submit --field 'name=value'` | | +| Wait for element to appear | `open-browser interact wait ` | `--timeout-ms 5000` | +| Check network requests | `open-browser navigate ` | `--network-log` | +| Get navigation graph | `open-browser navigate ` | `--format json --with-nav` | +| Map entire site | `open-browser map ` | `--depth 3 --output kg.json` | +| Run interactive session | `open-browser repl` | `--js` for JavaScript | +| Automate via WebSocket | `open-browser serve` | `--host --port` | +| Clean session data | `open-browser clean` | `--cookies-only` or `--cache-only` | +| View PDF content | `open-browser navigate ` | Works automatically | +| Use JavaScript execution | `open-browser navigate ` | `--js --wait-ms 5000` | + +## Workflow + +### Pattern 1: Page Content Verification + +``` +1. Navigate to URL: open-browser navigate --format json +2. Parse JSON output +3. Assert: check semantic_tree for expected headings, links, roles +4. Assert: check stats (landmarks, links, headings, actions counts) +``` + +### Pattern 2: Form Interaction Testing + +``` +1. Navigate to form page: open-browser navigate --interactive-only +2. Identify form fields by element IDs ([#1], [#2], etc.) +3. Type into fields: open-browser interact type-id +4. Submit form: open-browser interact submit 'form' --field 'name=value' +5. Verify result: check response page semantic tree +``` + +### Pattern 3: Navigation Flow Testing + +``` +1. Navigate to start page: open-browser navigate --format json --with-nav +2. Check navigation_graph for expected internal links +3. Click link: open-browser interact click-id +4. Verify navigation landed on correct page (check title, headings) +``` + +### Pattern 4: Site Coverage Mapping + +``` +1. Map site: open-browser map --depth 3 --output kg.json +2. Parse kg.json +3. Count states and transitions +4. Verify all expected routes are discovered +5. Check for unreachable pages (gaps in the graph) +``` + +### Pattern 5: Network Request Verification + +``` +1. Navigate with network logging: open-browser navigate --network-log --format json +2. Parse network_log from JSON output +3. Assert: expected requests were made (check URLs, status codes, timing) +4. Assert: no failed requests +5. Assert: subresource count matches expectations +``` + +### Pattern 6: Session Persistence Testing + +``` +1. Start REPL: open-browser repl +2. Navigate to login page: visit +3. Type username: type # +4. Submit form: submit +5. Navigate to protected page: visit +6. Verify session persisted (cookies maintained) +4. Navigate to protected page: visit +5. Verify session persisted (cookies maintained) +``` + +## Output Formats + +| Format | Flag | When to use | +|--------|------|-------------| +| Markdown | `--format md` (default) | Quick visual inspection, human-readable | +| Tree | `--format tree` | See hierarchy and nesting clearly | +| JSON | `--format json` | Programmatic parsing, assertions, CI | + +JSON output structure for assertions: +- `url` — final URL (after redirects) +- `title` — page title +- `semantic_tree.root` — tree with `role`, `children`, `text` +- `semantic_tree.stats` — `{ landmarks, links, headings, actions }` +- `navigation_graph.internal_links` — `[{ url, label }]` +- `navigation_graph.external_links` — `[url]` +- `navigation_graph.forms` — `[{ action, method, fields }]` +- `network_log.requests` — `[{ method, url, status, content_type, timing_ms }]` + +## Architecture Context + +``` +open-browser +├── crates/open-core Browser type, HTML parsing, semantic tree, interaction, tabs +├── crates/open-debug Network debugger, request recording, subresource discovery +├── crates/open-cdp CDP WebSocket server (14 domains) +├── crates/open-kg Knowledge Graph, BFS crawler, state fingerprinting +└── crates/open-cli CLI binary (navigate, interact, map, tab, serve, repl, clean) +``` + +**Key architectural facts for testing:** +- No rendering engine — HTTP GET + HTML parse only (except optional V8 via `--js`) +- Element IDs are per-page, not persistent across navigations +- Tab state does NOT persist across CLI invocations (use REPL or CDP for persistence) +- JS execution is limited: only inline scripts, no external scripts, setTimeout/setInterval are no-ops +- PDF detection is automatic (content-type sniffing), no flags needed + +## Known Limitations + +| Limitation | Impact | Workaround | +|---|---|---| +| No real JS rendering | Cannot test JS-heavy SPAs | Use `--js` for basic inline scripts, use Playwright for real JS | +| No screenshots | No visual regression testing | Use Playwright for visual tests | +| No CSS rendering | Cannot test CSS-dependent behavior | Use Playwright for layout tests | +| No cross-browser | Only tests server-rendered HTML | Use Playwright for cross-browser | +| External scripts not executed | Dynamic content from external JS won't appear | By design — only inline scripts | +| setTimeout/setInterval no-ops | Timed behaviors won't execute | By design — prevents infinite loops | +| Element IDs per-page | IDs change between navigations | Re-navigate to get fresh IDs before interacting | + +## Error Handling + +| Situation | Response | +|---|---| +| Element ID not found | Re-navigate to get fresh element IDs, use `--interactive-only` to list available IDs | +| Form submission fails | Check CSRF tokens in form fields, verify form action/method | +| Navigation timeout | Increase `--wait-ms`, check if URL is reachable with `curl` | +| Build failure on Linux | Install `cmake` and `clang`, set `LIBCLANG_PATH=/usr/lib` | +| JS execution hangs | Avoid complex JS, only inline scripts are supported | +| PDF not detected | Verify server sends `Content-Type: application/pdf` header | +| CDP connection refused | Check `serve` is running, verify host/port, check firewall | + +## Quick Tests + +**Should trigger:** +- "Test the login flow with open-browser" +- "Map the site structure for testing coverage" +- "Navigate to a URL and check the semantic tree" +- "Interact with a form using open-browser" +- "Start a CDP server for browser automation" +- "Check what headings are on this page" +- "Verify the navigation links on the homepage" +- "Extract the semantic structure of this URL" + +**Should not trigger:** +- "Take a screenshot of the page" +- "Test across Chrome, Firefox, and Safari" +- "Run visual regression tests" +- "Use Playwright to automate tests" +- "Check if the CSS layout is correct" +- "Test drag and drop functionality" +- "Verify pixel-perfect rendering" + +**Functional:** +- `open-browser navigate https://example.com --format json` returns valid JSON with semantic_tree +- `open-browser interact https://example.com click-id 1` follows link by element ID +- `open-browser map https://example.com --depth 1 --output kg.json` produces valid knowledge graph +- `open-browser repl` starts interactive session with persistent state +- `open-browser navigate ` automatically extracts PDF content + +## References + +- `references/installation.md` — Build from source, Arch Linux extras, Docker +- `references/cli-reference.md` — All commands, flags, output formats, examples +- `references/testing-patterns.md` — Playwright vs open-browser translations, assertion recipes +- `references/repl-and-cdp.md` — REPL commands, CDP server for automation +- `references/knowledge-graph.md` — Site mapping, state fingerprinting, transition types +- `references/semantic-roles.md` — HTML element to ARIA role/action mapping table +- `references/programmatic-usage.md` — Rust API, Browser type, tab management +- `references/examples.md` — End-to-end examples for common testing scenarios +- `references/troubleshooting.md` — Build issues, runtime errors, fixes, workarounds +## Examples + +See `references/examples.md` for complete end-to-end examples including: + +- Navigate and verify page content +- Login flow with REPL +- Form submission via CLI +- Network debugging +- Site mapping and coverage verification +- PDF extraction +- Tab management +- CDP automation with Python +- Smoke testing multiple pages +- SEO audit script + +## Best Practices + +- **Always get fresh element IDs** before interacting — IDs are per-page and not persistent across navigations +- **Use `--interactive-only` first** when you need to interact with elements, to quickly find available IDs +- **Use `--format json` in scripts** for reliable programmatic parsing +- **Use REPL for multi-step flows** that require session persistence (cookies, login state) +- **Use CDP server for external automation** from Python, Node.js, or other test frameworks +- **Use `--network-log` for performance assertions** and to verify subresource loading +- **Map the site first** when testing unknown sites, to understand state space and reachable pages +- **Prefer `--format tree`** for quick visual debugging, `--format md` for default readability +- **Check `--with-nav`** when you need form descriptors and internal link graphs +- **Use Playwright as fallback** for anything requiring screenshots, real JS rendering, or complex interactions + +## Related Skills + +| Skill | Purpose | When to use | +|---|---|---| +| **playwright-cli** | Full browser automation with real rendering | When you need screenshots, JS-heavy SPA testing, visual regression | +| **bowser** | Headless browsing, parallel sessions | When you need background browser automation | diff --git a/skills/openbrowser/references/cli-reference.md b/skills/openbrowser/references/cli-reference.md new file mode 100644 index 0000000..cdf5baf --- /dev/null +++ b/skills/openbrowser/references/cli-reference.md @@ -0,0 +1,178 @@ +# CLI Reference + +## Global Flags + +| Flag | Description | +| --- | --- | +| `--format ` | Output format (default: md) | +| `--interactive-only` | Show only interactive elements | +| `--js` | Enable JavaScript execution via V8/deno_core | +| `--wait-ms ` | Wait time for async JS rendering (default: varies) | +| `--header
` | Custom HTTP header (can be repeated) | +| `--network-log` | Capture network request table | +| `--with-nav` | Include navigation graph in JSON output | +| `-v` | Verbose logging | + +## Subcommands + +### navigate + +Navigate to a URL and output the semantic tree. + +```bash +open-browser navigate [--format md|tree|json] [--interactive-only] [--js] [--wait-ms N] [--header "Key: Value"] [--network-log] [--with-nav] [-v] +``` + +Examples: + +```bash +# Default markdown tree +open-browser navigate https://example.com + +# JSON with full data +open-browser navigate https://example.com --format json --with-nav + +# Only buttons, links, inputs +open-browser navigate https://example.com --interactive-only + +# With JavaScript execution +open-browser navigate https://example.com --js --wait-ms 5000 + +# Custom auth header +open-browser navigate https://api.example.com --header "Authorization: Bearer token" + +# Network debugging +open-browser navigate https://example.com --network-log --format json +``` + +JSON output fields: `url`, `title`, `semantic_tree` (`root`, `stats`), `navigation_graph` (`internal_links`, `external_links`, `forms`), `network_log` (`total_requests`, `total_bytes`, `total_time_ms`, `requests[]`) + +### interact + +Interact with page elements at HTTP level. + +```bash +open-browser interact [args] [--format json] [--js] [--wait-ms N] +``` + +Actions: + +- `click ` — Click element by CSS selector (follows links, submits forms) +- `click-id ` — Click element by its ID number (e.g., `click-id 1` for `[#1]`) +- `type ` — Type into field by CSS selector +- `type-id ` — Type into field by element ID +- `submit --field 'name=value'` — Submit form with field values +- `wait --timeout-ms N` — Wait for element to appear +- `scroll --direction down|up|to-top|to-bottom` — Scroll/detect pagination + +Examples: + +```bash +# Click link by element ID +open-browser interact https://example.com click-id 1 + +# Type into search field +open-browser interact https://example.com type-id 3 'search query' + +# Submit login form +open-browser interact https://example.com submit 'form' --field 'user=admin' --field 'pass=secret' + +# Wait for dynamic content with JS +open-browser interact https://example.com wait '.result-list' --js --timeout-ms 5000 + +# Scroll/paginate +open-browser interact 'https://example.com/news?page=1' scroll --direction down +``` + +How interactions work: + +- **click (link):** resolves href, HTTP GET, returns new page +- **click (button):** finds enclosing `
`, collects all fields including hidden CSRF tokens, submits via HTTP +- **type:** returns field selector + value +- **submit:** collects all form fields, merges with `--field` values, HTTP POST/GET +- **wait:** checks HTML for selector, re-fetches if not found +- **scroll:** detects pagination patterns in URL (`?page=`, `?offset=`, `/page/N`) + +### map + +Map a site's structure into a knowledge graph via BFS crawl. + +```bash +open-browser map [--depth N] [--max-pages N] [--output ] [--no-pagination] [-v] +``` + +| Flag | Default | Description | +| --- | --- | --- | +| `--depth` | 3 | Maximum crawl depth | +| `--max-pages` | 50 | Maximum pages to crawl | +| `--output` | required | Output JSON file path | +| `--no-pagination` | false | Skip pagination discovery | +| `-v` | — | Verbose logging | + +Examples: + +```bash +# Standard site map +open-browser map https://example.com --output kg.json + +# Shallow crawl +open-browser map https://example.com --depth 1 --output kg.json + +# Deep crawl +open-browser map https://example.com --depth 5 --max-pages 200 --output kg.json +``` + +### tab + +Manage multiple browser tabs. + +```bash +open-browser tab open [--js] +open-browser tab list +open-browser tab info +open-browser tab navigate +``` + +Note: Tab state does NOT persist across CLI invocations. Use REPL or CDP server for persistent tabs. + +### serve + +Start Chrome DevTools Protocol WebSocket server. + +```bash +open-browser serve [--host ] [--port ] [--timeout ] +``` + +| Flag | Default | Description | +| --- | --- | --- | +| `--host` | localhost | Bind address | +| `--port` | default | Port number | +| `--timeout` | none | Inactivity timeout in seconds | + +Implemented CDP domains: Browser, Target, Page, Runtime, DOM, Network, Emulation, Input, CSS, Log, Console, Security, Performance, Open (custom) + +### repl + +Start interactive session with persistent state. + +```bash +open-browser repl [--js] [--format md|tree|json] [--wait-ms N] +``` + +### clean + +Clean session data (cookies, cache, localStorage). + +```bash +open-browser clean [--cookies-only] [--cache-only] [--cache-dir ] +``` + +## PDF Support + +PDF URLs are detected automatically by Content-Type. No flags needed: + +```bash +open-browser navigate https://example.com/report.pdf +``` + +Works with all formats: `--format json`, `--format tree`, `--format md` diff --git a/skills/openbrowser/references/examples.md b/skills/openbrowser/references/examples.md new file mode 100644 index 0000000..93e3e7c --- /dev/null +++ b/skills/openbrowser/references/examples.md @@ -0,0 +1,264 @@ +# Examples + +Complete end-to-end examples for common testing scenarios with open-browser. + +## Example 1: Navigate and Verify Page Content + +Verify that `https://example.com` has the expected heading and at least one link. + +```bash +RESULT=$(open-browser navigate https://example.com --format json) + +# Check title +echo "$RESULT" | jq -r '.title' +# Expected: "Example Domain" + +# Check there is an h1 heading +# Count all headings (H1 specificity depends on JSON structure) +H1_COUNT=$(echo "$RESULT" | jq '[.. | objects | select(.role? == "heading")] | length') +echo "Heading count: $H1_COUNT" +# Expected: >= 1 (check output for h1 specifically) + +# Check there is at least 1 link +LINKS=$(echo "$RESULT" | jq '.semantic_tree.stats.links') +echo "Links: $LINKS" +# Expected: >= 1 +``` + +## Example 2: Login Flow with REPL + +Test a complete login flow with session persistence using the REPL. + +```bash +# Start REPL with JS enabled +open-browser repl --js + +# Inside the REPL: +visit https://example.com/login +# Output shows interactive elements with IDs +# e.g., [#1] textbox "Email" [action: fill] +# [#2] textbox "Password" [action: fill] +# [#3] button "Sign In" [action: click] + +type #1 user@example.com +type #2 mypassword +click #3 + +# Verify redirect to dashboard +visit https://example.com/dashboard +# Check dashboard headings are present + +exit +``` + +## Example 3: Form Submission via CLI + +Submit a contact form with fields and verify the thank-you page. + +```bash +# Step 1: Navigate and identify form fields +open-browser navigate https://example.com/contact --interactive-only +# Output shows: +# [#1] textbox "Name" [action: fill] +# [#2] textbox "Email" [action: fill] +# [#3] textbox "Message" [action: fill] +# [#4] button "Send" [action: click] + +# Step 2: Submit via interact +RESULT=$(open-browser interact https://example.com/contact submit 'form' \ + --field 'name=John Doe' \ + --field 'email=john@example.com' \ + --field 'message=Hello!' \ + --format json) + +# Step 3: Verify we landed on the thank-you page +FINAL_URL=$(echo "$RESULT" | jq -r '.url') +echo "Final URL: $FINAL_URL" +# Expected: "https://example.com/thank-you" + +# Step 4: Verify heading on thank-you page +THANK_YOU_HEADING=$(echo "$RESULT" | jq -r '.. | objects | select(.role? == "heading") | .text' | head -1) +echo "Heading: $THANK_YOU_HEADING" +# Expected: contains "Thank You" +``` + +## Example 4: Network Debugging + +Capture all network requests made when loading a page. + +```bash +RESULT=$(open-browser navigate https://example.com --network-log --format json) + +# Total requests and timing +echo "$RESULT" | jq '{total_requests: .network_log.total_requests, total_bytes: .network_log.total_bytes, total_time_ms: .network_log.total_time_ms, failed: .network_log.failed}' + +# List all request URLs +echo "$RESULT" | jq -r '.network_log.requests[] | "\(.method) \(.status) \(.timing_ms)ms \(.url)"' +``` + +Expected output: +``` +GET 200 142ms https://example.com/ +GET 200 45ms https://example.com/styles.css +GET 200 23ms https://example.com/script.js +``` + +## Example 5: Site Mapping + +Map a site and verify specific routes are reachable. + +```bash +# Generate knowledge graph +open-browser map https://example.com --depth 2 --output kg.json + +# Verify key pages were found +for PATH in "/" "/about" "/contact"; do + FOUND=$(jq -r --arg p "$PATH" '.states | to_entries[] | select(.value.url | endswith($p)) | .key' kg.json) + if [ -n "$FOUND" ]; then + echo "PASS: $PATH found" + else + echo "FAIL: $PATH not found" + fi +done + +# Verify there are no dead links (unverified transitions) +UNVERIFIED=$(jq '[.transitions[] | select(.verified == false)] | length' kg.json) +echo "Unverified transitions: $UNVERIFIED" +# Expected: 0 +``` + +## Example 6: PDF Extraction + +Navigate to a PDF URL and extract its structure. + +```bash +RESULT=$(open-browser navigate https://example.com/report.pdf --format json) + +# Check page title (from PDF metadata or first heading) +echo "$RESULT" | jq -r '.title' + +# Count headings extracted from PDF +HEADING_COUNT=$(echo "$RESULT" | jq '[.. | objects | select(.role? == "heading")] | length') +echo "Headings found: $HEADING_COUNT" + +# List all headings +echo "$RESULT" | jq -r '.. | objects | select(.role? == "heading") | "\(.level // ""): \(.text)"' +``` + +## Example 7: Tab Management + +Open multiple tabs in the REPL and switch between them. + +```bash +open-browser repl + +# Inside REPL: +visit https://example.com +# Now at example.com + +tab open https://httpbin.org +# Opened tab 2 + +tab list +# Tabs (2 total): +# * [2] Ready — httpbin.org — https://httpbin.org +# [1] Ready — Example Domain — https://example.com + +tab switch 1 +# Back to example.com + +click #1 +# Click first interactive element + +tab switch 2 +# Back to httpbin.org + +exit +``` + +## Example 8: CDP Automation with Python + +Start the CDP server and control it from Python. + +```bash +# Terminal 1: start CDP server +open-browser serve --host 127.0.0.1 --port 9222 +``` + +```python +# test_cdp.py +import asyncio +import websockets +import json + +async def test_navigation(): + uri = "ws://127.0.0.1:9222" + async with websockets.connect(uri) as ws: + # Navigate to example.com + await ws.send(json.dumps({ + "id": 1, + "method": "Page.navigate", + "params": {"url": "https://example.com"} + })) + response = await ws.recv() + print(response) + + # Evaluate title + await ws.send(json.dumps({ + "id": 2, + "method": "Runtime.evaluate", + "params": {"expression": "document.title"} + })) + response = await ws.recv() + print(response) + +asyncio.run(test_navigation()) +``` + +## Example 9: Smoke Test Multiple Pages + +Run a quick smoke test across multiple pages. + +```bash +#!/bin/bash +URLS=( + "https://example.com/" + "https://example.com/about" + "https://example.com/products" + "https://example.com/contact" +) + +for URL in "${URLS[@]}"; do + RESULT=$(open-browser navigate "$URL" --format json) + STATUS=$(echo "$RESULT" | jq -r '.semantic_tree.stats.headings') + if [ "$STATUS" -gt 0 ]; then + echo "PASS: $URL ($STATUS headings)" + else + echo "FAIL: $URL (no headings found)" + fi +done +``` + +## Example 10: SEO Audit + +Check basic SEO elements on a page. + +```bash +RESULT=$(open-browser navigate https://example.com --format json) + +# 1. Title exists and is not empty +TITLE=$(echo "$RESULT" | jq -r '.title') +[ -n "$TITLE" ] && echo "PASS: Title present ($TITLE)" || echo "FAIL: Missing title" + +# 2. Exactly one H1 (if JSON includes level field, filter with | select(.level? == 1)) +H1_COUNT=$(echo "$RESULT" | jq '[.. | objects | select(.role? == "heading")] | length') +[ "$H1_COUNT" -ge 1 ] && echo "PASS: At least one heading" || echo "FAIL: No headings found" + +# 3. Has navigation landmark +NAV=$(echo "$RESULT" | jq '[.. | objects | select(.role? == "navigation")] | length') +[ "$NAV" -ge 1 ] && echo "PASS: Navigation landmark present" || echo "FAIL: No navigation landmark" + +# 4. Has main landmark +MAIN=$(echo "$RESULT" | jq '[.. | objects | select(.role? == "main")] | length') +[ "$MAIN" -ge 1 ] && echo "PASS: Main landmark present" || echo "FAIL: No main landmark" +``` diff --git a/skills/openbrowser/references/installation.md b/skills/openbrowser/references/installation.md new file mode 100644 index 0000000..622c0ce --- /dev/null +++ b/skills/openbrowser/references/installation.md @@ -0,0 +1,83 @@ +# Installation + +## Prerequisites + +- **Rust nightly** required (deno_core uses `const_type_id` feature) +- Install: `rustup install nightly` + +## Build from Source + +```bash +# Install Rust nightly +rustup install nightly + +git clone https://github.com/JasonHonKL/Openbrowser.git +cd Openbrowser + +# Build with JavaScript support (V8 via deno_core) +cargo +nightly install --path crates/open-cli --features js + +# Or build without JavaScript support +cargo +nightly install --path crates/open-cli +``` + +Note: the repository directory is `Openbrowser`, but the installed binary is `open-browser`. + +## Build Dependencies + +Building open-browser requires several system-level dependencies. The `boring-sys2` crate (BoringSSL bindings) uses cmake and a C/C++ toolchain. The `bindgen` crate requires `libclang`. + +### Common Dependencies + +| Dependency | Purpose | Install (Debian/Ubuntu) | Install (Arch Linux) | Install (macOS) | +|---|---|---|---|---| +| cmake | Build system for BoringSSL | `sudo apt install cmake` | `sudo pacman -S cmake` | `brew install cmake` | +| clang + libclang | Required by bindgen for FFI bindings | `sudo apt install clang libclang-dev` | `sudo pacman -S clang` | `brew install llvm` | +| build-essential | C/C++ compiler and tools | `sudo apt install build-essential` | (included with base-devel) | `xcode-select --install` | +| pkg-config | Library detection | `sudo apt install pkg-config` | `sudo pacman -S pkgconf` | (included with Xcode) | +| libssl-dev | TLS support | `sudo apt install libssl-dev` | (included with openssl) | (included with macOS) | + +### Arch Linux Specific Notes + +On Arch Linux, `bindgen` may fail to find `libclang.so` even after installing `clang`. You need to set the `LIBCLANG_PATH` environment variable: + +```bash +# Add to ~/.bashrc or ~/.zshrc +export LIBCLANG_PATH=/usr/lib + +# Or for fish shell, add to ~/.config/fish/config.fish +set -gx LIBCLANG_PATH /usr/lib +``` + +Without this, you will see an error like: +``` +Unable to find libclang: "couldn't find any valid shared libraries matching: ['libclang.so', ...]" +``` + +The `libclang.so` library is installed to `/usr/lib/libclang.so` by the `clang` package on Arch, but `bindgen` does not search this path by default. + +### Troubleshooting Build Failures + +| Error | Cause | Fix | +|---|---|---| +| `is 'cmake' not installed?` | cmake not on PATH | `sudo pacman -S cmake` or `sudo apt install cmake` | +| `Unable to find libclang` | bindgen cannot find libclang.so | Set `LIBCLANG_PATH=/usr/lib` | +| `failed to run custom build command for boring-sys2` | Missing build dependencies | Install cmake, clang, and C++ compiler | +| `const_type_id` feature error | Not using nightly | Use `cargo +nightly` not `cargo` | + +## Docker + +```bash +docker build -t open-browser . +docker run --rm open-browser navigate https://example.com +``` + +## Verification + +After installation, verify it works: + +```bash +open-browser navigate https://example.com +``` + +You should see a semantic tree output with the page structure. diff --git a/skills/openbrowser/references/knowledge-graph.md b/skills/openbrowser/references/knowledge-graph.md new file mode 100644 index 0000000..579de12 --- /dev/null +++ b/skills/openbrowser/references/knowledge-graph.md @@ -0,0 +1,123 @@ +# Knowledge Graph (Site Mapping) + +Map a site's functional structure into a deterministic state graph. Nodes are view-states (semantic tree hash + resource fingerprint), edges are verified transitions. + +## Command + +```bash +open-browser map [--depth N] [--max-pages N] [--output ] [--no-pagination] [-v] +``` + +| Flag | Default | Description | +|---|---|---| +| `--depth` | 3 | Maximum crawl depth | +| `--max-pages` | 50 | Maximum pages to crawl | +| `--output` | required | Output JSON file path | +| `--no-pagination` | false | Skip pagination discovery | +| `-v` | — | Verbose logging | + +## How It Works + +1. **BFS crawl** — Starting from the root URL, visits pages breadth-first up to `--depth` and `--max-pages` +2. **State fingerprinting** — Each page gets a composite ID: blake3 hash of semantic tree structure (roles + interactivity, not text) + resource URLs + URL path +3. **Deduplication** — Pages with identical fingerprints are merged (same layout, different copy = same state) +4. **Transition discovery** — For each page, discovers: link clicks, hash navigation (`#section`), pagination (`?page=N`, `/page/N`), and optional form submissions +5. **Verification** — Each transition is followed and the target state is confirmed + +## Transition Types + +| Type | Trigger | Example | +|---|---|---| +| `link_click` | Click internal link | `About` | +| `hash_navigation` | Hash/anchor link | `Features` | +| `pagination` | URL-based pagination | `?page=2`, `/page/2`, `?offset=20` | +| `form_submit` | Form submission | `` | + +## Output Structure + +```json +{ + "root_url": "https://example.com", + "built_at": "2026-04-02T14:30:00Z", + "stats": { + "total_states": 12, + "total_transitions": 23, + "verified_transitions": 21, + "max_depth_reached": 3, + "pages_crawled": 12, + "crawl_duration_ms": 5420 + }, + "states": { + "": { + "url": "https://example.com/", + "title": "Example Corp", + "fingerprint": { + "url_path": "/", + "tree_hash": "...", + "resource_set_hash": "..." + }, + "semantic_tree": { ... }, + "resource_urls": ["..."] + } + }, + "transitions": [ + { + "from": "", + "to": "", + "trigger": { "type": "link_click", "url": "/about", "label": "About Us" }, + "verified": true, + "outcome": { "status": 200, "final_url": "...", "matched_prediction": true } + } + ] +} +``` + +## Usage Examples + +```bash +# Standard site map +open-browser map https://example.com --output kg.json + +# Shallow crawl (homepage only + direct links) +open-browser map https://example.com --depth 1 --output kg.json + +# Deep crawl with high page limit +open-browser map https://example.com --depth 5 --max-pages 200 --output kg.json + +# Skip pagination (only follow direct links) +open-browser map https://example.com --output kg.json --no-pagination + +# Verbose logging for debugging +open-browser map https://example.com -v --output kg.json +``` + +## Testing Use Cases + +### 1. Coverage Verification +Map the site, then assert all expected routes were discovered: +```bash +open-browser map https://example.com --depth 2 --output kg.json +EXPECTED=("/" "/about" "/products" "/contact") +for PAGE in "${EXPECTED[@]}"; do + FOUND=$(jq -r --arg p "$PAGE" '.states | to_entries[] | select(.value.url | endswith($p)) | .key' kg.json) + [ -n "$FOUND" ] && echo "PASS: $PAGE found" || echo "FAIL: $PAGE not found" +done +``` + +### 2. Dead Link Detection +```bash +open-browser map https://example.com --depth 3 --output kg.json +# Find unverified transitions (potential dead links) +jq '[.transitions[] | select(.verified == false)] | length' kg.json +``` + +### 3. Duplicate State Detection +```bash +# States with same tree_hash have identical layouts +jq '.states | to_entries | group_by(.value.fingerprint.tree_hash) | map(select(length > 1)) | length' kg.json +``` + +### 4. Transition Coverage Report +```bash +jq -r '.stats | "States: \(.total_states), Transitions: \(.total_transitions), Verified: \(.verified_transitions), Pages: \(.pages_crawled), Duration: \(.crawl_duration_ms)ms"' kg.json +``` diff --git a/skills/openbrowser/references/programmatic-usage.md b/skills/openbrowser/references/programmatic-usage.md new file mode 100644 index 0000000..634ba52 --- /dev/null +++ b/skills/openbrowser/references/programmatic-usage.md @@ -0,0 +1,95 @@ +# Programmatic Usage (Rust API) + +The `Browser` type from `open-core` unifies navigation, interaction, and tab management into a single API. + +## Setup + +```rust +use open_core::Browser; +use open_core::BrowserConfig; + +let mut browser = Browser::new(BrowserConfig::default()); +``` + +## Navigation + +```rust +// Navigate to URL (creates a tab automatically) +let tab = browser.navigate("https://example.com").await?; +``` + +## Interaction + +```rust +// Click using CSS selector — updates tab automatically if navigation occurs +let result = browser.click("a").await?; + +// Click using element ID — easier for AI agents +let result = browser.click_by_id(1).await?; // Click element with ID [#1] + +// Type using element ID +let result = browser.type_by_id(3, "search query").await?; // Type into element [#3] + +// Type using CSS selector +browser.type_text("input[name='q']", "search query")?; + +// Submit a form (FormState accumulates field values) +let state = browser.current_form_state()?; +browser.submit("form", &state).await?; +``` + +## Tab Management + +```rust +// Create a new tab +let id = browser.create_tab("https://example.com/page2"); + +// Switch to tab +browser.switch_to(id).await?; + +// Go back in history +browser.go_back().await?; +``` + +## Accessing State + +```rust +// Get current page +let page = browser.current_page().unwrap(); + +// Get semantic tree +let tree = page.semantic_tree(); + +// Find element by ID +if let Some(element) = page.find_by_element_id(1) { + println!("Element selector: {}", element.selector); +} +``` + +## Architecture + +The `Browser` type owns: +- HTTP client (reqwest) +- Tab state (multiple tabs with independent history) +- Session persistence (cookies, headers, localStorage) +- Optional JavaScript execution (deno_core) + +Internal pipeline: fetch via reqwest -> parse HTML with scraper -> build semantic tree with ARIA roles -> detect interactive elements and assign IDs. + +PDF URLs detected by content-type (`application/pdf`) are automatically routed to PDF extraction (pdf-extract/lopdf) instead of HTML parsing. + +## Crate Dependencies + +``` +open-browser +├── crates/open-core Browser type, HTML parsing, semantic tree, interaction, tabs +├── crates/open-debug Network debugger, request recording, subresource discovery +├── crates/open-cdp CDP WebSocket server (14 domains) +├── crates/open-kg Knowledge Graph, BFS crawler, state fingerprinting +└── crates/open-cli CLI binary +``` + +When using programmatically, you primarily interact with `open-core`. The other crates provide additional functionality: +- `open-debug` for network logging +- `open-cdp` for CDP server integration +- `open-kg` for site mapping diff --git a/skills/openbrowser/references/registry.json b/skills/openbrowser/references/registry.json new file mode 100644 index 0000000..019c833 --- /dev/null +++ b/skills/openbrowser/references/registry.json @@ -0,0 +1,52 @@ +{ + "name": "openbrowser", + "description": "Headless browser for AI agents - semantic tree output, page interaction, site mapping", + "author": "Chris Engelhard ", + "references": [ + { + "file": "installation.md", + "title": "Installation", + "description": "Build from source, system dependencies, Arch Linux extras, Docker, troubleshooting" + }, + { + "file": "cli-reference.md", + "title": "CLI Reference", + "description": "All subcommands, flags, output formats, and usage examples" + }, + { + "file": "testing-patterns.md", + "title": "Testing Patterns", + "description": "Playwright to open-browser translation table, assertion recipes, workflow templates" + }, + { + "file": "repl-and-cdp.md", + "title": "REPL and CDP Server", + "description": "Interactive REPL commands, CDP WebSocket server, when to use each" + }, + { + "file": "knowledge-graph.md", + "title": "Knowledge Graph", + "description": "Site mapping, BFS crawl, state fingerprinting, transition types, coverage testing" + }, + { + "file": "semantic-roles.md", + "title": "Semantic Roles", + "description": "HTML element to ARIA role mapping, action types, element ID system" + }, + { + "file": "programmatic-usage.md", + "title": "Programmatic Usage", + "description": "Rust API, Browser type, navigation, interaction, tab management" + }, + { + "file": "examples.md", + "title": "Examples", + "description": "End-to-end examples: navigation, forms, REPL, CDP, PDF, smoke tests, SEO audit" + }, + { + "file": "troubleshooting.md", + "title": "Troubleshooting", + "description": "Build issues, runtime errors, element IDs, form submission, JS execution, performance" + } + ] +} diff --git a/skills/openbrowser/references/repl-and-cdp.md b/skills/openbrowser/references/repl-and-cdp.md new file mode 100644 index 0000000..686f2d7 --- /dev/null +++ b/skills/openbrowser/references/repl-and-cdp.md @@ -0,0 +1,149 @@ +# REPL and CDP Server + +## Interactive REPL + +Start a persistent interactive session where browser state (tabs, pages, cookies, history) is preserved across commands. + +```bash +# Start REPL with defaults +open-browser repl + +# Enable JS execution by default +open-browser repl --js + +# Set default output format and JS wait time +open-browser repl --format json --wait-ms 5000 +``` + +The REPL prompt shows current URL context: +``` +open> visit https://example.com + document [role: document] + └── region [role: region] + ├── heading (h1) "Example Domain" + └── link "Learn more" → https://iana.org/domains/example + 0 landmarks, 1 links, 1 headings, 1 actions + +open [https://example.com]> tab open https://httpbin.org +Opened tab 2: httpbin.org + +open [https://httpbin.org]> tab list +Tabs (2 total): + * [2] Ready — httpbin.org — https://httpbin.org + [1] Ready — Example Domain — https://example.com + +open [https://httpbin.org]> tab switch 1 +Switched to tab 1: https://example.com + +open [https://example.com]> click 'a' +Navigated to: https://iana.org/domains/example + +open [https://iana.org/domains/example]> back +open [https://example.com]> exit +Bye. +``` + +### REPL Commands + +| Command | Description | +|---|---| +| `visit ` / `open ` | Navigate to URL | +| `click ` | Click an element using CSS selector | +| `click #` | Click an element by its ID (e.g., `click #1`) | +| `type ` | Type into a field using CSS selector | +| `type # ` | Type into a field by ID (e.g., `type #3 hello`) | +| `submit [name=value...]` | Submit a form | +| `scroll [down\|up\|to-top\|to-bottom]` | Scroll the page | +| `wait [timeout_ms]` | Wait for element | +| `back` / `forward` | Navigate history | +| `reload` | Reload current page | +| `tab list` | List all open tabs | +| `tab open ` | Open new tab | +| `tab switch ` | Switch to tab by ID | +| `tab close [id]` | Close tab | +| `tab info` | Show active tab info | +| `js [on\|off]` | Toggle JS execution | +| `format md\|tree\|json` | Change output format | +| `wait-ms ` | Set JS wait time | +| `help` | Show available commands | +| `exit` / `quit` | Exit REPL | + +### When to Use REPL vs CLI + +| Scenario | Use REPL | Use CLI | +|---|---|---| +| Login flow with session persistence | X | | +| Multi-step form interaction | X | | +| Quick single-page check | | X | +| CI/CD test automation | | X (or CDP) | +| Tab management across steps | X | | +| Scripting in bash/python | | X | + +## CDP Server + +Start a Chrome DevTools Protocol WebSocket server for automation. + +```bash +# Start on default host/port +open-browser serve + +# Custom host and port +open-browser serve --host 0.0.0.0 --port 9222 + +# With inactivity timeout +open-browser serve --timeout 60 +``` + +### CDP Flags + +| Flag | Default | Description | +|---|---|---| +| `--host` | localhost | Bind address | +| `--port` | default | Port number | +| `--timeout` | none | Inactivity timeout in seconds | + +### Implemented CDP Domains + +| Domain | Purpose | +|---|---| +| Browser | Browser-level operations | +| Target | Target/tab management | +| Page | Page navigation, lifecycle | +| Runtime | JavaScript execution | +| DOM | DOM tree access | +| Network | Network interception, monitoring | +| Emulation | Device emulation | +| Input | Input event dispatch | +| CSS | CSS access | +| Log | Log entry access | +| Console | Console API | +| Security | Security state | +| Performance | Performance metrics | +| Open | Custom extensions | + +### CDP Usage Pattern + +```bash +# 1. Start the server +open-browser serve --port 9222 & + +# 2. Connect via WebSocket +# Using wscat: wscat -c ws://localhost:9222 +# Using python: websockets.connect('ws://localhost:9222') +# Using node: new WebSocket('ws://localhost:9222') + +# 3. Send CDP commands +{"id": 1, "method": "Page.navigate", "params": {"url": "https://example.com"}} +{"id": 2, "method": "Runtime.evaluate", "params": {"expression": "document.title"}} +``` + +### When to Use CDP vs REPL vs CLI + +| Scenario | Use CDP | Use REPL | Use CLI | +|---|---|---|---| +| External automation (Python/Node) | X | | | +| Real-time event streaming | X | | | +| Persistent browser with programmatic control | X | | | +| Quick interactive debugging | | X | | +| Bash script integration | | | X | +| CI/CD pipeline tests | X | | X | diff --git a/skills/openbrowser/references/semantic-roles.md b/skills/openbrowser/references/semantic-roles.md new file mode 100644 index 0000000..5d5424a --- /dev/null +++ b/skills/openbrowser/references/semantic-roles.md @@ -0,0 +1,94 @@ +# Semantic Roles + +open-browser maps HTML elements to ARIA roles and actions. This is the complete mapping reference. + +## Element-to-Role Mapping + +| HTML Element | ARIA Role | Action Type | Notes | +|---|---|---|---| +| `` / `` | `document` | — | Root node | +| `
` | `banner` | — | Page header | +| `