Repository navigation
Local only - #167
Open
BHspeed wants to merge 23 commits into
Open
Local only#167BHspeed wants to merge 23 commits into
BHspeed wants to merge 23 commits into
Conversation
Replace CopilotKit Intelligence with the runtime's SSE mode and a SQLite-backed agent runner so conversations persist in the workspace database. Default the model endpoint to Ollama, disable Parallel web search and CopilotKit telemetry by default, and drop cloud-only Automatic Learning and Slack wiring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
localtools/server.py is an MCP server exposing file management, local vision (Ollama), YOLO object detection and mouse/keyboard control, with Windows confirmation dialogs for risky actions. OpenDots loads MCP servers from mcp.servers.json and gives their tools to every Dot. Raise run timeout and output limit for local reasoning models, and add start-local.ps1 and docs/LOCAL.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…riggers - Dots get a model (or role: @Planner, @default, @Vision) and granted tool groups; local MCP tools are filtered per Dot. - Delegation tools let a Dot hand tasks to specialists in linked sub-conversations (depth-limited); a starter team (Planner, Files, Vision, Operator, Researcher) is seeded once. - Long-term memory indexes conversations and folders with a local embedding model and adds search_memory / index_folder tools. - Tasks support daily and new-file triggers; runs carry trigger input. - DuckDuckGo-only web_search and fetch_page in localtools (opt-in). - Chat attachment upload endpoint; longer leases/timeouts for local models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nking switch - Dot dialog: model/role picker, tool group permissions, think toggle. - Settings: default, planner and vision model roles. - Schedule dialog: interval, daily-at-time and new-file triggers. - Chat: attach or drop files (saved locally) for Dots to open. - Per-Dot thinking switch (reasoning_effort none) for faster simple work. - localtools: treat zero/empty optional arguments as defaults. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dots could not open websites or programs: open_path only accepts files under the allowed roots, and the MCP SDK masked every exception as 'Error executing tool', so models invented causes (e.g. 'no browsers installed'). Tools now re-raise failures as readable ToolErrors, and new PC-control tools open URLs in the default or a named browser and launch installed apps (including Store apps) by Start-menu name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
localtools/browser.py drives a visible Chrome/Edge window with its own persistent profile. Pages come back as text plus numbered interactive elements, so models click/type by [ref] instead of pixels. Reading, scrolling and links run freely; typing (except search boxes) and submit/buy/send/delete/sign-in buttons ask the owner first. New 'Web browser' tool group, granted once to Dot, Operator and Researcher. open_url now uses the shell when the requested browser is the default (e.g. the DuckDuckGo Store app), so the URL opens directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Dots' Chrome exposes a localhost-only DevTools port. OpenDots attaches with Playwright, streams the active tab into a new Browser tab in the side panel (CDP screencast over SSE), and relays the owner's navigation, clicks, scrolling and typing while they have taken control. Takeover pauses agent browser actions (shared flag file); elements are outlined before the agent clicks or types. Fixed window size keeps the view readable on ultrawide screens. Disable the CopilotKit dev inspector overlay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Track the most recently opened/navigated tab instead of the first 'visible' one (separate windows all report visible), prefer non-blank tabs, and reuse a blank tab when an agent asks for a new one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Playwright-launched Chrome carries --enable-automation and --no-sandbox, which Google sign-in rejects as insecure. Launch Chrome as a regular process (sandbox on, no automation flags) with the local DevTools port and attach over CDP; navigator.webdriver is false. Add a 'Sign in normally' fallback that reopens the same profile without the DevTools port. Trading and money-movement buttons (buy/sell/higher/lower/call/ put/deposit/withdraw/trade...) always require the owner's approval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Runs were only saved on completion, so a server restart (dev watch mode restarted on any project file change) erased the owner's message and the work in progress. Checkpoint in-flight runs to SQLite and close interrupted ones with a visible note on rehydrate. - Dev watch restarts only on src changes; start-local runs without auto-restart. - Prompt browser Dots with the shared browser's current page and tell every Dot to try its tools (snapshot/read, screenshot+describe for charts and canvases) before saying it cannot see or access something. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Ollama context 16K: 32K overflowed the 16 GB GPU into shared memory (qwen3:14b went from 8 to 36 tokens/s). - Image tools use the configured vision model (gemma4-e4b); hide Ollama's internal engine copies from model pickers. - Browser: element refs stay stable across snapshots and actions, pointer-cursor div/span buttons are detected, in-view elements come first, icon buttons get alt-text names, and browser_find locates elements by text. Dots are told to use refs, never screen-position mouse clicks, on web pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dots with the new Self-improvement group save procedures that worked (save_skill, per Dot or shared), recall the most relevant ones each turn via local embeddings, and can propose new role instructions, which only apply after the owner clicks Apply in the Dot's settings. Skills never change tools, models or access. Dot settings show learned skills (view, forget) and any pending proposal. Existing Dots get the group once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
'Start OpenDots.cmd' runs start-local.ps1 (double-clicking a .ps1 opens Notepad). The script opens the browser only once the app answers, just opens the page if OpenDots is already running, and keeps the window open with the error if startup fails instead of closing silently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Developer Dot (gpt-oss-20b) edits a separate git worktree on its own branch with dev_* tools (overview, read, search, exact-text edit, write), runs type-check + lint + tests there, and submits a review. The new Changes tab shows each change's summary, check results and diff with Apply (merge) / Reject (delete branch). Files that govern permissions, approvals, tools or secrets are protected from Developer edits, and paths cannot leave the code tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A forced 'git worktree remove' follows the node_modules junction on Windows and deleted the main checkout's packages. Unlink the junction first, refuse to remove if node_modules is not a link, and prune afterwards. Test that discarding keeps the shared packages intact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New browser_visual_click / browser_visual_type tools take a plain
description ('the first email in the inbox'), screenshot the page with
numbered boxes over clickable things, let the local vision model pick
one (falling back to a lettered grid), then glide a visible cursor
there and click/type with real mouse and keyboard events. Approvals are
unchanged. The element list now also covers rows, list items and
jsaction buttons used by web apps, and Dots switch to the visual tools
after two failed searches.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Numbered marks picked by a small model hit the wrong element on dense pages (4/8 on a mock inbox). Ask a Qwen-VL grounding model (LOCALTOOLS_GROUNDING_MODEL, default qwen3-vl:8b) to point at the target in 0-1000 coordinates and click there (6/7 through the real tool), keeping marks and the lettered grid as fallbacks. Approval checks use the element actually under the click point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The non-thinking instruct model points correctly 8/8 on the dense mock inbox at ~3 s per click (vs 6/7 and 20-270 s for the thinking variant). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Some local models stream tool arguments twice ('{...}{...}'). Once that
is saved in a conversation, Ollama rejects every later request with
'400 invalid tool call arguments'. Before each turn, keep the first
complete JSON value of every assistant tool call (or '{}'), which also
unblocks conversations that already broke.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A long chat (58 messages, ~119K chars) filled the 16K context, leaving 32 tokens for the answer, so the reply came back empty. Each turn now keeps recent messages in full, shortens older tool output, then drops the oldest messages to stay within budget (context minus a reserve for instructions, tools and the answer), always keeping the latest request and never starting on an orphaned tool result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pocketOp trading added.
Records upstream main as merged so the local-only -> main PR has no conflicts. The local-only tree is kept unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.