Skip to content

README.md

Latest commit

 

History

History
105 lines (81 loc) · 4.76 KB

README.md

File metadata and controls

105 lines (81 loc) · 4.76 KB

Neural Computers (The Data Pipeline)

Neural Computer teaser diagram

📘 Blog   ·   🎬 Demo Gallery   ·   📄 Paper

News

  • 2026 April: First open-source release of Neural Computers, starting with the data pipeline for CLI and GUI trajectory generation.

Abstract

Neural Computers (NCs) are neural networks that unify computation, memory, and I/O in a single latent runtime state. The long-term goal is the Completely Neural Computer (CNC): a general-purpose neural computer that challenges the layered architecture of conventional computers. As an initial step, this project studies whether executable dynamics can be learned solely from collected I/O traces, without access to instrumented program state, using CLI and GUI trajectories as training and evaluation data.

Quick Start

Environment Python 3.9+ · Docker · asciinema · agg · ffmpeg

1. CLI / Asciinema

python3 main.py cligen asciinema record --command "python3 --version"
python3 main.py cligen asciinema cast-to-mp4 workspace/cligen_general/casts

2. CLI / VHS

python3 main.py cligen vhs build-image
python3 main.py cligen vhs generate-basic --count 10 --prefix demo
python3 main.py cligen vhs make-manifest
python3 main.py cligen vhs run-manifest

3. GUI / Synthetic

pip install -r engine/gui/synthetic_data_collection/requirements.txt
python3 main.py guiworld build-image
python3 main.py guiworld synthetic --count 1 --max-workers 1

Runtime Generations from Neural Computers

Using training data generated by the code released in this repository.

CLIGen (general / asciinema)

CLIGen general demo 1 CLIGen general demo 2 CLIGen general demo 3 CLIGen general demo 4 CLIGen general demo 5 CLIGen general demo 6

CLIGen (clean / vhs)

CLIGen clean demo 1 CLIGen clean demo 4 CLIGen clean demo 6 CLIGen clean demo 9 CLIGen clean demo 11 CLIGen clean demo 14

GUIWorld (GUI)

Alternating: Conventional Computer / Neural Computer

GUIWorld conventional computer demo 1 GUIWorld neural computer demo 1 GUIWorld conventional computer demo 10 GUIWorld neural computer demo 10 GUIWorld conventional computer demo 19 GUIWorld neural computer demo 19

Acknowledge

Our models (not released here) are built from Wan2.1 and Matrix-Game-2. The data engine for CLIGen (General) is built from Asciinema, the data engine for CLIGen (Clean) is from VHS, the data engine for GUIWorld (Random) is modified directly from Neural-OS, and the data engine for GUIWorld (CUA) is built according to Claude CUA.

Data Engine Source
CLIGen (General) Asciinema
CLIGen (Clean) VHS
GUIWorld (Random) Neural-OS
GUIWorld (CUA) Claude CUA