Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f15c7d9
Shared Hardware Resource Pool for Compute Environments (#1390)
alexcos20 Jun 20, 2026
2e892d2
Merge branch 'main' into next-4
alexcos20 Jun 20, 2026
4f2ac60
Merge branch 'main' into next-4
alexcos20 Jul 9, 2026
f207e5a
Add Services (#1402)
alexcos20 Jul 9, 2026
0e007d8
bump contract to 2.9.0 (#1410)
alexcos20 Jul 9, 2026
ea51b7c
cross-node auth tokens (#1404)
giurgiur99 Jul 9, 2026
6cb3ead
Update src/components/Auth/index.ts
alexcos20 Jul 9, 2026
d82a5b3
Update src/components/Auth/index.ts
alexcos20 Jul 9, 2026
1733e6d
Merge branch 'main' into next-4
alexcos20 Jul 10, 2026
f5a4353
fix network recreate (#1416)
alexcos20 Jul 14, 2026
7bcf7f0
add cpuList (#1417)
alexcos20 Jul 14, 2026
78fbc3b
services updates (#1422)
alexcos20 Jul 17, 2026
90fb2a9
remove axios (#1424)
alexcos20 Jul 17, 2026
c765ec6
Feature/use native sqlite (#1423)
alexcos20 Jul 17, 2026
38507db
resources constrains (#1425)
alexcos20 Jul 17, 2026
85fa61b
New updatedAt record for services (#1428)
giurgiur99 Jul 23, 2026
c483ea1
allow docker specs on service restart (#1429)
alexcos20 Jul 24, 2026
cfee6c3
add more templates
dnsi0 Jul 24, 2026
f25cf82
fix stable diffusion template
dnsi0 Jul 28, 2026
3dc2179
Merge pull request #1431 from oceanprotocol/add-templates
dnsi0 Jul 29, 2026
fb4085b
fix extend service (#1435)
alexcos20 Jul 30, 2026
879f495
new arg "release" for stop service (#1434)
giurgiur99 Jul 30, 2026
9b0a689
fix bugs
alexcos20 Aug 5, 2026
61285cb
fix comment
alexcos20 Aug 5, 2026
8d98cb5
Merge pull request #1443 from oceanprotocol/bug/fix_negative_runMaxWa…
bogdanfazakas Aug 11, 2026
e20dc15
add database retry
dnsi0 Aug 13, 2026
c34ad6f
add envs, validations, tests
dnsi0 Aug 13, 2026
d85d002
fix return type
dnsi0 Aug 13, 2026
b764761
remove timeout
dnsi0 Aug 14, 2026
3559492
fix review
dnsi0 Aug 14, 2026
682a59d
fix type
dnsi0 Aug 14, 2026
66ee749
Merge branch 'main' into next-4
alexcos20 Aug 18, 2026
9825ba3
Merge pull request #1447 from oceanprotocol/add-database-retry
dnsi0 Aug 19, 2026
d5de4fe
Authenticate `PolicyServerPassthrough` / `initializePSVerification`, …
alexcos20 Aug 20, 2026
c583736
Feature/docker stats (#1436)
alexcos20 Aug 20, 2026
787cf3d
remove serviceEndpoints (#1448)
alexcos20 Aug 20, 2026
32bf96a
use custom ocean.js/cli
alexcos20 Aug 20, 2026
730a784
fix merge errors
alexcos20 Aug 20, 2026
d52abe7
Dependency and toolchain refresh (#1453)
alexcos20 Aug 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ databases
.env.*
.git
.github
docs
docs/*
!docs/serviceTemplates
src/test
*.md
*.log
Expand Down
15 changes: 8 additions & 7 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,14 @@ export P2P_BOOTSTRAP_NODES=
export P2P_FILTER_ANNOUNCED_ADDRESSES=

## compute
# Each environment defines its own resources (CPU, RAM, disk, GPUs) with full configuration.
# CPU, RAM, and disk are per-env exclusive: inUse tracked only within the environment where the job runs.
# A global check ensures the aggregate usage across all environments does not exceed physical capacity.
# GPUs are shared-exclusive: if a job on envA uses gpu0, it shows as in-use on envB too.
# CPU cores are automatically partitioned across environments based on each env's cpu.total.
# CPU and RAM defaults are auto-detected from the system when not configured.
# export DOCKER_COMPUTE_ENVIRONMENTS='[{"socketPath":"/var/run/docker.sock","environments":[{"id":"envA","storageExpiry":604800,"maxJobDuration":3600,"minJobDuration":60,"resources":[{"id":"cpu","total":4,"max":4,"min":1,"type":"cpu"},{"id":"ram","total":16,"max":16,"min":1,"type":"ram"},{"id":"disk","total":500,"max":500,"min":10,"type":"disk"},{"id":"gpu0","total":1,"max":1,"min":0,"type":"gpu","init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":["0"],"Capabilities":[["gpu"]]}}}],"fees":{"1":[{"feeToken":"0x123","prices":[{"id":"cpu","price":1},{"id":"ram","price":0.1},{"id":"disk","price":0.01},{"id":"gpu0","price":5}]}]}}]}]'
# Resources are defined at the Docker-connection level (socketPath) and shared across all environments.
# cpu, ram, and disk are auto-detected from the host — omit them to use all available capacity,
# or include them to cap/reserve (e.g. limit an 8-core host to 6 cores for compute).
# GPUs and other hardware go in the connection-level "resources" array with kind:"discrete".
# Each environment references pool resources by id using lightweight refs {id, total?, min?, max?}.
# Dual-gate tracking for fungible resources: per-env ceiling (Gate 1) + engine-wide pool (Gate 2).
# Discrete resources (GPUs) are tracked globally — a GPU in use on envA shows as in-use on envB too.
# export DOCKER_COMPUTE_ENVIRONMENTS='[{"socketPath":"/var/run/docker.sock","resources":[{"id":"disk","total":500},{"id":"gpu0","kind":"discrete","type":"gpu","total":1,"description":"NVIDIA A100","platform":"nvidia","driverVersion":"570.195.03","init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":["GPU-uuid-a"],"Capabilities":[["gpu"]]}}}],"environments":[{"id":"envA","storageExpiry":604800,"maxJobDuration":3600,"minJobDuration":60,"resources":[{"id":"cpu"},{"id":"ram"},{"id":"disk","max":500},{"id":"gpu0"}],"fees":{"1":[{"feeToken":"0x123","prices":[{"id":"cpu","price":1},{"id":"ram","price":0.1},{"id":"disk","price":0.01},{"id":"gpu0","price":5}]}]}}]}]'
export DOCKER_COMPUTE_ENVIRONMENTS=


4 changes: 0 additions & 4 deletions .eslintignore

This file was deleted.

30 changes: 0 additions & 30 deletions .eslintrc

This file was deleted.

19 changes: 13 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 'v22.15.0'
node-version: '22.23.1'
- name: Cache node_modules
uses: actions/cache@v3
env:
Expand All @@ -42,7 +42,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
node: ['18.20.4', 'v20.19.0', 'v22.15.0']
node: ['22.23.1']

steps:
- uses: actions/checkout@v4
Expand All @@ -66,7 +66,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 'v22.15.0'
node-version: '22.23.1'
- name: Cache node_modules
uses: actions/cache@v3
env:
Expand Down Expand Up @@ -102,7 +102,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 'v22.15.0'
node-version: '22.23.1'
- name: Cache node_modules
uses: actions/cache@v3
env:
Expand All @@ -118,6 +118,7 @@ jobs:
with:
repository: 'oceanprotocol/barge'
path: 'barge'
ref: '43cfdfd21154a2bae00770b779e7c39390ff5043'
- name: Login to Docker Hub
if: ${{ env.DOCKERHUB_PASSWORDNONO && env.DOCKERHUB_USERNAMENONO }}
run: |
Expand All @@ -129,6 +130,8 @@ jobs:
working-directory: ${{ github.workspace }}/barge
run: |
bash -x start_ocean.sh --no-node --with-typesense 2>&1 > start_ocean.log &
env:
CONTRACTS_VERSION: '2.9.0'
- run: npm ci
- run: npm run build
- run: docker image ls
Expand Down Expand Up @@ -194,7 +197,7 @@ jobs:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 'v22.15.0'
node-version: '22.23.1'

- name: Cache node_modules
uses: actions/cache@v3
Expand All @@ -213,6 +216,7 @@ jobs:
with:
repository: 'oceanprotocol/barge'
path: 'barge'
ref: feature/node-v4
- name: Login to Docker Hub
if: ${{ env.DOCKERHUB_PASSWORD && env.DOCKERHUB_USERNAME }}
run: |
Expand All @@ -226,6 +230,8 @@ jobs:
working-directory: ${{ github.workspace }}/barge
run: |
bash -x start_ocean.sh --no-node --with-typesense 2>&1 > start_ocean.log &
env:
CONTRACTS_VERSION: '2.9.0'
- run: docker image ls
- name: Delete default runner images
run: |
Expand All @@ -235,12 +241,13 @@ jobs:
with:
repository: 'oceanprotocol/ocean-cli'
path: 'ocean-cli'
ref: deps/remove_web3_and_bump_deps
- name: Checkout Ocean-js
uses: actions/checkout@v4
with:
repository: 'oceanprotocol/ocean.js'
path: 'ocean.js'
ref: main
ref: feature/update_web3_deps
- name: Build ocean-js
working-directory: ${{ github.workspace }}/ocean.js
run: |
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
pull_request:
branches:
- 'main'
- 'next-4'

env:
DOCKERHUB_IMAGE: ${{ 'oceanprotocol/ocean-node' }}
Expand Down
2 changes: 1 addition & 1 deletion .nvmrc
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
22
22.23.1

58 changes: 40 additions & 18 deletions CLAUDE.md
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.


Ocean Node is the all-in-one backend for the Ocean Protocol stack. A single Node process
replaces three legacy components: **Provider** (data access / encryption / compute),
**Aquarius** (metadata cache) and the **subgraph** (on-chain event indexing). It is a
Expand All @@ -13,19 +12,32 @@ of which dispatch to the same set of command handlers.

## 1. Environment & tooling prerequisites

- **Node.js 22 is required** (`.nvmrc` pins `22`). Always run `nvm use` (or
`source ~/.nvm/nvm.sh && nvm use`) before any `npm`, build, or test command. The wrong
Node version fails with errors like `Unexpected token 'with'` or missing `GLIBC_2.38`.
- **Node.js ≥ 22.13 is required** (`.nvmrc` pins `22.23.1`, matching the Dockerfile and CI;
`package.json` `engines` requires `>=22.13.0`). The Dockerfile additionally pins each base
image by digest, so bumping the version means updating the `sha256:` alongside the tag —
resolve it with `docker buildx imagetools inspect node:<version>-trixie`, not from the tag
alone. Always run `nvm use` (or `source ~/.nvm/nvm.sh && nvm use`) before
any `npm`, build, or test command. The wrong Node version fails with errors like
`Unexpected token 'with'`, missing `GLIBC_2.38`, or — since the SQLite layer uses the
built-in `node:sqlite` module — `ERR_UNKNOWN_BUILTIN_MODULE: node:sqlite` on Node < 22.13.
This is enforced by `.cursor/rules/tests-nvm.mdc` and the in-repo `CLAUDE.md`.
- If `sqlite3` native bindings break after switching to Node 22, rebuild from source:
`npm_config_build_from_source=true npm rebuild sqlite3`.
- **`postinstall` runs `scripts/fix-libp2p-http-utils.js`** — a patch applied to a libp2p
dependency. Expect it to run on every `npm install`; don't remove it.
- **There is no `postinstall` step.** `npm install` is plain. (Historically a `postinstall`
ran `scripts/fix-libp2p-http-utils.js` to default a missing URL port to 443/80 in
`@libp2p/http-utils`; upstream shipped that fix in `2.0.3`, so both the hook and the
script were removed.)
- **Docker + docker-compose** are needed for the metadata database (Typesense or
Elasticsearch) and for C2D (Compute-to-Data) via the local Docker socket.
- TypeScript config: ESM (`module: esnext`, `target: ES2022`, `moduleResolution: node`),
`experimentalDecorators` + `emitDecoratorMetadata` enabled, `rootDir: ./src`,
`outDir: ./dist`. All local imports use the `.js` extension (compiled ESM convention).
- TypeScript config: **TypeScript 6**, ESM (`module: esnext`, `target: ES2022`,
`moduleResolution: node`), `experimentalDecorators` + `emitDecoratorMetadata` enabled,
`rootDir: ./src`, `outDir: ./dist`. Most local imports use the `.js` extension
(compiled ESM convention), though ~31 relative imports still omit it.
- `strict` is **on**, with two documented exceptions: `strictNullChecks: false` and
`useUnknownInCatchVariables: false`. Turning either on is a real project (~1324 and
~389 errors); do it subsystem by subsystem, not in passing.
- Two things must be done before TypeScript 7: `moduleResolution` has to move off
`node10` (`ignoreDeprecations: "6.0"` only defers it, and `nodenext` currently costs
~123 errors), and `typescript-eslint` has to support TS 7 — its peer range is
`>=4.8.4 <6.1.0`, so it pins us below 7 today.

### Only-mandatory config: `PRIVATE_KEY`

Expand Down Expand Up @@ -64,19 +76,27 @@ The Dockerfile uses the identical CMD. The compiled entry point is `dist/index.j
### Lint / format

```bash
npm run lint # eslint (.ts,.tsx) + type-check
npm run lint # eslint + type-check
npm run lint:fix # eslint --fix
npm run format # prettier --write '**/*.{js,jsx,ts,tsx}'
```

ESLint extends `oceanprotocol` + `prettier/recommended`. Notable rules: `require-await`
is an **error**, `no-unused-vars` is an **error**, empty catch blocks are allowed. Prettier:
ESLint 10 with flat config in `eslint.config.js` (there is no `.eslintrc`/`.eslintignore`).
`eslint-config-oceanprotocol` is **not** used — it is pinned to eslint ^8 and cannot follow
eslint to flat config, so the preset is composed in-repo from `@eslint/js`,
`typescript-eslint`, `eslint-plugin-security`, `eslint-plugin-promise` and
`eslint-plugin-prettier`. Only `**/*.ts` is linted; `.js` is ignored as build output.
Notable rules: `require-await` is an **error**, `no-unused-vars` is an **error** (with
`args: 'none'`, `caughtErrors: 'none'`), empty catch blocks are allowed. `@typescript-eslint`'s
`recommended` set is deliberately *not* extended, and a few rules that eslint 9/10 and the
newer plugins added are switched off — see the comments in `eslint.config.js`. Prettier:
no semicolons, single quotes, `printWidth: 90`, no trailing commas, 2-space tabs.

### Tests (important build quirk)

**Tests run against compiled JS in `dist/test/`, not the TypeScript source.** The
`test:*` scripts all call `npm run build-tests` first, which:

- compiles `src/` (incl. `src/test`) into `dist/`,
- copies `src/test/.env.test` and `.env.test2` into `dist/test`,
- copies `src/test/config.json` to `$HOME/config.json`.
Expand Down Expand Up @@ -189,7 +209,7 @@ the command, looks up the handler by `task.command`, and calls `handler.handle(t
Two front doors, one dispatcher:

- **HTTP `POST /directCommand`** (`src/components/httpRoutes/commands.ts`): validates the
body, then decides *local vs remote*. If the command targets this node (or no P2P), it
body, then decides _local vs remote_. If the command targets this node (or no P2P), it
calls `oceanNode.handleDirectProtocolCommand(...)`. If it targets another peer and P2P is
enabled, it forwards via `oceanNode.getP2PNode().sendTo(node, msg, multiAddrs)`. Responses
are streamed back to the client (binary or text).
Expand Down Expand Up @@ -225,6 +245,7 @@ register it in the `CoreHandlersRegistry` constructor; add param validation; opt
REST route in `src/components/httpRoutes/` and mount it in `httpRoutes/index.ts`.

Handler source is grouped under `src/components/core/`:

- `handler/` — general handlers (ddo, download, encrypt, fees, nonce, query, status, p2p,
auth, accessList, escrow, fileInfo, persistentStorage, policyServer, getJobs).
- `compute/` — C2D command handlers: `initialize`, `startCompute` (paid), `freeStartCompute`,
Expand Down Expand Up @@ -263,13 +284,14 @@ Handler source is grouped under `src/components/core/`:
`getComputeResult` (+ `getComputeStreamableLogs`) → `stopCompute`. Paid compute settles via
the `Escrow` component; `serviceResourceMatching.ts` maps requested cpu/ram/disk/gpu against
environment pools (dual-gate: per-env ceiling + engine-wide pool; GPUs tracked globally).
See `docs/compute-pricing.md`, `docs/GPU.md`.
See `docs/compute.md`.
- **database/** — `Database.init()` factory (`index.ts`, `DatabaseFactory.ts`). The metadata
DB backend is pluggable: **Typesense or Elasticsearch** (chosen by `DB_TYPE`) for DDOs,
indexer state, logs, orders, ddoState, access lists, escrow events — behind the
`Abstract*Database` interfaces in `BaseDatabase.ts`. **SQLite** is always used for the
nonce DB, config DB, C2D job DB, and auth-token DB (works even with no metadata DB
configured). See `docs/database.md`.
configured) — via Node's built-in `node:sqlite` module (no native addon), wrapped by
`SqliteClient` in `src/components/database/sqliteClient.ts`. See `docs/database.md`.
- **KeyManager/** — provider-abstraction over the node key (`docs/KeyManager.md`). Currently
`RawPrivateKeyProvider` (from `PRIVATE_KEY`); derives the libp2p peerId/keys and the EVM
address, and caches the ethers signer. Designed to add KMS providers (GCP/AWS) later.
Expand Down Expand Up @@ -316,5 +338,5 @@ can talk to `/var/run/docker.sock`. Deployment options (Docker, local Docker bui
`Arhitecture.md` (note the spelling), `API.md` (full HTTP API reference — very large, plus a
Postman collection), `env.md` (authoritative env-var reference), `database.md`,
`Storage.md` / `persistentStorage.md`, `KeyManager.md`, `PolicyServer.md`, `services.md`
(Service-on-Demand), `compute-pricing.md` / `GPU.md` (C2D), `networking.md`, `Logs.md`,
(Service-on-Demand), `compute.md` (C2D configuration: resources, GPUs, constraints, pricing), `networking.md`, `Logs.md`,
`Publishing.md`, `testing.md`, `dockerDeployment.md`.
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:22.22.2-trixie@sha256:17ccc50fade521c62e2acefd0c975bf5eb2a09632b8717fa7f8b1c2b4e967a07 AS builder
FROM node:22.23.1-trixie@sha256:3145536027ca5268e24654f7efebf1dbdd684cda3708324e6c53f4ad61af8710 AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 \
build-essential \
Expand All @@ -14,7 +14,7 @@ COPY . .
RUN npm run build && npm prune --omit=dev


FROM node:22.22.2-trixie-slim@sha256:76043ed3132293c26b960ede4358d3c8ba424ee64662cd2d56318b76fcc51c4c AS runner
FROM node:22.23.1-trixie-slim@sha256:e6d9a389d34ff9678438af985c9913fbd1eb6ed36e80fea56644f4b4f6dd70ba AS runner
RUN apt-get update && apt-get install -y --no-install-recommends \
dumb-init \
gosu \
Expand Down Expand Up @@ -43,6 +43,9 @@ COPY --chown=node:node --from=builder /usr/src/app/node_modules ./node_modules
COPY --chown=node:node --from=builder /usr/src/app/schemas ./schemas
COPY --chown=node:node --from=builder /usr/src/app/package.json ./
COPY --chown=node:node --from=builder /usr/src/app/config.json ./
# Ship the operator service-on-demand templates so SERVICE_TEMPLATES_PATH=docs/serviceTemplates/
# resolves inside the image (the rest of docs/ stays excluded via .dockerignore).
COPY --chown=node:node --from=builder /usr/src/app/docs/serviceTemplates ./docs/serviceTemplates

RUN mkdir -p databases c2d_storage logs

Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ This command will run you through the process of setting up the environmental va

> [!NOTE]
> The quickstart script attempts to automatically detect GPUs (NVIDIA via `nvidia-smi`, others via `lspci`) and appends them to your `DOCKER_COMPUTE_ENVIRONMENTS`.
> Detected GPUs are added to `DOCKER_COMPUTE_ENVIRONMENTS[0].resources` (the connection-level resource pool), and a lightweight ref is added to `DOCKER_COMPUTE_ENVIRONMENTS[0].environments[0].resources`.
> If you choose to manually configure `DOCKER_COMPUTE_ENVIRONMENTS` before running the script (e.g. via environment variable), be aware that auto-detected GPUs will be **merged** into your configuration, which could lead to duplication if you already manually defined them.
> For most users, it is recommended to let the script handle GPU detection automatically.

Expand Down Expand Up @@ -153,5 +154,5 @@ Your node is now running. To start additional nodes, repeat these steps in a new
- [Network Configuration](docs/networking.md)
- [Logging & accessing logs](docs/networking.md)
- [Docker Deployment Guide](docs/dockerDeployment.md)
- [C2D GPU Guide](docs/GPU.md)
- [Compute pricing](docs/compute-pricing.md)
- [Compute (C2D) Configuration — resources, GPUs, constraints, pricing](docs/compute.md)
- [Services (Service-on-Demand)](docs/services.md)
15 changes: 14 additions & 1 deletion config.json
Original file line number Diff line number Diff line change
Expand Up @@ -94,18 +94,31 @@
"validateUnsignedDDO": true,
"jwtSecret": "ocean-node-secret",
"enableBenchmark": false,
"serviceTemplatesPath": "docs/serviceTemplates/",
"dockerComputeEnvironments": [
{
"socketPath": "/var/run/docker.sock",
"resources": [
{
"id": "disk",
"total": 1
}
],
"environments": [
{
"storageExpiry": 604800,
"maxJobDuration": 3600,
"minJobDuration": 60,
"resources": [
{
"id": "cpu"
},
{
"id": "ram"
},
{
"id": "disk",
"total": 1
"max": 1
}
],
"access": {
Expand Down
Loading
Loading