Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,28 @@ cp .env.example .env.local # Add your API keys
bun run src/index.ts run -p supermemory -b locomo
```

LongMemEval-V2 uses the build-aware workflow because multiple questions share
the same exact haystack and require multimodal, benchmark-owned evaluation:

```bash
bun run src/index.ts lme-v2 --help
bun run src/index.ts lme-v2 dry-run \
--run-id lme-v2-dry-run \
--dataset data/benchmarks/longmemeval-v2
```

See [docs/LONGMEMEVAL_V2.md](docs/LONGMEMEVAL_V2.md) for the architecture,
download/preparation commands, MemoryBuild semantics, live-service preflight,
resume behavior, and validation status.

To use the browser workflow, run `bun run src/index.ts serve --no-open`, open
**New Run**, and choose **LongMemEval-V2** from the **Benchmark** dropdown. Its
guided form defaults to an offline Plan and exposes exact-haystack limits,
reader/evaluator models, a V2 memory-provider dropdown, and advanced
bounded-ingestion controls. Supermemory, Filesystem, and local RAG support live
V2 stages; Mem0 and Zep are clearly labelled Plan-only until their exact
interruption/reconciliation contracts are implemented.

## Configuration

```bash
Expand Down
20 changes: 20 additions & 0 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Third-party notices

## LongMemEval-V2

The LongMemEval-V2 integration is based on the public
[`xiaowu0162/longmemeval-v2`](https://huggingface.co/datasets/xiaowu0162/longmemeval-v2)
dataset and the LongMemEval-V2 benchmark repository.

- Reference revision: `f152293e235517d504809563c833d7190b8c713b`
- Supermemory adapter oracle: `feat/supermemory` commit
`2fa6616dce77e0385d7e1c44510dfde8aa3c46e3`
- Structured Accessibility Converter oracle:
`supermemory_adapter/approaches/Approach_1.py`
- Oracle SHA-256:
`22cff05fafa9f882040afa8296439da0f911f800c107424de105ab3af5e69236`
- License: Apache License 2.0

MemoryBench downloads the dataset's own `LICENSE` file as part of the pinned,
checksum-verified snapshot. The TypeScript converter is a new implementation
whose behavior is tested against the cited reference.
Loading