Skip to content

feat(persistence): add embedded seekDB support - #1293

Open
Evenss wants to merge 10 commits into
oceanbase:masterfrom
Evenss:master
Open

feat(persistence): add embedded seekDB support#1293
Evenss wants to merge 10 commits into
oceanbase:masterfrom
Evenss:master

Conversation

@Evenss

@Evenss Evenss commented Aug 20, 2026

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

None

Rationale for this change

PowerContext currently supports SQLite and externally managed OceanBase deployments. This PR adds an optional embedded seekDB backend for users who need OceanBase-compatible persistence without running a separate database server.

The integration uses pylibseekdb only to manage the embedded database lifecycle, while reusing the existing OceanBase-compatible persistence, full-text search, and vector search paths.

What changes are included in this PR?

  • Add an optional powercontext[seekdb] extra with pylibseekdb.
  • Introduce SeekDBConfig and SeekDBProfile.
  • Start and stop the embedded seekDB instance together with the PowerContext runtime.
  • Connect to the local seekDB instance through its asynchronous MySQL Unix socket.
  • Reuse the existing OceanBase-compatible full-text and vector index implementations.
  • Add seekdb to the runtime database discriminator and persistence composition.
  • Disable session autocommit to preserve SQLAlchemy transaction behavior.
  • Add a seekDB-specific SQLAlchemy close path to avoid socket reset errors during graceful shutdown.
  • Add cancellation-safe startup cleanup and close the SQLAlchemy connection pool before stopping the embedded instance.
  • Support POWERCONTEXT_SERVER_DATABASE_KIND=seekdb.
  • Use the Server data directory's seekdb subdirectory when DATABASE_PATH is omitted or blank.
  • Support custom seekDB paths and database names.
  • Extend the real-experience E2E harness to discover, count, and purge seekDB-backed scopes.
  • Add Server settings coverage for default, blank, and custom seekDB paths.
  • Update .env.example with an opt-in embedded seekDB configuration example.
  • Update uv.lock with the optional seekDB dependency.

Are there any user-facing changes?

Yes. This PR adds a new opt-in persistence backend.

Install PowerContext with embedded seekDB support:

pip install "powercontext[server,seekdb]"

Configure the Server:

POWERCONTEXT_SERVER_DATABASE_KIND=seekdb

# Optional. When omitted, the Server uses the seekdb subdirectory
# under its data directory, such as $POWERCONTEXT_HOME/seekdb.
POWERCONTEXT_SERVER_DATABASE_PATH=.powercontext/seekdb

The seekDB database name defaults to test.

This is not a breaking change:

  • SQLite remains the default database.
  • Existing SQLite and OceanBase behavior is unchanged.
  • pylibseekdb is not installed by the default Server extra.
  • The native seekDB binding is loaded only when kind=seekdb is selected.
  • Settings parsing does not create the seekDB directory or start an embedded instance.

Embedded seekDB currently requires a supported Linux or macOS pylibseekdb wheel. Windows users can continue using SQLite or OceanBase, but embedded seekDB is not currently available on Windows.

How was this change tested?

  • uv lock --check: passed.
  • Ruff checks on the changed Python sources and focused tests: passed.
  • Type checking: passed.
  • git diff --check: passed.
  • Targeted Server database settings tests: 7 passed.
  • Manually started PowerContext with an embedded seekDB data directory.
  • Verified /health/live returned a healthy status.
  • Verified /health/ready reported both the runtime and database as ready.
  • Verified database schema initialization through the existing persistence path.
  • Verified graceful Ctrl+C shutdown exited with code 0.
  • Verified shutdown produced no ConnectionResetError, SQLAlchemy close warning, or traceback.

AI usage statement

OpenAI Codex (GPT-5.6) was used to assist with implementation, debugging, code review, test execution, and drafting this PR.

Comment thread src/powercontext/builtin/persistence/seekdb/profile.py Outdated
Comment thread pyproject.toml
"pyobvector>=0.2.28,<0.3",
"sqlalchemy[asyncio]>=2,<3",
]
seekdb = [

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This extra also changes the root package metadata recorded in e2e/bub/uv.lock, but that lockfile was not regenerated. On this head, uvx --from uv==0.10.12 uv lock --project e2e/bub --locked exits with The lockfile needs to be updated; the same command passes on the base commit. Please regenerate and commit e2e/bub/uv.lock. It may also be worth making the E2E validation run the locked check, since the current sync step can update the file silently.

Comment thread src/powercontext/builtin/persistence/seekdb/profile.py
Comment thread .env.example
@Evenss

Evenss commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Thanks for the review @Teingi . I have addressed all of the feedback:

  • Made seekDB shutdown cancellation-safe. If cancellation occurs during shutdown, PowerContext now waits for active transactions to finish and for the engine to be disposed before closing the seekDB instance, then re-raises the cancellation.
  • Added a regression test that cancels shutdown while an active transaction is held.
  • Regenerated e2e/bub/uv.lock with uv 0.10.12 and the official PyPI index. The locked validation now passes.
  • Restricted embedded seekDB to its built-in test database. The storage path remains configurable, while other database names are rejected during configuration validation.
  • Added embedded seekDB setup sections to the English and Chinese user documentation, covering installation, supported platforms, environment variables, startup, and verification.

Validation completed:

  • seekDB profile tests: 6 passed
  • Server seekDB configuration tests: 5 passed
  • Ruff, ty, prek, and git diff --check: passed
  • Strict documentation build: passed
  • Real temporary seekDB startup and connection smoke test: SELECT DATABASE() returned test

I kept the GitHub workflows unchanged to keep this PR focused on the minimal seekDB integration.

try:
await asyncio.shield(close_task)
except asyncio.CancelledError:
await close_task

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep the cleanup task shielded after the first cancellation? The initial asyncio.shield(close_task) protects database.close(), but the except branch awaits close_task directly. A second cancellation therefore cancels the close, and the outer finally still calls instance.close() while an active transaction is running.

I reproduced this on the current head with pylibseekdb==1.3.0.post4: hold an insert transaction, start __aexit__, then cancel the shutdown task twice. The transaction fails with OperationalError 2013 (Connection reset by peer), and the inserted row is missing after reopening the database. With one cancellation, the row is preserved.

Please wait for the cleanup task through repeated shielded awaits, defer and re-raise cancellation only after cleanup completes, and add a regression test covering multiple cancellations. _open_instance() has the same unshielded follow-up await.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an optional embedded seekDB persistence backend to PowerContext’s Server/runtime, enabling OceanBase-compatible storage without running an external database process.

Changes:

  • Introduces SeekDBConfig/SeekDBProfile and wires seekdb into runtime database selection and composition.
  • Adds settings support for default/blank/custom seekDB paths, plus E2E harness support for discovering and purging seekDB-backed scopes.
  • Adds the powercontext[seekdb] extra (with pylibseekdb) and documents how to install/configure embedded seekDB.

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/powercontext/builtin/persistence/seekdb/profile.py Implements embedded seekDB lifecycle + SQLAlchemy async engine creation over a local Unix socket.
src/powercontext/builtin/persistence/seekdb/__init__.py Exposes seekDB config/profile/errors as a package API.
src/powercontext/builtin/runtime/composition.py Allows the builtin runtime to open either OceanBase or embedded seekDB profiles for OceanBase-compatible indexes.
src/powercontext/builtin/runtime/config.py Extends the database discriminator union to include SeekDBConfig.
src/powercontext/server/settings.py Adds settings normalization so seekDB path defaults correctly when omitted/blank.
src/powercontext/paths.py Adds default_seekdb_path() and exports it.
tests/builtin/persistence/test_seekdb_profile.py Adds unit tests for seekDB profile lifecycle, cancellation safety, and SQLAlchemy dialect close behavior.
tests/test_server.py Adds ServerSettings coverage for selecting seekDB and for default/blank/custom path handling.
tests/e2e/real_experience_skill/harness.py Extends the E2E harness DB utilities to support seekDB profiles.
pyproject.toml Defines the seekdb optional extra and its dependency on pylibseekdb.
uv.lock Locks the new optional pylibseekdb dependency and adds the seekdb extra metadata.
e2e/bub/uv.lock Updates the E2E lock metadata to include the seekdb extra requirements.
docs/en/docs/how-to/install-and-run.md Documents embedded seekDB installation/configuration.
docs/zh/docs/how-to/install-and-run.md Chinese documentation for embedded seekDB installation/configuration.
.env.example Adds an opt-in embedded seekDB configuration example.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .env.example
# POWERCONTEXT_SERVER_DATABASE_URL=mysql+aoceanbase://user:password@host:2881/powercontext?charset=utf8mb4

# To use embedded seekDB instead, install powercontext[server,seekdb] and replace the SQLite database values.
# Omit DATABASE_PATH to use the Server data directory's seekdb subdirectory ($POWERCONTEXT_HOME/seekdb when set).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants