Design proposal: existing-file open, native key providers, and validated open

[skostad]

Summary

Would the maintainers be open to discussing several independent, generally useful APIs for applications with strict encrypted-database lifecycle requirements?

The proposal preserves existing behavior by default and separates four capabilities:

1. writable existing-file open without database creation;
2. native/opaque encryption-key providers;
3. validation before a database object is exposed to JavaScript;
4. stable redacted native error categories.

A fifth topic, race-safe create-new semantics, is presented as a design question rather than a prescribed implementation.

These capabilities would help applications that must fail closed when a database is missing, keep SQLCipher key bytes out of JavaScript, and reject an unexpected or unsupported database before application code can use it.

1. Explicit writable existing-file open

Motivation

The current non-read-only path permits database creation. Some applications need an existing encrypted database to be writable but must treat a missing file as an error rather than silently creating an empty replacement.

Proposed capability

Add an explicit open mode while preserving current behavior:

type DatabaseOpenMode =
  | 'createOrOpen'          // existing/default behavior
  | 'openExistingReadWrite'
  | 'readOnly';

openExistingReadWrite would:

• use writable SQLite open flags without SQLITE_OPEN_CREATE;
• return a stable missing-file error when the file does not exist;
• never create a replacement database;
• close any non-null native handle returned on a failed open.

This can be considered independently from the other proposal sections.

Compatibility

• Existing API/default behavior remains createOrOpen.
• Applications opt into the stricter mode explicitly.
• Existing encryption-key behavior need not change for this capability.

2. Native or opaque key provider

Motivation

Some threat models prohibit raw SQLCipher key bytes from entering JavaScript strings, buffers, application state, logs, or crash metadata. Platform-native code may already own custody or unwrap operations.

Proposed capability

Allow native application integration to register a key provider against an opaque reference:

JavaScript open options contain an opaque key reference only
-> platform-native provider resolves key bytes
-> native SQLCipher open path applies the bytes directly
-> key lifecycle and cleanup remain native

Conceptual JavaScript shape:

open({
  name: 'example.db',
  keyReference: 'application-defined-opaque-reference',
});

Conceptual native contract:

resolve(reference, openContext) -> bounded native key bytes or redacted error
release/erase adapter-owned copies after key application

The core library should not prescribe Android Keystore, iOS Keychain, biometrics, recovery, or application key hierarchy. Applications register platform-native providers and own those policies.

Required properties

• Raw key bytes never pass through JavaScript.
• Provider registration is native and scoped to the application/runtime.
• Unknown references fail before a usable database object exists.
• Keys are supplied through the direct SQLCipher native key API, not SQL string interpolation.
• Provider and adapter errors are redacted.
• Runtime/bridge teardown invalidates provider registrations and outstanding opens safely.

This capability could coexist with the current encryptionKey option for backward compatibility.

3. Validation before object publication

Motivation

Some applications must verify cipher activation, application identity, metadata, and supported schema lineage before JavaScript receives any usable database object.

Validating after open() is too late for that boundary because a host object has already been registered and returned.

Proposed capability

Provide a generic native validation hook or validated-open contract that runs:

native database open
-> SQLCipher key application
-> native validation
-> register and return database object only on success

Validation failure must:

• close the native SQLite handle, including non-null handles from failed opens;
• avoid registering the database in the public object registry;
• return a stable redacted error;
• expose no usable database object to JavaScript.

The generic mechanism should allow an application-native validator to inspect bounded metadata such as:

• SQLCipher/cipher availability;
• application_id;
• application metadata table;
• database format/version;
• supported schema lineage.

The core need not understand application schemas. A callback, registered validator ID, or bounded native validation command list may be appropriate; maintainer guidance on the architecture-compatible form would be valuable.

4. Redacted structured errors

Motivation

Stable categories improve recovery and testing without exposing keys, SQL, paths, native pointers, or database contents.

Proposed capability

Return a structured error with fields such as:

interface DatabaseOpenError {
  category:
    | 'database_missing'
    | 'database_unreadable_or_key_invalid'
    | 'key_unavailable'
    | 'validation_failed'
    | 'unsupported_schema'
    | 'native_internal_failure';
  retryable: boolean;
  operationId?: string;
  message: string; // redacted and stable
}

The public error should not expose:

• raw encryption keys or references that contain secrets;
• SQL text or parameters;
• arbitrary/full filesystem paths;
• native pointers;
• raw database contents;
• detailed wrong-key-versus-corruption distinctions that applications do not need.

Detailed native diagnostics, if retained, should be explicitly gated and redacted.

5. Race-safe create-new semantics: design question

Some applications need a separate create-new operation with this invariant:

• never overwrite, replace, adopt, or ambiguously open an unrelated existing file;
• concurrent create attempts cannot both report success;
• failure cleanup affects only artifacts created by the current operation.

SQLITE_OPEN_EXCLUSIVE is not proposed as a solution because SQLite documents it as a no-op for sqlite3_open_v2.

Would maintainers prefer race-safe create-new semantics to live:

1. inside OP-SQLite as a generic creation API;
2. in a lower-level filesystem/publication hook;
3. entirely in application-native code, followed by strict existing-file open?

Possible designs may involve operation-specific temporary files plus safe publication or platform-native exclusive filesystem primitives. This proposal intentionally does not prescribe an implementation before platform and failure-mode review.

6. Suggested review sequence

The capabilities can be reviewed independently:

1. openExistingReadWrite and stable missing-file error.
2. Redacted structured errors.
3. Native/opaque key-provider extension point.
4. Pre-publication native validation extension point.
5. Separate discussion of race-safe create-new semantics.

Questions for maintainers:

• Are these capabilities aligned with OP-SQLite's architecture and roadmap?
• Which capabilities belong in the core versus platform/application-native integration?
• Would a design document or proof-of-concept contribution be useful for any accepted direction?
• Are there existing internal hooks or planned APIs that already address part of this boundary?
• What compatibility and maintenance constraints should a contribution respect?

No code or application-specific solution is requested in this proposal. The immediate goal is to establish whether the general API direction is acceptable and how it could fit upstream cleanly.

[ospfranco]

1. You can already use any fs library to check for the existence of the DB file before trying to open it. It's a nice API improvement, but implementable on your own.

2. I don't see how can this be implemented, maybe you can provide a simple PoC. Additionally, letting people handle their own encryption per platform is a bad idea, most people don't know what they are doing. As explained in the docs using op-s2 or expo encrypted secret libs is a far better idea, because of the time spent making the libraries secure and using platform APIs. The bytes of the key touching JS was never an issue, talking here from working with larger companies are receiving security audits, it's not perfect, but nothing ever is and as long as the key is quickly used and discarded, it was always considered secure enough. Moving things to purely native presents so many challenges that will just bloat op-sqlite IMO. But feel free to expand here.

3. I do not know enough of any of this mechanisms to have an opinion. I again have doubts this can be made to work seamlessly, but if you can provide a PoC maybe I will get a clearer picture.

4. Is op-sqlite exposing any secrets here? I don't see the threat any important threat here?

5. You mean an open call here? Any open call should execute in the same thread, this should never happen on op-sqlite, only queries are executed in a separate thread.