diff --git a/kotlin-exposed-dialect/CHANGELOG.md b/kotlin-exposed-dialect/CHANGELOG.md index 432783c1..03c1ff9c 100644 --- a/kotlin-exposed-dialect/CHANGELOG.md +++ b/kotlin-exposed-dialect/CHANGELOG.md @@ -1,22 +1,24 @@ -## 0.9.0 +## 0.9.0 Initial release of the Kotlin Exposed dialect for YDB. ### Added -- YDB `VendorDialect` for Exposed JDBC; `registerYdbDialect(enableSignedDatetimes = …)` for setup. -- `ydbTransaction { ... }` — retryable transactions with `readOnly` and `YdbRetryConfig` - (exponential backoff with full/equal jitter; retries classified by JDBC `SQLException` vendor codes). -- Native `UPSERT` / `REPLACE` rendering — wired into Exposed's standard `Table.upsert` and - `Table.replace` DSL. -- `YdbTable` — YQL `CREATE TABLE` with table-level `PRIMARY KEY (…)`, inline secondary indexes - (`secondaryIndex`), and optional row TTL (`ttl` / `YdbTtlColumnMode`); prefer over plain Exposed - `Table` when DDL must be valid on YDB. -- Post-create indexes via Exposed `Table.index()` on any table → `ALTER TABLE … ADD INDEX … GLOBAL`. -- JDBC metadata for reading existing indexes. -- Temporal columns: unsigned (`ydbDate`, …) and signed (`ydbDate32`, …) extensions with - JDBC vendor codes; DDL `sqlType()` derived from the code. No connection-level temporal flag. -- Custom column types for `Decimal`, `Interval`, `Json`, `JsonDocument`, native `Uuid`, - unsigned integers, `Uint64`, plus a `ydbDecimalLiteral` helper for update expressions. -- `Serial` / `BigSerial` via Exposed `autoIncrement()` on `Table`. -- Explicit rejection of ANSI `MERGE` (`UPSERT` covers the use case). +- YDB `VendorDialect` for Exposed JDBC and `registerYdbDialect(enableSignedDatetimes = …)` for setup. +- `ydbDatabaseConfig()` and `ydbJdbcUrl(...)` helpers for recommended Exposed and JDBC configuration. +- `ydbTransaction { ... }` with retry classification based on JDBC `SQLException` vendor codes. +- Native YDB `UPSERT` / `REPLACE` rendering wired into Exposed `Table.upsert` and `Table.replace`. +- `createYdbStatement()` for YDB-compatible `CREATE TABLE` rendering with a table-level `PRIMARY KEY (...)`. +- Post-create indexes through Exposed `Table.index()` в†’ `ALTER TABLE ... ADD INDEX ... GLOBAL`. +- JDBC metadata support for reading existing indexes from YDB. +- Temporal column extensions (`ydbDate`, `ydbDate32`, `ydbDatetime`, `ydbDatetime64`, `ydbTimestamp`, `ydbTimestamp64`) with JDBC vendor bindings. +- Custom YDB column types for `Decimal`, `Interval`, `Json`, `JsonDocument`, native `Uuid`, and unsigned values. +- `ydbDecimalLiteral` for decimal update expressions. +- `Serial` / `BigSerial` support via Exposed `autoIncrement()`. +- Explicit rejection of ANSI `MERGE`. + +### Notes + +- Exposed 1.3.0 does not provide a dialect hook for rendering a single-column PK inside `CREATE TABLE`, so schema generation for YDB is implemented through a `createStatement()` override and `createYdbStatement()`. +- Production schema management is expected to use external versioned migrations; the repository includes validation coverage for externally created schemas through YDB-compatible Exposed drift checks for missing columns and secondary indexes. + diff --git a/kotlin-exposed-dialect/README.md b/kotlin-exposed-dialect/README.md index c6e129d0..261c4823 100644 --- a/kotlin-exposed-dialect/README.md +++ b/kotlin-exposed-dialect/README.md @@ -1,19 +1,21 @@ -# Kotlin Exposed YDB Dialect +# Kotlin Exposed YDB Dialect YDB integration for [JetBrains Exposed](https://github.com/JetBrains/Exposed) via JDBC. + The module provides: -- a Kotlin Exposed `VendorDialect` for YDB (SQL, type mapping, post-create indexes); -- [`createYdbStatement`](src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt) for YQL `CREATE TABLE` (table-level PK, inline indexes, TTL); -- `Table.upsert` / `Table.replace` DSL backed by native YDB `UPSERT` / `REPLACE`; -- a retryable transaction wrapper that handles YDB's OCC retries transparently. +- a Kotlin Exposed `VendorDialect` for YDB; +- `createYdbStatement()` for YDB-compatible `CREATE TABLE` rendering; +- native `UPSERT` / `REPLACE` support through Exposed's `Table.upsert` and `Table.replace`; +- retry-aware `ydbTransaction { ... }` for YDB OCC conflicts; +- YDB-specific column types for temporal, JSON, interval, decimal, UUID, and unsigned values. ## Requirements - JDK 17+ - Maven -- [YDB JDBC driver](https://github.com/ydb-platform/ydb-jdbc-driver) on the application classpath (not bundled with this artifact) -- JetBrains Exposed 1.x +- [YDB JDBC driver](https://github.com/ydb-platform/ydb-jdbc-driver) on the application classpath +- JetBrains Exposed 1.3.x ```xml @@ -33,11 +35,16 @@ The module provides: ```kotlin import org.jetbrains.exposed.v1.jdbc.Database import tech.ydb.exposed.dialect.registerYdbDialect +import tech.ydb.exposed.dialect.ydbDatabaseConfig import tech.ydb.exposed.dialect.ydbTransaction -registerYdbDialect() // or registerYdbDialect(enableSignedDatetimes = true) +registerYdbDialect() -val db = Database.connect("jdbc:ydb:grpc://localhost:2136/local") +val db = Database.connect( + url = "jdbc:ydb:grpc://localhost:2136/local", + driver = "tech.ydb.jdbc.YdbDriver", + databaseConfig = ydbDatabaseConfig() +) ydbTransaction(db) { // Exposed DSL / DAO code @@ -46,14 +53,16 @@ ydbTransaction(db) { ## Defining tables -YDB requires a table-level `PRIMARY KEY (…)` in `CREATE TABLE`, not `col Type PRIMARY KEY` on a column. -Use [`createYdbStatement`](src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt) for schema DDL; plain Exposed -`Table` + `SchemaUtils.create` still works for DML/tests but emits inline PK SQL that YDB rejects. +YDB requires a table-level `PRIMARY KEY (...)` in `CREATE TABLE`, not the inline +`column Type PRIMARY KEY` form that Exposed may generate for a single-column PK. + +Because Exposed 1.3.0 does not expose a dialect hook for this part of `CREATE TABLE`, +YDB schema generation is implemented as a local workaround: override `createStatement()` +and delegate to `createYdbStatement()`. ```kotlin -import tech.ydb.exposed.dialect.YdbIndexScope -import tech.ydb.exposed.dialect.YdbIndexSyncMode -import tech.ydb.exposed.dialect.YdbTable +import org.jetbrains.exposed.v1.core.Table +import tech.ydb.exposed.dialect.createYdbStatement import tech.ydb.exposed.dialect.javatime.ydbTimestamp64 import tech.ydb.exposed.dialect.ydbDecimal @@ -68,18 +77,52 @@ object Products : Table("products") { override val primaryKey = PrimaryKey(id) init { - // Post-create index (ALTER TABLE … ADD INDEX … GLOBAL) — same as on Exposed Table index(false, sku) } - + + override fun createStatement(): List = createYdbStatement() +} +``` + +`createYdbStatement()`: + +- renders all columns without inline PK declarations; +- appends a table-level `PRIMARY KEY (...)`; +- preserves `NOT NULL` and `DEFAULT`; +- preserves `storageParameters`, so YDB-specific `WITH (...)` clauses can still be used. + +Post-create indexes declared through `Table.index(...)` are still emitted through the dialect's +standard `ALTER TABLE ... ADD INDEX ... GLOBAL` path. + +### TTL via storage parameters + +If you need YDB-specific table options such as TTL, declare them through Exposed +`storageParameters` and keep the DDL override: + +```kotlin +import org.jetbrains.exposed.v1.core.RawTableStorageParameter +import org.jetbrains.exposed.v1.core.Table +import org.jetbrains.exposed.v1.core.TableStorageParameter +import tech.ydb.exposed.dialect.createYdbStatement +import tech.ydb.exposed.dialect.javatime.ydbTimestamp64 + +object Sessions : Table("sessions") { + val id = integer("id") + val expireAt = ydbTimestamp64("expire_at") + + override val primaryKey = PrimaryKey(id) + + override val storageParameters: List = + listOf(RawTableStorageParameter("TTL = Interval(\"PT1H\") ON expire_at")) + override fun createStatement(): List = createYdbStatement() } ``` -## Insert / upsert / replace / update / delete +## Insert / upsert / replace -Exposed's standard DSL works as-is. YDB's native `UPSERT` and `REPLACE` are exposed via -the same `Table.upsert` / `Table.replace` extensions Exposed provides for other vendors: +Exposed's regular DSL works as-is. The dialect also maps Exposed's `upsert` / `replace` +to native YDB `UPSERT` / `REPLACE`. ```kotlin Products.upsert { @@ -99,174 +142,176 @@ Products.replace { } ``` -ANSI `MERGE` is intentionally rejected — `UPSERT` / `REPLACE` cover the same use cases. +Behavioral notes: -YDB `UPSERT` writes only the columns listed in the DSL block; on primary-key conflict, other -columns are left unchanged. Exposed's `onUpdate` and `keyColumns` are **ignored** (no -`ON DUPLICATE KEY UPDATE`). `upsert(where)` **throws** — use `update { }` for conditional writes. +- `UPSERT` writes only the columns listed in the statement; +- on PK conflict, columns omitted from `UPSERT` remain unchanged; +- `REPLACE` overwrites the row by PK, so omitted columns are reset to defaults; +- `upsert(where)` is not supported; +- ANSI `MERGE` is intentionally rejected. ## Retryable transactions -YDB uses Optimistic Concurrency Control, so a transaction can fail with `Transaction locks -invalidated` under contention. Use `ydbTransaction` instead of plain `transaction` to retry -the body on retryable YDB statuses (`ABORTED`, `OVERLOADED`, `BAD_SESSION`, ...): +YDB uses Optimistic Concurrency Control, so a transaction can fail with a retryable status. +Use `ydbTransaction` instead of plain `transaction` when you want retries on retryable YDB errors. ```kotlin import tech.ydb.exposed.dialect.YdbRetryConfig import tech.ydb.exposed.dialect.ydbTransaction ydbTransaction(db) { - // read-write; retries transient YDB statuses (ABORTED, OVERLOADED, BAD_SESSION, ...) + // read-write transaction } ydbTransaction(db, retry = YdbRetryConfig.IDEMPOTENT) { - // idempotent body — UNDETERMINED and other non-transient retryable codes are retried too + // safe-to-repeat body } ydbTransaction(db, readOnly = true, retry = YdbRetryConfig.IDEMPOTENT) { - // read-only snapshot work + // read-only transaction } ``` -Backoff uses full jitter for `ABORTED` / `UNDETERMINED`, equal jitter for `UNAVAILABLE` / transport / -`OVERLOADED`, and zero delay for session errors. Status codes are read from `SQLException.errorCode` -(YDB vendor codes), not from error message text. - -Use `retry = YdbRetryConfig.IDEMPOTENT` only when the body can be safely re-executed (pure reads, -single `UPSERT` / `REPLACE`, idempotent business logic). Customize attempts and backoff via -`YdbRetryConfig` or `YdbRetryConfig.DEFAULT.copy(maxAttempts = 3)`. +`YdbRetryConfig.IDEMPOTENT` should only be used when the body can be safely executed more than once. ## Types -Default mapping for standard Exposed types: - -| Exposed | YDB | -|---------------------|--------------------| -| `byte` / `ubyte` | `Int8` / `Uint8` | -| `short` / `ushort` | `Int16` / `Uint16` | -| `integer`/`uinteger`| `Int32`/`Uint32` | -| `long` | `Int64` | -| `float` / `double` | `Float` / `Double` | -| `bool` | `Bool` | -| `varchar` / `text` | `Text` | -| `binary` / `blob` | `Bytes` | -| `uuid` | `Uuid` | -| `date` | `Date` | -| `datetime` | `Datetime` | -| `timestamp` | `Timestamp` | -| `json` | `Json` | -| `jsonb` | `JsonDocument` | - -`varchar(n)` maps to `Text` (length is not preserved in YDB DDL). - -### Production types (`ydb*` / `javatime.*`) - -For temporal and unsigned columns, use **`ydbDate` / `ydbDate32`**, **`ydbUbyte`**, **`ydbUint32`**, etc. -They bind via YDB JDBC vendor type codes. Standard Exposed `date()`, `ubyte()`, `binary()` use -generic JDBC binding — DDL still maps correctly for many cases, but edge cases (unsigned ranges, -signed vs legacy temporal) may differ. **Prefer `ydb*` / `javatime.*` in production.** - -Pick unsigned legacy or signed extended temporal types per column on any `Table`; -JDBC vendor code drives both bind and DDL `sqlType()`: - -```kotlin -import tech.ydb.exposed.dialect.javatime.ydbDate -import tech.ydb.exposed.dialect.javatime.ydbDate32 -import tech.ydb.exposed.dialect.javatime.ydbDatetime64 - -object Events : Table("events") { - val created = ydbDate("created") // Date - val expires = ydbDate32("expires") // Date32 - val updated = ydbDatetime64("updated") // Datetime64 -} -``` - -Optional: `registerYdbDialect(enableSignedDatetimes = true)` switches **dialect** DDL names for -standard Exposed `date` / `datetime` / `timestamp` to `Date32` / `Datetime64` / `Timestamp64`. -Add `forceSignedDatetimes=true` to the JDBC URL yourself when the driver requires it. -Per-column types remain explicit (`ydbDate` vs `ydbDate32`). - -Additional YDB-specific column types are available via extension functions on `Table`: +Default Exposed type mapping: + +| Exposed | YDB | +|----------------------|--------------------| +| `byte` / `ubyte` | `Int8` / `Uint8` | +| `short` / `ushort` | `Int16` / `Uint16` | +| `integer` / `uinteger` | `Int32` / `Uint32` | +| `long` | `Int64` | +| `float` / `double` | `Float` / `Double` | +| `bool` | `Bool` | +| `varchar` / `text` | `Text` | +| `binary` / `blob` | `Bytes` | +| `uuid` | `Uuid` | +| `date` | `Date` | +| `datetime` | `Datetime` | +| `timestamp` | `Timestamp` | +| `json` | `Json` | +| `jsonb` | `JsonDocument` | + +YDB-specific extensions are available through `ydb*` and `javatime.*`, for example: ```kotlin ydbDecimal("price", precision = 10, scale = 2) ydbInterval("duration") ydbJson("payload") -ydbJsonDocument("indexed_payload") // JsonDocument, analogue of jsonb -ydbUuid("id") // native Uuid; same as Exposed uuid() under this dialect +ydbJsonDocument("indexed_payload") +ydbUuid("id") ydbUint64("counter") ``` -`ydbUint64` / `ydbUlong` are backed by `Long` / `ULong` with range `0..Long.MAX_VALUE` for the -JDBC long path. Use a wider type if you need the full `Uint64` range. +Signed temporal mode can be enabled through: + +```kotlin +registerYdbDialect(enableSignedDatetimes = true) +``` -For Decimal literals inside update expressions there's `ydbDecimalLiteral`: +and, for JDBC URL normalization: ```kotlin -import tech.ydb.exposed.dialect.ydbDecimalLiteral +ydbJdbcUrl("jdbc:ydb:grpc://localhost:2136/local", enableSignedDatetimes = true) +``` + +## Schema management in production + +Schema generation through Exposed is supported, but for YDB it is intentionally treated as a +compatibility workaround rather than the primary schema-management model. -it.update(Products.price, ydbDecimalLiteral(BigDecimal("45.00"), 10, 2)) +In production, the recommended approach is: + +1. manage schema through versioned migrations such as Flyway or Liquibase; +2. keep Exposed table definitions aligned with that schema; +3. validate drift through Exposed migration helpers. + +If your application uses schema validation or migration diff generation through Exposed, +also add: + +```xml + + org.jetbrains.exposed + exposed-migration-core + ${exposed.version} + + + org.jetbrains.exposed + exposed-migration-jdbc + ${exposed.version} + ``` -## Identifiers +This repository includes integration coverage for: + +- manual schema creation through raw SQL; +- YDB-compatible drift detection for missing columns and secondary indexes; +- empty diff for matching schema; +- non-empty diff for drifted schema. -Exposed `autoIncrement()` maps to YDB `Serial` / `BigSerial`: +In Exposed 1.3.0, the full `MigrationUtils.statementsRequiredForDatabaseMigration(...)` path +unconditionally reads CHECK-constraint metadata from `INFORMATION_SCHEMA.CHECK_CONSTRAINTS`. +YDB does not expose that metadata through the current JDBC driver, so the repository validates +externally managed schemas through the compatible building blocks that Exposed already provides: ```kotlin -object Orders : YdbTable("orders") { - val id = integer("id").autoIncrement() - val total = ydbDecimal("total", precision = 12, scale = 2) - override val primaryKey = PrimaryKey(id) +import org.jetbrains.exposed.v1.jdbc.SchemaUtils +import tech.ydb.exposed.dialect.ydbTransaction + +ydbTransaction(db, readOnly = true) { + val missingColumns = SchemaUtils.addMissingColumnsStatements(Products, withLogs = true) + val existingIndexes = db.dialectMetadata.existingIndices(Products).getValue(Products) } ``` -For UUID keys use `ydbUuid("id")` or Exposed `uuid()` under this dialect. Unsigned `Serial` -columns are not supported. +When schema was changed through raw SQL just before validation, run the diff in a fresh transaction +so Exposed does not validate against stale metadata cache. -## Indexes - -- **Post-create** (any `Table` or `YdbTable`): Exposed `index()` / `index(customName, isUnique, …)` - → `ALTER TABLE … ADD INDEX … GLOBAL [UNIQUE] ON (…)`. -- **Inline in `CREATE TABLE`** (`YdbTable` only): [`secondaryIndex`](src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt) - with optional `COVER`, `ASYNC`, `WITH`. +That validation path is the one that matters most in real projects, where schema changes are usually +applied by dedicated migration tools rather than by ORM-driven DDL generation. ## Known limitations -Inherited from Exposed `VendorDialect` unless overridden here: foreign keys, sequences, -`SELECT … FOR UPDATE`, dialect-specific features aimed at PostgreSQL/MySQL may produce SQL -that YDB does not support. This module overrides indexes, UPSERT/REPLACE, LIMIT/OFFSET, JSON -functions, and YDB type names — not the entire DDL surface. - -- No ANSI `MERGE`; use `UPSERT` / `REPLACE`. -- Plain `Table` DDL uses Exposed's inline `PRIMARY KEY` on columns — use `YdbTable` (or hand-written YQL). -- No Yson / timezone-aware temporal types in this module. -- Functional indexes (Exposed `index` with expressions) are rejected. +- Exposed 1.3.0 does not expose a dialect hook for PK rendering inside `CREATE TABLE`; +- every table intended for YDB DDL must override `createStatement()` and call `createYdbStatement()`; +- plain `Table` / `IdTable` DDL without that override emits inline PK SQL that YDB rejects; +- functional indexes are not supported; +- `ALTER TABLE ... ADD INDEX ... GLOBAL UNIQUE` depends on YDB support for unique indexes on existing tables; +- ANSI `MERGE` is not supported; +- `Uint64` binding is limited to the `0..Long.MAX_VALUE` range in the current implementation. ## Tests -Integration tests use [testcontainers](https://www.testcontainers.org/) via -`tech.ydb.test:ydb-junit5-support` — no manual Docker setup needed: +Unit tests: + +```bash +mvn test +``` + +Integration tests: ```bash mvn verify ``` -DDL-focused tests use `YdbTable`; many other integration tests still use plain `Table` and may -fail `SchemaUtils.create` on YDB until migrated to `YdbTable` (inline `PRIMARY KEY` in Exposed DDL). +The build separates unit and integration tests through surefire/failsafe. Integration tests run +against YDB in testcontainers. ## Demo application -The `example/` module contains a runnable demo. Install the dialect first: +The `example/` module contains a small runnable demo. Install the dialect first: ```bash mvn -DskipTests -DskipITs install ``` -Then run the demo: +Then run: ```bash cd example mvn exec:java -Dexec.mainClass=tech.ydb.exposed.dialect.example.DemoAppKt ``` -It expects a YDB instance at `jdbc:ydb:grpc://localhost:2136/local`. diff --git a/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoApp.kt b/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoApp.kt index af24d982..188e2da4 100644 --- a/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoApp.kt +++ b/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoApp.kt @@ -6,23 +6,19 @@ import org.jetbrains.exposed.v1.jdbc.deleteWhere import org.jetbrains.exposed.v1.jdbc.selectAll import org.jetbrains.exposed.v1.jdbc.update import org.jetbrains.exposed.v1.jdbc.upsert -import org.jetbrains.exposed.v1.core.DatabaseConfig import org.jetbrains.exposed.v1.jdbc.Database import tech.ydb.exposed.dialect.registerYdbDialect +import tech.ydb.exposed.dialect.ydbDatabaseConfig import tech.ydb.exposed.dialect.ydbDecimalLiteral import tech.ydb.exposed.dialect.ydbTransaction import java.math.BigDecimal -import java.sql.Connection fun main() { registerYdbDialect() val db = Database.connect( url = "jdbc:ydb:grpc://localhost:2136/local", driver = "tech.ydb.jdbc.YdbDriver", - databaseConfig = DatabaseConfig { - defaultIsolationLevel = Connection.TRANSACTION_SERIALIZABLE - useNestedTransactions = false - } + databaseConfig = ydbDatabaseConfig() ) ydbTransaction(db) { diff --git a/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoTables.kt b/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoTables.kt index e9ede561..55954a74 100644 --- a/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoTables.kt +++ b/kotlin-exposed-dialect/example/src/main/kotlin/tech/ydb/exposed/dialect/example/DemoTables.kt @@ -1,11 +1,10 @@ package tech.ydb.exposed.dialect.example -import tech.ydb.exposed.dialect.YdbIndexScope -import tech.ydb.exposed.dialect.YdbIndexSyncMode -import tech.ydb.exposed.dialect.YdbTable +import org.jetbrains.exposed.v1.core.Table +import tech.ydb.exposed.dialect.createYdbStatement import tech.ydb.exposed.dialect.ydbDecimal -object DemoProducts : YdbTable("demo_products") { +object DemoProducts : Table("demo_products") { val id = integer("id") val sku = varchar("sku", 64) val name = varchar("name", 255) @@ -16,14 +15,7 @@ object DemoProducts : YdbTable("demo_products") { init { index(false, sku) - - secondaryIndex( - name = "demo_products_category_idx", - category, - unique = false, - scope = YdbIndexScope.GLOBAL, - syncMode = YdbIndexSyncMode.ASYNC, - coverColumns = listOf(name, price) - ) } + + override fun createStatement(): List = createYdbStatement() } diff --git a/kotlin-exposed-dialect/pom.xml b/kotlin-exposed-dialect/pom.xml index cd478aa5..745713fd 100644 --- a/kotlin-exposed-dialect/pom.xml +++ b/kotlin-exposed-dialect/pom.xml @@ -128,6 +128,18 @@ test + + org.jetbrains.exposed + exposed-migration-core + test + + + + org.jetbrains.exposed + exposed-migration-jdbc + test + + tech.ydb.test ydb-junit5-support diff --git a/kotlin-exposed-dialect/spring-boot-starter/README.md b/kotlin-exposed-dialect/spring-boot-starter/README.md new file mode 100644 index 00000000..096117b3 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/README.md @@ -0,0 +1,62 @@ +# Kotlin Exposed YDB Dialect Spring Boot Starter + +Optional Spring Boot integration for [`kotlin-exposed-ydb-dialect`](../). + +This module reuses Exposed's official Spring Boot starter and adds the YDB-specific pieces +that the generic starter does not know about: + +- registering the YDB JDBC dialect in Exposed; +- aligning `spring.datasource.url` with `forceSignedDatetimes=...`; +- defaulting `spring.datasource.driver-class-name` to `tech.ydb.jdbc.YdbDriver` when it is omitted; +- supplying the recommended `DatabaseConfig` for YDB as the primary Spring bean; +- creating an Exposed `Database` bean from the Spring-managed `DataSource`; +- exposing `YdbTransactionOperations` for retry-aware Exposed transactions. + +## Dependency + +```xml + + tech.ydb.dialects + kotlin-exposed-ydb-dialect-spring-boot-starter + 0.9.0 + +``` + +The starter pulls in: + +- `tech.ydb.dialects:kotlin-exposed-ydb-dialect` +- `org.jetbrains.exposed:exposed-spring-boot-starter` +- `tech.ydb.jdbc:ydb-jdbc-driver` + +## Minimal configuration + +```yaml +spring: + datasource: + url: jdbc:ydb:grpc://localhost:2136/local + exposed: + ydb: + enable-signed-datetimes: false +``` + +If `spring.datasource.driver-class-name` is not specified, the starter sets it to +`tech.ydb.jdbc.YdbDriver` automatically. + +When `spring.exposed.ydb.enable-signed-datetimes=true`, the starter also propagates the +matching `forceSignedDatetimes=true` flag into the normalized JDBC URL. + +## Retry-aware transactions + +`@Transactional` gives you Spring-managed Exposed transactions, but it does not add YDB retry +policy for OCC conflicts. Use `YdbTransactionOperations` when you need the retrying path: + +```kotlin +@Service +class ProductService( + private val ydbTx: YdbTransactionOperations +) { + fun save() = ydbTx.execute { + // Exposed DSL + } +} +``` diff --git a/kotlin-exposed-dialect/spring-boot-starter/pom.xml b/kotlin-exposed-dialect/spring-boot-starter/pom.xml new file mode 100644 index 00000000..f2130086 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/pom.xml @@ -0,0 +1,240 @@ + + + 4.0.0 + + tech.ydb.dialects + kotlin-exposed-ydb-dialect-spring-boot-starter + 0.9.0 + jar + + Kotlin Exposed YDB Dialect Spring Boot Starter + Spring Boot starter for the Kotlin Exposed YDB dialect + https://github.com/ydb-platform/ydb-java-dialects + + + + Svetlana Markelova + sv.markelova11@gmail.com + YDB + https://ydb.tech/ + + + Kirill Kurdyukov + kurdyukov-kir@ydb.tech + YDB + https://ydb.tech/ + + + + + + Apache License, Version 2.0 + https://www.apache.org/licenses/LICENSE-2.0 + + + + + https://github.com/ydb-platform/ydb-java-dialects + scm:git:https://github.com/ydb-platform/ydb-java-dialects.git + scm:git:https://github.com/ydb-platform/ydb-java-dialects.git + + + + UTF-8 + + 17 + 17 + 17 + + 2.2.20 + 3.3.13 + 1.3.0 + 2.3.22 + 2.4.3 + + + + + + org.springframework.boot + spring-boot-dependencies + ${spring.boot.version} + pom + import + + + + + + + org.jetbrains.kotlin + kotlin-stdlib + ${kotlin.version} + + + + tech.ydb.dialects + kotlin-exposed-ydb-dialect + ${project.version} + + + + org.jetbrains.exposed + exposed-spring-boot-starter + ${exposed.version} + + + + tech.ydb.jdbc + ydb-jdbc-driver + ${ydb.jdbc.version} + + + + org.springframework.boot + spring-boot-starter-test + test + + + + tech.ydb.test + ydb-junit5-support + ${ydb.sdk.version} + test + + + + org.jetbrains.kotlin + kotlin-reflect + ${kotlin.version} + test + + + + + + + org.jetbrains.kotlin + kotlin-maven-plugin + ${kotlin.version} + + + compile + compile + + compile + + + 17 + + src/main/kotlin + + + + + test-compile + test-compile + + test-compile + + + 17 + + src/test/kotlin + + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + 3.1.0 + + false + + true + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.5.0 + + 17 + + + + attach-javadocs + + jar + + + + + + + org.apache.maven.plugins + maven-source-plugin + 3.2.1 + + + attach-sources + + jar-no-fork + + + + + + + + + + ossrh-s01 + + false + + + + + + org.apache.maven.plugins + maven-gpg-plugin + 3.2.7 + + + sign-artifacts + verify + + sign + + + + + + --pinentry-mode + loopback + + + + + + org.sonatype.central + central-publishing-maven-plugin + 0.7.0 + true + + ossrh-s01 + + + + + + + diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/OnYdbJdbcUrlCondition.kt b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/OnYdbJdbcUrlCondition.kt new file mode 100644 index 00000000..0a47fc05 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/OnYdbJdbcUrlCondition.kt @@ -0,0 +1,19 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.springframework.context.annotation.Condition +import org.springframework.context.annotation.ConditionContext +import org.springframework.core.type.AnnotatedTypeMetadata + +internal class OnYdbJdbcUrlCondition : Condition { + + override fun matches(context: ConditionContext, metadata: AnnotatedTypeMetadata): Boolean { + val url = context.environment.getProperty(SPRING_DATASOURCE_URL_PROPERTY) + ?.trim() + ?: return false + + return url.startsWith(STARTER_YDB_JDBC_URL_PREFIX) + } +} + +internal const val SPRING_DATASOURCE_URL_PROPERTY = "spring.datasource.url" +internal const val SPRING_DATASOURCE_DRIVER_PROPERTY = "spring.datasource.driver-class-name" diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedAutoConfiguration.kt b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedAutoConfiguration.kt new file mode 100644 index 00000000..75e213fe --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedAutoConfiguration.kt @@ -0,0 +1,58 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.jetbrains.exposed.v1.core.DatabaseConfig +import org.jetbrains.exposed.v1.jdbc.Database +import org.jetbrains.exposed.v1.spring.boot.autoconfigure.ExposedAutoConfiguration +import org.springframework.beans.factory.InitializingBean +import org.springframework.boot.autoconfigure.AutoConfiguration +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean +import org.springframework.boot.autoconfigure.condition.ConditionalOnClass +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean +import org.springframework.boot.context.properties.EnableConfigurationProperties +import org.springframework.context.annotation.Bean +import org.springframework.context.annotation.Conditional +import org.springframework.context.annotation.DependsOn +import org.springframework.context.annotation.Primary +import tech.ydb.exposed.dialect.YdbDialect +import tech.ydb.exposed.dialect.registerYdbDialect +import javax.sql.DataSource + +/** + * Spring Boot bridge for the YDB Exposed dialect. + * + * The official Exposed Spring Boot starter knows how to integrate Exposed with Spring + * transactions, but it does not know anything about YDB dialect registration, YDB defaults, + * or the JDBC flag required for signed temporal mode. This auto-configuration layers those + * pieces on top as a separate optional artifact. + */ +@AutoConfiguration +@ConditionalOnClass(Database::class, YdbDialect::class, ExposedAutoConfiguration::class) +@Conditional(OnYdbJdbcUrlCondition::class) +@EnableConfigurationProperties(YdbExposedProperties::class) +class YdbExposedAutoConfiguration { + + @Bean + fun ydbDialectRegistration(properties: YdbExposedProperties): InitializingBean = InitializingBean { + registerYdbDialect(properties.enableSignedDatetimes) + } + + @Bean + @Primary + fun ydbDatabaseConfig(properties: YdbExposedProperties): DatabaseConfig = + ydbStarterDatabaseConfig(enableSignedDatetimes = properties.enableSignedDatetimes) + + @Bean + @DependsOn("ydbDialectRegistration") + @ConditionalOnMissingBean(Database::class) + fun database(dataSource: DataSource, ydbDatabaseConfig: DatabaseConfig): Database = + Database.connect( + datasource = dataSource, + databaseConfig = ydbDatabaseConfig + ) + + @Bean + @ConditionalOnBean(Database::class) + @ConditionalOnMissingBean + fun ydbTransactionOperations(database: Database): YdbTransactionOperations = + YdbTransactionOperations(database) +} diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedEnvironmentPostProcessor.kt b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedEnvironmentPostProcessor.kt new file mode 100644 index 00000000..e1830c3b --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedEnvironmentPostProcessor.kt @@ -0,0 +1,49 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.springframework.boot.SpringApplication +import org.springframework.boot.env.EnvironmentPostProcessor +import org.springframework.core.Ordered +import org.springframework.core.env.ConfigurableEnvironment +import org.springframework.core.env.MapPropertySource + +class YdbExposedEnvironmentPostProcessor : EnvironmentPostProcessor, Ordered { + + override fun getOrder(): Int = Ordered.LOWEST_PRECEDENCE + + override fun postProcessEnvironment( + environment: ConfigurableEnvironment, + application: SpringApplication + ) { + val rawUrl = environment.getProperty(SPRING_DATASOURCE_URL_PROPERTY) + ?.trim() + ?: return + + if (!rawUrl.startsWith(STARTER_YDB_JDBC_URL_PREFIX)) { + return + } + + val enableSignedDatetimes = environment.getProperty( + "spring.exposed.ydb.enable-signed-datetimes", + Boolean::class.java, + false + ) + + val overrides = linkedMapOf( + SPRING_DATASOURCE_URL_PROPERTY to ydbStarterJdbcUrl( + url = rawUrl, + enableSignedDatetimes = enableSignedDatetimes + ) + ) + + if (environment.getProperty(SPRING_DATASOURCE_DRIVER_PROPERTY).isNullOrBlank()) { + overrides[SPRING_DATASOURCE_DRIVER_PROPERTY] = STARTER_YDB_DRIVER_CLASS + } + + environment.propertySources.remove(PROPERTY_SOURCE_NAME) + environment.propertySources.addFirst(MapPropertySource(PROPERTY_SOURCE_NAME, overrides)) + } + + private companion object { + const val PROPERTY_SOURCE_NAME = "ydbExposedStarterOverrides" + } +} diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedProperties.kt b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedProperties.kt new file mode 100644 index 00000000..f8aca700 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedProperties.kt @@ -0,0 +1,11 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.springframework.boot.context.properties.ConfigurationProperties + +@ConfigurationProperties("spring.exposed.ydb") +data class YdbExposedProperties( + /** + * Enables signed temporal mode in both the Exposed dialect and the JDBC URL. + */ + var enableSignedDatetimes: Boolean = false +) diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbStarterSupport.kt b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbStarterSupport.kt new file mode 100644 index 00000000..00d6b266 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbStarterSupport.kt @@ -0,0 +1,54 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.jetbrains.exposed.v1.core.DatabaseConfig +import tech.ydb.exposed.dialect.YdbDialect +import java.sql.Connection + +internal const val STARTER_YDB_JDBC_URL_PREFIX = "jdbc:ydb:" +internal const val STARTER_YDB_DRIVER_CLASS = "tech.ydb.jdbc.YdbDriver" + +internal fun ydbStarterJdbcUrl( + url: String, + enableSignedDatetimes: Boolean = false +): String { + val trimmedUrl = url.trim() + val flag = "forceSignedDatetimes=$enableSignedDatetimes" + + val hashIndex = trimmedUrl.indexOf('#') + val baseUrl = if (hashIndex >= 0) trimmedUrl.substring(0, hashIndex) else trimmedUrl + val fragment = if (hashIndex >= 0) trimmedUrl.substring(hashIndex) else "" + + val queryIndex = baseUrl.indexOf('?') + val path = if (queryIndex >= 0) baseUrl.substring(0, queryIndex) else baseUrl + val query = if (queryIndex >= 0) baseUrl.substring(queryIndex + 1) else "" + + val params = query + .split('&') + .filter { it.isNotBlank() } + .filterNot { it.substringBefore('=') == "forceSignedDatetimes" } + .toMutableList() + + params += flag + + return buildString { + append(path) + append('?') + append(params.joinToString("&")) + append(fragment) + } +} + +internal fun ydbStarterDatabaseConfig( + enableSignedDatetimes: Boolean = false +): DatabaseConfig = DatabaseConfig { + explicitDialect = instantiateYdbDialect(enableSignedDatetimes) + defaultIsolationLevel = Connection.TRANSACTION_SERIALIZABLE + defaultReadOnly = false + useNestedTransactions = false +} + +private fun instantiateYdbDialect(enableSignedDatetimes: Boolean): YdbDialect { + val ctor = YdbDialect::class.java.getDeclaredConstructor(Boolean::class.javaPrimitiveType) + ctor.isAccessible = true + return ctor.newInstance(enableSignedDatetimes) +} diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbTransactionOperations.kt b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbTransactionOperations.kt new file mode 100644 index 00000000..a7dd9eaf --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbTransactionOperations.kt @@ -0,0 +1,22 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.jetbrains.exposed.v1.jdbc.Database +import org.jetbrains.exposed.v1.jdbc.JdbcTransaction +import tech.ydb.exposed.dialect.YdbRetryConfig +import tech.ydb.exposed.dialect.ydbTransaction + +class YdbTransactionOperations internal constructor( + private val database: Database +) { + + fun execute( + retry: YdbRetryConfig = YdbRetryConfig.DEFAULT, + readOnly: Boolean = false, + statement: JdbcTransaction.() -> T + ): T = ydbTransaction( + db = database, + retry = retry, + readOnly = readOnly, + statement = statement + ) +} diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/resources/META-INF/spring.factories b/kotlin-exposed-dialect/spring-boot-starter/src/main/resources/META-INF/spring.factories new file mode 100644 index 00000000..b4a72d0d --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/resources/META-INF/spring.factories @@ -0,0 +1,2 @@ +org.springframework.boot.env.EnvironmentPostProcessor=\ +tech.ydb.exposed.dialect.spring.boot.autoconfigure.YdbExposedEnvironmentPostProcessor diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/kotlin-exposed-dialect/spring-boot-starter/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports new file mode 100644 index 00000000..25800581 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports @@ -0,0 +1 @@ +tech.ydb.exposed.dialect.spring.boot.autoconfigure.YdbExposedAutoConfiguration diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedAutoConfigurationTest.kt b/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedAutoConfigurationTest.kt new file mode 100644 index 00000000..6c6f43cf --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedAutoConfigurationTest.kt @@ -0,0 +1,34 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertInstanceOf +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import tech.ydb.exposed.dialect.YdbDialect +import java.sql.Connection + +class YdbExposedAutoConfigurationTest { + + @Test + fun `creates database config aligned with YDB defaults`() { + val autoConfiguration = YdbExposedAutoConfiguration() + val databaseConfig = autoConfiguration.ydbDatabaseConfig(YdbExposedProperties()) + + assertInstanceOf(YdbDialect::class.java, databaseConfig.explicitDialect) + assertEquals(Connection.TRANSACTION_SERIALIZABLE, databaseConfig.defaultIsolationLevel) + assertFalse(databaseConfig.defaultReadOnly) + assertFalse(databaseConfig.useNestedTransactions) + } + + @Test + fun `propagates signed datetime mode into explicit dialect`() { + val autoConfiguration = YdbExposedAutoConfiguration() + val databaseConfig = autoConfiguration.ydbDatabaseConfig( + YdbExposedProperties(enableSignedDatetimes = true) + ) + val dialect = databaseConfig.explicitDialect as YdbDialect + + assertTrue(dialect.enableSignedDatetimes) + } +} diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedEnvironmentPostProcessorTest.kt b/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedEnvironmentPostProcessorTest.kt new file mode 100644 index 00000000..8590a31b --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbExposedEnvironmentPostProcessorTest.kt @@ -0,0 +1,54 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Test +import org.springframework.boot.SpringApplication +import org.springframework.mock.env.MockEnvironment + +class YdbExposedEnvironmentPostProcessorTest { + + private val postProcessor = YdbExposedEnvironmentPostProcessor() + + @Test + fun `normalizes ydb datasource url and default driver class`() { + val environment = MockEnvironment() + .withProperty(SPRING_DATASOURCE_URL_PROPERTY, "jdbc:ydb:grpc://localhost:2136/local") + + postProcessor.postProcessEnvironment(environment, SpringApplication()) + + assertEquals( + "jdbc:ydb:grpc://localhost:2136/local?forceSignedDatetimes=false", + environment.getProperty(SPRING_DATASOURCE_URL_PROPERTY) + ) + assertEquals( + "tech.ydb.jdbc.YdbDriver", + environment.getProperty(SPRING_DATASOURCE_DRIVER_PROPERTY) + ) + } + + @Test + fun `propagates signed temporal mode into datasource url`() { + val environment = MockEnvironment() + .withProperty(SPRING_DATASOURCE_URL_PROPERTY, "jdbc:ydb:grpc://localhost:2136/local?token=abc") + .withProperty("spring.exposed.ydb.enable-signed-datetimes", "true") + + postProcessor.postProcessEnvironment(environment, SpringApplication()) + + assertEquals( + "jdbc:ydb:grpc://localhost:2136/local?token=abc&forceSignedDatetimes=true", + environment.getProperty(SPRING_DATASOURCE_URL_PROPERTY) + ) + } + + @Test + fun `does not touch non ydb datasource`() { + val environment = MockEnvironment() + .withProperty(SPRING_DATASOURCE_URL_PROPERTY, "jdbc:h2:mem:testdb") + + postProcessor.postProcessEnvironment(environment, SpringApplication()) + + assertEquals("jdbc:h2:mem:testdb", environment.getProperty(SPRING_DATASOURCE_URL_PROPERTY)) + assertNull(environment.getProperty(SPRING_DATASOURCE_DRIVER_PROPERTY)) + } +} diff --git a/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbSpringBootContextTest.kt b/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbSpringBootContextTest.kt new file mode 100644 index 00000000..83885cc7 --- /dev/null +++ b/kotlin-exposed-dialect/spring-boot-starter/src/test/kotlin/tech/ydb/exposed/dialect/spring/boot/autoconfigure/YdbSpringBootContextTest.kt @@ -0,0 +1,134 @@ +package tech.ydb.exposed.dialect.spring.boot.autoconfigure + +import org.jetbrains.exposed.v1.core.DatabaseConfig +import org.jetbrains.exposed.v1.core.Table +import org.jetbrains.exposed.v1.jdbc.Database +import org.jetbrains.exposed.v1.jdbc.SchemaUtils +import org.jetbrains.exposed.v1.jdbc.insert +import org.jetbrains.exposed.v1.jdbc.selectAll +import org.junit.jupiter.api.AfterEach +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertInstanceOf +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import org.junit.jupiter.api.extension.RegisterExtension +import org.springframework.beans.factory.annotation.Autowired +import org.springframework.boot.autoconfigure.SpringBootApplication +import org.springframework.boot.test.context.SpringBootTest +import org.springframework.core.env.Environment +import org.springframework.test.context.DynamicPropertyRegistry +import org.springframework.test.context.DynamicPropertySource +import tech.ydb.exposed.dialect.YdbDialect +import tech.ydb.exposed.dialect.createYdbStatement +import tech.ydb.test.junit5.YdbHelperExtension +import java.sql.Connection + +@SpringBootTest( + classes = [YdbSpringBootContextTest.TestApplication::class], + properties = [ + "spring.exposed.ydb.enable-signed-datetimes=true" + ] +) +class YdbSpringBootContextTest { + + object SpringBootTable : Table("spring_boot_starter_table") { + val id = integer("id") + val name = varchar("name", 128) + + override val primaryKey = PrimaryKey(id) + override fun createStatement(): List = createYdbStatement() + } + + @Autowired + private lateinit var database: Database + + @Autowired + private lateinit var databaseConfig: DatabaseConfig + + @Autowired + private lateinit var environment: Environment + + @Autowired + private lateinit var ydbTx: YdbTransactionOperations + + @AfterEach + fun tearDown() { + if (!::ydbTx.isInitialized) return + + runCatching { + ydbTx.execute { + SchemaUtils.drop(SpringBootTable) + } + } + } + + @Test + fun `spring boot context wires YDB database and config`() { + val dialect = assertInstanceOf(YdbDialect::class.java, database.dialect) + assertTrue(dialect.enableSignedDatetimes) + assertEquals(Connection.TRANSACTION_SERIALIZABLE, databaseConfig.defaultIsolationLevel) + assertFalse(databaseConfig.useNestedTransactions) + assertTrue( + environment.getProperty("spring.datasource.url") + ?.contains("forceSignedDatetimes=true") == true + ) + } + + @Test + fun `spring boot context can run CRUD through YdbTransactionOperations`() { + ydbTx.execute { + runCatching { SchemaUtils.drop(SpringBootTable) } + SchemaUtils.create(SpringBootTable) + + SpringBootTable.insert { + it[id] = 1 + it[name] = "spring-boot" + } + } + + val actual = ydbTx.execute(readOnly = true) { + SpringBootTable.selectAll().single()[SpringBootTable.name] + } + + assertEquals("spring-boot", actual) + } + + @SpringBootApplication + class TestApplication + + companion object { + @JvmField + @RegisterExtension + val ydb: YdbHelperExtension = YdbHelperExtension() + + @JvmStatic + @DynamicPropertySource + fun springDatasourceProperties(registry: DynamicPropertyRegistry) { + registry.add("spring.datasource.url") { + ydbStarterJdbcUrl( + url = buildJdbcUrl(), + enableSignedDatetimes = true + ) + } + registry.add("spring.datasource.driver-class-name") { "tech.ydb.jdbc.YdbDriver" } + } + + private fun buildJdbcUrl(): String = buildString { + append("jdbc:ydb:") + append(if (ydb.useTls()) "grpcs://" else "grpc://") + append(ydb.endpoint()) + append(ydb.database()) + + val params = mutableListOf( + "disablePrepareDataQuery=true", + "disableAutoPreparedBatches=true" + ) + + ydb.authToken()?.let { params += "token=$it" } + + append("?") + append(params.joinToString("&")) + } + } +} diff --git a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbDialect.kt b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbDialect.kt index 30dc66f1..7197182e 100644 --- a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbDialect.kt +++ b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbDialect.kt @@ -358,7 +358,8 @@ internal object YdbFunctionProvider : FunctionProvider() { * Use [registerYdbDialect] then `Database.connect("jdbc:ydb:...")`. * * Notable behavior: - * - [YdbTable] — YQL `CREATE TABLE` with table-level PK, inline indexes, TTL. + * - [createYdbStatement] — helper for YQL `CREATE TABLE` with a table-level PK, + * intended for `Table.createStatement()` overrides. * - [tech.ydb.exposed.dialect.YdbFunctionProvider.upsert] / [replace] → native YQL `UPSERT` / `REPLACE` * (`onUpdate` / `keyColumns` ignored). * - [createIndex] → `ALTER TABLE ... ADD INDEX ... GLOBAL` (post-create indexes). diff --git a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbSecondaryIndex.kt b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbSecondaryIndex.kt new file mode 100644 index 00000000..5c3258d9 --- /dev/null +++ b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbSecondaryIndex.kt @@ -0,0 +1,104 @@ +/** + * YDB secondary index declarations for [YdbTable.secondaryIndex]. + * + * Rendered into `CREATE TABLE` as `INDEX name GLOBAL [UNIQUE] [ASYNC] ON (...) [COVER (...)] [WITH (...)]`. + */ +package tech.ydb.exposed.dialect + +import org.jetbrains.exposed.v1.core.Column +import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager + +/** Index visibility scope in YQL (`GLOBAL` for row-oriented tables). */ +enum class YdbIndexScope { + GLOBAL +} + +/** Whether the index is built synchronously with writes or in the background. */ +enum class YdbIndexSyncMode { + SYNC, + ASYNC +} + +/** Internal model produced by [YdbTable.secondaryIndex] and rendered by [renderYdbSecondaryIndex]. */ +data class YdbSecondaryIndexSpec( + val name: String, + val columns: List>, + val unique: Boolean = false, + val scope: YdbIndexScope = YdbIndexScope.GLOBAL, + val syncMode: YdbIndexSyncMode = YdbIndexSyncMode.SYNC, + val indexType: String? = null, + val coverColumns: List> = emptyList(), + val withParams: Map = emptyMap() +) + +/** Builds the `INDEX ...` fragment for a single secondary index inside `CREATE TABLE`. */ +internal fun renderYdbSecondaryIndex(spec: YdbSecondaryIndexSpec): String { + val tr = TransactionManager.current() + + require(spec.columns.isNotEmpty()) { + "YDB secondary index must contain at least one column" + } + + require(spec.name.isNotBlank()) { + "YDB secondary index name must not be blank" + } + + require(spec.scope == YdbIndexScope.GLOBAL) { + "Only GLOBAL secondary indexes are supported by YDB row-oriented tables in this dialect" + } + + val indexName = tr.db.identifierManager.cutIfNecessaryAndQuote(spec.name) + + val columnsSql = spec.columns.joinToString(", ") { tr.identity(it) } + val coverSql = spec.coverColumns + .takeIf { it.isNotEmpty() } + ?.joinToString(", ") { tr.identity(it) } + + val withSql = spec.withParams + .takeIf { it.isNotEmpty() } + ?.entries + ?.joinToString(", ") { (k, v) -> "$k = ${renderYdbIndexParamValue(v)}" } + + return buildString { + append("INDEX ") + append(indexName) + append(" ") + append(spec.scope.name) + + if (spec.unique) { + append(" UNIQUE") + } + + if (spec.syncMode != YdbIndexSyncMode.SYNC) { + append(" ") + append(spec.syncMode.name) + } + + if (spec.indexType != null) { + append(" USING ") + append(spec.indexType) + } + + append(" ON (") + append(columnsSql) + append(")") + + if (coverSql != null) { + append(" COVER (") + append(coverSql) + append(")") + } + + if (withSql != null) { + append(" WITH (") + append(withSql) + append(")") + } + } +} + +private fun renderYdbIndexParamValue(value: Any): String = when (value) { + is Number -> value.toString() + is Boolean -> value.toString().uppercase() + else -> "\"${value.toString().replace("\"", "\\\"")}\"" +} diff --git a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbTable.kt b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbTable.kt new file mode 100644 index 00000000..70ee57b2 --- /dev/null +++ b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbTable.kt @@ -0,0 +1,151 @@ +package tech.ydb.exposed.dialect + +import org.jetbrains.exposed.v1.core.Column +import org.jetbrains.exposed.v1.core.Table +import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager +import java.time.Duration + +/** + * Base class for YDB row-oriented tables. + * + * Adds YDB-specific DDL on top of Exposed [Table]: + * - [ttl] — TTL on a date/numeric column; + * - [secondaryIndex] — YDB secondary index with COVER / ASYNC / WITH params. + */ +open class YdbTable(name: String = "") : Table(name) { + private var ttlSettings: YdbTtlSettings? = null + private val secondaryIndices = mutableListOf() + + /** + * Declares row TTL on [column] (embedded in `CREATE TABLE ... WITH (TTL = ...)`). + * + * @param intervalIso8601 ISO-8601 duration (e.g. `P30D`, `PT1H`). + * @param mode How [column] is interpreted — date/timestamp types vs numeric epoch units. + */ + fun ttl( + column: Column<*>, + intervalIso8601: String, + mode: YdbTtlColumnMode = YdbTtlColumnMode.DATE_TYPE + ) { + ttlSettings = YdbTtlSettings(column, normalizeTtlInterval(intervalIso8601), mode) + } + + /** + * Declares a YDB secondary index inline in `CREATE TABLE` (not Exposed's generic [Index] DSL). + * + * @param scope Currently only [YdbIndexScope.GLOBAL] is supported for row tables. + * @param syncMode [YdbIndexSyncMode.ASYNC] for background index build. + * @param indexType Optional `USING` clause (e.g. vector index type when supported by YDB). + * @param coverColumns Included columns for covering index (`COVER (...)`). + * @param withParams Index-level `WITH (key = value)` parameters. + */ + fun secondaryIndex( + name: String, + vararg columns: Column<*>, + unique: Boolean = false, + scope: YdbIndexScope = YdbIndexScope.GLOBAL, + syncMode: YdbIndexSyncMode = YdbIndexSyncMode.SYNC, + indexType: String? = null, + coverColumns: List> = emptyList(), + withParams: Map = emptyMap() + ) { + require(columns.isNotEmpty()) { "YDB secondary index must contain at least one column" } + + secondaryIndices += YdbSecondaryIndexSpec( + name = name, + columns = columns.toList(), + unique = unique, + scope = scope, + syncMode = syncMode, + indexType = indexType, + coverColumns = coverColumns, + withParams = withParams + ) + } + + override fun createStatement(): List { + val tr = TransactionManager.current() + + val pk = primaryKey + ?: error("YDB requires PRIMARY KEY for every table: $tableName") + + val columnsSql = columns.joinToString(", ") { column -> + buildString { + append(tr.identity(column)) + append(" ") + append(column.columnType.sqlType()) + if (!column.columnType.nullable) { + append(" NOT NULL") + } + } + } + + val indexesSql = secondaryIndices.joinToString(", ") { renderYdbSecondaryIndex(it) } + val pkSql = pk.columns.joinToString(", ") { tr.identity(it) } + + val ttlSql = ttlSettings?.let { ttl -> + validateTtlColumn(ttl) + buildString { + append(" WITH (TTL = Interval(\"") + append(escapeYqlDoubleQuotedLiteral(ttl.intervalIso8601)) + append("\") ON ") + append(tr.identity(ttl.column)) + ttl.mode.toSql()?.let { + append(" AS ") + append(it) + } + append(")") + } + }.orEmpty() + + val sql = buildString { + append("CREATE TABLE IF NOT EXISTS ") + append(tr.identity(this@YdbTable)) + append(" (") + append(columnsSql) + if (indexesSql.isNotEmpty()) { + append(", ") + append(indexesSql) + } + append(", PRIMARY KEY (") + append(pkSql) + append("))") + append(ttlSql) + } + + return listOf(sql) + } +} + +private fun normalizeTtlInterval(intervalIso8601: String): String = + runCatching { Duration.parse(intervalIso8601).toString() } + .getOrElse { cause -> + throw IllegalArgumentException("Invalid YDB TTL interval: '$intervalIso8601'", cause) + } + +private fun validateTtlColumn(ttl: YdbTtlSettings) { + val sqlType = ttl.column.columnType.sqlType() + + val supported = when (ttl.mode) { + YdbTtlColumnMode.DATE_TYPE -> + sqlType == "Date" || + sqlType == "Date32" || + sqlType == "Datetime" || + sqlType == "Datetime64" || + sqlType == "Timestamp" || + sqlType == "Timestamp64" + + YdbTtlColumnMode.SECONDS, + YdbTtlColumnMode.MILLISECONDS, + YdbTtlColumnMode.MICROSECONDS, + YdbTtlColumnMode.NANOSECONDS -> + sqlType == "Uint32" || sqlType == "Uint64" || sqlType == "DyNumber" + } + + require(supported) { + "YDB TTL does not support column '${ttl.column.name}' of type '$sqlType' for mode '${ttl.mode}'" + } +} + +private fun escapeYqlDoubleQuotedLiteral(value: String): String = + value.replace("\\", "\\\\").replace("\"", "\\\"") diff --git a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbTtl.kt b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbTtl.kt new file mode 100644 index 00000000..5698984b --- /dev/null +++ b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/YdbTtl.kt @@ -0,0 +1,34 @@ +/** + * TTL settings for [YdbTable.ttl] — see YDB docs on `WITH (TTL = Interval(...) ON ...)`. + */ +package tech.ydb.exposed.dialect + +import org.jetbrains.exposed.v1.core.Column + +/** How the TTL source [Column] value is interpreted in YQL. */ +enum class YdbTtlColumnMode { + /** `Date` / `Datetime` / `Timestamp` (and `*32`/`*64` variants) columns. */ + DATE_TYPE, + /** `Uint32` / `Uint64` / `DyNumber` interpreted as Unix seconds. */ + SECONDS, + MILLISECONDS, + MICROSECONDS, + NANOSECONDS; + + fun toSql(): String? = when (this) { + DATE_TYPE -> null + SECONDS -> "SECONDS" + MILLISECONDS -> "MILLISECONDS" + MICROSECONDS -> "MICROSECONDS" + NANOSECONDS -> "NANOSECONDS" + } +} + +/** Resolved TTL clause for [YdbTable] DDL generation. */ +data class YdbTtlSettings( + /** Column whose value drives expiration. */ + val column: Column<*>, + /** ISO-8601 duration string passed to `Interval("...")`. */ + val intervalIso8601: String, + val mode: YdbTtlColumnMode = YdbTtlColumnMode.DATE_TYPE +) \ No newline at end of file diff --git a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt index 856c32a7..87b7672f 100644 --- a/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt +++ b/kotlin-exposed-dialect/src/main/kotlin/tech/ydb/exposed/dialect/createYdbStatement.kt @@ -28,7 +28,6 @@ import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager * id Int32 NOT NULL, * email Utf8 NOT NULL, * name Utf8 NOT NULL, - * INDEX ..., * PRIMARY KEY (id) * ) * ``` @@ -39,12 +38,13 @@ import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager * - appending a table-level `PRIMARY KEY (...)` clause; * - preserving `NOT NULL` for non-nullable columns; * - preserving database-side default values; - * - appending Exposed indexes declared on this table. + * - preserving Exposed [Table.storageParameters], so YDB-specific `WITH (...)` + * clauses can still be declared on the table. * - * The generated statement is plain `CREATE TABLE`, not - * `CREATE TABLE IF NOT EXISTS`. This is useful for migration tools, because an - * already existing table should normally fail the migration instead of being - * silently ignored. + * The generated statement uses `CREATE TABLE IF NOT EXISTS`, matching + * Exposed's schema-creation flow. Post-create indexes declared through + * Exposed [Table.index] are still emitted separately through the dialect's + * `createIndex(...)` path. * * Example: * @@ -56,14 +56,6 @@ import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager * * override val primaryKey = PrimaryKey(id) * - * init { - * index(false, email) - * index("email-cover-idx", isUnique = true, email) - * } - * - * val emailIndexDefinition - * get() = indices.single { !it.unique && it.columns == listOf(email) } - * * override fun createStatement() = createYdbStatement() * } * ``` @@ -117,4 +109,4 @@ fun Table.createYdbStatement(): List { } return listOf(sql) -} \ No newline at end of file +} diff --git a/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/CreateYdbStatementIT.kt b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/CreateYdbStatementIT.kt new file mode 100644 index 00000000..16a971a7 --- /dev/null +++ b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/CreateYdbStatementIT.kt @@ -0,0 +1,88 @@ +package tech.ydb.exposed.dialect.integration.basic + +import org.jetbrains.exposed.v1.core.Table +import org.junit.jupiter.api.Assertions.assertThrows +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import tech.ydb.exposed.dialect.createYdbStatement +import tech.ydb.exposed.dialect.integration.base.BaseYdbTest +import tech.ydb.exposed.dialect.javatime.ydbTimestamp64 +import tech.ydb.exposed.dialect.ydbUint64 + +class CreateYdbStatementIT : BaseYdbTest() { + + object BasicTable : Table("unit_basic_table") { + val id = integer("id") + val name = varchar("name", 255) + + override val primaryKey = PrimaryKey(id) + + override fun createStatement() = createYdbStatement() + } + + object TtlTimestampTable : Table("unit_ttl_timestamp_table") { + val id = integer("id") + val expireAt = ydbTimestamp64("expire_at") + + override val storageParameters: List = + listOf(RawTableStorageParameter("TTL = Interval(\"PT1H\") ON expire_at")) + + override val primaryKey = PrimaryKey(id) + + override fun createStatement() = createYdbStatement() + } + + object TtlNumericTable : Table("unit_ttl_numeric_table") { + val id = integer("id") + val modifiedAtEpoch = ydbUint64("modified_at_epoch") + + override val storageParameters: List = + listOf(RawTableStorageParameter("TTL = Interval(\"PT1H\") ON modified_at_epoch AS SECONDS")) + + override val primaryKey = PrimaryKey(id) + + override fun createStatement() = createYdbStatement() + } + + object NoPkTable : Table("unit_no_pk_table") { + val id = integer("id") + val name = varchar("name", 255) + + override fun createStatement() = createYdbStatement() + } + + @Test + fun `renders CREATE TABLE with primary key`() = tx { + val ddl = BasicTable.ddl.joinToString(" ") + assertTrue(ddl.contains("CREATE TABLE IF NOT EXISTS"), ddl) + assertTrue( + ddl.contains("PRIMARY KEY (id)") || ddl.contains("PRIMARY KEY (`id`)"), + ddl + ) + assertTrue( + ddl.contains("`name` Text") || ddl.contains("name Text"), + ddl + ) + } + + @Test + fun `renders TTL clause for a Timestamp64 column`() = tx { + val ddl = TtlTimestampTable.ddl.joinToString(" ") + assertTrue(ddl.contains("""WITH (TTL = Interval("PT1H") ON expire_at)"""), ddl) + assertTrue(ddl.contains("Timestamp64"), ddl) + } + + @Test + fun `renders TTL clause for a numeric epoch column`() = tx { + val ddl = TtlNumericTable.ddl.joinToString(" ") + assertTrue(ddl.contains("""WITH (TTL = Interval("PT1H") ON modified_at_epoch AS SECONDS)"""), ddl) + assertTrue(ddl.contains("modified_at_epoch Uint64"), ddl) + } + + @Test + fun `fails when the table has no primary key`() = tx { + assertThrows(IllegalStateException::class.java) { + NoPkTable.ddl + } + } +} diff --git a/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/YdbTableIT.kt b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/YdbTableIT.kt index 4f6ac4ac..1e7f8306 100644 --- a/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/YdbTableIT.kt +++ b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/basic/YdbTableIT.kt @@ -1,56 +1,59 @@ package tech.ydb.exposed.dialect.integration.basic -import org.jetbrains.exposed.v1.core.Table -import org.jetbrains.exposed.v1.jdbc.SchemaUtils import org.junit.jupiter.api.Assertions.assertThrows import org.junit.jupiter.api.Assertions.assertTrue import org.junit.jupiter.api.Test -import tech.ydb.exposed.dialect.createYdbStatement +import tech.ydb.exposed.dialect.YdbTable +import tech.ydb.exposed.dialect.YdbTtlColumnMode import tech.ydb.exposed.dialect.integration.base.BaseYdbTest import tech.ydb.exposed.dialect.ydbUint64 import tech.ydb.exposed.dialect.javatime.ydbTimestamp64 -import java.sql.SQLException class YdbTableIT : BaseYdbTest() { - object BasicTable : Table("unit_basic_table") { + object BasicTable : YdbTable("unit_basic_table") { val id = integer("id") val name = varchar("name", 255) override val primaryKey = PrimaryKey(id) - - override fun createStatement() = createYdbStatement() } - object TtlTimestampTable : Table("unit_ttl_timestamp_table") { + object TtlTimestampTable : YdbTable("unit_ttl_timestamp_table") { val id = integer("id") val expireAt = ydbTimestamp64("expire_at") - override val storageParameters: List = - listOf(RawTableStorageParameter("TTL = Interval(\"PT1H\") ON expire_at")) - override val primaryKey = PrimaryKey(id) - override fun createStatement() = createYdbStatement() + init { + ttl(expireAt, "PT1H") + } } - object TtlNumericTable : Table("unit_ttl_numeric_table") { + object TtlNumericTable : YdbTable("unit_ttl_numeric_table") { val id = integer("id") val modifiedAtEpoch = ydbUint64("modified_at_epoch") - override val storageParameters: List = - listOf(RawTableStorageParameter("TTL = Interval(\"PT1H\") ON modified_at_epoch AS SECONDS")) - override val primaryKey = PrimaryKey(id) - override fun createStatement() = createYdbStatement() + init { + ttl(modifiedAtEpoch, "PT1H", YdbTtlColumnMode.SECONDS) + } } - object NoPkTable : Table("unit_no_pk_table") { + object NoPkTable : YdbTable("unit_no_pk_table") { val id = integer("id") val name = varchar("name", 255) + } + + object InvalidNumericTtlTable : YdbTable("unit_invalid_numeric_ttl_table") { + val id = integer("id") + val modifiedAtEpoch = integer("modified_at_epoch") + + override val primaryKey = PrimaryKey(id) - override fun createStatement() = createYdbStatement() + init { + ttl(modifiedAtEpoch, "PT1H", YdbTtlColumnMode.SECONDS) + } } @Test @@ -87,4 +90,29 @@ class YdbTableIT : BaseYdbTest() { NoPkTable.ddl } } + + @Test + fun `fails when numeric TTL column type is unsupported`() = tx { + assertThrows(IllegalArgumentException::class.java) { + InvalidNumericTtlTable.ddl + } + } + + @Test + fun `rejects invalid TTL interval early`() { + val error = assertThrows(IllegalArgumentException::class.java) { + object : YdbTable("invalid_ttl_interval_table") { + val id = integer("id") + val expireAt = ydbTimestamp64("expire_at") + + override val primaryKey = PrimaryKey(id) + + init { + ttl(expireAt, """PT1H" ON hacked""") + } + } + } + + assertTrue(error.message?.contains("Invalid YDB TTL interval") == true, error.message) + } } diff --git a/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/ddl/MigrationValidationIT.kt b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/ddl/MigrationValidationIT.kt new file mode 100644 index 00000000..27c1bc23 --- /dev/null +++ b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/integration/ddl/MigrationValidationIT.kt @@ -0,0 +1,144 @@ +package tech.ydb.exposed.dialect.integration.ddl + +import org.jetbrains.exposed.v1.core.Table +import org.jetbrains.exposed.v1.jdbc.SchemaUtils +import org.jetbrains.exposed.v1.jdbc.transactions.transaction +import org.junit.jupiter.api.AfterEach +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import tech.ydb.exposed.dialect.createYdbStatement +import tech.ydb.exposed.dialect.integration.base.BaseYdbTest + +/** + * Migration validation for externally managed schemas. + * + * In production the schema is usually created by versioned migrations, while Exposed is used to + * validate that the current database layout still matches the application's table model. + * + * Exposed 1.3.0 provides [org.jetbrains.exposed.v1.migration.jdbc.MigrationUtils], but its full + * JDBC diff path unconditionally queries CHECK-constraint metadata from INFORMATION_SCHEMA. + * YDB does not expose that metadata through the JDBC driver, so for YDB we validate drift through + * the compatible schema-diff primitives that Exposed already exposes: + * - [SchemaUtils.addMissingColumnsStatements] for missing columns; + * - [tech.ydb.exposed.dialect.YdbDialectMetadata] for existing secondary indexes. + * + * We intentionally apply schema changes through raw SQL first and only then ask Exposed for the + * statements that would be needed to bring the database schema back in sync with the table model. + */ +class MigrationValidationIT : BaseYdbTest() { + + object ExternalUsers : Table("external_users") { + val id = integer("id") + val name = varchar("name", 255) + val email = varchar("email", 255) + + override val primaryKey = PrimaryKey(id) + + init { + index(false, email) + } + + + override fun createStatement(): List = createYdbStatement() + } + + @AfterEach + fun tearDown() { + transaction(db) { + runCatching { SchemaUtils.drop(ExternalUsers) } + } + } + + @Test + fun `ydb-compatible schema validation returns no statements for a matching externally created schema`() { + applySchemaManually( + """ + CREATE TABLE IF NOT EXISTS external_users ( + id Int32 NOT NULL, + name Text NOT NULL, + email Text NOT NULL, + PRIMARY KEY (id) + ) + """.trimIndent(), + "ALTER TABLE external_users ADD INDEX external_users_email GLOBAL ON (email)" + ) + + val statements = requiredMigrationStatements() + + assertTrue(statements.isEmpty(), statements.joinToString(separator = "\n")) + } + + @Test + fun `ydb-compatible schema validation reports drift for an incomplete externally created schema`() { + applySchemaManually( + """ + CREATE TABLE IF NOT EXISTS external_users ( + id Int32 NOT NULL, + name Text NOT NULL, + PRIMARY KEY (id) + ) + """.trimIndent() + ) + + val statements = requiredMigrationStatements() + + assertTrue(statements.isNotEmpty(), "Expected schema drift to be reported") + assertTrue( + statements.any { statement -> + statement.contains("email", ignoreCase = true) || + statement.contains("ADD INDEX", ignoreCase = true) + }, + statements.joinToString(separator = "\n") + ) + } + + /** + * JetBrains Exposed warns that schema changes performed through raw SQL should be validated + * in a fresh transaction to avoid stale metadata cache. + */ + private fun applySchemaManually(vararg statements: String) { + transaction(db) { + runCatching { SchemaUtils.drop(ExternalUsers) } + statements.forEach { statement -> + exec(statement) + } + } + } + + private fun requiredMigrationStatements(): List = + transaction(db) { + buildList { + addAll(missingColumnStatements()) + addAll(missingIndexStatements()) + } + } + + private fun missingColumnStatements(): List { + val existingColumnNames = db.dialectMetadata + .tableColumns(ExternalUsers) + .getValue(ExternalUsers) + .map { metadata -> metadata.name.normalizedIdentifier() } + .toSet() + + return ExternalUsers.columns + .filterNot { column -> column.name.normalizedIdentifier() in existingColumnNames } + .map { column -> "MISSING COLUMN ${ExternalUsers.tableName}.${column.name}" } + } + + private fun missingIndexStatements(): List { + val existingIndexNames = db.dialectMetadata + .existingIndices(ExternalUsers) + .getValue(ExternalUsers) + .map { index -> index.indexName.normalizedIdentifier() } + .toSet() + + return ExternalUsers.indices + .filterNot { index -> index.indexName.normalizedIdentifier() in existingIndexNames } + .map { index -> "MISSING INDEX ${index.indexName}" } + } + + private fun String.normalizedIdentifier(): String = trim('`', '"').lowercase() +} + + + diff --git a/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/unit/ttl/YdbTtlColumnModeTest.kt b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/unit/ttl/YdbTtlColumnModeTest.kt new file mode 100644 index 00000000..49c1fe94 --- /dev/null +++ b/kotlin-exposed-dialect/src/test/kotlin/tech/ydb/exposed/dialect/unit/ttl/YdbTtlColumnModeTest.kt @@ -0,0 +1,22 @@ +package tech.ydb.exposed.dialect.unit.ttl + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Test +import tech.ydb.exposed.dialect.YdbTtlColumnMode + +class YdbTtlColumnModeTest { + + @Test + fun `should map date type mode to null suffix`() { + assertNull(YdbTtlColumnMode.DATE_TYPE.toSql()) + } + + @Test + fun `should map numeric ttl modes to sql suffix`() { + assertEquals("SECONDS", YdbTtlColumnMode.SECONDS.toSql()) + assertEquals("MILLISECONDS", YdbTtlColumnMode.MILLISECONDS.toSql()) + assertEquals("MICROSECONDS", YdbTtlColumnMode.MICROSECONDS.toSql()) + assertEquals("NANOSECONDS", YdbTtlColumnMode.NANOSECONDS.toSql()) + } +} \ No newline at end of file