Add SQL Server (mssql2019) persistence plugin#11063
Conversation
Implements a new SQL plugin backed by github.com/microsoft/go-mssqldb supporting both the default persistence store and the visibility store on SQL Server 2019+ / Azure SQL: - Plugin package with per-table query implementations translated from the postgresql plugin: MERGE WITH (HOLDLOCK) upserts (executed per-row for slices, since sqlx cannot expand MERGE), UPDLOCK/REPEATABLEREAD table hints for locking reads, TOP/OFFSET-FETCH pagination, expanded row-value tuple comparisons, and central ?->@pn rebinding. - Session layer with sqlserver:// DSNs, passwordCommand support via RefreshingConnector, and TLS-to-DSN-parameter mapping. - T-SQL schemas for main store (37 tables) and single-table visibility with PERSISTED computed search-attribute columns, a persisted coalesce_close_time ordering column, SHA2_256 hash PK for chasm_node_maps (900-byte key limit), and full-text search over Text attributes via a row_key IDENTITY surrogate key. - Unified and legacy visibility query converters (OPENJSON for KeywordList, CONTAINS for Text, BIT-safe boolean literal rewriting via a new optional pluginBoolValueConverterLegacy hook). - Admin operations issue single-statement batches and classify Azure SQL transient fault codes for Azure SQL Database compatibility.
- Blank-import the mssql plugin in the server and temporal-sql-tool binaries so init() registration runs. - Include schema/mssql in PathsByDB so --schema-name validation accepts the embedded mssql schema directories. - Add mssql2019 to AllowListForValidation (SQL visibility stores do not support list-of-values custom search attributes) and to the functional-test UseSQLVisibility driver switch. - Add GetMSSQLTestClusterOption and MSSQL_SEEDS/MSSQL_PORT environment helpers for the persistence test harness.
- SQL Server 2022 Developer container (includes Full-Text Search, which the mssql visibility schema requires) in the dev docker-compose. - make install-schema-mssql2019 target with MSSQL_USER/MSSQL_PASSWORD variables matching the container's SA credentials. - config/development-mssql.yaml mirroring the other development-*.yaml configs for running the server against local SQL Server.
Found by applying the schema to SQL Server 2022 in Docker:
- Remove the IF NOT EXISTS guards from the full-text catalog/index:
Temporal's schema statement splitter parses IF as a PL/pgSQL block
needing END IF, so T-SQL IF guards produce 'unmatched i'. To re-apply,
drop and recreate the database instead of setup-schema --overwrite.
- Make coalesce_close_time deterministic so it can be PERSISTED:
CAST('...' AS DATETIME2) is non-deterministic (depends on session
language); use CONVERT(DATETIME2(6), '...', 126) (ISO 8601) instead.
- Document that the stock SQL Server container does not bundle Full-Text
Search and must have mssql-server-fts installed.
Verified end to end: schema applies cleanly, the server boots on
mssql2019, and a workflow runs to completion and is returned by
visibility list and count queries.
Mirror the postgresql test entry files so the shared persistence suites run against SQL Server via -run TestMSSQL. Requires a running SQL Server with mssql-server-fts installed (see docker-compose). All 44 suites pass against SQL Server 2022: 521 subtests, 0 failures — covering shard/execution/history/task/queue/metadata/cluster-metadata CRUD, every per-table history suite, and the full visibility filter, count, group-by and pagination paths.
|
Note for maintainers on the failing Happy to adjust whatever the internal run actually flags — and to add a public CI matrix entry for SQL Server if you'd like this exercised in this repo's CI as well. |
What was changed
Adds a Microsoft SQL Server persistence plugin (
mssql2019, SQL Server 2019+ / Azure SQL) that can serve both the default persistence store and the visibility store, following the existing mysql/postgresql/sqlite plugin architecture (registered viasql.RegisterPlugin, selected withpluginName: "mssql2019").common/persistence/sql/sqlplugin/mssql/): per-table query implementations translated from the postgresql plugin —MERGE ... WITH (HOLDLOCK)upserts (executed per row for slices, since sqlx cannot expand MERGE),UPDLOCK/REPEATABLEREADlocking hints,TOP/OFFSET-FETCHpagination, expanded row-value tuple comparisons, central?→@pNrebinding,passwordCommandsupport viaRefreshingConnector, TLS-to-DSN mapping for go-mssqldb.schema/mssql/v2019/): all 37 main-store tables; single-table visibility (postgres layout) with PERSISTED computed search-attribute columns, a persistedcoalesce_close_timeordering column, an SHA2_256 hash PK forchasm_node_maps(SQL Server's 900-byte index key limit), and full-text search over Text attributes via arow_key IDENTITYsurrogate key.CONTAINS()for Text). T-SQL has no boolean literals, so a small optional hook (pluginBoolValueConverterLegacy) was added to the legacy converter to rewritetrue/false→1/0for BIT columns; the unified path uses the existingConvertComparisonExprhook.schema.PathsByDB,AllowListForValidation, persistence test harness wiring, docker-compose service,install-schema-mssql2019Makefile target,development-mssql.yaml.github.com/microsoft/go-mssqldb v1.10.0(registered with sqlx as"sqlserver").Why
SQL Server is the remaining major relational database without Temporal support. Organizations standardized on the Microsoft stack (licensing, DBA tooling, compliance) currently have no supported way to run Temporal on the database they operate, on-prem or on Azure SQL.
How it was tested
All against a real SQL Server 2022 instance (Docker, with
mssql-server-fts):temporal-sql-tool(setup-schema + update-schema).mssql2019for both stores: all shards acquired, system namespace/workflows running, no driver errors.ListWorkflowExecutions(includingWorkflowType = '...' AND ExecutionStatus = 'Completed') andCountWorkflowExecutionsreturn correct results.TestMSSQLentry (common/persistence/tests/mssql_test.go), covering shard, execution mutable state (+tasks), every per-table history suite, matching tasks v1/v2, task queues, queue v1/v2, nexus endpoints, metadata, cluster metadata, and the full visibility filter/count/group-by/pagination paths.go build ./...,go vet, andgofmtare clean across the tree.Notes for reviewers
mssql-server-fts) if you want this exercised in CI, and to split this PR into smaller pieces if that's easier to review.