diff --git a/architecture/client-architecture.mdx b/architecture/client-architecture.mdx index c0358675..2f886d95 100644 --- a/architecture/client-architecture.mdx +++ b/architecture/client-architecture.mdx @@ -126,7 +126,7 @@ It is possible for different [buckets](/architecture/powersync-service#bucket-sy ## Writing Data (via SQLite Database and Upload Queue) -Any mutations on the SQLite database, namely updates, deletes and inserts, are immediately reflected in the SQLite database, and also also automatically placed into an **upload queue** by the Client SDK. +Any mutations on the SQLite database, namely updates, deletes and inserts, are immediately reflected in the SQLite database, and also automatically placed into an **upload queue** by the Client SDK. The upload queue is a blocking [FIFO](https://en.wikipedia.org/wiki/FIFO_%28computing_and_electronics%29) queue. @@ -136,7 +136,7 @@ The Client SDK processes the upload queue by invoking an `uploadData()` function The reason why we designed PowerSync this way is that it allows you to apply your own backend business logic, validations and authorization to any mutations going to your source database. -The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. +The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. For a detailed breakdown of when `uploadData()` is called, including throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). diff --git a/client-sdks/reference/capacitor.mdx b/client-sdks/reference/capacitor.mdx index 21729a01..b818e3e8 100644 --- a/client-sdks/reference/capacitor.mdx +++ b/client-sdks/reference/capacitor.mdx @@ -186,7 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/dotnet.mdx b/client-sdks/reference/dotnet.mdx index 797382c9..efd294de 100644 --- a/client-sdks/reference/dotnet.mdx +++ b/client-sdks/reference/dotnet.mdx @@ -205,7 +205,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.FetchCredentials](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L50) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/flutter.mdx b/client-sdks/reference/flutter.mdx index d5910d3b..3bbc72f8 100644 --- a/client-sdks/reference/flutter.mdx +++ b/client-sdks/reference/flutter.mdx @@ -192,7 +192,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/fetchCredentials.html) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/javascript-web.mdx b/client-sdks/reference/javascript-web.mdx index d78a5680..da00e160 100644 --- a/client-sdks/reference/javascript-web.mdx +++ b/client-sdks/reference/javascript-web.mdx @@ -191,7 +191,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/kotlin.mdx b/client-sdks/reference/kotlin.mdx index ad44c077..df439c25 100644 --- a/client-sdks/reference/kotlin.mdx +++ b/client-sdks/reference/kotlin.mdx @@ -164,7 +164,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnector.fetchCredentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/node.mdx b/client-sdks/reference/node.mdx index a9c2f43e..c1388483 100644 --- a/client-sdks/reference/node.mdx +++ b/client-sdks/reference/node.mdx @@ -101,7 +101,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/react-native-and-expo.mdx b/client-sdks/reference/react-native-and-expo.mdx index 33ecb516..a044874f 100644 --- a/client-sdks/reference/react-native-and-expo.mdx +++ b/client-sdks/reference/react-native-and-expo.mdx @@ -186,8 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- Use this to upload client-side changes to your app backend. -\-> See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/rust.mdx b/client-sdks/reference/rust.mdx index 78813e96..524f033a 100644 --- a/client-sdks/reference/rust.mdx +++ b/client-sdks/reference/rust.mdx @@ -254,7 +254,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/swift.mdx b/client-sdks/reference/swift.mdx index 5b116198..ea3df87d 100644 --- a/client-sdks/reference/swift.mdx +++ b/client-sdks/reference/swift.mdx @@ -127,7 +127,7 @@ Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnectorProtocol.fetchCredentials` - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/tauri.mdx b/client-sdks/reference/tauri.mdx index dbfaccb3..a3449967 100644 --- a/client-sdks/reference/tauri.mdx +++ b/client-sdks/reference/tauri.mdx @@ -189,7 +189,7 @@ Please [let us know](/resources/contact-us) if you want to implement a backend c Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/writing-data.mdx b/client-sdks/writing-data.mdx index 23776b2d..9edd6e9d 100644 --- a/client-sdks/writing-data.mdx +++ b/client-sdks/writing-data.mdx @@ -126,7 +126,9 @@ PowerSync automatically queues writes and uploads them to your backend. The uplo | `PATCH` | Update existing row | Contains the row `id`, and value of each changed column. | Generated by `UPDATE` statements. | | `DELETE` | Delete existing row | Contains the row `id` | Generated by `DELETE` statements. | -For details on how writes are uploaded to your backend, see [Writing Client Changes](/handling-writes/writing-client-changes). +The SDK automatically processes this queue by calling your `uploadData()` function. For a detailed breakdown of when and how `uploadData()` is called, including triggers, throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). + +For details on how to structure your backend to accept these writes, see [Writing Client Changes](/handling-writes/writing-client-changes). ## Advanced Topics diff --git a/configuration/app-backend/client-side-integration.mdx b/configuration/app-backend/client-side-integration.mdx index e8f8d257..634a0b18 100644 --- a/configuration/app-backend/client-side-integration.mdx +++ b/configuration/app-backend/client-side-integration.mdx @@ -20,11 +20,100 @@ Accordingly, you must pass a _backend connector_ as an argument when you call `c | Purpose | Function | Description | |---------|----------|-------------| -| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an _upload queue_ by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See [Writing Data](/client-sdks/writing-data) in the _Client SDKs_ section for more details on the implementation of `uploadData()`. | +| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an _upload queue_ by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See the [detailed behavior below](#when-uploaddata-is-called) and [Writing Data](/client-sdks/writing-data) in the _Client SDKs_ section for more details on the implementation of `uploadData()`. | | **Authentication integration:** | `fetchCredentials()` | Called by the PowerSync Client SDK to obtain a JWT and the endpoint URL for your PowerSync Service instance. The SDK uses the JWT to authenticate against the PowerSync Service. `fetchCredentials()` typically returns an object with `token` (JWT) and `endpoint` fields. See [Authentication Setup](/configuration/auth/overview) for more details on JWT authentication. | Some authentication providers generate JWTs for users which PowerSync can verify directly, and in that case, your `fetchCredentials()` function implementation can simply return that JWT from client-side state. Your `fetchCredentials()` implementation only needs to retrieve a JWT from your backend if you are using [Custom Authentication](/configuration/auth/custom) integration. See the [Authentication Overview](/configuration/auth/overview) for more background. +### When `uploadData()` is Called + +The PowerSync Client SDK calls `uploadData()` automatically - you never call it directly. It is invoked in the following scenarios: + +1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. +2. **On initial connection / reconnection** - When `connect()` establishes (or re-establishes) a sync stream and the first message is received from the PowerSync Service, the SDK triggers `uploadData()` to flush any writes that were made while offline or disconnected. +3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (every 20 seconds, with slight jitter). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred. +4. **After an error, with retry** - If `uploadData()` throws an error, the SDK waits for a configurable delay (default: 5 seconds, controlled by `retryDelayMs`) and then retries. This continues until the upload succeeds or the sync stream is disconnected. + +#### Upload Loop Behavior + +The SDK calls `uploadData()` in a **loop** - not just once per trigger. After each successful `uploadData()` call, the SDK checks whether there are more items in the upload queue. If there are, it calls `uploadData()` again immediately. The loop continues until the queue is empty. This means your `uploadData()` implementation only needs to process one batch (or one transaction) per call. + +Once the queue is empty, the SDK updates an internal write checkpoint used for [consistency](/architecture/consistency) tracking. + +#### Throttling + +To avoid excessive calls, upload triggers are **throttled**. Rapid local writes (e.g., multiple `INSERT` statements in quick succession) are coalesced so that `uploadData()` is invoked at most once per throttle interval. If an upload is already in progress when a new write occurs, the SDK will trigger another upload after the current one completes. + +The default throttle interval varies by SDK: + +| SDK | Default | Option Name | +|-----|---------|-------------| +| JavaScript / React Native / Node / Capacitor / Tauri | 1,000 ms | `crudUploadThrottleMs` | +| Kotlin | 1,000 ms | `crudThrottleMs` | +| Swift | 1 second | `crudThrottle` (seconds) | +| .NET | 1,000 ms | `CrudUploadThrottleMs` | +| Dart / Flutter | 10 ms | `crudThrottleTime` | + +The throttle interval is configurable via options when calling `connect()`: + + +```typescript TypeScript +await db.connect(connector, { crudUploadThrottleMs: 500 }); +``` + +```dart Dart/Flutter +db.connect( + connector: connector, + options: SyncOptions( + crudThrottleTime: Duration(milliseconds: 500), + ), +); +``` + +```kotlin Kotlin +database.connect(connector, crudThrottleMs = 500L) +``` + +```swift Swift +try await database.connect( + connector: connector, + options: ConnectOptions(crudThrottle: 0.5) // seconds +) +``` + +```csharp .NET +await database.Connect(connector, new PowerSyncConnectionOptions( + crudUploadThrottleMs: 500 +)); +``` + + +#### Error Handling + + +If your `uploadData()` throws an error (e.g. due to a `4xx` or `5xx` response from your backend), the SDK will **retry** the same upload indefinitely, effectively blocking the upload queue. Your backend should return `2xx` for validation errors or write conflicts, and reserve error responses for transient failures. See [Writing Client Changes](/handling-writes/writing-client-changes#recommendations) and [Handling Write / Validation Errors](/handling-writes/handling-write-validation-errors) for recommended patterns. + + +When `uploadData()` throws an error: +1. The SDK logs the error and updates the sync status with an `uploadError`. +2. It waits for the retry delay (default: 5 seconds). +3. It retries the upload from the beginning of the queue. +4. If the sync stream disconnects during the retry wait, the upload loop exits and will resume when the connection is re-established. + +#### Stalled Upload Queue Detection + +If the SDK detects that the same CRUD entry is at the front of the queue across consecutive upload iterations (i.e., `uploadData()` returned without error but didn't call `.complete()` on the batch or transaction), it logs a warning: + +> _"Potentially previously uploaded CRUD entries are still present in the upload queue. Make sure to handle uploads and complete CRUD transactions or batches by calling and awaiting their \[.complete()\] method. The next upload iteration will be delayed."_ + +After logging this warning, the SDK **throws an internal error**, which causes it to enter the retry delay path (default: 5 seconds) before attempting the upload again. This prevents a tight loop when `uploadData()` consistently fails to process entries. + +This typically means your `uploadData()` implementation is not calling `.complete()` after successfully processing entries. Always call `.complete()` on the `CrudBatch` or `CrudTransaction` to remove processed entries from the queue. + + +`uploadData()` is only called while the sync stream is connected. If the device is offline, writes accumulate in the upload queue and are uploaded automatically when connectivity is restored and the sync stream reconnects. + + ### When `fetchCredentials()` is Called The PowerSync Client SDK **caches credentials internally** and `fetchCredentials` is called in the following scenarios: diff --git a/handling-writes/handling-write-validation-errors.mdx b/handling-writes/handling-write-validation-errors.mdx index aa2109c7..49497cf5 100644 --- a/handling-writes/handling-write-validation-errors.mdx +++ b/handling-writes/handling-write-validation-errors.mdx @@ -1,6 +1,6 @@ --- title: "Handling Write / Validation Errors" -description: "The general approach is that for transient errors (e.g. server or database unavailable), the changes are kept in the client-side upload queue, and retried at 5 second intervals, keeping the original order. In the future it will be possible to control the retry behavior." +description: "The general approach is that for transient errors (e.g. server or database unavailable), the changes are kept in the client-side upload queue and retried automatically. The retry delay is configurable — see When uploadData() is Called for details." --- For validation errors or write conflicts (see the definition of this below in [Technical Details](/handling-writes/handling-write-validation-errors#additional-technical-details)), changes are automatically rolled back on the client. diff --git a/handling-writes/writing-client-changes.mdx b/handling-writes/writing-client-changes.mdx index 30486530..27cf2351 100644 --- a/handling-writes/writing-client-changes.mdx +++ b/handling-writes/writing-client-changes.mdx @@ -4,7 +4,7 @@ description: "Your backend application needs to expose an API endpoint to apply --- - Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details. + Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details, including [when `uploadData()` is called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) and its retry behavior.