diff --git a/build/data/versions.yaml b/build/data/versions.yaml index 99d18867e..6ffb59369 100644 --- a/build/data/versions.yaml +++ b/build/data/versions.yaml @@ -1,2 +1,2 @@ -specification: "6.1.1" +specification: "6.2.0" protocol: 6 diff --git a/specifications/features.md b/specifications/features.md index d73708b24..a6c74be37 100644 --- a/specifications/features.md +++ b/specifications/features.md @@ -651,7 +651,10 @@ The threading and/or asynchronous model for each realtime library will vary by l - `(RTN19b)` If there are any pending channels i.e. in the `ATTACHING` or `DETACHING` state, the respective `ATTACH` or `DETACH` message should be resent to Ably - `(RTN23)` Heartbeats - `(RTN23a)` If a transport does not receive any indication of activity on a transport for a period greater than the sum of the `maxIdleInterval` (which will be sent in the `connectionDetails` of the most recent `CONNECTED` message received on that transport) and the [`realtimeRequestTimeout`](#TO3l11), that transport should be disconnected. Any message (or non-message indicator, see `RTN23b`) received counts as an indication of activity and should reset the timer, not merely heartbeat messages. However, it must be received (that is, sent from the server to the client); client-sent data does not count. - - `(RTN23b)` When initiating a connection, the client may send a `heartbeats` param in the querystring, with value `true` or `false`. If the value is true, the server will use Ably protocol messages (for example, a message with a `HEARTBEAT` action) to satisfy the `maxIdleInterval` requirement. If it is false or unspecified, the server is permitted to use any transport-level mechanism (for example, [websocket](https://ably.com/topic/websockets) ping frames) to satisfy this. So for example, for [websocket transports](https://ably.com/topic/websockets), if the client is able to observe websocket pings, then it should send `heartbeats=false`. If not, it should send `heartbeats=true`. + - `(RTN23b)` When initiating a connection, the client may send a `heartbeats` param in the querystring, with value `true`, `false` or `bounce`. If the value is `true`, the server will use Ably protocol messages (for example, a message with a `HEARTBEAT` action) to satisfy the `maxIdleInterval` requirement. If it is `false` or unspecified, the server is permitted to use any transport-level mechanism (for example, [websocket](https://ably.com/topic/websockets) ping frames) to satisfy this. For `bounce`, see [RTN23c](#RTN23c). So for example, for [websocket transports](https://ably.com/topic/websockets), if the client is able to observe websocket pings, then it should send `heartbeats=false`. If not, it should send `heartbeats=true`, unless [RTN23c](#RTN23c) applies. + - `(RTN23c)` In an environment where client code execution might be suspended while leaving the transport itself alive (for example, a browser which may freeze a background tab but continue to respond to websocket ping frames on the tab's behalf), the client should specify `heartbeats=bounce`. In this mode, the server will send `PING` `ProtocolMessage`s instead of `HEARTBEAT`s to satisfy the `maxIdleInterval` requirement. + - `(RTN23c1)` On receiving a `ProtocolMessage` with action `PING` on a transport, the client must send a `ProtocolMessage` with action `PONG` on that same transport. The client must do this regardless of the value (if any) of the `heartbeats` param that it sent when initiating that transport. + - `(RTN23c2)` If the `PING` contains an `id` ([TR4n](#TR4n)), the `PONG` must contain the same `id`. Otherwise, the `PONG` must not contain an `id`. - `(RTN24)` A connected client may receive a `CONNECTED` `ProtocolMessage` from Ably at any point (though is typically triggered by a reauth, see `RTC8a`). The `connectionDetails` in the `ProtocolMessage` must override any stored details, see `RTN21`. The `Connection` should emit an `UPDATE` event with a `ConnectionStateChange` object, which should have both `previous` and `current` attributes set to `CONNECTED`, and the `reason` attribute set to to the `error` member of the `CONNECTED` `ProtocolMessage` (if any). (Note that `UPDATE` should be the only event emitted: in particular, the library must not emit an `CONNECTED` event if the client was already connected, see `RTN4h`). - `(RTN25)` `Connection#errorReason` attribute is an optional `ErrorInfo` object which is set by the library when an error occurs on the connection, as described by [RSA4c1](#RSA4c1), [RSA4d](#RSA4d), [RTN11d](#RTN11d), [RTN14a](#RTN14a), [RTN14b](#RTN14b), [RTN14e](#RTN14e), [RTN14g](#RTN14g), [RTN15c7](#RTN15c7), [RTN15c4](#RTN15c4), [RTN15d](#RTN15d), [RTN15h](#RTN15h), [RTN15i](#RTN15i), [RTN16e](#RTN16e). - `(RTN26)` `Connection#whenState` function: @@ -1588,7 +1591,7 @@ The core SDK provides an API for wrapper SDKs to supply Ably with analytics info #### ProtocolMessage - `(TR1)` A `ProtocolMessage` represents the type used to send and receive messages over the Realtime protocol. A ProtocolMessage always relates either to the connection or to a single channel only, but can contain multiple individual Messages or PresenceMessages. -- `(TR2)` `ProtocolMessage` `Action` enum has the following values in order from zero: `HEARTBEAT`, `ACK`, `NACK`, `CONNECT`, `CONNECTED`, `DISCONNECT`, `DISCONNECTED`, `CLOSE`, `CLOSED`, `ERROR`, `ATTACH`, `ATTACHED`, `DETACH`, `DETACHED`, `PRESENCE`, `MESSAGE`, `SYNC`, `AUTH`, `ACTIVATE`, `OBJECT`, `OBJECT_SYNC`, `ANNOTATION` +- `(TR2)` `ProtocolMessage` `Action` enum has the following values in order from zero: `HEARTBEAT`, `ACK`, `NACK`, `CONNECT`, `CONNECTED`, `DISCONNECT`, `DISCONNECTED`, `CLOSE`, `CLOSED`, `ERROR`, `ATTACH`, `ATTACHED`, `DETACH`, `DETACHED`, `PRESENCE`, `MESSAGE`, `SYNC`, `AUTH`, `ACTIVATE`, `OBJECT`, `OBJECT_SYNC`, `ANNOTATION`, `PING`, `PONG` - `(TR3)` `ProtocolMessage` `Flag` enum has the following values, where a flag with value `n` is defined to be set if the bitwise AND of the `flags` field with `2ⁿ` is nonzero - `(TR3a)` 0: `HAS_PRESENCE` - `(TR3b)` 1: `HAS_BACKLOG` @@ -1756,7 +1759,7 @@ The core SDK provides an API for wrapper SDKs to supply Ably with analytics info - `(CD2e)` `maxInboundRate` is the maximum allowable number of requests per second from a client or Ably. In the case of a realtime connection, this restriction applies to the number of `ProtocolMessage` objects sent, whereas in the case of REST, it is the total number of REST requests per second - `(CD2f)` `connectionStateTtl` is the duration that Ably will persist the connection state when a Realtime client is abruptly disconnected - `(CD2g)` `serverId` string is a unique identifier for the front-end server that the client has connected to. This server ID is only used for the purposes of debugging - - `(CD2h)` `maxIdleInterval` is the maximum length of time in milliseconds that the server will allow no activity to occur in the server-\>client direction. After such a period of inactivity, the server will send a `HEARTBEAT` or transport-level ping to the client. If the value is 0, the server will allow arbitrarily-long levels of inactivity. + - `(CD2h)` `maxIdleInterval` is the maximum length of time in milliseconds that the server will allow no activity to occur in the server-\>client direction. After such a period of inactivity, the server will send a `HEARTBEAT` or `PING` `ProtocolMessage`, or a transport-level ping, to the client (which of these depends on the `heartbeats` param, see [RTN23b](#RTN23b) and [RTN23c](#RTN23c)). If the value is 0, the server will allow arbitrarily-long levels of inactivity. - `(CD2i)` `objectsGCGracePeriod` integer - the length of time, in milliseconds, that the client library must wait before releasing resources for tombstoned objects and map entries (see [RTO10](../objects-features#RTO10)) - `(CD2j)` `siteCode` string - an identifier for the site that the client has connected to\ h4. ChannelProperties @@ -2709,6 +2712,9 @@ Each type, method, and attribute is labelled with the name of one or more clause ACTIVATE // TR2 OBJECT // TR2 OBJECT_SYNC // TR2 + ANNOTATION // TR2 + PING // TR2 + PONG // TR2 class AuthDetails: // AD* accessToken: String // AD2, RTC8a diff --git a/specifications/protocol.md b/specifications/protocol.md index b1a454505..761857bf2 100644 --- a/specifications/protocol.md +++ b/specifications/protocol.md @@ -123,6 +123,14 @@ Each Protocol Message has an `action` that indicates the nature of the message. - ANNOTATION (21) := The \`annotations\` field of the Protocol Message has a payload of one or more `Annotation` messages, all associated with a single channel. They may be sent in either direction. The channel associated with these annotations is indicated in the channel field.`
`{=html}`
`{=html} + + +- PING (22) := Sent by either peer to check that the other is alive and able to process Protocol Messages (as opposed to merely having a live transport). The recipient must respond with a `PONG` on the same transport, carrying the same `id` as the `PING` if it has one. The service only sends `PING` messages on connections whose client requested `heartbeats=bounce` when initiating the connection, in which case it closes the transport if it does not receive a `PONG` in time; see [client library spec RTN23c](/client-lib-development-guide/features#RTN23c). Like heartbeats, `PING` messages are not exposed to the client app. No other message fields are populated in a `PING` message. + + + +- PONG (23) := Sent by either peer in response to a `PING`, with the same `id` as the `PING` if it had one. A `PONG` never elicits a response, and is not exposed to the client app. No other message fields are populated in a `PONG` message. + ## Protocol Message fields ProtocolMessages are populated with one or more of the following fields. diff --git a/uts/docs/completion-status.md b/uts/docs/completion-status.md index e58bb40f8..c475bc838 100644 --- a/uts/docs/completion-status.md +++ b/uts/docs/completion-status.md @@ -187,7 +187,7 @@ This matrix lists all spec items from the [Ably features spec](../../specificati | RTN20 | OS network change handling (RTN20a–RTN20c) | Yes — `realtime/unit/connection/network_change_test.md` | | RTN21 | ConnectionDetails override defaults | Partial — `realtime/unit/connection/update_events_test.md` covers RTN21; `realtime/integration/connection_lifecycle_test.md` covers RTN21 | | RTN22 | Re-authentication request handling (RTN22a) | Yes — `realtime/unit/connection/server_initiated_reauth_test.md`; `realtime/integration/proxy/auth_reauth.md` covers RTN22 | -| RTN23 | Heartbeats (RTN23a–RTN23b) | Yes — `realtime/unit/connection/heartbeat_test.md` | +| RTN23 | Heartbeats (RTN23a–RTN23c, incl. PING/PONG) | Yes — `realtime/unit/connection/heartbeat_test.md` | | RTN24 | UPDATE event on CONNECTED while connected | Yes — `realtime/unit/connection/update_events_test.md` | | RTN25 | Connection#errorReason attribute | Yes — `realtime/unit/connection/error_reason_test.md` | | RTN26 | Connection#whenState function (RTN26a–RTN26b) | Yes — `realtime/unit/connection/when_state_test.md` | diff --git a/uts/realtime/unit/connection/heartbeat_test.md b/uts/realtime/unit/connection/heartbeat_test.md index 1e46a3a62..636a090aa 100644 --- a/uts/realtime/unit/connection/heartbeat_test.md +++ b/uts/realtime/unit/connection/heartbeat_test.md @@ -1,6 +1,6 @@ # Heartbeat Tests (RTN23) -Spec points: `RTN23`, `RTN23a`, `RTN23b` +Spec points: `RTN23`, `RTN23a`, `RTN23b`, `RTN23c` ## Test Type Unit test with mocked WebSocket client @@ -19,7 +19,9 @@ RTN23 defines how the client detects connection liveness: 1. **HEARTBEAT protocol messages** (`heartbeats=true` in connection URL) - for platforms where the WebSocket client does NOT surface ping events 2. **WebSocket ping frames** (`heartbeats=false` or omitted) - for platforms where the WebSocket client CAN surface ping events -A concrete implementation should implement either RTN23a with HEARTBEAT messages OR RTN23b with ping frames, depending on platform capabilities. The test cases below cover both approaches. +- **RTN23c**: A third value, `heartbeats=bounce`, asks the server to check liveness at the protocol level using PING messages instead of HEARTBEATs. Clients whose code may be suspended while the transport stays alive (browsers) should send this. Every client, whatever `heartbeats` value it sent, must answer a PING with a PONG on the same transport (RTN23c1), echoing the PING's `id` (RTN23c2). + +A concrete implementation should implement either RTN23a with HEARTBEAT messages OR RTN23b with ping frames, depending on platform capabilities (browser implementations additionally send `heartbeats=bounce`, per RTN23c). All implementations must implement RTN23c1. The test cases below cover all of these. ### Verifying Transient States @@ -1164,6 +1166,286 @@ CLOSE_CLIENT(client) --- +# RTN23c Tests (heartbeats=bounce and PING/PONG) + +The first test applies to platforms where the client library's code may be suspended while the transport itself remains alive and continues to answer transport-level pings (browsers, per RTN23c). Such clients should send `heartbeats=bounce` in the connection URL. The remaining tests apply to **all** platforms: the obligation to answer a PING with a PONG does not depend on the `heartbeats` value the client sent. + +--- + +## RTN23c - Client sends heartbeats=bounce when its code may be suspended independently of the transport + +**Test ID**: `realtime/unit/RTN23c/heartbeats-bounce-query-param-0` + +**Spec requirement:** A client running in an environment where the transport may keep answering transport-level liveness checks while the client's own code is suspended (e.g. a web browser) should send `heartbeats=bounce`. + +Tests that the client requests protocol-level PING/PONG liveness checking. Only applies to browser (or equivalent) builds of an SDK. + +### Setup + +```pseudo +captured_url = null + +mock_ws = MockWebSocket( + onConnectionAttempt: (conn) => { + captured_url = conn.url + conn.respond_with_success(ProtocolMessage( + action: CONNECTED, + connectionId: "connection-id", + connectionKey: "connection-key", + connectionDetails: ConnectionDetails( + connectionKey: "connection-key", + maxIdleInterval: 15000, + connectionStateTtl: 120000 + ) + )) + } +) +install_mock(mock_ws) + +client = Realtime(options: ClientOptions( + key: "appId.keyId:keySecret", + autoConnect: false +)) +``` + +### Test Steps + +```pseudo +client.connect() +AWAIT_STATE client.connection.state == ConnectionState.connected +``` + +### Assertions + +```pseudo +ASSERT captured_url.query_params["heartbeats"] == "bounce" +CLOSE_CLIENT(client) +``` + +--- + +## RTN23c1 - Client responds to PING with PONG echoing id + +**Test ID**: `realtime/unit/RTN23c1/ping-pong-echo-id-0` + +**Spec requirement:** On receiving a PING, the client must send a PONG on the same transport (RTN23c1). If the PING has an `id`, the PONG must carry the same `id`; otherwise the PONG must have no `id` (RTN23c2). A PONG does not expect an ACK, so must not have a `msgSerial` (RTN7b). + +### Setup + +```pseudo +mock_ws = MockWebSocket( + onConnectionAttempt: (conn) => { + conn.respond_with_success(ProtocolMessage( + action: CONNECTED, + connectionId: "connection-id", + connectionKey: "connection-key", + connectionDetails: ConnectionDetails( + connectionKey: "connection-key", + maxIdleInterval: 15000, + connectionStateTtl: 120000 + ) + )) + } +) +install_mock(mock_ws) + +client = Realtime(options: ClientOptions( + key: "appId.keyId:keySecret", + autoConnect: false +)) +``` + +### Test Steps + +```pseudo +client.connect() +AWAIT_STATE client.connection.state == ConnectionState.connected + +# PING with an id +mock_ws.active_connection.send_to_client(ProtocolMessage( + action: PING, + id: "ping-1" +)) +pong_with_id = AWAIT mock_ws.await_next_message_from_client() + +# PING without an id +mock_ws.active_connection.send_to_client(ProtocolMessage( + action: PING +)) +pong_without_id = AWAIT mock_ws.await_next_message_from_client() +``` + +### Assertions + +```pseudo +ASSERT pong_with_id.action == PONG +ASSERT pong_with_id.id == "ping-1" +ASSERT pong_with_id.msgSerial IS NULL + +ASSERT pong_without_id.action == PONG +ASSERT pong_without_id.id IS NULL +ASSERT pong_without_id.msgSerial IS NULL + +# A PONG is a bare liveness response; nothing else should be populated +FOR pong IN [pong_with_id, pong_without_id]: + ASSERT pong.channel IS NULL + ASSERT pong.messages IS NULL + ASSERT pong.presence IS NULL + +# The client sent nothing else in response to the PINGs (e.g. no HEARTBEAT) +client_messages = mock_ws.events.filter(e => e.type == MESSAGE_FROM_CLIENT) +ASSERT client_messages.length == 2 +CLOSE_CLIENT(client) +``` + +--- + +## RTN23c1 - Client responds to PING regardless of heartbeats param + +**Test ID**: `realtime/unit/RTN23c1/pong-regardless-of-heartbeats-param-1` + +**Spec requirement:** The client must answer a PING with a PONG regardless of the `heartbeats` value it sent when initiating the transport. + +Uses `transportParams` (RTC1f1) to override whatever `heartbeats` value the platform sends by default, so that the test exercises a value other than the platform's own. + +### Setup + +```pseudo +captured_url = null + +mock_ws = MockWebSocket( + onConnectionAttempt: (conn) => { + captured_url = conn.url + conn.respond_with_success(ProtocolMessage( + action: CONNECTED, + connectionId: "connection-id", + connectionKey: "connection-key", + connectionDetails: ConnectionDetails( + connectionKey: "connection-key", + maxIdleInterval: 15000, + connectionStateTtl: 120000 + ) + )) + } +) +install_mock(mock_ws) + +client = Realtime(options: ClientOptions( + key: "appId.keyId:keySecret", + transportParams: { "heartbeats": "false" }, + autoConnect: false +)) +``` + +### Test Steps + +```pseudo +client.connect() +AWAIT_STATE client.connection.state == ConnectionState.connected + +ASSERT captured_url.query_params["heartbeats"] == "false" + +mock_ws.active_connection.send_to_client(ProtocolMessage( + action: PING, + id: "ping-1" +)) +pong = AWAIT mock_ws.await_next_message_from_client() +``` + +### Assertions + +```pseudo +ASSERT pong.action == PONG +ASSERT pong.id == "ping-1" +CLOSE_CLIENT(client) +``` + +--- + +## RTN23a - PING message resets idle timer + +**Test ID**: `realtime/unit/RTN23a/ping-resets-timer-6` + +**Spec requirement:** Any message from the server, including PING messages, resets the idle timer (RTN23a). A client using `heartbeats=bounce` relies on this, since the server sends PINGs rather than HEARTBEATs. + +### Setup + +```pseudo +connection_attempt_count = 0 + +mock_ws = MockWebSocket( + onConnectionAttempt: (conn) => { + connection_attempt_count++ + conn.respond_with_success(ProtocolMessage( + action: CONNECTED, + connectionId: "connection-id-" + connection_attempt_count, + connectionKey: "connection-key-" + connection_attempt_count, + connectionDetails: ConnectionDetails( + connectionKey: "connection-key-" + connection_attempt_count, + maxIdleInterval: 3000, # 3 seconds + connectionStateTtl: 120000 + ) + )) + } +) +install_mock(mock_ws) + +client = Realtime(options: ClientOptions( + key: "appId.keyId:keySecret", + realtimeRequestTimeout: 1000, # 1 second + disconnectedRetryTimeout: 500, + autoConnect: false +)) +``` + +### Test Steps + +```pseudo +enable_fake_timers() + +client.connect() +AWAIT_STATE client.connection.state == ConnectionState.connected + +ASSERT connection_attempt_count == 1 + +# Advance time (not enough to trigger timeout: 3000 + 1000 = 4000ms) +ADVANCE_TIME(2000) +# Send PING from server - resets timer +mock_ws.active_connection.send_to_client(ProtocolMessage( + action: PING, + id: "ping-1" +)) +# Advance time again (2000ms since PING, still within threshold) +ADVANCE_TIME(2000) +# Connection should still be alive - no reconnection triggered +ASSERT client.connection.state == ConnectionState.connected +ASSERT connection_attempt_count == 1 + +# Advance time past the timeout window (4100ms since last PING) +ADVANCE_TIME(2100) + +# After idle timeout fires, the client enters DISCONNECTED and waits +# disconnectedRetryTimeout before reconnecting. If using fake timers, +# ensure time is advanced past both the idle timeout AND the retry delay. + +# Wait for reconnection to complete +AWAIT_STATE client.connection.state == ConnectionState.connected +``` + +### Assertions + +```pseudo +# Verify reconnection happened +ASSERT connection_attempt_count == 2 + +# Verify the client closed the first WebSocket connection +client_close_events = mock_ws.events.filter(e => e.type == CLIENT_CLOSE) +ASSERT client_close_events.length == 1 +CLOSE_CLIENT(client) +``` + +--- + # Implementation Notes > **Implementation note:** Some SDKs perform an internet connectivity check (RTN17j) @@ -1186,6 +1468,13 @@ A concrete SDK implementation should: - Expect HEARTBEAT protocol messages from server - Implement RTN23a tests +4. **If the library's code may be suspended while the transport stays alive (browsers)**: + - Send `heartbeats=bounce` in connection URL instead of `true` (RTN23c) + - Expect PING protocol messages from server; these reset the idle timer like any other message + - Implement the RTN23c test + +5. **In all cases**: answer PING with PONG (RTN23c1 tests) + ### Platform-Specific Notes **Dart:** The standard `dart:io` WebSocket does **not** surface ping frames to the application layer. The ping/pong mechanism is handled automatically and internally - there is no `onPing` callback. Therefore, Dart implementations must use **RTN23a** (HEARTBEAT protocol messages) for idle timeout detection. The RTN23b tests do not apply to Dart. diff --git a/uts/realtime/unit/helpers/mock_websocket.md b/uts/realtime/unit/helpers/mock_websocket.md index 345496aa3..63bc7deaa 100644 --- a/uts/realtime/unit/helpers/mock_websocket.md +++ b/uts/realtime/unit/helpers/mock_websocket.md @@ -191,6 +191,7 @@ mock_ws.active_connection.send_ping_frame() **Connection URL query parameter:** - If the client sends `heartbeats=true`, it expects HEARTBEAT protocol messages - If the client sends `heartbeats=false` (or omits it), the server may use ping frames +- If the client sends `heartbeats=bounce` (RTN23c), it expects PING protocol messages, which it must answer with PONG (RTN23c1). Browser builds send this. - The test should verify which parameter the client sends based on platform capabilities ## Protocol Message Templates @@ -226,6 +227,13 @@ ERROR_MESSAGE(code, message) = ProtocolMessage( HEARTBEAT_MESSAGE = ProtocolMessage( action: HEARTBEAT ) + +# Server-initiated liveness check (RTN23c1); the client must reply with +# ProtocolMessage(action: PONG, id: id) +PING_MESSAGE(id) = ProtocolMessage( + action: PING, + id: id +) ``` ## Example: Handler Pattern with State