refactor: one array class

changes/4011.feature.md

@@ -0,0 +1,3 @@
+`Array` now owns its own state and accepts a keyword-only `runner` argument for plugging in a custom event loop. Every asynchronous array operation is available as a `*_async` method on `Array` (for example `Array.getitem_async`). `Array.async_array` is deprecated; use the `*_async` methods instead.
+
+The `Array` constructor signature changed from `Array(async_array)` to `Array(metadata, store_path, config=None, *, runner=None)`, mirroring `AsyncArray`. The legacy `Array(async_array)` form still works for now but is deprecated and will be removed in a future release; construct an `Array` directly with the new signature instead.

docs/user-guide/arrays.md

@@ -74,6 +74,40 @@ print(z[:])
 More information about NumPy-style indexing can be found in the
 [NumPy documentation](https://numpy.org/doc/stable/user/basics.indexing.html).
 
+### Asynchronous access
+
+The indexing and I/O operations shown above are synchronous: they block until the
+data has been read or written. Every such operation also has an asynchronous
+counterpart on the `Array`, named with an `_async` suffix, which returns a
+coroutine you can `await`. These are useful for issuing many requests
+concurrently from `async` code without going through Zarr's synchronous wrapper.
+
+```python exec="true" session="arrays" source="above" result="ansi"
+import asyncio
+
+
+async def read_corner(arr):
+    return await arr.getitem_async((0, 0))
+
+
+print(asyncio.run(read_corner(z)))
+```
+
+Counterparts exist for the full read/write surface, including the advanced
+indexers described below — for example `getitem_async`, `setitem_async`,
+`get_orthogonal_selection_async`, `get_coordinate_selection_async`,
+`get_block_selection_async`, and the matching setters, as well as `resize_async`,
+`append_async`, and `update_attributes_async`. Each synchronous method is
+implemented by running its `_async` counterpart through the array's runner; see
+[Custom event loops with runner](performance.md#custom-event-loops-with-runner)
+in the performance guide for how to control which event loop executes them.
+
+!!! note
+
+    Earlier versions of Zarr exposed the asynchronous API through a separate
+    `AsyncArray` object reachable via the `Array.async_array` property. That
+    property is now deprecated in favor of the `_async` methods on `Array`.
+
 ## Persistent arrays
 
 In the examples above, compressed data for each chunk of the array was stored in

docs/user-guide/performance.md

@@ -246,6 +246,43 @@ thread pool (see the Dask section below). Increasing it may improve throughput
 in CPU-bound workloads where many synchronous-to-async dispatches happen
 concurrently.
 
+### Custom event loops with `runner`
+
+Every `Array` method that touches storage is implemented as an asynchronous
+coroutine. A synchronous call like `z[...]` runs that coroutine to completion
+through the array's *runner*. By default the runner is a `SyncRunner`, which
+submits the coroutine to Zarr's shared background event loop (the same mechanism
+described above, governed by `threading.max_workers`).
+
+You can supply your own runner to control which event loop executes the
+coroutines — for example to reuse an event loop you already manage, or to
+integrate with another async framework. A runner is any object with a
+`run(coro)` method that awaits the coroutine and returns its result:
+
+```python exec="true" session="performance" source="above"
+import zarr
+from zarr.core.sync import SyncRunner
+
+
+class MyRunner:
+    def run(self, coro):
+        # Execute `coro` on the event loop of your choice and return its result.
+        return SyncRunner().run(coro)
+
+
+z = zarr.create_array(store={}, shape=(100,), chunks=(10,), dtype="i4")
+z_custom = zarr.Array(
+    metadata=z.metadata,
+    store_path=z.store_path,
+    config=z.config,
+    runner=MyRunner(),
+)
+```
+
+`Runner` is a [`typing.Protocol`][], so a custom runner does not need to subclass
+anything — it only needs a compatible `run` method. When `runner` is omitted (or
+`None`), the array uses the default `SyncRunner`.
+
 ### Using Zarr with Dask
 
 [Dask](https://www.dask.org/) is a popular parallel computing library that works well with Zarr for processing large arrays. When using Zarr with Dask, it's important to consider the interaction between Dask's thread pool and Zarr's concurrency settings.

src/zarr/api/synchronous.py

@@ -221,7 +221,7 @@ def open(
         )
     )
     if isinstance(obj, AsyncArray):
-        return Array(obj)
+        return Array._from_async_array(obj)
     else:
         return Group(obj)
 
@@ -391,7 +391,7 @@ def array(data: npt.ArrayLike | AnyArray, **kwargs: Any) -> AnyArray:
         The new array.
     """
 
-    return Array(sync(async_api.array(data=data, **kwargs)))
+    return Array._from_async_array(sync(async_api.array(data=data, **kwargs)))
 
 
 def group(
@@ -760,7 +760,7 @@ def create(
     z : Array
         The array.
     """
-    return Array(
+    return Array._from_async_array(
         sync(
             async_api.create(
                 shape=shape,
@@ -944,7 +944,7 @@ def create_array(
     # <Array memory://... shape=(100, 100) dtype=int32>
     ```
     """
-    return Array(
+    return Array._from_async_array(
         sync(
             zarr.core.array.create_array(
                 store,
@@ -1165,7 +1165,7 @@ def from_array(
     array([[0, 0], [0, 0]])
     """
 
-    return Array(
+    return Array._from_async_array(
         sync(
             zarr.core.array.from_array(
                 store,
@@ -1214,7 +1214,7 @@ def empty(shape: tuple[int, ...], **kwargs: Any) -> AnyArray:
     retrieve data from an empty Zarr array, any values may be returned,
     and these are not guaranteed to be stable from one access to the next.
     """
-    return Array(sync(async_api.empty(shape, **kwargs)))
+    return Array._from_async_array(sync(async_api.empty(shape, **kwargs)))
 
 
 # TODO: move ArrayLike to common module
@@ -1241,7 +1241,7 @@ def empty_like(a: ArrayLike, **kwargs: Any) -> AnyArray:
     retrieve data from an empty Zarr array, any values may be returned,
     and these are not guaranteed to be stable from one access to the next.
     """
-    return Array(sync(async_api.empty_like(a, **kwargs)))
+    return Array._from_async_array(sync(async_api.empty_like(a, **kwargs)))
 
 
 # TODO: add type annotations for kwargs and fill_value
@@ -1262,7 +1262,9 @@ def full(shape: tuple[int, ...], fill_value: Any, **kwargs: Any) -> AnyArray:
     Array
         The new array.
     """
-    return Array(sync(async_api.full(shape=shape, fill_value=fill_value, **kwargs)))
+    return Array._from_async_array(
+        sync(async_api.full(shape=shape, fill_value=fill_value, **kwargs))
+    )
 
 
 # TODO: move ArrayLike to common module
@@ -1282,7 +1284,7 @@ def full_like(a: ArrayLike, **kwargs: Any) -> AnyArray:
     Array
         The new array.
     """
-    return Array(sync(async_api.full_like(a, **kwargs)))
+    return Array._from_async_array(sync(async_api.full_like(a, **kwargs)))
 
 
 # TODO: add type annotations for kwargs
@@ -1301,7 +1303,7 @@ def ones(shape: tuple[int, ...], **kwargs: Any) -> AnyArray:
     Array
         The new array.
     """
-    return Array(sync(async_api.ones(shape, **kwargs)))
+    return Array._from_async_array(sync(async_api.ones(shape, **kwargs)))
 
 
 # TODO: add type annotations for kwargs
@@ -1320,7 +1322,7 @@ def ones_like(a: ArrayLike, **kwargs: Any) -> AnyArray:
     Array
         The new array.
     """
-    return Array(sync(async_api.ones_like(a, **kwargs)))
+    return Array._from_async_array(sync(async_api.ones_like(a, **kwargs)))
 
 
 # TODO: update this once async_api.open_array is fully implemented
@@ -1356,7 +1358,7 @@ def open_array(
     AsyncArray
         The opened array.
     """
-    return Array(
+    return Array._from_async_array(
         sync(
             async_api.open_array(
                 store=store,
@@ -1387,7 +1389,7 @@ def open_like(a: ArrayLike, path: str, **kwargs: Any) -> AnyArray:
     AsyncArray
         The opened array.
     """
-    return Array(sync(async_api.open_like(a, path=path, **kwargs)))
+    return Array._from_async_array(sync(async_api.open_like(a, path=path, **kwargs)))
 
 
 # TODO: add type annotations for kwargs
@@ -1406,7 +1408,7 @@ def zeros(shape: tuple[int, ...], **kwargs: Any) -> AnyArray:
     Array
         The new array.
     """
-    return Array(sync(async_api.zeros(shape=shape, **kwargs)))
+    return Array._from_async_array(sync(async_api.zeros(shape=shape, **kwargs)))
 
 
 # TODO: add type annotations for kwargs
@@ -1425,4 +1427,4 @@ def zeros_like(a: ArrayLike, **kwargs: Any) -> AnyArray:
     Array
         The new array.
     """
-    return Array(sync(async_api.zeros_like(a, **kwargs)))
+    return Array._from_async_array(sync(async_api.zeros_like(a, **kwargs)))