feat: let callers opt out of index creation with create_index=False

docs/api/exceptions.rst

@@ -158,6 +158,10 @@ there raises ``redis.exceptions.NoPermissionError`` itself rather than a wrapped
 :class:`RedisSearchError`. See :doc:`/user_guide/installation` for the ACL categories
 RedisVL needs.
 
+When the credential genuinely cannot run ``FT.INFO``, this error is not something to
+handle: construct the extension with ``create_index=False`` instead, which skips the
+existence check entirely. See :doc:`/user_guide/installation`.
+
 Telling "the index is missing" apart from other failures
 --------------------------------------------------------
 

docs/user_guide/installation.md

@@ -186,30 +186,102 @@ The Sentinel URL format supports:
 
 ## Redis permissions (ACLs)
 
-RedisVL works through Redis Search commands, so a connecting credential needs the `@search` ACL category, or the individual `FT.*` commands. Reading an index additionally requires key permissions covering its prefix: the [ACL documentation](https://redis.io/docs/latest/operate/oss_and_stack/management/security/acl/#command-categories) describes this rule for creating, modifying, and reading an index, and in practice `FT.INFO`, `FT.SEARCH`, and `FT.AGGREGATE` are denied when the index prefix falls outside the allowed key patterns. `FT.CREATE` is not checked this way, so a credential can create an index it is then unable to read.
+RedisVL reaches Redis through Redis Search commands, but not all of them need the `@search` category. Querying and loading work under an ordinary `+@read +@write` role; it is the commands that inspect or manage an index — `FT.INFO`, `FT.CREATE`, `FT._LIST` — that need `@search` or an explicit grant.
 
-One command needs more than `@search`. Redis tags `FT._LIST` as `@admin` as well as `@search` and `@slow`, and ACL rules are applied left to right — so a rule that grants search access and then takes back administrative commands, such as `+@search -@admin` or `+@all -@admin`, denies it:
+The command-to-category mapping below was measured against live servers rather than quoted from published documentation, which does not list ACL categories per `FT.*` command. It was identical on Redis 8.0.6, 8.2.7, 8.4.5, 8.6.4, 8.8.0 and 8.8.1. Check your own deployment with `COMMAND INFO ft.info`, `ACL CAT search`, or `ACL DRYRUN <user> FT.INFO <index>`.
+
+### What each operation needs
+
+| Operation | Redis command | `+@all -@admin` | `+@read +@write` |
+|---|---|---|---|
+| `index.query()`, `index.search()`, `index.aggregate()` | `FT.SEARCH`, `FT.AGGREGATE` | Yes | Yes |
+| `index.load()` | `HSET` or `JSON.SET` (needs key access) | Yes | Yes |
+| `index.exists()`, `index.info()`, `index.clear()`, `SearchIndex.from_existing()`, `rvl index info`, `rvl stats` | `FT.INFO` | Yes | **No** |
+| `index.create()` | `FT.CREATE` | Yes | **No** |
+| `index.delete()`, `rvl index delete`, `rvl index destroy` | `FT.DROPINDEX` | Yes | Yes |
+| Enumerating indexes (see below) | `FT._LIST` | **No** | **No** |
+
+Except for `FT.CREATE`, every `Yes` above assumes key patterns that cover the index prefix — see [Key permissions](#key-permissions). `FT.CREATE` is not checked against those patterns, so a credential can create an index it cannot query. Adding `-@dangerous` to the second column additionally denies `FT.DROPINDEX`, so `index.delete()` becomes `No`. An SVS-VAMANA schema needs more than `FT.CREATE`: `index.create()` first probes capabilities with `INFO` (`@slow @dangerous`) and `MODULE LIST` (`@admin @slow @dangerous`), so both `-@admin` and `-@dangerous` policies break creation for those schemas.
+
+Two of the rows above deserve their own explanation.
+
+`FT._LIST` is tagged `@admin` as well as `@search` and `@slow`, and ACL rules are applied left to right — so a rule that grants search access and then takes back administrative commands, such as `+@search -@admin` or `+@all -@admin`, denies it:
 
 ```text
 User <name> has no permissions to run the 'FT._LIST' command
 ```
 
-Note that `FT._LIST` does not *require* `@admin`: granting `+@search` on its own permits it. Only rules that subtract `@admin` after granting search are affected. To keep such a policy and still enumerate indexes, grant the command back explicitly with `+ft._list`.
+`FT._LIST` does not *require* `@admin`: granting `+@search` on its own permits it. Only rules that subtract `@admin` after granting search are affected. To keep such a policy and still enumerate indexes, grant the command back explicitly with `+ft._list`. Enumeration is reached by `SearchIndex.listall()` and `AsyncSearchIndex.listall()`, by `rvl index listall`, and by the migration entry points that discover indexes for you: `rvl migrate helper`, `rvl migrate wizard` when no `-i/--index` is given, and `rvl migrate batch-plan --pattern`.
+
+`FT.DROPINDEX` is tagged `@dangerous` and `@write` as well as `@search`, so a policy that subtracts `@dangerous` denies `index.delete()`, `rvl index delete`, and `rvl index destroy`.
+
+### Roles built from `@read` and `@write`
+
+`FT.INFO` is in neither `@read` nor `@write` — its only category is `@search` — and `FT.CREATE` is the same. So an application role assembled from `+@read +@write -@dangerous` — a natural least-privilege shape for a runtime that must query and load but must not manage indexes — can query and load, but cannot ask whether an index exists and cannot create or drop one. Subtracting `@dangerous` is not what denies `FT.INFO` or `FT.CREATE`: `+@all -@dangerous` permits both. Those commands are simply never granted by `@read` or `@write`.
+
+Every extension constructor checks whether its index exists, so under such a credential all of them fail while being constructed:
+
+```text
+RedisSearchError: Error while fetching llmcache index info:
+User <name> has no permissions to run the 'FT.INFO' command
+```
+
+### "no permissions to run the 'FT.INFO' command"
+
+RedisVL does not guess its way around this. A credential that cannot ask whether the index exists also cannot create one, so there is nothing useful to infer — instead, tell RedisVL that the index is already there:
+
+```python
+cache = SemanticCache(
+    name="llmcache",
+    redis_url="redis://localhost:6379",
+    create_index=False,
+)
+```
+
+`create_index=False` is available on `SemanticCache`, `MessageHistory`, `SemanticMessageHistory` and `SemanticRouter`. It skips the existence check, the comparison of your schema against the live index, and index creation — the constructor issues no index command at all. Pass it when the index is managed externally, or when the credential cannot run `FT.INFO`. It cannot be combined with `overwrite=True`, which asks for the opposite.
+
+A `SearchIndex` used directly needs nothing special: build it with `from_dict()` or `from_yaml()`, then load and query. Two of its methods stay unavailable, because both read index metadata: `from_existing()`, which reconstructs a schema out of Redis, and `clear()`, which starts by calling `info()`.
+
+The flag also skips the SVS-VAMANA capability probe described above, since that runs inside `create()`.
+
+### Provisioning a router
+
+`SemanticRouter` with `create_index=False` writes nothing at all: not the reference vectors for its routes, and not the stored route config that `SemanticRouter.from_existing()` reads. Preparing a router for this mode therefore means constructing it once with a privileged credential — a hand-written `FT.CREATE` is not enough, because the reference vectors have to be embedded and written too. Without them the router matches nothing, which looks like a distance-threshold problem rather than an empty index.
+
+Afterwards, `SemanticRouter.from_existing(name, create_index=False)` is the way to attach to it: it recovers the routes and thresholds with `JSON.GET` and needs no `FT.INFO`. The stored config must contain the full route set. Each route's distance threshold is applied from that recovered list, so an incomplete stored config silently narrows matching — and `add_route()` and `remove_route()` rewrite the stored config from the same list, so mutating an incomplete config permanently drops the omitted routes from the config every other client reads.
 
-| Operation | Redis command | Permitted by `+@all -@admin` |
-|---|---|---|
-| `index.create()`, `index.exists()` | `FT.CREATE`, `FT.INFO` | Yes |
-| `index.query()`, `index.search()`, `index.aggregate()` | `FT.SEARCH`, `FT.AGGREGATE` | Yes |
-| `index.info()`, `rvl index info`, `rvl stats` | `FT.INFO` | Yes |
-| `index.load()` | `HSET` or `JSON.SET` (needs `@write` and key access) | Yes |
-| `index.delete()`, `rvl index delete`, `rvl index destroy` | `FT.DROPINDEX` | Yes |
-| Enumerating indexes (see below) | `FT._LIST` | No |
+### When the schema diverges
 
-`SemanticCache`, `SemanticMessageHistory`, `MessageHistory`, and `SemanticRouter` each call `index.create()` while being constructed, so they are covered by the first row.
+With `create_index=False` nothing verifies that the live index matches the schema you described. Some mismatches are loud on first use, and several are silent:
 
-Index enumeration is the only thing an `-@admin` rule breaks. It is reached by `SearchIndex.listall()` and `AsyncSearchIndex.listall()`, by `rvl index listall`, and by the migration entry points that discover indexes for you: `rvl migrate helper`, `rvl migrate wizard` when no `-i/--index` is given, and `rvl migrate batch-plan --pattern`.
+| Mismatch | What happens |
+|---|---|
+| The index does not exist | `RedisSearchError` on the first query, naming the missing index |
+| Vector dimensions disagree | `Error parsing vector similarity query: query vector blob size (32) does not match index's expected size (16)` — but only once the index holds a document. On an empty index the same query returns nothing, so a freshly provisioned index hides this until the first write lands |
+| The prefix does not cover your keys | **Silent.** Documents are written but never indexed, so queries return nothing, forever |
+| The index is `ON JSON` and you write hashes (or the reverse) | **Silent**, the same way |
+| The datatype or distance metric differs | **Silent.** Neither is restated by a query, so nothing compares them — results come back ranked by the index's metric, not yours |
 
-Other categories gate different operations. `FT.DROPINDEX` is tagged `@dangerous` and `@write` as well as `@search`, so a policy that subtracts `@dangerous` denies `index.delete()`, `rvl index delete`, and `rvl index destroy`.
+For the silent cases the tell is `FT.INFO`'s `key_type`, `prefixes` and `attributes` — not `hash_indexing_failures`, which stays `0` because those keys were never indexing candidates. Diagnosing it therefore needs a credential that can run `FT.INFO`.
+
+An extension constructed with `create_index=False` refuses index-wide `delete()` and `clear()` operations (and their async cache equivalents). This protects an externally managed index — including an index reached through an alias — from being destroyed through an attach-only instance. Targeted operations such as dropping a specific cache entry or message remain available. Perform lifecycle-wide destructive operations through the privileged provisioning path that owns the index.
+
+### Key permissions
+
+Command categories are only half of it. Redis also scopes the search commands by key pattern: [the ACL documentation](https://redis.io/docs/latest/operate/oss_and_stack/management/security/acl/#command-categories) states that only users with access to a *superset* of the prefixes defined at index creation can create, modify, or read an index.
+
+Measured on 8.4.5 against an index prefixed `doc:`, with the command categories held constant at `+@all`:
+
+| Key patterns | `FT.SEARCH`, `FT.INFO`, `FT.AGGREGATE` |
+|---|---|
+| `~doc:*` (superset) | Permitted |
+| `%R~doc:*` (read permission only) | Permitted |
+| `~doc:1` (partial overlap) | `NOPERM User does not have the required permissions to query the index` |
+| `~other:*` (no overlap) | The same denial |
+
+Partial overlap is worth emphasising: it fails exactly like no overlap at all, rather than returning the subset you can read. `FT.CREATE` is not checked this way, so a credential can create an index it is then unable to query.
+
+`create_index=False` does not help here — the very commands it lets you avoid are joined by the ones it cannot, so widen the key patterns instead.
 
 Outside of Redis Search, RedisVL identifies itself on connect with `CLIENT SETINFO`. That command is tagged `@connection` and `@slow`, and belongs to neither `@read` nor `@write`, so a rule built up from those categories never grants it. A credential that cannot run it still connects: identification only populates the `lib-name` field that `CLIENT LIST` and `CLIENT INFO` display, so a refusal is ignored (and logged, if you have configured logging at debug level). Grant `+client|setinfo` if you want RedisVL to appear as the connecting library there — note that this labels the connection RedisVL opens, while redis-py labels the rest of the pool as plain `redis-py`.
 
@@ -222,4 +294,6 @@ Both manage ACLs through their own control plane rather than the `ACL SETUSER` c
 - **Redis Cloud** provides three predefined ACL rules that cannot be edited — Full-Access, Read-Write ("read and write commands and excludes dangerous commands"), and Read-Only — which you assign to a data access role. See [Configure permissions with Redis ACLs](https://redis.io/docs/latest/operate/rc/security/access-control/data-access-control/configure-acls/). Custom rules use the same syntax as above.
 - **Redis Software** ships one predefined ACL, Full Access, and you define others in the Cluster Manager UI or with a [`POST /v1/redis_acls`](https://redis.io/docs/latest/operate/rs/security/access-control/create-db-roles/) request. It [does not support every `ACL` command](https://redis.io/docs/latest/operate/rs/security/access-control/redis-acl-overview/#acl-command-support), nor nested selectors, nor `(` and `)` in key patterns.
 
-Because the predefined rules' exact command sets are not published, confirm a credential against the database rather than inferring what its policy name implies. Redis Software's documentation uses `+@read +FT.INFO +FT.SEARCH` as an example rule, which is a good illustration: it permits querying and `index.exists()`, but not `index.create()` or index enumeration. Grant `FT.CREATE` explicitly when the application creates its own index, which `SemanticCache`, `SemanticMessageHistory`, `MessageHistory`, and `SemanticRouter` all do.
+Because the predefined rules' exact command sets are not published, confirm a credential against the database rather than inferring what its policy name implies — `ACL DRYRUN <user> FT.INFO <index>` answers it directly. Read the descriptions carefully before assuming you are unaffected: Read-Only allows read commands, and Read-Write "allows read and write commands and excludes dangerous commands", so both read as `@read`/`@write`-shaped — the shape that denies `FT.INFO` and `FT.CREATE` and wants `create_index=False`. Only Full-Access is clearly unaffected.
+
+Redis Software's documentation uses `+@read +FT.INFO +FT.SEARCH` as an example rule, which is a good illustration: it permits querying and `index.exists()`, but not `index.create()` or index enumeration. Grant `FT.CREATE` explicitly when the application creates its own index. When the index is provisioned for the application instead, leave it out and construct with `create_index=False`.