fix: stabilise the export harness and publish what was assembled

.github/workflows/assemble.yml

@@ -79,6 +79,13 @@ jobs:
         with:
           name: assembled-site
           path: build/site
+          # Without this the upload silently drops dot-prefixed paths, so the
+          # published branch would be missing files the assembler produced —
+          # docs/.cursor/ went missing from the first published build that way.
+          # The assembler must publish what it assembled, editor config included:
+          # deciding a file is "not really content" is how a byte-comparability
+          # guarantee turns into a list of exceptions.
+          include-hidden-files: true
           retention-days: 7
 
   publish:

docs/agent-branch-experiments.mdx

@@ -58,7 +58,7 @@ Use a different column name for each model. Within an experiment, use the same
 model for the stored text and the search queries.
 </Note>
 
-## Apply the winning experiment to `main`
+## Apply the winning experiment to `main` {#apply-the-winning-experiment-to-main}
 
 The branch experiments leave you with the results side by side, and they
 deliberately never touch `main`. Once you've picked a winner, apply it to
@@ -76,7 +76,7 @@ the losing branch around until you're confident in the result.
 See [Branches](/tables/branching) for more on how branches, versions, and tags
 relate.
 
-## More experiments you can run
+## More experiments you can run {#more-experiments-you-can-run}
 
 Swapping embedding models on a branch while working with agents is only one example
 of what you can do with the LanceDB plugin. The table below shows other experiments

docs/api-reference/index.mdx

@@ -13,7 +13,7 @@ If you're looking for a REST API reference, visit the [REST API](/api-reference/
 If you're looking for conceptual and practical namespace guidance before diving into method signatures, see
 [Namespaces and Catalog Model](/namespaces) and [Using Namespaces in SDKs](/namespaces/usage).
 
-## Supported SDKs
+## Supported SDKs {#supported-sdks}
 
 Python, Typescript and Rust SDKs are officially supported by LanceDB. You can use these SDKs to interact with both LanceDB OSS and Enterprise deployments.
 
@@ -23,7 +23,7 @@ Python, Typescript and Rust SDKs are officially supported by LanceDB. You can us
 | <Icon icon="square-js" /> [Typescript](https://lancedb.github.io/lancedb/js/) | A TypeScript wrapper around the Rust library, built with `napi-rs`
 | <Icon icon="rust" /> [Rust](https://docs.rs/lancedb/latest/lancedb/index.html) | Native Rust library with persistent-storage and high performance |
 
-## REST API SDKs
+## REST API SDKs {#rest-api-sdks}
 
 <Badge color="red">Enterprise</Badge>
 
@@ -33,7 +33,7 @@ REST API-based SDKs provide a convenient way to interact with LanceDB Enterprise
 |:--------------|-------------------|
 | <Icon icon="java" /> [Java](https://lancedb.github.io/lancedb/java/java/)| REST API Enterprise SDK in Java |
 
-## Community-driven SDKs
+## Community-driven SDKs {#community-driven-sdks}
 
 In addition to the officially supported SDKs, the LanceDB community may contribute SDKs in other languages.
 These SDKs may not have the same level of support or feature parity as the official ones supported by LanceDB, but they can be an option

docs/api-reference/rest/index.mdx

@@ -15,7 +15,7 @@ additional endpoints for managing tables and data.
 If you have specific needs or questions about the Enterprise REST API Namespace,
 please [contact us](mailto:support@lancedb.com).
 
-## Authentication
+## Authentication {#authentication}
 
 <Badge color="red">Enterprise</Badge>
 
@@ -24,7 +24,7 @@ must be encoded as JSON or Arrow RPC.
 
 To authenticate to the Enterprise REST API, you need the endpoint for your deployment and a valid API key for that deployment.
 
-### Get your Enterprise credentials
+### Get your Enterprise credentials {#get-your-enterprise-credentials}
 
 1. Obtain the following values from your LanceDB administrator or the LanceDB team that provisioned your Enterprise deployment:
    - an API key
@@ -41,7 +41,7 @@ export LANCEDB_DATABASE="your-database-name"
 
 3. If your Enterprise deployment is private, connect through the private network endpoint provided for your deployment. For example, Azure Private Link deployments commonly use a private IP or an internal DNS name as the endpoint.
 
-### Verify authentication
+### Verify authentication {#verify-authentication}
 
 4. Check that you can reach the deployment and list tables:
 

docs/build-with-ai-agents.mdx

@@ -77,14 +77,14 @@ Not every agent supports project scope; `npx plugins targets` lists what it
 found and what each target supports.
 </Accordion>
 
-## Get started with the LanceDB agent plugin
+## Get started with the LanceDB agent plugin {#get-started-with-the-lancedb-agent-plugin}
 
 This tutorial uses the Camelot dataset from the [quickstart](/quickstart), with
 a portrait added for each character. Each LanceDB row contains validated
 metadata and raw JPEG bytes. Text, images, and any embeddings you add later
 remain in the same table.
 
-### 1. Download the multimodal dataset
+### 1\. Download the multimodal dataset {#1-download-the-multimodal-dataset}
 
 From a new project directory, download the JSON file and portraits:
 
@@ -133,7 +133,7 @@ Pydantic models before writing it.
 After the agent writes the pipeline, inspect the schema, batching, and write
 path rather than assuming it followed the plugin's guidance correctly.
 
-### 2. Prompt your agent to build the pipeline
+### 2\. Prompt your agent to build the pipeline {#2-prompt-your-agent-to-build-the-pipeline}
 
 Install the Python packages used by the example:
 
@@ -173,7 +173,7 @@ simply ask it to use the `lancedb` plugin in the prompt, as shown above.
 That should be enough! The agent will create `ingest_multimodal.py`, or similar.
 The following sections inspect the script to verify that it follows the plugin's guidance.
 
-#### Data validation
+#### Data validation {#data-validation}
 
 The plugin encourages the agent to validate each record with Pydantic before writing it.
 The agent should ideally define a schema for the table and a nested schema for the
@@ -186,7 +186,7 @@ The agent should ideally define a schema for the table and a nested schema for t
 In this case, our agent correctly defined `Character` and `Stats` Pydantic models
 and validated the JSON before adding it to the table.
 
-#### Batched ingestion
+#### Batched ingestion {#batched-ingestion}
 
 Naively calling `table.add()` once per row is slow, and is considered an anti-pattern
 in LanceDB. The plugin encourages the agent to collect incoming rows into batches and
@@ -202,7 +202,7 @@ batch. If validation fails, that batch is never written. The function yields up
 to `batch_size` rows at a time, providing an iterable of batches for the ingestion step,
 shown next.
 
-#### Table maintenance
+#### Table maintenance {#table-maintenance}
 For LanceDB OSS, the plugin instructs the agent to call
 `table.optimize()` after the ingestion loop. This compacts small fragments, cleans up
 old versions according to the retention policy, and incorporates new data into indexes.
@@ -220,7 +220,7 @@ This example dataset has only eight rows, so the default batch size writes it in
 one call. Larger inputs should still avoid single-row write commits.
 </Note>
 
-### 3. Run the OSS pipeline
+### 3\. Run the OSS pipeline {#3-run-the-oss-pipeline}
 
 ```bash
 uv run python ingest_multimodal.py
@@ -238,7 +238,7 @@ Here are the first three rows:
 Once your pipeline works, you can [run experiments on branches](/agent-branch-experiments)
 to try new embedding models, parsers, or search settings without touching `main`.
 
-## Takeaways
+## Takeaways {#takeaways}
 
 The example in this tutorial was small, but similar ideas apply to other workflows, too.
 Give the agent the data source, the constraints it must respect, and the
@@ -247,7 +247,7 @@ artifacts it should return.
 The plugin supplies LanceDB-specific guidance, but it's the user's responsibility to
 ensure the output makes sense for the application.
 
-### Try the plugin with your own dataset
+### Try the plugin with your own dataset {#try-the-plugin-with-your-own-dataset}
 
 The plugin shown in this tutorial should generalize reasonably well to other use cases.
 If you find any issues, open [an issue](https://github.com/lancedb/lancedb-agent-plugins/issues)

docs/demos/index.mdx

@@ -13,7 +13,7 @@ App | Description
 [Video Search](#video-search) | A video search application that allows searching through a library of videos using natural language queries.
 
 
-## Semantic.art
+## Semantic.art {#semantic-art}
 
 <Badge color="green">multimodal</Badge>
 <Badge color="red">hybrid-search</Badge>
@@ -37,7 +37,7 @@ Read in detail about how Semantic.art is built in this blog post.
 
 </Card>
 
-## Wikipedia 41M Hybrid Search
+## Wikipedia 41M Hybrid Search {#wikipedia-41m-hybrid-search}
 
 <Badge color="green">multimodal</Badge>
 <Badge color="red">hybrid-search</Badge>
@@ -61,7 +61,7 @@ Read in detail about how the Wikipedia 41M hybrid search demo is built in this b
 
 </Card>
 
-## Video Search
+## Video Search {#video-search}
 
 <Badge color="green">multimodal</Badge>
 <Badge color="orange">video-search</Badge>

docs/embedding/index.mdx

@@ -34,7 +34,7 @@ that automatically generates vector embeddings during data ingestion. Automatic
 generation is available in LanceDB OSS, with SDK-specific query ergonomics. The API abstracts
 embedding generation, allowing you to focus on your application logic.
 
-## Embedding Registry
+## Embedding Registry {#embedding-registry}
 
 You can get a supported embedding function from the registry, and then use it in your table schema. 
 Once configured, the embedding function will automatically generate embeddings when you insert data
@@ -55,7 +55,7 @@ while Rust examples typically compute query embeddings explicitly before vector
     </CodeBlock>
 </CodeGroup>
 
-### Using an embedding function
+### Using an embedding function {#using-an-embedding-function}
 
 Create an embedding function before you attach it to table or schema metadata. Python and TypeScript fetch
 provider implementations from the embedding registry, while Rust constructs the provider embedding function
@@ -123,7 +123,7 @@ For non-sensitive settings such as inference device selection, you can also use
 
 Find the full list of arguments for each provider in the [integrations](/integrations/embedding) section.
 
-## Multiple embedding columns
+## Multiple embedding columns {#multiple-embedding-columns}
 
 A single table can include more than one embedding definition when you want to store multiple semantic views
 of the same data, or generate embeddings from different source columns. In practice, each embedding definition
@@ -136,11 +136,11 @@ In TypeScript, automatic query embedding currently uses the first embedding func
 table metadata. If a table has multiple embedding definitions and you need to query a specific vector
 column, compute the query embedding explicitly and pass the vector to the search builder.
 
-## Embedding model providers
+## Embedding model providers {#embedding-model-providers}
 
 LanceDB supports most popular embedding providers.
 
-### Text embeddings
+### Text embeddings {#text-embeddings}
 
 | Provider | Model ID | Default Model |
 |----------|----------|---------------|
@@ -150,7 +150,7 @@ LanceDB supports most popular embedding providers.
 | Cohere | `cohere` | `embed-english-v3.0` |
 | ... | ... | ... |
 
-### Multimodal embedding
+### Multimodal embedding {#multimodal-embedding}
 
 | Provider | Model ID | Supported Inputs |
 |----------|----------|------------------|
@@ -160,7 +160,7 @@ LanceDB supports most popular embedding providers.
 
 You can find all supported embedding models in the [integrations](/integrations/embedding) section.
 
-## Embeddings in LanceDB Enterprise
+## Embeddings in LanceDB Enterprise {#embeddings-in-lancedb-enterprise}
 <Badge color="red">Enterprise</Badge>
 
 In LanceDB Enterprise, embedding generation during data ingestion is client-side and the resulting vectors are
@@ -171,7 +171,7 @@ The Enterprise server does not currently generate embeddings from query text on
 query-time embedding happens on the client side.
 </Info>
 
-### How string queries are interpreted
+### How string queries are interpreted {#how-string-queries-are-interpreted}
 
 For the Python remote client, `table.search("hello")` can take two different paths:
 
@@ -218,7 +218,7 @@ want full control over query-time behavior.
     </CodeBlock>
 </CodeGroup>
 
-## Custom Embedding Functions
+## Custom Embedding Functions {#custom-embedding-functions}
 
 You can always implement your own embedding function:
 - Python/TypeScript: subclass `TextEmbeddingFunction` (text) or `EmbeddingFunction` (multimodal).

docs/embedding/quickstart.mdx

@@ -17,7 +17,7 @@ import {
 LanceDB will automatically vectorize the data both at ingestion and query time. All you need to do is specify which model to use.
 Popular embedding models like OpenAI, Hugging Face, Sentence Transformers, CLIP, and more, are supported.
 
-## Step 1: Import Required Libraries
+## Step 1: Import Required Libraries {#step-1-import-required-libraries}
 
 First, import the necessary LanceDB components:
 
@@ -40,7 +40,7 @@ from lancedb.embeddings import get_registry
 - TypeScript uses `lancedb.embedding.getRegistry()` and `lancedb.embedding.LanceSchema()` for the same registry/schema workflow
 - In TypeScript, import the provider module before calling `getRegistry().get(...)`; the provider import is what registers names such as `"huggingface"` or `"openai"`
 
-## Step 2: Connect to LanceDB
+## Step 2: Connect to LanceDB {#step-2-connect-to-lancedb}
 
 Establish a connection to your LanceDB OSS directory or Enterprise cluster:
 
@@ -55,7 +55,7 @@ db = lancedb.connect(...)
     </CodeBlock>
 </CodeGroup>
 
-## Step 3: Initialize the Embedding Function
+## Step 3: Initialize the Embedding Function {#step-3-initialize-the-embedding-function}
 
 Choose and configure your embedding model:
 
@@ -76,7 +76,7 @@ This creates an embedding function from the local embedding registry. The Python
 - Modify the model name for different embedding models
 - Set `device="cuda"` for GPU acceleration if available
 
-## Step 4: Define Your Schema
+## Step 4: Define Your Schema {#step-4-define-your-schema}
 
 Create a Pydantic model that defines your table structure:
 
@@ -97,7 +97,7 @@ class Words(LanceModel):
 - `model.ndims()`: Sets vector dimensions for your model
 - In TypeScript, use `model.sourceField(...)` and `model.vectorField()` inside `LanceSchema(...)`
 
-## Step 5: Create Table and Ingest Data
+## Step 5: Create Table and Ingest Data {#step-5-create-table-and-ingest-data}
 
 Create a table with your schema and add data:
 
@@ -124,7 +124,7 @@ If your input already includes the vector column, automatic embedding only runs
 absent or entirely null for the batch. Partially supplied vectors are treated as manual data, so
 LanceDB preserves them instead of filling only the missing rows.
 
-## Step 6: Query with Automatic Embedding
+## Step 6: Query with Automatic Embedding {#step-6-query-with-automatic-embedding}
 
 Note: On LanceDB Enterprise, the server does not generate embeddings from query text. In the Python remote
 client, `table.search("greetings")` can still work when the table schema includes embedding metadata, because

docs/enterprise/architecture.mdx

@@ -30,13 +30,13 @@ flowchart TB
     CP -.->|Govern and configure| IX
 ```
 
-## Compute-storage separation
+## Compute-storage separation {#compute-storage-separation}
 
 In LanceDB Enterprise, storage and compute are deliberately decoupled. Table data and index artifacts live in object storage, while query-serving and background workers read from and write to that shared durable layer. This means compute can be replaced, scaled, or specialized without making any individual node the owner of the dataset.
 
 This design has practical consequences. Query fleets can scale for interactive traffic without also scaling background indexing capacity. Heavy indexing and compaction work can run on dedicated workers instead of stealing resources from user-facing queries. Caches can accelerate hot reads without becoming the source of truth. And because the data remains in object storage, durability does not depend on the lifecycle of a particular server or local disk.
 
-## Architecture
+## Architecture {#architecture}
 
 At a high level, the control plane governs the system and the data plane executes the work. The control plane is responsible for configuration, service discovery, identity integration, policy, and cluster lifecycle. It determines how the system should behave, but it is not the layer serving table data or executing user queries -- that's the role of the data plane.
 
@@ -80,15 +80,15 @@ flowchart TB
     WAL -->|Persist| OS
 ```
 
-## Remote tables
+## Remote tables {#remote-tables}
 
 A [remote table](/tables-and-namespaces#understanding-tables) is the user-facing abstraction over this architecture. From the client side, you connect to a logical storage layer and table over the network by providing a `db://...` connection identifier. The system then resolves that logical name to the underlying storage-backed table and executes the operation inside the cluster.
 
 This is why Enterprise feels familiar at the API level while operationally behaving differently. Your application still issues table operations and queries, but it is no longer coupled to a local storage path or a single host. Instead, the cluster takes responsibility for execution, coordination, and background upkeep. In SDK terms, `open_table(...)` returns a `RemoteTable`. Architecturally, a remote table is the bridge between the client-facing API and the storage-backed system behind it.
 
 This design makes LanceDB Enterprise suitable for catalog-backed layouts, see [Namespaces and the Catalog Model](/namespaces) for more details. For the basic application flow, see the shared [quickstart](/quickstart).
 
-## Read path
+## Read path {#read-path}
 
 When a client issues a query against a remote table, the path is straightforward:
 
@@ -100,7 +100,7 @@ When a client issues a query against a remote table, the path is straightforward
 
 This separation is what lets Enterprise combine a clean remote API with a serving layer that can scale horizontally and keep hot data close to execution.
 
-## Write path
+## Write path {#write-path}
 
 Writes follow a different path because durability comes first:
 
@@ -110,7 +110,7 @@ Writes follow a different path because durability comes first:
 
 Keeping the commit path centered on object storage ensures that the durable record of the table lives outside any single query node. Regardless of whether you're using Lance namespaces or an external catalog, the catalog's role is mainly to resolve table names and provide access details -- the table’s actual data and index artifacts remain in object storage.
 
-## Background work
+## Background work {#background-work}
 
 Indexing, compaction, and cleanup are intentionally moved off the user request path. After table changes are committed, the system can determine that additional work is needed and assign it to background workers built for heavyweight processing.
 
@@ -122,7 +122,7 @@ In practice, that usually looks like this:
 
 This separation is one of the clearest architectural reasons to use LanceDB Enterprise: the same query-serving infrastructure does not have to handle every expensive indexing or compaction task itself.
 
-## What this means for users
+## What this means for users {#what-this-means-for-users}
 
 For teams using LanceDB Enterprise, the architecture changes the _operational model_ more than the programming model. You still work with tables and queries, but the cluster now takes responsibility for distributed execution, cache-aware reads, and long-running background jobs.