Skip to content

feat: list namespaces endpoint - #232

Open
gdccyuen wants to merge 1 commit into
Ontos-AI:mainfrom
gdccyuen:feat/gdccyuen/list-namespaces-endpoint
Open

feat: list namespaces endpoint#232
gdccyuen wants to merge 1 commit into
Ontos-AI:mainfrom
gdccyuen:feat/gdccyuen/list-namespaces-endpoint

Conversation

@gdccyuen

Copy link
Copy Markdown
Contributor

What

Adds GET /api/v1/documents/namespaces — returns the authenticated user's namespaces with active document counts.

{
  "namespaces": [
    {"namespace": "alpha", "document_count": 2},
    {"namespace": "beta",  "document_count": 1}
  ]
}

Why

Currently a user has no way to discover which namespaces they have uploaded documents to. Namespace is a free-form string label set at job creation; without a list endpoint, callers must track namespaces themselves (issue raised during self-hosted deployment review). The composite index idx_documents_user_namespace_status already covers the grouping query, so the cost is a single indexed scan.

Changes

  • apps/api/app/repositories/document_repository.py — new list_namespace_counts_for_user (GROUP BY namespace over non-archived docs, covered by idx_documents_user_namespace_status).
  • apps/api/app/services/documents/lifecycle_service.py — new list_namespaces service method.
  • apps/api/app/api/v1/routes/documents.py — new GET /namespaces route, declared before /{document_id} so FastAPI matches the literal path first.
  • apps/api/tests/contract/test_documents_contract.py — 2 contract tests:
    • per-user isolation: another user's namespaces are not visible; archived docs are excluded from counts.
    • empty case: returns {"namespaces": []} when the user has no documents.

Conventions matched

  • Filter is status != "archived" (matches existing list_by_user_namespace), not status = "active".
  • Auth via Depends(with_current_user); user-scoped at the repository layer.
  • Response shape parallels list_documents: top-level dict with a list field.

Verification

uv run pytest apps/api/tests/contract/test_documents_contract.py -q
19 passed in 14.96s

(includes 17 existing tests — no regressions.)

Add GET /api/v1/documents/namespaces returning the authenticated
user's namespaces with active document counts. Per-user isolation and
archived doc exclusion follow the existing list_documents conventions;
query is covered by idx_documents_user_namespace_status.

Previously users had no way to discover namespaces they had uploaded to
since namespace is a free-form string label on document creation.
@suguanYang

Copy link
Copy Markdown
Contributor

Thanks for the contribution. We understand that the immediate requirement is to recover namespace values from documents already stored in Knowhere, particularly when a self-hosted client has not retained those values itself.

However, this appears to conflict with the intended ownership of namespace in Knowhere.

A namespace is not currently a first-class Knowhere resource. It has no independent identity, creation operation, metadata, authorization policy, or lifecycle. It is a free-form isolation label supplied
and owned by the client, then stored on documents and related retrieval records so document operations and retrieval can be scoped correctly.

The proposed endpoint therefore does not really list namespaces. It lists distinct namespace strings that currently have non-archived documents. A value appears only after a document is created, disappears after its last document is archived, cannot represent an empty namespace, and may include accidental values caused by client input errors.

Under the original contract, the client that chooses a namespace should also retain it. For example, a client using tenant or workspace IDs as namespaces already owns the authoritative tenant/workspace registry; Knowhere should not become a secondary and incomplete registry for those identifiers.

Could you describe the concrete workflow and actor that require namespace discovery? In particular:

  • Why does that client not already know the namespace values it supplied?
  • Is this intended for normal application behavior or administrative recovery in a self-hosted deployment?
  • Should archived-only and empty namespaces exist in the result?
  • Is the credential expected to enumerate every tenant label and document count under the account?

If the requirement is administrative reconciliation, we can consider an explicitly administrative document-aggregation endpoint. We do not think we should present the current GROUP BY documents.namespace result as a namespace resource without first changing the domain contract.

There are also two implementation details to address if we retain this direction:

  • The PR describes “active document counts,” but the query counts every status except archived.
  • The v1 document router is mounted under v2, so the change also exposes /api/v2/documents/namespaces, while only v1 is documented and tested.

For now, We would prefer clarification of the business requirement before merging.

@gdccyuen

Copy link
Copy Markdown
Contributor Author

In a broader context, I intend to build an internal analysis platform for an organization that runs entirely on-premise and supports collaborative teamwork. Each dashboard account represents a specific domain of knowledge, and within that, documents can be organized into folders (i.e., namespaces). Authentication for the notebook has been replaced with an independent, pluggable system, allowing a single user to hold one or more API keys (associated with different dashboard accounts) to access content across multiple domains.

Now, to address your specific questions:

  • Why doesn't the client already know the namespace values it supplied?
    The user responsible for preparing and uploading knowledge or documents is often not the same as the one actively working within that domain.

  • Is this intended for normal application behavior or administrative recovery in a self-hosted deployment?
    I believe it is intended for normal application behavior.

  • Should archived-only and empty namespaces appear in the results?
    Archived-only namespaces should be skipped. An empty namespace should be treated as equivalent to default.

  • Is the credential expected to enumerate every tenant label and document count under the account?
    I suggest we simply follow the current API key (credential) and enumerate all active (non-archived) documents and namespaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants