Skip to content

Enable transactions extensions env var #374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Added configurable landing page ID `STAC_FASTAPI_LANDING_PAGE_ID` [#352](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/352)
- Added support for `S_CONTAINS`, `S_WITHIN`, `S_DISJOINT` spatial filter operations [#371](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/issues/371)
- Introduced the `DATABASE_REFRESH` environment variable to control whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. [#370](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/370)
- Added the `ENABLE_TRANSACTIONS_EXTENSIONS` environment variable to enable or disable the Transactions and Bulk Transactions API extensions. When set to `false`, endpoints provided by `TransactionsClient` and `BulkTransactionsClient` are not available. This allows for flexible deployment scenarios and improved API control. [#374](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/374)

### Changed

Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ You can customize additional settings in your `.env` file:
| `ENABLE_DIRECT_RESPONSE` | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false` | Optional
| `RAISE_ON_BULK_ERROR` | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` Optional |
| `DATABASE_REFRESH` | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
| `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. If set to `false`, the POST `/collections` route and related transaction endpoints (including bulk transaction operations) will be unavailable in the API. This is useful for deployments where mutating the catalog via the API should be prevented. | `true` | Optional |

> [!NOTE]
> The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, and `ES_VERIFY_CERTS` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch.
Expand Down
42 changes: 29 additions & 13 deletions stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""FastAPI application."""

import logging
import os
from contextlib import asynccontextmanager

Expand All @@ -23,6 +24,7 @@
from stac_fastapi.core.rate_limit import setup_rate_limit
from stac_fastapi.core.route_dependencies import get_route_dependencies
from stac_fastapi.core.session import Session
from stac_fastapi.core.utilities import get_bool_env
from stac_fastapi.elasticsearch.config import ElasticsearchSettings
from stac_fastapi.elasticsearch.database_logic import (
DatabaseLogic,
Expand All @@ -39,6 +41,12 @@
)
from stac_fastapi.extensions.third_party import BulkTransactionExtension

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)

settings = ElasticsearchSettings()
session = Session.create_from_settings(settings)

Expand All @@ -60,19 +68,6 @@
aggregation_extension.GET = EsAggregationExtensionGetRequest

search_extensions = [
TransactionExtension(
client=TransactionsClient(
database=database_logic, session=session, settings=settings
),
settings=settings,
),
BulkTransactionExtension(
client=BulkTransactionsClient(
database=database_logic,
session=session,
settings=settings,
)
),
FieldsExtension(),
QueryExtension(),
SortExtension(),
Expand All @@ -81,6 +76,27 @@
FreeTextExtension(),
]

if TRANSACTIONS_EXTENSIONS:
search_extensions.insert(
0,
TransactionExtension(
client=TransactionsClient(
database=database_logic, session=session, settings=settings
),
settings=settings,
),
)
search_extensions.insert(
1,
BulkTransactionExtension(
client=BulkTransactionsClient(
database=database_logic,
session=session,
settings=settings,
)
),
)

extensions = [aggregation_extension] + search_extensions

database_logic.extensions = [type(ext).__name__ for ext in extensions]
Expand Down
43 changes: 30 additions & 13 deletions stac_fastapi/opensearch/stac_fastapi/opensearch/app.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""FastAPI application."""

import logging
import os
from contextlib import asynccontextmanager

Expand All @@ -23,6 +24,7 @@
from stac_fastapi.core.rate_limit import setup_rate_limit
from stac_fastapi.core.route_dependencies import get_route_dependencies
from stac_fastapi.core.session import Session
from stac_fastapi.core.utilities import get_bool_env
from stac_fastapi.extensions.core import (
AggregationExtension,
FilterExtension,
Expand All @@ -39,6 +41,12 @@
create_index_templates,
)

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)

settings = OpensearchSettings()
session = Session.create_from_settings(settings)

Expand All @@ -60,19 +68,6 @@
aggregation_extension.GET = EsAggregationExtensionGetRequest

search_extensions = [
TransactionExtension(
client=TransactionsClient(
database=database_logic, session=session, settings=settings
),
settings=settings,
),
BulkTransactionExtension(
client=BulkTransactionsClient(
database=database_logic,
session=session,
settings=settings,
)
),
FieldsExtension(),
QueryExtension(),
SortExtension(),
Expand All @@ -81,6 +76,28 @@
FreeTextExtension(),
]


if TRANSACTIONS_EXTENSIONS:
search_extensions.insert(
0,
TransactionExtension(
client=TransactionsClient(
database=database_logic, session=session, settings=settings
),
settings=settings,
),
)
search_extensions.insert(
1,
BulkTransactionExtension(
client=BulkTransactionsClient(
database=database_logic,
session=session,
settings=settings,
)
),
)

extensions = [aggregation_extension] + search_extensions

database_logic.extensions = [type(ext).__name__ for ext in extensions]
Expand Down
47 changes: 47 additions & 0 deletions stac_fastapi/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
)
from stac_fastapi.core.rate_limit import setup_rate_limit
from stac_fastapi.core.route_dependencies import get_route_dependencies
from stac_fastapi.core.utilities import get_bool_env

if os.getenv("BACKEND", "elasticsearch").lower() == "opensearch":
from stac_fastapi.opensearch.config import AsyncOpensearchSettings as AsyncSettings
Expand Down Expand Up @@ -479,3 +480,49 @@ async def route_dependencies_client(route_dependencies_app):
base_url="http://test-server",
) as c:
yield c


def build_test_app():
TRANSACTIONS_EXTENSIONS = get_bool_env(
"ENABLE_TRANSACTIONS_EXTENSIONS", default=True
)
settings = AsyncSettings()
aggregation_extension = AggregationExtension(
client=EsAsyncAggregationClient(
database=database, session=None, settings=settings
)
)
aggregation_extension.POST = EsAggregationExtensionPostRequest
aggregation_extension.GET = EsAggregationExtensionGetRequest
search_extensions = [
SortExtension(),
FieldsExtension(),
QueryExtension(),
TokenPaginationExtension(),
FilterExtension(),
FreeTextExtension(),
]
if TRANSACTIONS_EXTENSIONS:
search_extensions.insert(
0,
TransactionExtension(
client=TransactionsClient(
database=database, session=None, settings=settings
),
settings=settings,
),
)
extensions = [aggregation_extension] + search_extensions
post_request_model = create_post_request_model(search_extensions)
return StacApi(
settings=settings,
client=CoreClient(
database=database,
session=None,
extensions=extensions,
post_request_model=post_request_model,
),
extensions=extensions,
search_get_request_model=create_get_request_model(search_extensions),
search_post_request_model=post_request_model,
).app
35 changes: 34 additions & 1 deletion stac_fastapi/tests/resources/test_collection.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
import copy
import os
import uuid

import pytest
from httpx import AsyncClient
from stac_pydantic import api

from ..conftest import create_collection, delete_collections_and_items, refresh_indices
from ..conftest import (
build_test_app,
create_collection,
delete_collections_and_items,
refresh_indices,
)

CORE_COLLECTION_PROPS = [
"id",
Expand Down Expand Up @@ -36,6 +43,32 @@ async def test_create_and_delete_collection(app_client, load_test_data):
assert resp.status_code == 204


@pytest.mark.asyncio
async def test_create_collection_transactions_extension(load_test_data):
test_collection = load_test_data("test_collection.json")
test_collection["id"] = "test"

os.environ["ENABLE_TRANSACTIONS_EXTENSIONS"] = "false"
app_disabled = build_test_app()
async with AsyncClient(app=app_disabled, base_url="http://test") as client:
resp = await client.post("/collections", json=test_collection)
assert resp.status_code in (
404,
405,
501,
), f"Expected failure, got {resp.status_code}"

os.environ["ENABLE_TRANSACTIONS_EXTENSIONS"] = "true"
app_enabled = build_test_app()
async with AsyncClient(app=app_enabled, base_url="http://test") as client:
resp = await client.post("/collections", json=test_collection)
assert resp.status_code == 201
resp = await client.delete(f"/collections/{test_collection['id']}")
assert resp.status_code == 204

del os.environ["ENABLE_TRANSACTIONS_EXTENSIONS"]


@pytest.mark.asyncio
async def test_create_collection_conflict(app_client, ctx):
"""Test creation of a collection which already exists"""
Expand Down