Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions python/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,29 @@ This project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased]

### What's New

#### MCAP imports

`client.data_import` now supports MCAP (`.mcap`) files with ROS 2 (`ros2msg`/`cdr`) topics.

```python
job = client.data_import.import_from_path("recording.mcap", asset=my_asset)
```

Importing without a config ingests every supported channel. Topics that cannot be decoded fail the import unless `McapParseErrorPolicy.IGNORE_ERROR` is set.

`detect_config` reads the file's channels locally without decoding messages, returning an `McapImportConfig` whose `data` holds one entry per field. Edit it to select, rename, or retype channels before importing.

```python
config = client.data_import.detect_config("recording.mcap")
config.complex_types_import_mode = McapComplexTypesImportMode.STRING
```

Variable-cardinality fields (dynamic and bounded arrays) are typed `BYTES`. As with Parquet, `complex_types_import_mode` on the config decides what each becomes: Arrow IPC bytes, a JSON string under `<name>.json`, both (the default), or neither.

Reading a file locally needs the new `mcap` extra (`pip install sift-stack-py[mcap]`), so both `detect_config` and importing without a config require it.

## [v0.20.0] - August 25, 2026

### What's New
Expand Down
10 changes: 5 additions & 5 deletions python/docs/guides/pytest_plugin/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,11 +100,11 @@ The plugin runs in one of three modes, picked at invocation.
| **Offline** | `--sift-offline` | No; records to a log file for later replay | Environments without Sift access. |
| **Disabled** | `--sift-disabled` | No | Local dev. Bounds still evaluate and return a real pass/fail. |

Online mode pings Sift once at session start and aborts if Sift is unreachable or the credentials are invalid,
so a misconfigured job fails immediately instead of silently producing no report.
During the run, every create and update is appended to a JSONL log file.
A background worker uploads new entries to Sift incrementally.
If the connection drops mid-test, the test keeps running and the log keeps writing locally.
Online mode pings Sift once at session start and aborts if Sift is unreachable or the credentials are invalid,
so a misconfigured job fails immediately instead of silently producing no report.
During the run, every create and update is appended to a JSONL log file.
A background worker uploads new entries to Sift incrementally.
If the connection drops mid-test, the test keeps running and the log keeps writing locally.
The remaining entries can be uploaded afterward by running import-test-result-log, which the plugin prints on exit. That command resumes into the report the interrupted run created rather than starting a second one.

See [Running Modes](running_modes.md) for the log-file and replay pipeline,
Expand Down
79 changes: 79 additions & 0 deletions python/examples/data_import/mcap/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
"""Import an MCAP (.mcap) file into Sift.

MCAP files are self-describing, so the import needs no column mapping: every
channel of every supported topic (ros2msg schemas with cdr messages) is
imported. Channel names combine the topic and flattened field path; the
bundled sample_data.mcap produces "/imu/data.angular_velocity.x",
"/imu/data.linear_acceleration.x", "/battery.voltage", and their siblings.

Swap sample_data.mcap for your own .mcap recording.
"""

import os

from dotenv import load_dotenv
from sift_client import SiftClient

if __name__ == "__main__":
load_dotenv()

grpc_uri = os.getenv("SIFT_GRPC_URI")
assert grpc_uri, "expected 'SIFT_GRPC_URI' environment variable to be set"

rest_uri = os.getenv("SIFT_REST_URI")
assert rest_uri, "expected 'SIFT_REST_URI' environment variable to be set"

apikey = os.getenv("SIFT_API_KEY")
assert apikey, "expected 'SIFT_API_KEY' environment variable to be set"

asset_name = os.getenv("ASSET_NAME")
assert asset_name, "expected 'ASSET_NAME' environment variable to be set"

client = SiftClient(api_key=apikey, grpc_url=grpc_uri, rest_url=rest_uri)

# Auto-detect the config and import the file.
import_job = client.data_import.import_from_path(
"sample_data.mcap",
asset=asset_name,
)

import_job.wait_until_complete()

# If auto-detect doesn't quite match your file, inspect the config and patch
# it before importing. Common fixes: drop channels you don't need, rename or
# retype a channel, skip undecodable topics, set a start time for logs on a
# non-Unix epoch, or pick metadata records to import.
#
# from datetime import datetime, timezone
#
# from sift_client.sift_types.data_import import (
# McapComplexTypesImportMode,
# McapParseErrorPolicy,
# )
#
# config = client.data_import.detect_config("sample_data.mcap")
# print(config) # inspect every detected channel
#
# # Example: import array fields only as JSON strings, instead of the
# # default of both JSON and Arrow IPC bytes
# config.complex_types_import_mode = McapComplexTypesImportMode.STRING
#
# # Example: import only the IMU topic
# config.data = [d for d in config.data if d.topic == "/imu/data"]
#
# # Example: skip undecodable topics and records instead of failing
# config.parse_error_policy = McapParseErrorPolicy.IGNORE_ERROR
#
# # Example: reinterpret log_time as elapsed nanoseconds from an explicit
# # start; only for recorders whose clock did not track Unix time
# config.relative_start_time = datetime(2026, 1, 1, tzinfo=timezone.utc)
#
# # Example: import every key of a named metadata record as run metadata
# config.metadata_records = ["calibration"]
#
# import_job = client.data_import.import_from_path(
# "sample_data.mcap",
# asset=asset_name,
# config=config,
# )
# import_job.wait_until_complete()
2 changes: 2 additions & 0 deletions python/examples/data_import/mcap/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
python-dotenv
sift-stack-py[mcap]
Binary file added python/examples/data_import/mcap/sample_data.mcap
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
CsvImportConfig,
Hdf5ImportConfig,
ImportConfig,
McapImportConfig,
ParquetFlatDatasetImportConfig,
ParquetSingleChannelPerRowImportConfig,
TdmsImportConfig,
Expand Down Expand Up @@ -47,6 +48,8 @@ def _set_config_on_request(
request.hdf5_config.CopyFrom(config._to_proto())
elif isinstance(config, UlogImportConfig):
request.ulog_config.CopyFrom(config._to_proto())
elif isinstance(config, McapImportConfig):
request.mcap_config.CopyFrom(config._to_proto())
else:
raise TypeError(f"Unsupported import config type: {type(config).__name__}")

Expand Down
Loading
Loading