ChangeLog

----------------------------------------------------------------------------

DTLMod (0.5) May 25, 2026

Improvements:
  - Per-transaction compression ratio variability is now active
    - CompressionReductionMethod::get_reduced_variable_global_size() and
      get_reduced_variable_local_size() now accept an optional transaction_id
      parameter and call get_effective_ratio(transaction_id) to compute the
      actual ratio, which may be perturbed around the nominal value when
      ratio_variability > 0
    - Engine::put() passes the current transaction id to both calls, so the
      simulated transfer size varies realistically from transaction to transaction
      when ratio_variability is set
    - The same transaction_id always produces the same effective ratio
      (deterministic hash of variable name and transaction id), making
      simulations reproducible
    - ReductionMethod base-class virtuals and all overrides (Compression,
      Decimation) updated accordingly; Decimation ignores the parameter as its
      reduced sizes are transaction-independent
    - Python bindings updated: get_reduced_variable_global_size() and
      get_reduced_variable_local_size() accept an optional transaction_id
      keyword argument (default 0)
  - Transaction cancellation support
    - New Engine::cancel_transaction(transaction_id) allows an external actor
      to cancel a specific in-flight transaction, unblocking both its
      publishers and subscribers
    - Publishers and subscribers receive a TransactionCanceledException;
      catching it allows them to survive the cancellation and continue to
      subsequent transactions
    - Works with both File and Staging engines across any number of publishers
      and subscribers
  - Memory efficiency: progressive eviction of metadata transaction entries
    - Once all subscribers have consumed a transaction, its entries are evicted
      from the in-memory Metadata::transaction_infos_ map
    - When metadata export is enabled and publishers and subscribers coexist
      (file streaming), evicted entries are progressively flushed to
      per-variable temporary files; the final metadata file is assembled at
      pub_close() from those files and any remaining in-memory entries,
      preserving the existing format and transaction count
    - When the stream is opened by subscribers only after all publishers have
      closed (sequential scenario), memory-only eviction is performed (the
      metadata file has already been written by pub_close())
    - Memory footprint of the File engine now grows as O(N_pub) instead of
      O(N_pub × N_transactions) for long-running concurrent streaming workloads
  - Improved test coverage
    - Comprehensive C++ and Python test suite for transaction cancellation,
      covering cancellation while waiting for a subscriber, while waiting for
      a publisher, and mid-transaction for both File and Staging engines
    - Tests for the 3-parameter form of DTL::add_stream
    - Tests for propagation of reduction parameters to subscribers for both
      compression and decimation methods

Bug Fixes:
  - SZ compression model: the ratio formula incorrectly used -log10(accuracy)
    instead of accuracy^beta, inverting the accuracy-to-ratio relationship;
    constants recalibrated to give ratio ≈ 7 at accuracy = 1e-3 and ratio ≈ 2
    at accuracy = 1e-6
  - CompressionReductionMethod did not register the subscriber-side variable
    during inquire_variable, preventing Engine::get() from simulating
    decompression costs
  - DecimationReductionMethod did not propagate its parameterization to
    subscribers, preventing them from querying reduced variable sizes after
    inquire_variable
  - Transaction cancellation in StagingEngine: CancelException and
    NetworkFailureException thrown during pub_close and sub_close were not
    caught when a cancellation was in progress, leaving in-flight activities
    in an inconsistent state
  - StagingEngine::begin_sub_transaction did not decrement
    num_subscribers_starting_ when a transaction was canceled, causing an
    imbalance in subsequent synchronization
  - CompressionReductionMethod::ParameterizedCompression stored a raw pointer
    to the associated Variable; replaced with the variable name string to
    eliminate potential dangling references

API Changes:
  - New Engine method:
    - Engine::cancel_transaction(unsigned int transaction_id) cancels a
      specific transaction performed by this engine. This method must be
      called from an external actor not involved in the transaction. It raises
      a TransactionCanceledException that publishers and subscribers must catch
      to survive the cancellation and continue to subsequent transactions. If
      both sides have already moved past the given transaction ID, the call is
      a no-op to avoid accidentally canceling a subsequent transaction.
  - New Stream helper methods:
    - Stream::get_engine_type() and Stream::get_transport_method() respectively
      return the enum value of the engine type and transport method for a
      Stream.
    - As a consequence, in the Python bindings, the Stream.engine_type and
      Stream.transport_method readonly properties that return the engine type
      and transport method as a string have been renamed to
      Stream.engine_type_str and Stream.transport_method_str; the existing
      properties now return the enum values.
    - Stream::get_reduction_method(name) retrieves a reduction method
      associated to the stream by name, returning std::nullopt if not found.
  - New ReductionMethod virtual method:
    - ReductionMethod::propagate_for_subscriber(publisher_var, subscriber_var)
      copies the publisher-side parameterization to the subscriber variable;
      the default implementation is a no-op; both DecimationReductionMethod
      and CompressionReductionMethod provide overrides called by
      Stream::inquire_variable.
  - Behavior modifications:
    - DTL::add_stream can now take an Engine::Type and a Transport::Method as
      optional parameters (both defaulting to Undefined if not specified).
    - Reduction methods can now be declared in the DTL configuration file via
      a "reduction_methods" array in the stream definition.

----------------------------------------------------------------------------

DTLMod (0.4) February 16, 2026

Major improvements:
  - Data Reduction framework
    - New abstract ReductionMethod interface for extensible data reduction strategies
    - Decimation method: spatial subsampling with per-dimension stride, optional
      interpolation (linear, quadratic, cubic), configurable computational cost,
      and support for both publisher-side and subscriber-side application
    - Compression method: size reduction with preserved shape, supporting three
      compressor profiles (fixed ratio, SZ-inspired, and ZFP-inspired models),
      separate compression and decompression costs, and per-transaction ratio
      variability
    - Publisher-side reduction state is propagated to subscribers through
      inquire_variable, enabling detection and prevention of conflicting
      double reductions
    - Reduction operations work with both File and Staging engines
  - New documentation pages for the reduction feature (Reduction, Decimation,
    and Compression) and updated Main Concepts page
  - Improved test coverage
    - Comprehensive C++ and Python tests for both reduction methods
    - Coverage of error handling, parameter validation, re-parameterization,
      and publisher-subscriber workflows for each reduction method
    - New test for subscriber-first arrival pattern in File engine
  - Code quality and CI improvements
    - Improved coverage reporting with SonarQube and CodeFactor integration
    - Added subscriber synchronization barrier in File engine
    - Removed unnecessary defensive guards and race condition checks that
      cannot occur under SimGrid's maestro orchestration

API Changes:
  - New Stream method:
    - Stream::define_reduction_method(name) creates a named reduction method
      ("decimation" or "compression") for the stream
  - New Variable methods and properties:
    - Variable::set_reduction_operation(method, parameters) applies a reduction
      with key-value parameters to a variable
    - Variable::is_reduced(), is_reduced_by_publisher(), is_reduced_by_subscriber()
      query the reduction state
    - Variable::get_reduction_method() retrieves the applied ReductionMethod
  - New ReductionMethod class with query methods:
    - get_reduced_variable_global_size(), get_reduced_variable_local_size()
    - get_reduced_variable_shape()
    - get_flop_amount_to_reduce_variable(), get_flop_amount_to_decompress_variable()
  - Full Python bindings for all reduction operations, including nine new
    exception types for parameter validation errors
  - Engine::put() now automatically simulates reduction cost and transports the
    reduced data size when the variable is reduced
  - Engine::get() now automatically simulates decompression cost after receiving
    compressed data

----------------------------------------------------------------------------

DTLMod (0.3) January 19, 2026

Major improvements:
  - Enhanced code quality with comprehensive refactoring 
    - Improved modern C++17 usage throughout the codebase
    - Reduced code complexity and technical debt
  - Memory safety and robustness improvements
    - Fixed critical memory safety issue in DTL connection management
    - Fixed race condition in Engine creation
    - Plugged memory leaks
  - Major refactoring with improved memory management, encapsulation, and move-only semantics
  - Improved CI/CD infrastructure
    - Added Valgrind and sanitizer checks (AddressSanitizer, UndefinedBehaviorSanitizer)
    - Enhanced test coverage and validation
  - Comprehensive proofread of the documentation

API Changes:
  - DTL::get_stream_by_name_or_null() renamed to DTL::get_stream_by_name()
    Returns std::optional<std::shared_ptr<Stream>> instead of raw pointer
    Python: dtl.get_stream_by_name() now returns None if stream not found
  - Metadata management transferred from Engine to Stream
    Stream::get_metadata_file_name() replaces Engine::get_metadata_file_name()
    Python: stream.metadata_file_name replaces engine.metadata_file_name
  - Stream method chaining improvements
    set_engine_type() and set_transport_method() now return Stream& instead of Stream*
    Enables cleaner fluent interface: stream.set_engine_type(...).set_transport_method(...)
  - DTL::create() now accepts std::string_view and has empty default parameter
  - String parameters changed to std::string_view for better performance:
    - DTL::add_stream(), DTL::get_stream_by_name()
    - Stream and Variable constructors and methods
  - get_engine_type_str() and get_transport_method_str() now return std::optional<const char*>

----------------------------------------------------------------------------

DTLMod (0.2) November 11, 2025

  - New comprehensive documentation
  - Full python bindings
  - Improved code coverage
  - Reduced technical debt
  - Fixed deadlocks in File engine
  - Update to SimGrid 4.1 and FSMod 0.4
  - API:
    - Make metadata export configurable at stream level.
    - Have a simpler version of Engine::put() that automatically uses the
      local size of the Variable.
    - Stream::remove_variable now throws an UnknownVariableException rather
      than returning a bool

----------------------------------------------------------------------------

DTLMod (0.1) July 4, 2025

First Release

----------------------------------------------------------------------------