Add reservation benchmark

[jerbaroo]

This PR adds a benchmark which can be run with just bench-chunks-select. It produces the SVG contained in this PR. Output from the Rust bench is written to a CSV (not committed) which is then read by Python to produce a graph. The Rust bench also prints the information to stdout, which looks like:

SHAPE                          STRATEGY                            SIZE         MEDIAN     P10 / P90
---------------------------------------------------------------------------------------------------------
Seeding database with shape=FewSubmissionsManyChunks and total_chunks=100:
    Metadata values: 10
    Submissions per metadata value: 1
    Chunks per submission: 10
Beginning benchmark of strategy Random
FewSubmissionsManyChunks       Random                                 100          56.5       51.4 / 65.9
Beginning benchmark of strategy PreferDistinct { meta_key: "metadata_value", underlying: Oldest }
FewSubmissionsManyChunks       PreferDistinct(metadata_value, Oldest) 100          111.1      103.1 / 130.8
Seeding database with shape=FewSubmissionsManyChunks and total_chunks=500:
    Metadata values: 10
    Submissions per metadata value: 1
    Chunks per submission: 50
Beginning benchmark of strategy Random
FewSubmissionsManyChunks       Random                                 500          55.8       53.1 / 66.6
Beginning benchmark of strategy PreferDistinct { meta_key: "metadata_value", underlying: Oldest }
FewSubmissionsManyChunks       PreferDistinct(metadata_value, Oldest) 500          182.6      174.3 / 256.1
Seeding database with shape=FewSubmissionsManyChunks and total_chunks=1000:
    Metadata values: 10
    Submissions per metadata value: 1
    Chunks per submission: 100
Beginning benchmark of strategy Random
FewSubmissionsManyChunks       Random                                 1000         55.9       52.1 / 63.2
Beginning benchmark of strategy PreferDistinct { meta_key: "metadata_value", underlying: Oldest }
FewSubmissionsManyChunks       PreferDistinct(metadata_value, Oldest) 1000         344.3      260.8 / 384.3
Seeding database with shape=FewSubmissionsManyChunks and total_chunks=2000:
    Metadata values: 10
    Submissions per metadata value: 1
    Chunks per submission: 200
Beginning benchmark of strategy Random
FewSubmissionsManyChunks       Random                                 2000         55.3       50.9 / 62.7
Beginning benchmark of strategy PreferDistinct { meta_key: "metadata_value", underlying: Oldest }
FewSubmissionsManyChunks       PreferDistinct(metadata_value, Oldest) 2000         499.6      438.2 / 681.2
Seeding database with shape=FewSubmissionsManyChunks and total_chunks=4000:
    Metadata values: 10
    Submissions per metadata value: 1
    Chunks per submission: 400
Beginning benchmark of strategy Random

[ReinierMaas]

I didn't review opsqueue/benches/chunks_select.rs yet, only the top block comment. But I think I already have some good advice that can be helpful.

[ReinierMaas]

Which dependency needed this? And at what time of it lifecycle build or runtime?

Just enabling the benchmarks probably caused it to show-up for you to add it here. What I am thinking is should we also ensure that our packaged opsqueue server has this library path available.

[jerbaroo]

This is added just to the pkgs.mkShell, it fixes the following error from attempting to plot the benchmark results with matplotlib:

Traceback (most recent call last):
  File "/home/jeremy-barisch-rooney/channable/opsqueue/opsqueue/benches/plot_chunks_select.py", line 10, in <module>
    import matplotlib.pyplot as plt  # noqa: E402
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeremy-barisch-rooney/channable/opsqueue/libs/opsqueue_python/.venv-928fc105951d55f0ef3cc1c72c15d87e2ac058c1/lib/python3.13/site-packages/matplotlib/__init__.py", line 161, in <module>
    from . import _api, _version, cbook, rcsetup
  File "/home/jeremy-barisch-rooney/channable/opsqueue/libs/opsqueue_python/.venv-928fc105951d55f0ef3cc1c72c15d87e2ac058c1/lib/python3.13/site-packages/matplotlib/cbook.py", line 24, in <module>
    import numpy as np
  File "/home/jeremy-barisch-rooney/channable/opsqueue/libs/opsqueue_python/.venv-928fc105951d55f0ef3cc1c72c15d87e2ac058c1/lib/python3.13/site-packages/numpy/__init__.py", line 109, in <module>
    from numpy.__config__ import show_config
  File "/home/jeremy-barisch-rooney/channable/opsqueue/libs/opsqueue_python/.venv-928fc105951d55f0ef3cc1c72c15d87e2ac058c1/lib/python3.13/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy._core._multiarray_umath import (
    ...<3 lines>...
    )
  File "/home/jeremy-barisch-rooney/channable/opsqueue/libs/opsqueue_python/.venv-928fc105951d55f0ef3cc1c72c15d87e2ac058c1/lib/python3.13/site-packages/numpy/_core/__init__.py", line 85, in <module>
    raise ImportError(msg) from exc
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python 3.13 from "/home/jeremy-barisch-rooney/channable/opsqueue/libs/opsqueue_python/.venv-928fc105951d55f0ef3cc1c72c15d87e2ac058c1/bin/python"
  * The NumPy version is: "2.5.1"

and make sure that they are the versions you expect.

Please carefully study the information and documentation linked above.
This is unlikely to be a NumPy issue but will be caused by a bad install
or environment on your machine.

Original error was: libstdc++.so.6: cannot open shared object file: No such file or directory

[ReinierMaas]

If you flip FOR EACH amount of chunks and FOR EACH strategy you can extend chunks/submissions instead of generating an entirely new dataset for each measurement. This can greatly speed-up your test runs.

[jerbaroo]

I have updated the code to support extending with chunks. Though I didn't flip the order, so now it looks like:

/// For each shape:
///   For each amount of chunks:
///     Seed (all chunks) or extend (additional chunks) the database
///     FOR EACH strategy:
///         Create new dispatcher.
///         Run benchmark
///         Calculate stats
///         Write result

Notably we cannot do the extending for the FewSubmissionsManyChunks shape.