Drop DLPack dependency; reorganize examples as perf benchmarks

AGENTS.md

@@ -39,7 +39,7 @@ that lazily converts observations to Julia types).
   `from_csv`/`from_json`/`from_parquet`) that re-wrap results in the default
   `"julia"` format.
 - `src/transforms.jl` — `py2jl` / `numpy2jl` / `jl2numpy` / `jl2py`. `py2jl`
-  recursively converts Python containers, numpy arrays (copyless via DLPack), and
+  recursively converts Python containers, numpy arrays (copyless, zero-copy), and
   PIL images into Julia types; `jl2py` is the write-path dual. The `"julia"`
   format is numpy-backed, so numeric array columns decode to real N-D Julia arrays
   and image columns decode to raw numeric arrays (not `Colorant` colorviews).

CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed
+- Dropped the `DLPack` dependency. `numpy2jl` (the zero-copy numpy → Julia conversion behind
+  `py2jl`) now shares the buffer through PythonCall directly (`PyArray` + `unsafe_wrap`), still
+  returning a genuine `Array` so results stay on Julia's fast paths (BLAS, GPU host→device copies).
+  Behavior is unchanged — zero-copy, write-back, reversed axes; read-only/non-contiguous buffers are
+  still copied. A bonus fix: buffer cleanup now routes through PythonCall's GIL-deferred decref
+  instead of DLPack's GIL-reentering finalizer, so a numpy buffer freed on a `DataLoader`
+  `parallel=true` worker thread can no longer deadlock against a thread compiling under the GIL.
+
 ### Added
 - `MLUtils.DataLoader(ds::Dataset; num_workers=N)` works out of the box for process-parallel
   loading, including over `mapobs`/`ObsView`-wrapped datasets. Building on the `Serialization`

Project.toml

@@ -9,7 +9,6 @@ projects = ["test", "docs"]
 [deps]
 Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
 CondaPkg = "992eb4ea-22a4-4c89-a5bb-47a3300528ab"
-DLPack = "53c2dc0f-f7d5-43fd-8906-6c0220547083"
 ImageCore = "a09fc81d-aa75-5fe9-8630-4744c3626534"
 MLCore = "c2834f40-e789-41da-a90e-33b280584a8c"
 PythonCall = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d"
@@ -19,7 +18,6 @@ Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
 [compat]
 Compat = "4.10"
 CondaPkg = "0.2"
-DLPack = "0.3"
 ImageCore = "0.9, 0.10"
 MLCore = "1.1"
 PythonCall = "0.9"

README.md

@@ -26,7 +26,7 @@ pkg> add HuggingFaceDatasets
 
 HuggingFaceDatasets.jl provides wrappers around types from the `datasets` python package (e.g. `Dataset` and `DatasetDict`) along with a few related methods.
 
-Check out the [examples/](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/tree/main/examples) folder for usage examples.
+Check out the [perf/](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/tree/main/perf) folder for usage examples and data-loading benchmarks.
 
 Observations are returned in the `"julia"` format by default, i.e. converted to native julia types on access:
 

docs/src/guide.md

@@ -293,7 +293,7 @@ Note that the format and the custom transform share the same slot: setting the `
 format installs `py2jl` as the transform, and `with_jltransform` then replaces it — which
 is why the transform above calls `py2jl` itself. For layering additional per-batch
 processing on top of the `"julia"` format, prefer `MLUtils.mapobs` (see below), as in the
-[`examples/flux_mnist/flux_mnist.jl`](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/blob/main/examples/flux_mnist/flux_mnist.jl)
+[`perf/flux_mnist/flux_mnist.jl`](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/blob/main/perf/flux_mnist/flux_mnist.jl)
 script.
 
 The order of operations when you index is:
@@ -367,7 +367,7 @@ A few things to note:
 
 !!! warning "Arrays come back transposed"
     numpy is **row-major**, Julia is **column-major**. The zero-copy conversion
-    ([`numpy2jl`](@ref), via DLPack) therefore returns an array whose **dimensions are
+    ([`numpy2jl`](@ref)) therefore returns an array whose **dimensions are
     reversed** relative to the Python side: a numpy array of shape `(d₁, …, dₙ)` becomes a
     Julia array of size `(dₙ, …, d₁)`.
 
@@ -598,4 +598,4 @@ guide for the mechanics and tradeoffs.
 
 A complete example — `mapobs` transform, `num_workers`, and a training loop, benchmarked
 against PyTorch — lives in
-[`examples/flux_mnist/`](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/tree/main/examples/flux_mnist).
+[`perf/flux_mnist/`](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/tree/main/perf/flux_mnist).

docs/src/index.md

@@ -86,7 +86,7 @@ Python: {'label': 5}
 See the [Guide](@ref) for the transform workflow, method forwarding, array/image
 orientation, and integration with MLUtils/Flux data loaders. Runnable examples
 live in the
-[`examples/`](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/tree/main/examples)
+[`perf/`](https://github.com/JuliaGenAI/HuggingFaceDatasets.jl/tree/main/perf)
 folder.
 
 ## Troubleshooting