[SYSTEMDS-3524] Add multi-threaded unique in LibMatrixSketch by shieru1214 · Pull Request #2542 · apache/systemds

shieru1214 · 2026-07-11T23:05:41Z

Summary

This PR adds a multi-threaded path for unique in LibMatrixSketch.

The existing single-threaded implementation is kept and is still used as the baseline/fallback. I added a new overload: getUniqueValues(MatrixBlock blkIn, Types.Direction dir, int k), where k controls the requested number of threads.

The implementation follows the existing SystemDS unique direction semantics:

RowCol: unique scalar values over the input matrix, returned as one column
Row: row-wise unique scalar values, preserving the number of rows
Col: column-wise unique scalar values, preserving the number of columns

Implementation

For k > 1 and sufficiently large inputs, the code uses balanced row or column ranges and CommonThreadPool.

The parallel logic is direction-specific:

RowCol: each worker builds a local HashSet<Double> over its row range, and the local sets are merged afterwards.
Row: rows are partitioned across workers. The code first computes the maximum number of unique values per row, allocates the output, and then fills the row-wise unique values in parallel.
Col: columns are partitioned across workers. The code first computes the maximum number of unique values per column, allocates the output, and then fills the column-wise unique values in parallel.

There are two parallel paths:

regular parallel deduplication, used when the estimated local memory usage is safe
batched parallel deduplication, used when full thread-local temporary data may be too large

The batched path processes a limited number of ranges at a time, clears temporary local data earlier, and then continues with the next batch. If the input is small, k <= 1, or batching is not useful, the code falls back to the original single-threaded path.

Testing

I added a focused JUnit test:

src/test/java/org/apache/sysds/runtime/matrix/data/LibMatrixSketchUniqueParallelTest.java

It checks RowCol, Row, and Col, and compares the multi-threaded result with the single-threaded baseline.

I ran the following tests:

mvn -Dtest=LibMatrixSketchUniqueParallelTest test
mvn -Dtest=UniqueRow,UniqueRowCol test

The focused test completed with 4 tests passing, and the existing unique tests completed with 9 tests passing. I also ran the focused test with -Xmx64m to exercise the memory-aware batched path.

Runtime experiments

I ran local runtime experiments with fixed k=4, comparing against the k=1 baseline.

The experiments were run locally on an Apple Silicon Mac using Java 17. The JVM heap was fixed at 3 GB, and all inputs used dense MatrixBlock storage.

For each case, I used three warm-up iterations followed by five measured iterations. The order of k=1 and k=4 was alternated between iterations to reduce ordering effects, and the median runtime is reported. Input generation is not included in the measurements.

The experiments cover:

directions: RowCol, Row, and Col
input sizes: small and large
unique shares: 1%, 10%, 50%, and 100%
thread counts: k=1 and k=4

In the tables, Unique Values has a different meaning for each direction:

for RowCol, it is the total number of distinct scalar values in the input
for Row, it is the number of distinct scalar values within each row
for Col, it is the number of distinct scalar values within each column

The exact integer count used for each case is shown in the tables. Speedup is calculated as k=1 time / k=4 time, so a value above 1 means that the multi-threaded path was faster.

The 3 GB heap limit was chosen to keep the large dense inputs within memory while allowing the memory guard to select the batched path for the large Row and Col cases. With this setting, the smaller cases use the regular parallel path, while the large Row and Col cases use batched parallel processing.

These are local measurements and are intended as an initial comparison rather than a full benchmark.

RowCol

Size	Input	Unique Share	Unique Values	k=1 (ms)	k=4 (ms)	Speedup
small	100000x1	1%	1000	1.598	2.362	0.68
small	100000x1	10%	10000	2.041	4.897	0.42
small	100000x1	50%	50000	2.145	5.307	0.40
small	100000x1	100%	100000	3.092	7.047	0.44
large	2000000x1	1%	20000	23.294	20.875	1.12
large	2000000x1	10%	200000	32.340	112.501	0.29
large	2000000x1	50%	1000000	79.177	115.331	0.69
large	2000000x1	100%	2000000	60.601	110.218	0.55

Row

Size	Input	Unique Share	Unique Values per Row	k=1 (ms)	k=4 (ms)	Speedup
small	10000x64	1%	1	10.561	3.722	2.84
small	10000x64	10%	6	12.057	4.279	2.82
small	10000x64	50%	32	14.747	5.811	2.54
small	10000x64	100%	64	18.601	9.271	2.01
large	200000x64	1%	1	209.592	66.916	3.13
large	200000x64	10%	6	237.736	85.111	2.79
large	200000x64	50%	32	291.955	112.381	2.60
large	200000x64	100%	64	362.697	165.816	2.19

Col

Size	Input	Unique Share	Unique Values per Column	k=1 (ms)	k=4 (ms)	Speedup
small	64x10000	1%	1	14.640	4.453	3.29
small	64x10000	10%	6	15.888	5.031	3.16
small	64x10000	50%	32	20.366	6.899	2.95
small	64x10000	100%	64	22.102	9.930	2.23
large	64x200000	1%	1	296.105	87.217	3.40
large	64x200000	10%	6	323.869	99.900	3.24
large	64x200000	50%	32	418.123	135.996	3.07
large	64x200000	100%	64	447.553	220.738	2.03

The row-wise and column-wise cases show consistent speedups because rows and columns can be processed independently. Their speedup generally decreases as the number of unique values grows, since larger sets and outputs require more work.

RowCol behaves differently because its thread-local sets must still be merged into one global set. The large input with a 1% unique share shows a small improvement, but in the other tested cases the allocation and merge overhead is greater than the saved parallel work.

codecov · 2026-07-20T10:52:53Z

Codecov Report

❌ Patch coverage is 9.43396% with 240 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.48%. Comparing base (ae9a59c) to head (bcf2d75).
⚠️ Report is 58 commits behind head on main.

Files with missing lines	Patch %	Lines
...che/sysds/runtime/matrix/data/LibMatrixSketch.java	9.43%	237 Missing and 3 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2542      +/-   ##
============================================
+ Coverage     71.41%   71.48%   +0.06%     
- Complexity    48774    49849    +1075     
============================================
  Files          1571     1602      +31     
  Lines        188912   193298    +4386     
  Branches      37067    37836     +769     
============================================
+ Hits         134917   138184    +3267     
- Misses        43544    44317     +773     
- Partials      10451    10797     +346

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

gaturchenko

Thank you for the contribution @shieru1214. It looks good overall, but there are some issues I would like you to go over.

First, the most critical problem is that the AggregateUnary instruction will always call the single-threaded implementation. With multi-threading in place, it should be only used as a fallback.

Second, the batched multi-threading path has a number of problems related to memory estimation and guards against excessive usage. Below is the summary of suggested fixes:

Int overflow: compute as int end = (int) Math.min((long) pos + config._taskLen, config._len);
The safety condition for Row and Col is numThreads × cellsPerIndex × 64 ≤ budget (one live row/column set per thread). Otherwise, you force batched execution where it can be safely avoided
You have to keep in mind that for batched RowCol the merged HashSet grows toward the distinct count. So you should size taskLen against a fraction of the budget with the remainder reserved for the merged set. If the worst-case simply doesn't fit, fall back to sequential execution.
I'm pretty sure batched tests do not test the respective path in our CI since the JVM heap is never sufficiently low (the worst-case requirement is ~77 MiB). You can, e.g., add an overload like maxLocalBytes, or account for it in any other way you prefer

Also, the formatting job is failing, you need to run the dev/format-changed.sh script. Your fork doesn't have it, so please rebase on the current main so that it's present in your repository as well.

Sign in to view

-		//similar to R's unique, this operation takes a matrix and computes the
-		//unique values (or rows in case of multiple column inputs)
-
+		return getUniqueValues(blkIn, dir, 1);


Sign in to view

+			for( int pos = 0; pos < config._len; ) {
+				ArrayList<UniqueValueTask> tasks = new ArrayList<>();
+				for( int i = 0; i < config._numThreads && pos < config._len; i++ ) {
+					int end = Math.min(pos + config._taskLen, config._len);


Sign in to view

+		for( int pos = 0; pos < config._len; ) {
+			ArrayList<UniqueCountTask> tasks = new ArrayList<>();
+			for( int i = 0; i < config._numThreads && pos < config._len; i++ ) {
+				int end = Math.min(pos + config._taskLen, config._len);


Sign in to view

+		for( int pos = 0; pos < config._len; ) {
+			ArrayList<UniqueOutputTask> tasks = new ArrayList<>();
+			for( int i = 0; i < config._numThreads && pos < config._len; i++ ) {
+				int end = Math.min(pos + config._taskLen, config._len);


Sign in to view

+
+		ExecutorService pool = CommonThreadPool.get(config._numThreads);
+		try {
+			HashSet<Double> hashSet = new HashSet<>();


Sign in to view

+		if( numThreads <= 1 )
+			return null;
+
+		int taskLen = Math.max(1, (int) Math.min(Integer.MAX_VALUE, maxBatchIndexes / numThreads));


Sign in to view

+	 * @return true if local deduplication is small enough for the parallel path
+	 */
+	private static boolean isLocalDedupMemoryBudgetSafe(MatrixBlock blkIn, Types.Direction dir) {
+		return getMaxLocalDedupIndexes(blkIn, dir) >= getPartitionLength(blkIn, dir);


Sign in to view

+ * under the License.
+ */
+
+package org.apache.sysds.runtime.matrix.data;


Sign in to view

+			return 0;
+
+		long bytesPerIndex = cellsPerIndex * Double.BYTES * PAR_UNIQUE_LOCAL_BYTES_OVERHEAD;
+		long maxLocalBytes = Runtime.getRuntime().maxMemory() / PAR_UNIQUE_MAX_LOCAL_BYTES_FRACTION;


…hed memory guards - AggregateUnaryCPInstruction now passes op.getNumThreads() into LibMatrixSketch.getUniqueValues, so the multi-threaded implementation is actually used instead of always falling back to the single-threaded path. - Fix int overflow in the batched task-range computation ((int) Math.min((long) pos + taskLen, len)). - Correct the memory guard for the row-wise and column-wise paths: their workers reuse one set that is cleared per row/column, so only one live set per thread is charged (numThreads * cellsPerIndex * 64 <= budget) instead of charging all indexes, which forced batched execution where it is safely avoidable. - For batched RowCol reserve the all-distinct worst case of the merged set up front and size taskLen against the remaining budget; fall back to sequential execution when even the merged set does not fit. - Add a getUniqueValues overload with an injectable maxLocalBytes budget, move the parallel test into functions.unique so CI actually runs it, and add cases that deterministically exercise all three batched paths and their sequential fallbacks (the heap-derived default budget never triggers them in CI).

Derive the transient deduplication budget from InfrastructureAnalyzer.getLocalMaxMemory() instead of Runtime.getRuntime().maxMemory(). The analyzer reflects the local JVM memory that parfor adjusts per worker, so the raw JVM maximum would over-estimate the budget available to a parfor worker. This also matches the convention used by the neighbouring matrix libraries (e.g. LibMatrixMult). Also expand the direction dispatch into explicit if/return branches so the formatter does not wrap the calls mid-argument.

Adds a benchmark in the existing performance package that compares unique() against the k=1 baseline across different shares of unique values, for all three directions (RowCol, Row, Col) and a small and a large input, at a fixed degree of parallelism. The share of unique values is interpreted per direction, matching what that direction deduplicates: distinct values over the whole input for RowCol, per row for Row and per column for Col. Each configuration is warmed up and then measured repeatedly, the median is reported, the order of the baseline and parallel runs is alternated to reduce ordering effects, and input generation is excluded from the measurements.

Isso-W · 2026-07-26T20:05:00Z

Update: review feedback addressed

Thanks for the detailed review. All points are addressed:

1. AggregateUnary always called the single-threaded implementation.
AggregateUnaryCPInstruction now passes op.getNumThreads() into LibMatrixSketch.getUniqueValues(...). The thread count was already plumbed into UnarySketchOperator by InstructionUtils, it just was not forwarded, so the multi-threaded path was unreachable from DML and the 2-argument overload is now only the k = 1 fallback.

2. Int overflow in the batched task ranges.
All three occurrences now compute int end = (int) Math.min((long) pos + config._taskLen, config._len);.

3. Safety condition for Row and Col.
The row-wise and column-wise workers reuse a single set that is cleared per row/column, so only one live set per thread has to be charged. The guard is now numThreads * cellsPerIndex * 64 <= budget instead of charging every index, which no longer forces batched execution where it can safely be avoided.

4. Batched RowCol and the merged set.
The RowCol workers accumulate over their whole range and are merged into a set that grows toward the total distinct count. The batched configuration now reserves the all-distinct worst case for the merged set up front and sizes taskLen against the remaining budget only. If not even the merged set fits, batching cannot help and the code falls back to sequential execution.

5. The batched paths were not covered in CI.
There is now a getUniqueValues(blkIn, dir, k, maxLocalBytes) overload, so tests can inject a small budget instead of relying on a low JVM heap. The test also moved from org.apache.sysds.runtime.matrix.data to org.apache.sysds.test.functions.unique, because the CI test selection only scans org.apache.sysds.test.* and the test was therefore never executed. Added cases now exercise all three batched paths and their sequential fallbacks deterministically.

Additionally, the memory budget is now derived from InfrastructureAnalyzer.getLocalMaxMemory() rather than Runtime.getRuntime().maxMemory(), so it reflects the local JVM memory that parfor adjusts per worker, matching the convention used in the neighbouring matrix libraries.

The branch is rebased on current main, and dev/format-changed.sh has been applied.

Testing

src/test/java/org/apache/sysds/test/functions/unique/LibMatrixSketchUniqueParallelTest.java

mvn -Dtest=LibMatrixSketchUniqueParallelTest test   ->  10 tests, 0 failures
mvn -Dtest=UniqueRow,UniqueRowCol test              ->   9 tests, 0 failures

19 tests pass in total. Checkstyle reports 0 violations and the format check on the PR-edited lines is clean.

Runtime experiments

The benchmark is included as src/test/java/org/apache/sysds/performance/matrix/UniquePerf.java, so the numbers below can be reproduced:

java -Xmx3g --add-modules jdk.incubator.vector \
  -cp target/classes:target/test-classes:<deps> \
  org.apache.sysds.performance.matrix.UniquePerf 4 5

The experiments were run locally on an Intel Core i7-13700H (20 logical cores) using Java 17, with the JVM heap fixed at 3 GB and dense MatrixBlock inputs throughout.

For each configuration I used three warm-up iterations followed by five measured iterations. The order of the k=1 and k=4 runs is alternated between iterations to reduce ordering effects, and the median runtime is reported. Input generation is not included in the measurements.

The experiments cover the directions RowCol, Row and Col, a small and a large input, unique shares of 1%, 10%, 50% and 100%, and thread counts k=1 and k=4. Unique Values is the number of distinct scalar values in the whole input for RowCol, per row for Row and per column for Col. Speedup is k=1 time divided by k=4 time, so a value above 1 means the multi-threaded path was faster.

These are local measurements and are intended as an initial comparison rather than a full benchmark.

RowCol

Size	Input	Unique Share	Unique Values	k=1 (ms)	k=4 (ms)	Speedup
small	100000x1	1%	1000	3.186	21.061	0.15
small	100000x1	10%	10000	3.164	7.690	0.41
small	100000x1	50%	50000	2.407	6.757	0.36
small	100000x1	100%	100000	3.228	5.975	0.54
large	2000000x1	1%	20000	23.134	23.172	1.00
large	2000000x1	10%	200000	31.379	58.022	0.54
large	2000000x1	50%	1000000	180.886	248.605	0.73
large	2000000x1	100%	2000000	248.350	325.830	0.76

Row

Size	Input	Unique Share	Unique Values per Row	k=1 (ms)	k=4 (ms)	Speedup
small	10000x64	1%	1	41.429	13.344	3.10
small	10000x64	10%	6	44.689	15.397	2.90
small	10000x64	50%	32	64.297	32.751	1.96
small	10000x64	100%	64	88.566	55.402	1.60
large	200000x64	1%	1	821.775	220.942	3.72
large	200000x64	10%	6	921.537	260.261	3.54
large	200000x64	50%	32	1290.565	568.493	2.27
large	200000x64	100%	64	1837.356	1175.834	1.56

Col

Size	Input	Unique Share	Unique Values per Column	k=1 (ms)	k=4 (ms)	Speedup
small	64x10000	1%	1	46.585	15.993	2.91
small	64x10000	10%	6	52.774	16.794	3.14
small	64x10000	50%	32	71.740	34.172	2.10
small	64x10000	100%	64	105.375	58.113	1.81
large	64x200000	1%	1	995.851	289.124	3.44
large	64x200000	10%	6	1117.002	335.624	3.33
large	64x200000	50%	32	1553.361	798.953	1.94
large	64x200000	100%	64	2253.123	1512.326	1.49

Row and Col show consistent speedups, decreasing as the share of unique values grows: at a low share the output is narrow and the runtime is dominated by the parallel scan, while at 100% the second pass writes a full-size output and becomes memory-bound.

RowCol still does not benefit. Its thread-local sets must be merged into one global set, and the merge cost grows with the total distinct count, so at a 100% share the merge re-inserts every distinct value and effectively repeats the sequential work on top of the parallel pass. At low shares the merge is cheap, but the concurrent construction of the per-thread HashSet<Double> instances costs more than the saved scan time. Making RowCol scale would need a different strategy — a shared concurrent set, a tree-shaped merge, or a primitive-keyed hash map to avoid boxing — which seems out of scope for this PR.

gaturchenko

Thank you for the fixes @shieru1214 @Isso-W, we appreciate your effort. I added a comment regarding the test file, please address when you can.

gaturchenko · 2026-07-27T12:44:42Z

This test file is rather inconsistent with the general structure of function tests. The biggest issue is that you don't actually invoke a DML script but rather call the instruction directly. So essentially instead of end-to-end you have unit tests, and we generally prefer the former. Therefore, please have another look at how existing unique tests are implemented. If necessary, you can extend the UniqueBase class. Regarding the tests that constrain the memory budget to trigger the batched path, you can in principle leave them as they are, but I'd still appreciate if you could find a way to make them DML-based as well.

Replaces the unit-style parallel test with end-to-end tests that follow the structure of the existing unique function tests, i.e. they invoke the DML script instead of calling the instruction directly. - adds uniqueCol.dml and UniqueCol, so the column-wise direction has script level coverage at all - extends UniqueBase with an ordered comparison, for cases where every row or column of the result holds a single value and the hash set iteration order cannot affect the outcome, and with a variant that temporarily reduces the local memory budget - adds multi-threaded cases for all three directions, and a batched RowCol case that reaches the memory-aware path from a script run - keeps a small unit test for the batched row-wise and column-wise paths only. Those select batching when numThreads * cellsPerIndex * 64 exceeds the budget, which from an end-to-end run would require the budget to fall into a narrow window depending on the thread count of the executing machine.

The multi-threaded cases so far only covered dense inputs. This adds a sparse counterpart for each direction, following the dense/sparse split of the countDistinct tests. Only every eighth row, every eighth column or every tenth cell is populated, so the input is read in sparse format while every row or column still holds a single distinct value and the expected result stays unambiguous.

Isso-W · 2026-07-28T02:27:11Z

Update: tests are now script-based

Thanks, that was a fair point. The parallel test was indeed a unit test in disguise. It is removed and replaced with end-to-end tests that follow the existing unique tests.

What changed

Deleted LibMatrixSketchUniqueParallelTest (344 lines of direct LibMatrixSketch calls).
Added uniqueCol.dml and UniqueCol. The column-wise direction had no script-level coverage at all before.
Extended UniqueBase with two variants:
- uniqueTestOrdered(...) compares row by row instead of ignoring row order. Usable where every row or column of the result holds a single value, so the hash set iteration order cannot affect the outcome.
- uniqueTestConstrainedMemory(...) temporarily lowers InfrastructureAnalyzer.setLocalMaxMemory(...) around the script run and restores it in a finally block, following the pattern already used in ParForNaiveBayesTest and CompilerTestBase.
Added multi-threaded and sparse cases for all three directions. The sparse inputs populate only every eighth row, every eighth column or every tenth cell, which mirrors the dense/sparse split of the countDistinct tests.

24 cases in total, 22 of them run the DML script.

Batched paths

RowCol batching is now reachable end-to-end: UniqueRowCol.testBatchedCP runs the script under a reduced budget and selects the batched path.

For Row and Col I did not manage to do the same. Those select batching only when numThreads * cellsPerIndex * 64 exceeds the budget, and reaching at least two batches additionally requires budget >= 2 * cellsPerIndex * 64. Both together put the budget into a [2, numThreads) window that depends on the thread count of the executing machine, and it is empty for numThreads = 2. That is a consequence of the corrected guard: after it, Row and Col only need batching when a single row or column is very large. So those two paths are covered by a small unit test, UniqueBatchedPathTest, in the same spirit as CountDistinctRowOrColBase.testCPSparseLarge, which also calls the library directly to reach a state that DML cannot express.

Happy to drop that test if you would rather have the two paths uncovered.

Verification

I instrumented the path selection temporarily to confirm the tests reach what they claim: the script runs use k = 20, the RowCol budget case selects the batched path, and the sparse inputs are read in sparse format. All 24 cases pass, Checkstyle reports 0 violations, and the format check is clean.

gaturchenko

Hey @shieru1214 @Isso-W, some more comments from my side. NB: since we are already past the project submission deadline, please feel free to take as much time as you need to make the changes. Also, if some of my comments are unclear or you disagree with them, feel absolutely free to respond, we can certainly have a discussion about this.

gaturchenko · 2026-07-28T09:07:37Z

+
+		boolean localDedupMemorySafe = isLocalDedupMemoryBudgetSafe(blkIn, dir, k, maxLocalBytes);
+		switch(dir) {
+			case RowCol:


Sorry, I missed this one in my prior review. Based on your benchmarking results, it seems that multi-threading yields a slowdown for RowCol. Can we then just fall back to single-threaded here?

gaturchenko · 2026-07-28T09:14:35Z

+	 * unambiguous, i.e. where every row or column of the result holds a single value and the iteration order of the
+	 * internal hash sets cannot affect the outcome.
+	 */
+	protected void uniqueTestOrdered(double[][] inputMatrix, double[][] expectedMatrix, Types.ExecType instType,


Why do you need this separate function? The existing uniqueTest function compares the results out-of-order that works both when the output is deterministic or random. Am I missing something here?

gaturchenko · 2026-07-28T09:22:22Z

 			writeExpectedMatrix("A", expectedMatrix);

-			compareResultsRowsOutOfOrder(epsilon);
+			if(orderedComparison)


Again as per my comment above, I don't really understand the purpose of orderedComparison

gaturchenko · 2026-07-28T09:39:47Z

Why is this still a unit test? You have limited the memory with InfrastructureAnalyzer but you can limit concurrency as well by calling InfrastructureAnalyzer.setLocalPar(), so I don't quite see what holds you back from exercising the batched path for all 3 cases end-to-end

gaturchenko · 2026-07-28T09:41:45Z

+	 * is one row and independent of any hash set iteration order.
+	 */
+	@Test
+	public void testMultiThreadedCP() {


Thank you for covering the Col case as well. However, what do you mean by large enough? The multi-threaded execution should be picked up by default unless the constraints are violated. Why are the 4 existing tests insufficient?

gaturchenko · 2026-07-28T09:45:09Z

+	 * against needlessly falling back to batched or sequential execution.
+	 */
+	@Test
+	public void testReducedMemoryBudgetCP() {


This should rather be in the batched test, although in principle you compare the results here, but do not actually check that the batched path was selected. So maybe if there is no way to verify it we don't need this test at all

gaturchenko · 2026-07-28T09:45:35Z

+	 * one column and independent of any hash set iteration order.
+	 */
+	@Test
+	public void testMultiThreadedCP() {


Same comment as for Col

gaturchenko · 2026-07-28T09:46:49Z

+	 * needlessly falling back to batched or sequential execution.
+	 */
+	@Test
+	public void testReducedMemoryBudgetCP() {


Same comment as for Col

gaturchenko · 2026-07-28T09:47:21Z

+	 * one column.
+	 */
+	@Test
+	public void testSparseMultiThreadedCP() {


What do you verify with this test? Why is this case different from the other tests?

gaturchenko · 2026-07-28T09:48:09Z

If multi-threading is not beneficial for RowCol, we don't need any new tests here

github-project-automation Bot added this to SystemDS PR Queue Jul 11, 2026

github-project-automation Bot moved this to In Progress in SystemDS PR Queue Jul 11, 2026

gaturchenko suggested changes Jul 20, 2026

View reviewed changes

github-project-automation Bot moved this from In Progress to In Review in SystemDS PR Queue Jul 20, 2026

shieru1214 and others added 7 commits July 25, 2026 03:22

Add multi-threaded unique in LibMatrixSketch

968e63f

Refine multi-threaded unique memory handling

101074e

Account for object overhead in parallel unique memory guard

70954fc

Add batched parallel unique handling

59da7b7

Align unique parallel path with SystemDS semantics

4a89ac7

Isso-W force-pushed the SYSTEMDS-3524 branch from 7c399c7 to 36f48c9 Compare July 25, 2026 02:54

gaturchenko suggested changes Jul 27, 2026

View reviewed changes

Isso-W added 2 commits July 27, 2026 21:31

gaturchenko suggested changes Jul 28, 2026

View reviewed changes

Uh oh!

Conversation

shieru1214 commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implementation

Testing

Runtime experiments

RowCol

Row

Col

Uh oh!

codecov Bot commented Jul 20, 2026

Codecov Report

Uh oh!

gaturchenko left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Isso-W commented Jul 26, 2026

Update: review feedback addressed

Testing

Runtime experiments

RowCol

Row

Col

Uh oh!

gaturchenko left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Isso-W commented Jul 28, 2026

Uh oh!

gaturchenko left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

shieru1214 commented Jul 11, 2026 •

edited

Loading

gaturchenko left a comment •

edited

Loading

gaturchenko left a comment •

edited

Loading