output only unique rows

[k4lizen]

Description of the issue

The output of my query has some duplicated rows (due to the logic of the query). It there a way to tell codeql to remove duplicates?

I don't see anything here: https://codeql.github.com/docs/ql-language-reference/queries/#select-clauses

[oscarsj]

A CodeQL query returns a set of tuples: if your results look duplicated, then they’re not actually identical tuples — there’s some value differing that may not be obvious in the UI (a common culprit is a hidden/distinguishing Location component when viewing results in VS Code).

To remove the apparent duplication, the usual fix is to project away the distinguishing information you don’t care about, i.e. change what you select so that only the relevant “key” is returned:

If you’re selecting an element with an attached Location, but you only care about the textual representation, select a string instead, e.g. x.toString().
Or select just the columns that define uniqueness for your use case (for example: file + line + message), and omit anything that varies across the “duplicate-looking” rows.

[k4lizen]

A CodeQL query returns a set of tuples: if your results look duplicated, then they’re not actually identical tuples — there’s some value differing that may not be obvious in the UI (a common culprit is a hidden/distinguishing Location component when viewing results in VS Code).

I'm not looking at the UI, just directly at the output of the select statement (i.e. what you get with codeql bqrs decode / codeql query run). If I run the output file through sort | uniq about a 1/3 of the rows are dropped.

If you’re selecting an element with an attached Location, but you only care about the textual representation, select a string instead, e.g. x.toString().

This fixed it though, thanks.