Summary
Any select with a take: clause returns empty strings for STR values
that live in the column's byte pool (i.e. longer than the 12-byte inline
limit). The values are correct without take:, and correct again if the query
also sorts (asc: / desc:). Since take: is how LIMIT is expressed, every
limited query over a wide-string column is affected.
Reproduction
(set t (table [s v] (list ["long-string-value-zero" "long-string-value-one" "long-string-value-two"] [0 1 2])))
(println (format "where, no take -> [%]" (at (at (select {from: t s: s where: (> v 0)}) 's) 0)))
(println (format "where + take -> [%]" (at (at (select {from: t s: s where: (> v 0) take: 1}) 's) 0)))
(println (format "take, no where -> [%]" (at (at (select {from: t s: s take: 1}) 's) 0)))
(println (format "where + take + asc -> [%]" (at (at (select {from: t s: s where: (> v 0) asc: s take: 1}) 's) 0)))
(println (format "identity + take -> [%]" (at (at (select {from: t take: 1}) 's) 0)))
(exit 0)
Output:
where, no take -> [long-string-value-one]
where + take -> []
take, no where -> []
where + take + asc -> [long-string-value-one]
identity + take -> []
So (select {from: t take: 1}) — no filter, no projection, just a row limit —
already loses the string.
Expected
take: slices rows; it should not change their values. The STR column should
carry the same strings the unlimited query returns.
Notes
- Only pool-backed values are lost. Strings of 12 bytes or fewer, stored inline
in ray_str_t, survive take: — which is why this can hide in tables of
short strings.
- Reproduces identically on an in-memory table, on a table round-tripped
through .db.splayed.set / .db.splayed.get, and on a splayed table written
by .csv.splayed, at every row count tried (3 to 2M).
strlen over the same column in the same query returns the correct lengths,
so the descriptors survive the slice. It looks like the per-vector pool is
simply not carried over to the sliced result
(col_propagate_str_pool).
- The result is silently wrong rather than an error, which is what makes it
worth prioritising: a LIMIT query over a URL or title column just reports
empty strings.
Versions
Reproduced on v2.5.14 (release tag, make release) and on dev at
95defb7aef2ff0863e70ab3b79785036cd529b1d, Linux aarch64.
Found while porting Rayforce to
ClickBench.
Summary
Any
selectwith atake:clause returns empty strings forSTRvaluesthat live in the column's byte pool (i.e. longer than the 12-byte inline
limit). The values are correct without
take:, and correct again if the queryalso sorts (
asc:/desc:). Sincetake:is howLIMITis expressed, everylimited query over a wide-string column is affected.
Reproduction
Output:
So
(select {from: t take: 1})— no filter, no projection, just a row limit —already loses the string.
Expected
take:slices rows; it should not change their values. TheSTRcolumn shouldcarry the same strings the unlimited query returns.
Notes
in
ray_str_t, survivetake:— which is why this can hide in tables ofshort strings.
through
.db.splayed.set/.db.splayed.get, and on a splayed table writtenby
.csv.splayed, at every row count tried (3 to 2M).strlenover the same column in the same query returns the correct lengths,so the descriptors survive the slice. It looks like the per-vector pool is
simply not carried over to the sliced result
(
col_propagate_str_pool).worth prioritising: a
LIMITquery over a URL or title column just reportsempty strings.
Versions
Reproduced on
v2.5.14(release tag,make release) and ondevat95defb7aef2ff0863e70ab3b79785036cd529b1d, Linux aarch64.Found while porting Rayforce to
ClickBench.