Skip to content

Commit 558530c

Browse files
authored
Test that a coerced TYPE_CONST_STRING argument stays in place (#213)
## Problem `TYPE_CONST_STRING` stores a pointer into a Ruby String: ```c case TYPE_CONST_STRING: ... else { dst->pointer = rb_string_value_cstr(src); } ``` A String argument is safe. `argv` holds it and keeps it in place. A `to_str` object is not. `rb_string_value_cstr` writes the coerced String back into `src`. `argv` still holds the original object, not the String. The next loop turn overwrites `src`. After that, only `converted_args` holds the String. The pointer must stay valid until `ffi_call` returns. `ffi_call` runs under `rb_thread_call_without_gvl`, so another thread can compact the heap during the call. `converted_args` must keep the String **in place**, not only alive. Master does this. `converted_args` sits in the `ALLOCV` buffer. The GC scans that buffer conservatively, and a conservative scan pins the object. ## Gap No test covers this. Before #205, `converted_args` was a Ruby Array. The GC may move an Array's elements, so the String moved while the pointer stayed, and the C function read the vacated slot. #205 replaced the Array with the buffer and fixed the fault as a side effect. Nothing records that the buffer is required. ## Change 1. `test_call_const_string_from_to_str_after_compaction` in `test/fiddle/test_function.rb`. 2. A comment at `converted_args` in `ext/fiddle/function.c` stating the requirement. The test calls `strcspn` as `[TYPE_CONST_STRING, TYPE_VOIDP] -> TYPE_SIZE_T`. Arg 0 is a `to_str` object returning a fresh 100-byte String. Arg 1 is a `to_ptr` object, which makes Fiddle call `Pointer.[]`; that allocation lets the GC run after the pointer is stored. The subject holds no `"Z"`; the reject set is `"Z"`. A correct call returns 100. A stale pointer returns 0, because a vacated slot holds zero bytes. Three details are load-bearing. Remove any one and the test passes on a faulty build; each has a comment: - Churn is the subject's byte size, half of it dropped. The compactor only evacuates non-full pages. - `to_ptr` returns a **new** `Pointer` each time. An existing `Pointer` gives Fiddle nothing to allocate. - The intact-case call runs **after** the loop. A call before it prepares the CIF, and that preparation is part of what opens the window. ## Result | tree | arm64-darwin, ruby 3.4.10 | aarch64-linux, ruby 3.4.7 | |---|---|---| | `d389bbf` (before #205) | fail | fail | | master | pass | pass | Three runs per tree on darwin, same result each time. Full suite on master with this change: 242 tests, 668 assertions, 0 failures, 1 error, 3 omissions. The error is `test_nogvl_poll` needing `envutil`; it fails the same way without this change. Environment: ruby 3.4.10 [arm64-darwin23]; ruby 3.4.7 [aarch64-linux], glibc 2.36, libffi 3.4.4. ## Note Released versions still have the fault. I measured 1.1.6 and 1.1.8 on both platforms: a two-argument call of this shape returns the wrong result on every iteration under `GC.stress` with `GC.auto_compact`. Master is correct. You may want a release. I report this as latent. The trigger is the declared argument type plus a `to_str` object in place of a String. In one application that loads fiddle at boot, all 227 bundled gems put every `Fiddle::Function.new` behind a Windows-only branch.
1 parent 8dc7c05 commit 558530c

2 files changed

Lines changed: 87 additions & 1 deletion

File tree

ext/fiddle/function.c

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,18 @@ function_call(int argc, VALUE argv[], VALUE self)
304304
(func->is_variadic ? sizeof(int) * n_call_args : 0));
305305
args.values = (void **)((char *)generic_args +
306306
sizeof(fiddle_generic) * n_call_args);
307-
/* GC-scanned (conservatively) as part of the ALLOCV buffer */
307+
/* GC-scanned (conservatively) as part of the ALLOCV buffer.
308+
*
309+
* The conservative scan both keeps these values alive and holds them in
310+
* place, and both properties are required. generic_args can hold a pointer
311+
* into one of them: TYPE_CONST_STRING stores rb_string_value_cstr(&src),
312+
* and for a to_str object that is the coerced String, which argv does not
313+
* hold. ffi_call then runs without the GVL, where another thread can compact
314+
* the heap.
315+
*
316+
* Storing these values somewhere the GC may move them instead, such as a
317+
* Ruby Array, keeps them alive but not in place. The C function then reads
318+
* the slot the String left. Keep them in this buffer. */
308319
converted_args = (VALUE *)((char *)args.values +
309320
sizeof(void *) * (n_call_args + 1));
310321
MEMZERO(converted_args, VALUE, n_call_args);

test/fiddle/test_function.rb

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,81 @@ def test_strcpy
179179
assert_equal("123", str.to_s)
180180
end
181181

182+
# A TYPE_CONST_STRING argument that comes from a to_str object is coerced inside
183+
# Fiddle. The caller's argv then holds that object, not the String that the
184+
# char * points into, so argv does not keep the String in place. Fiddle keeps the
185+
# coerced String in converted_args instead, and the conservative scan of the
186+
# ALLOCV buffer pins it there.
187+
#
188+
# That pin is what this test protects. A converted_args that the garbage
189+
# collector can move, such as a Ruby Array, lets compaction move the String after
190+
# the char * is stored. The C function then reads the slot the String left.
191+
#
192+
# The second argument must be neither a String nor a Pointer. Only then does
193+
# Fiddle call Pointer.[] on it, and that allocation is what lets the garbage
194+
# collector run while the char * is already in place.
195+
def test_call_const_string_from_to_str_after_compaction
196+
omit("Need CRuby") unless RUBY_ENGINE == "ruby"
197+
omit("Need GC.auto_compact=") unless GC.respond_to?(:auto_compact=)
198+
199+
length = 100
200+
subject_class = Class.new do
201+
define_method(:to_str) { "Q" * length }
202+
end
203+
# 4096 bytes keep these bytes in a malloc'd buffer, which compaction does not
204+
# move. A short String would hold its bytes in the object itself, and then this
205+
# pointer could dangle and report a failure that this test is not about.
206+
reject = "Z" + ("\0" * 4095)
207+
# to_ptr must build a new Pointer on every call. A Pointer that already exists
208+
# gives Fiddle nothing to allocate, the garbage collector then does not run
209+
# while the char * is in place, and this test cannot fail.
210+
opener_class = Class.new do
211+
define_method(:to_ptr) { Pointer[reject] }
212+
end
213+
214+
strcspn = Function.new(@libc["strcspn"],
215+
[TYPE_CONST_STRING, TYPE_VOIDP],
216+
TYPE_SIZE_T)
217+
# The subject holds no "Z", so strcspn stops at the terminator and reports the
218+
# full length. A stale char * reports 0, because the slot the String left holds
219+
# zero bytes, and churn content also reports 0.
220+
221+
churn = []
222+
results = []
223+
auto_compact = GC.auto_compact
224+
10.times do
225+
# Churn of the same byte size as the subject, with half of it dropped, so
226+
# that the subject's size pool holds sparsely occupied pages. The compactor
227+
# only evacuates such pages. Without this the subject never moves, and then
228+
# this test cannot fail.
229+
churn.clear
230+
2000.times { churn << ("Z" * length) }
231+
churn.each_index { |i| churn[i] = nil if i.even? }
232+
GC.start
233+
234+
# A short burst of same size allocations that this loop drops at once.
235+
# It leaves a partly filled page in the subject's size pool. The
236+
# compactor only moves objects out of pages that are not full, so
237+
# without this burst the subject can land in a full page. It then never
238+
# moves, and this test cannot fail.
239+
500.times { "y" * length }
240+
241+
begin
242+
GC.auto_compact = true
243+
GC.stress = true
244+
results << strcspn.call(subject_class.new, opener_class.new)
245+
ensure
246+
GC.stress = false
247+
GC.auto_compact = auto_compact
248+
end
249+
end
250+
assert_equal([length] * 10, results)
251+
# The intact case, checked after the loop: a call before it would prepare the
252+
# CIF, and that preparation is part of what lets the collector run inside the
253+
# window on the first measured call.
254+
assert_equal(length, strcspn.call("Q" * length, opener_class.new))
255+
end
256+
182257
def call_proc(string_to_copy)
183258
buff = +"000"
184259
str = yield(buff, string_to_copy)

0 commit comments

Comments
 (0)