hurd: clean up module#5242
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
I just saw the one thing then noticed the links, are you sure that's the most recent source? Per https://cgit.git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=t/sysvshm the branch you're linking hasn't been updated since 2021, and none of the branches at https://cgit.git.savannah.gnu.org/cgit/hurd/glibc.git/refs/heads are any newer.
There was a problem hiding this comment.
"hurd: remove duplicate routines": LGTM @ 62ccc91
| #[cfg(target_pointer_width = "32")] | ||
| pub l_type: c_int, | ||
| #[cfg(target_pointer_width = "32")] | ||
| pub l_whence: c_int, | ||
| #[cfg(target_pointer_width = "64")] | ||
| pub l_type: c_short, | ||
| #[cfg(target_pointer_width = "64")] | ||
| pub l_whence: c_short, | ||
| pub l_start: __off_t, | ||
| pub l_len: __off_t, | ||
| pub l_start: off_t, | ||
| pub l_len: off_t, | ||
| pub l_pid: __pid_t, | ||
| } | ||
|
|
||
| pub struct flock64 { | ||
| #[cfg(target_pointer_width = "32")] | ||
| pub l_type: c_int, | ||
| #[cfg(target_pointer_width = "32")] | ||
| pub l_whence: c_int, | ||
| #[cfg(target_pointer_width = "64")] | ||
| pub l_type: c_short, | ||
| #[cfg(target_pointer_width = "64")] | ||
| pub l_whence: c_short, | ||
| pub l_start: __off_t, | ||
| pub l_start: __off64_t, | ||
| pub l_len: __off64_t, | ||
| pub l_pid: __pid_t, |
There was a problem hiding this comment.
"hurd: use gnu_file_offset_bits64 to expose lfs": Is this accurate? Looking at https://github.com/bminor/glibc/blob/04e750e75b73957cf1c791535a3f4319534a52fc/sysdeps/mach/hurd/bits/types/struct_flock.h and https://github.com/bminor/glibc/blob/04e750e75b73957cf1c791535a3f4319534a52fc/sysdeps/mach/hurd/i386/bits/types/struct_flock.h, it looks like c_short is the right type on non-x86-32.
There was a problem hiding this comment.
just reviewed it with the updated glibc sources.
There was a problem hiding this comment.
yes, only i386 is to keep the non-posix-compatible int type for l_type/whence
|
Yeah - the author of #4127 is the lead Hurd developer as I understand it, and that PR links to the regular glibc sourceware repos. I assume that at some point the deviations were resolved and they just switched to developing upstream. @rustbot author since there's probably some things to revisit with that in mind. |
|
Reminder, once the PR becomes ready for a review, use |
There was a problem hiding this comment.
"hurd: deprecate lfs types and routines": Note that like the others, I am not comfortable deprecating the time bindings while users don't have a clean-cut solution.
There was a problem hiding this comment.
so, to a first approximation, how would a clean-cut solution look like?
There was a problem hiding this comment.
I think that what's at #4805 is the best bet, then we can find+replace these back once everything else is ready.
There was a problem hiding this comment.
I'm a bit surprised to see these deprecation warnings indeed, since the Linux port does not have them on its *64 variants.
There was a problem hiding this comment.
Claiming The definitions are equivalent when compiling on a 64bit target is wrong: yes on the 64b target they are equivalent, but then the programmer will migrate to the non-64 version and thusly break the 32bit builds unless gnu_file_offset_bits64 is used. So at most the warnings should be enabled only when the programmer did request for gnu_file_offset_bits64, in which case, yes, using the *64 variants is spurious.
There was a problem hiding this comment.
but again, better do this on all targets (Linux too notably) altogether rather than just Hurd
There was a problem hiding this comment.
those deprecations have now been replaced with comments. they were made before the plan in #4805 had been formulated. now they're just comments tagging some item for future deprecation.
also,
Claiming The definitions are equivalent when compiling on a 64bit target is wrong: yes on the 64b target they are equivalent, but then the programmer will migrate to the non-64 version and thusly break the 32bit builds unless gnu_file_offset_bits64 is used. So at most the warnings should be enabled only when the programmer did request for gnu_file_offset_bits64, in which case, yes, using the *64 variants is spurious.
aren't libc crate users just meant to take care of having the unsuffixed types be used on 64-bit targets, and of having gnu_file_offset_bits64 set on 32-bit targets?
if they're building for multiple targets, surely they must take into account platform differences.
There was a problem hiding this comment.
if they're building for multiple targets, surely they must take into account platform differences.
That's theory...
In practice, people don't think about everything, only really test on 64b platforms, etc. And notably:
of having gnu_file_offset_bits64 set on 32-bit targets?
AFAIK that's a new thing, not everybody will be doing this for a long time still, so we have to care for them not getting shot in the foot too much.
There was a problem hiding this comment.
Noted. I believe the plan at #4805 should better align with having library
users get more comfortable over time with these options.
This comment has been minimized.
This comment has been minimized.
6be7cc8 to
7159a66
Compare
|
@rustbot ready |
| pub type __squad_type = crate::__int64_t; | ||
| pub type __uquad_type = crate::__uint64_t; | ||
| pub type __squad_type = __int64_t; | ||
| pub type __uquad_type = __uint64_t; |
| pub struct fpos_t { | ||
| pub __pos: off_t, | ||
| pub __state: __mbstate_t, | ||
| } | ||
|
|
||
| pub struct __mbstate_t { | ||
| pub __count: c_int, | ||
| pub __value: __c_anonymous___mbstate_t___value, | ||
| } |
There was a problem hiding this comment.
"hurd: fill in definition for fpos_t": let's match linux-gnu and keep these fields private, similarly the union.
libc/src/unix/linux_like/linux/gnu/mod.rs
Line 343 in 328f309
There was a problem hiding this comment.
"hurd: add proper handling of lfs": looks pretty reasonable to me but I'd like the maintainer to ack
| cfg_if! { | ||
| if #[cfg(not(gnu_file_offset_bits64))] { | ||
| const STAT__SPARE_SIZE: usize = { | ||
| if size_of::<__fsid_t>() == size_of::<c_int>() { | ||
| 12 | ||
| } else { | ||
| 11 | ||
| } | ||
| }; | ||
| } else { | ||
| const STAT__SPARE_SIZE: usize = { | ||
| if size_of::<__fsid_t>() == size_of::<c_int>() { | ||
| 9 | ||
| } else { | ||
| 8 | ||
| } | ||
| }; | ||
| } | ||
| } |
There was a problem hiding this comment.
"hurd: add proper handling of lfs": nit, you can avoid the cfg-if here and instead do ... : usize = { if cfg!(gnu_file_offset_bits64) && size_of::<__fsid_t>() == size_of::<c_int>() { ... } to save a nesting.
Also, looks like there's an extra _ between STAT and SPARE (unless this is in the source?)
There was a problem hiding this comment.
done.
the identifier upstream is simply _SPARE_SIZE. i decided to add another underscore because there's already a "canonical" underscore used for separation between STAT and the actual identifier. considering this is not a public symbol, i decided to opt out of making it match upstream.
| pub st_spare: [c_int; 8usize], | ||
| st_spare: Padding< | ||
| [c_int; if size_of::<__fsid_t>() == size_of::<c_int>() { | ||
| 9 | ||
| } else { | ||
| 8 | ||
| }], | ||
| >, |
There was a problem hiding this comment.
"hurd: add proper handling of lfs": just put this in a const STAT64_SPARE_SIZE like the other, in case we need to reuse that value
| bits: [u32; 32], | ||
| #[cfg(not(all(target_pointer_width = "32", not(target_arch = "x86_64"))))] | ||
| bits: [u64; 16], | ||
| __bits: [__cpu_mask; 1024 / (8 * size_of::<__cpu_mask>())], |
There was a problem hiding this comment.
"hurd: adjust cpu_set_t definition": use __CPU_SETSIZE and __NCPUBITS intermediates like they do rather than hardcoding, just as nonpublic constants.
There was a problem hiding this comment.
decided i would make them public. __CPU_SETSIZE was already public before this pr so i made __NCPUBITS also public.
There was a problem hiding this comment.
"hurd: remove duplicate definitions": lgtm @ 8b36f4b
|
@rustbot author |
This comment has been minimized.
This comment has been minimized.
No, it's not.... The last commit there was 6 years ago... |
| pub st_fstype: c_int, | ||
| pub st_dev: __fsid_t, /* Actually st_fsid */ | ||
| pub st_ino: __ino_t, | ||
| pub st_fsid: __fsid_t, |
There was a problem hiding this comment.
No, we really want st_dev here, to match what the rest of the world expects. st_fsid is the real name in the structure, but st_dev is an alias for it, and that's what people expect. See #3785
|
|
||
| pub fn dirfd(dirp: *mut crate::DIR) -> c_int; | ||
|
|
||
| #[link_name = "__xpg_strerror_r"] |
There was a problem hiding this comment.
Why dropping this?
See https://sourceware.org/git/?p=glibc.git;a=blob;f=string/string.h;h=743395b3e3b2dd52194b1a6b8c34df4f8b56d8a1;hb=HEAD#l464 and linux does the same (except on musl, which is not ported to GNU/Hurd)
There was a problem hiding this comment.
did that because that one declaration exists when !__USE_GNU 1, but throughout the libc crate there's definitions affecting the hurd which assume __USE_GNU.
Footnotes
There was a problem hiding this comment.
assuming gnu functions are available is not the same as assuming that it's the gnu behavior that is desired.
Really, the linux crate does use the xpg redirection: https://github.com/rust-lang/libc/blob/main/src/unix/linux_like/linux_l4re_shared.rs#L1699 , and thus programs expect the xpg behavior, not the gnu behavior.
There was a problem hiding this comment.
i still don't get it. the linux definitions don't affect the hurd but i'm assuming you mean that as an example. why should we provide bindings to the non-gnu version if we have assumed otherwise in other parts of the codebase? granted, the current signature would have to change to that of the gnu version, but that can be readily done as we should be capable of breaking the api on tier 3 targets.
There was a problem hiding this comment.
the linux definitions don't affect the hurd
Sure. But the linux definition affects what people believe the correct behavior is supposed to be.
And in glibc, the definition of strerror_r is exactly the same on Linux and on the Hurd, so it makes full sense to expose the same interface in the linux crate and in the hurd crate.
why should we provide bindings to the non-gnu version if we have assumed otherwise in other parts of the codebase?
Because as I said above, “assuming gnu functions are available is not the same as assuming that it's the gnu behavior that is desired”. And I indeed took the Linux case as an example of this, but it's not only that: it's the very same function, with the very same file inside glibc. So really no reason to diverge from Linux on this.
There was a problem hiding this comment.
So IIUC, we're skipping the actual definition to match the Linux definition we
already expose so that users aren't surprised by the GNU/Hurd definitions
differing? If so, done.
There was a problem hiding this comment.
by the GNU/Hurd definitions differing?
The GNU/Hurd definition of strerror_r does not differ from the GNU/Linux definition. Both Linux and Hurd implementations of strerror_r come from the same glibc source code.
done
Thanks.
There was a problem hiding this comment.
So we're basically assumming that both GNU/Linux and the GNU/Hurd do not have
_GNU_SOURCE defined and would always want the POSIX version of this routine?
I understand that this seems to be the default from 1, but I'm not
sure.
Footnotes
There was a problem hiding this comment.
So we're basically assumming that both GNU/Linux and the GNU/Hurd do not have
_GNU_SOURCE defined and would always want the POSIX version of this routine?
I'm basically assuming that what people will want is what people are currently getting: all ports are currently returning an int, i.e. the posix behavior. So better just align on that, otherwise we will get unexpected issues.
Really, in general any divergence from what the rest of the world is currently doing only brings troubles.
|
|
||
| #[cfg_attr( | ||
| all(gnu_time_bits64, not(gnu_file_offset_bits64)), | ||
| link_name = "__glob64_time64" |
There was a problem hiding this comment.
? this doesn't match the conditions of https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/glob.h;h=4c824eb0d30ec557f9c5d5e1d85d823e1a1900c6;hb=HEAD#l158
And note that GNU/Hurd doesn't define __USE_TIME64_REDIRECTS
There was a problem hiding this comment.
fixed. there's one thing to note, though: we recently made it so that our cfgs to handle redirects of time64 and fileoffset64 could not be set separately (though at present we only issue a warning at compile time.) this may conflict with the behavior in upstream glibc where you seem to be capable of opting into one or the other.
|
|
||
| #[cfg_attr( | ||
| all(gnu_time_bits64, not(gnu_file_offset_bits64)), | ||
| link_name = "__globfree64_time64" |
There was a problem hiding this comment.
|
|
||
| #[cfg_attr( | ||
| all(gnu_time_bits64, not(gnu_file_offset_bits64)), | ||
| link_name = "__glob64_time64" |
There was a problem hiding this comment.
| pub fn globfree64(pglob: *mut glob64_t); | ||
| #[cfg_attr( | ||
| all(gnu_time_bits64, not(gnu_file_offset_bits64)), | ||
| link_name = "__globfree64_time64" |
There was a problem hiding this comment.
|
Thanks for this work! I'd rather actually test this natively before letting it merged, unfortunately I won't have time to do this before at least tuesday. |
| pub type timezone; | ||
| } | ||
|
|
||
| const STAT__SPARE_SIZE: usize = { |
There was a problem hiding this comment.
it's a bit sad to make these far from the structure that uses it. Is it not possible to embed the condition in the structure itself? for instance by emitting several st_spare definitions, each conditioned appropriately. We don't really strictly need to define a STAT__SPARE_SIZE condition just to mimick what glibc uses, _SPARE_SIZE is not part of the API.
There was a problem hiding this comment.
those constants weren't public anyway. i've inlined the logic in the structure now.
i forgot to update the pr description, but i was also told about that little after submitting the patches so the current state should reflect upstream glibc. |
0cb1b12 to
0e14b9b
Compare
| #[deprecated( | ||
| since = "0.2.187", | ||
| note = "Use `glob_t` instead. The definitions are equivalent." | ||
| )] |
There was a problem hiding this comment.
is this a leftover?
There was a problem hiding this comment.
I see that it's an explicit commit fd26718
But here again, the same happens in linux. The cleanup should really be done together in a separate PR.
I mean: linux and hurd use the same glibc, only a few functions really differ. We should really avoid divergence as much as possible in rust-libc since it's just the same codebase behind ; the more we diverge, the more difficult it becomes to maintain them.
There was a problem hiding this comment.
We don't have a glob_t on Linux 1. Open to getting rid of these
deprecations, but I think it's best if I don't add a type nobody has requested.
Footnotes
There was a problem hiding this comment.
We don't have a glob_t on Linux 1.
Mmm, then I am lost. Your deprecation warning is telling people to use glob_t, but Linux does not provide one? Are people supposed to either use glob_t or glob64_t depending on the OS? (only hurd and linux seem to be providing glob64_t)
There was a problem hiding this comment.
So it seems. That's why I mentioned in my last message that I was hesitant to
add another type (referring to glob_t on Linux.)
You think that I should remove the deprecation concerning the GNU/Hurd bindings
until somebody requests glob_t on Linux and then I can deprecate them all?
There was a problem hiding this comment.
You think
At this point, I don't think anything. I'm very surprised that glob_t not be available on Linux but glob64_t be.
If anything, I think we should align on what is already there for other OSes, and not try to clean up the Hurd crate more than the rest, and rather clean all creates on this point in the same PR when it becomes clear what is aimed for.
There was a problem hiding this comment.
I did a quick search in the debian source code, there does not see to be any rust package there that calls globfree or globfree64, so it does look like the functions have never been really used, and thus the issue never actually raised. But possibly that is so just because it is not available on Linux (which is really a gold reference to people), and if it was available on Linux, people would start using it.
In coherence with the lfs cleanup, we should indeed aim for telling people to use glob_t. But I don't see why doing so in a different way than the other lfs functions?
There was a problem hiding this comment.
The glob_t matter is separate from the LFS matters because these are
concerned with non-immediate changes that require first documenting them with a
small FIXME and later on (hopefully a few months from now) deprecating them.
This is because file offset types are way too many and have too much potential
impact on end users to deprecate in a single go.
glob64_t, on the other hand, is more easily "deprecatable." It's only on the
GNU/Hurd bindings and it's already got a perfect equivalent. I believe this
type we can already deprecate and eventually remove in favor of glob_t.
There was a problem hiding this comment.
glob64_t, on the other hand, is more easily "deprecatable." It's only on the GNU/Hurd bindings
No, it's also in the Linux bindings.
There was a problem hiding this comment.
Forgive my wording; I meant to say that the only bindings in which both
glob_t and glob64_t are exposed are those of the GNU/Hurd. That's why we
can simply go ahead and deprecate the latter; Users have an alternative with
glob_t.
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
Add GNU/Hurd definition and skip the opaque type definition in the top-level `unix` module. The definition fits that of [^1]. [^1]: https://sourceware.org/git/?p=glibc.git;a=blob;f=libio/bits/types/__fpos_t.h;h=bb04576651b9097b3027e4299cc30c88f334535f;hb=b5eecf35d91a3b29cff80b731bbc6bcce6863827
| )] | ||
| #[cfg_attr( | ||
| all(gnu_file_offset_bits64, not(gnu_time_bits64)), | ||
| link_name = "glob64" |
There was a problem hiding this comment.
the link_names must be glob64free, not glob64
There was a problem hiding this comment.
I believe it's actually none of those; globfree64 seems to be the right
choice 1. Fixed now.
Footnotes
| flags: c_int, | ||
| errfunc: Option<extern "C" fn(epath: *const c_char, errno: c_int) -> c_int>, | ||
| pglob: *mut glob64_t, | ||
| pglob: *mut glob_t, |
There was a problem hiding this comment.
the type must remain glob64_t, otherwise this will be bogus in the non-gnu_file_offset_bits64 case
There was a problem hiding this comment.
Will it, now? For all we care on the Rust side of things, the types for
glob_t and glob64_t are the same across FFI boundaries. The function
pointer fields are the only ones that differ upstream, but those we currently
choose to expose as both *mut c_void and private.
There was a problem hiding this comment.
But don't we want to enforce through typing that a glob_t returned by glob is freed by globfree, and a glob64_t returned by glob64 is freed by globfree64?
There was a problem hiding this comment.
I mean, this is a set of C bindings. Actually providing good typing would
likely involve setting up a higher-level abstraction that enforces relatively
well-defined semantics. But this is not really a point of contention for me, so
I can change it if it seems more coherent to you.
There was a problem hiding this comment.
Yes, I believe coherency is important, to avoid people getting surprised by what they read
| since = "0.2.187", | ||
| note = "Use `globfree` instead. Their definitions are equivalent." | ||
| )] | ||
| pub fn globfree64(pglob: *mut glob_t); |
There was a problem hiding this comment.
the type must remain glob64_t, otherwise this will be bogus in the non-gnu_file_offset_bits64 case
Add uses of the `gnu_file_offset_bits64` `cfg` to the `hurd` module. GNU/Hurd systems use glibc and that includes both the file offset and `time_t` feature test macros. This patch ensures the right definition is exposed on 64-unsuffixed types. Affected records don't directly use the `cfg` to define their fields conditionally. They instead use the types not prefixed with two underscores to define their fields. These types are themselves conditionally defined. This avoids cluttering both the type alias definition and the record definition.
Modify definition of `cpu_set_t` to use the same constant expression as used upstream [^1]. This resolves to the same values as before but relies on the conditional definition of the type `__cpu_mask` is defined in terms of. [^1]: https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/bits/cpu-set.h;h=ddb79cefc2e5d93003f8da84eca6302f99153a22;hb=b5eecf35d91a3b29cff80b731bbc6bcce6863827
Remove routine bindings that were already present in the `unix` module. Those in the top-level `unix` module were already outfit with the corresponding link name redirection under one of `gnu_file_offset_bits64` or `gnu_time_bits64`. They have been preferred over those in the `hurd` submodule.
Add deprecation notices to "glob64_t" and 64-suffixed routines. The bindings we expose have no difference whatsoever between `glob_t` and `glob64_t`. Differing fields are exposed here as `void *`s.
Add deprecation notice to "fallocate64" because this routine is only exposed under Linux in glibc [^1]. [^1]: https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/bits/fcntl-linux.h;h=587b815124318864320973b864899b19466e46b8;hb=HEAD#l450
Add comments annotating file offset types and routines whose identifier is 64-suffixed as slated for deprecation and eventual removal. this holds only for 64-bit targets, so the comment adds a "parameter" to the FIXME that will be found+replaced with a deprecation attribute later on. These types otherwise provide redundant definitions.
Description
This PR consists of an overall clean up of the GNU Hurd module. It adds proper handling of types and routines when
_USE_FILE_OFFSET64is defined by using the existinggnu_file_offset_bits64cfg. It also fixes a type definition and more closely mirrors those of upstream.It also adds a number of future deprecation annotations. These apply only to 64-bit targets on type aliases. They apply to both 64-bit targets and any target with the afore mentioned
cfgset in the case of certain records.Finally, there were some routines that were duplicated in both the
unixmodule and theunix/hurdmodule. These have ben removed off of the latter. This decision was made because theunixmodule's definitions already included the right link name redirection for bindings where the actual identifier is the 64-bit suffixed one.Sources
Checklist
libc-test/semverhave been updated*LASTor*MAXare included (see #3131)cd libc-test && cargo test --target mytarget); especially relevant for platforms that may not be checked in CI@rustbot label +stable-nominated