Repository navigation
Ensure floats are returned losslessly by the C ABI on 32-bit x86 - #161950
Conversation
|
|
This comment has been minimized.
This comment has been minimized.
|
@bors delegate try |
|
✌️ @beetrees, you can now perform try builds on this pull request! You can now post |
9749c87 to
477f3c8
Compare
|
@bors try |
This comment has been minimized.
This comment has been minimized.
Ensure floats are returned losslessly by the C ABI on 32-bit x86 try-job: i686-*
This comment has been minimized.
This comment has been minimized.
68fdb80 to
bb9b569
Compare
| let fraction = self.zext(fraction, self.type_ix(80)); | ||
|
|
||
| let is_nan_res = self.or(exp_and_sign, fraction); | ||
| let is_nan_res = self.bitcast(is_nan_res, self.type_x86_fp80()); |
There was a problem hiding this comment.
slightly confusing that this works, but apparently it does? (because when stored x86_fp80 would use 12 or 16 bytes).
There was a problem hiding this comment.
The LLVM x86_fp80 type is exactly 80 bits - padding is added when needed by Clang AFAICT:
| @@ -44,6 +72,12 @@ where | |||
| // float aggregates directly in a floating-point register. | |||
| if fn_abi.ret.layout.is_single_fp_element(cx) { | |||
There was a problem hiding this comment.
hmm, is this actually what we want for is_single_fp_element? Should it not also return true on f16 and f128 (or have a clearer name)?
rust/compiler/rustc_abi/src/layout/ty.rs
Lines 158 to 176 in ee4a4a6
There was a problem hiding this comment.
Looking into it, it definitely isn't what we want, and at least on s390x #[repr(C)] struct Foo(f16) is passed incorrectly.
The rust version moves ldgr %f0, %r0 from a general into a float register. With the C compilers the value is already there.
I'll try to clean that up.
There was a problem hiding this comment.
I've left it as is in this PR as I didn't want to put to many changes in one PR, but is_single_fp_element is definitely not correct here. Both Clang and GCC also match f16 on x86, and they also require the struct to have no padding (which isn't checked here at the moment). is_single_fp_element also doesn't handle #[repr(transparent)] structs containing 1ZST fields. is_single_fp_element is also used in the s390x ABI, where GCC and Clang compilers currently implement slightly different rules: padding is fine as long as the overall size if either 2, 4 bytes or 8 bytes. This appears to be GCC and Clang compilers violating the s390x ABI specification, which says "A structure equivalent to one of the above. A structure is equivalent to a type 𝑇 if and only if it has exactly one member, which is either of type 𝑇 itself or a structure equivalent to type 𝑇.", which implies that padding should actually be ignored.
(EDIT: That's strange, GitHub didn't show me your second comment until after I'd posted mine.)
There was a problem hiding this comment.
Oh yeah is_single_fp_element is completely wrong.
Strange that tests/ui/abi/compatibility.rs didn't catch it, it's meant exactly for things like this. Even when I add an f32 test, which will check whether a repr(transparent) wrapper around f32 has the same ABI as f32, nothing fails: #161991
26bbbba to
552a4cb
Compare
This comment has been minimized.
This comment has been minimized.
552a4cb to
ec5132b
Compare
This comment has been minimized.
This comment has been minimized.
ec5132b to
0e8b455
Compare
| // CHECK: ucomiss [[XMM]], [[XMM]] | ||
| // CHECK: jp [[NAN_LABEL:.*]] | ||
| // CHECK: movss dword ptr [esp + [[#OFFSET:]]], [[XMM]] | ||
| // CHECK: fld dword ptr [esp + [[#OFFSET]]] | ||
| // CHECK: ret | ||
| // CHECK: [[NAN_LABEL]]: | ||
| // CHECK: movd [[BITS:.*]], [[XMM]] | ||
| // CHECK: mov dword ptr [esp + [[#OFFSET:]]], 0 | ||
| // CHECK: mov e[[SIGN_AND_EXP:.*]], [[BITS]] | ||
| // CHECK: shl [[BITS]], 8 | ||
| // CHECK: shr e[[SIGN_AND_EXP]], 16 | ||
| // CHECK: mov dword ptr [esp + [[#OFFSET+4]]], [[BITS]] | ||
| // CHECK: or e[[SIGN_AND_EXP]], 32767 | ||
| // CHECK: mov word ptr [esp + [[#OFFSET+8]]], [[SIGN_AND_EXP]] | ||
| // CHECK: fld tbyte ptr [esp + [[#OFFSET]]] | ||
| // CHECK: ret |
There was a problem hiding this comment.
Would it make sense to extract the NaN branch into a function to not blow up the assembly so much?
There was a problem hiding this comment.
It would definitely be possible. The downsides would be that if the function called in the NaN branch was compiled separately (e.g. compiler-builtins) LLVM wouldn't be able to constant fold NaNs, and it would require leaf functions to do more stack setup, negating some of the benefit (a quick check based on the IR for extern "C" fn f(x: f64) -> f64 { x } shows that calling a function in the NaN branch would save a net 4 instructions in that case). Given that extern "C" functions that return f32/f64 are rare in Rust programs (as opposed to C where extern "C" is the default ABI) I don't think it's worth the extra complexity (part of my aim with this PR was to minimise the extra compiler complexity that goes towards supporting 32-bit x86 as much as possible). For comparison, the VaList setup prelude of a extern "C" fn f(x: ...) { f2(x) } function on x86_64-unknown-linux-gnu takes over 20 instructions.
0e8b455 to
3381af6
Compare
|
💔 Test for bf9ccb1 failed: CI. Failed jobs:
|
e40dd5f to
44927e5
Compare
|
@bors try jobs=-i686- |
This comment has been minimized.
This comment has been minimized.
Ensure floats are returned losslessly by the C ABI on 32-bit x86 try-job: *-i686-*
|
@rustbot review |
|
@bors r+ |
Ensure floats are returned losslessly by the C ABI on 32-bit x86 The x86 C ABI returns `f32` and `f64` using the x87 extended precision floating point format (henceforth `x86_fp80`) on the x87 floating point stack. Currently, LLVM uses regular `fld`/`fstp` instructions to convert `f32`/`f64` to and from `x86_fp80` when returning from a function with the C ABI, however these instructions perform floating point format conversions that quieten signalling NaNs, which breaks Rust's (and LLVM's) guarantees. This is a long-standing LLVM bug (llvm/llvm-project#66803) but is difficult to fix on the LLVM side as it ties into larger problems with the way LLVM handles x87 registers holding `f32`/`f64`, combined with the general lack of developer interest in 32-bit x86 without SSE (although the ABI bug specifically also affects targets with SSE2 enabled). This problem has been tracked on the Rust side in rust-lang#115567, with rust-lang#123351 ensuring that Rustic ABIs avoid the x87 stack altogether and rust-lang#113053 adding notes to the platform support page detailing how the targets are non-complaint (including the tier 1 targets `i686-unknown-linux-gnu` and `i686-pc-windows-msvc`). This PR fixes rust-lang#115567 by manually converting NaNs to and from `x86_fp80` when a function returns `f32`/`f64` (this was briefly discussed in [#t-compiler > &rust-lang#96;x87_f80&rust-lang#96; is weird @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/.60x87_f80.60.20is.20weird/near/619651653)). r? @tgross35 cc @RalfJung try-job: i686-*
Ensure floats are returned losslessly by the C ABI on 32-bit x86 The x86 C ABI returns `f32` and `f64` using the x87 extended precision floating point format (henceforth `x86_fp80`) on the x87 floating point stack. Currently, LLVM uses regular `fld`/`fstp` instructions to convert `f32`/`f64` to and from `x86_fp80` when returning from a function with the C ABI, however these instructions perform floating point format conversions that quieten signalling NaNs, which breaks Rust's (and LLVM's) guarantees. This is a long-standing LLVM bug (llvm/llvm-project#66803) but is difficult to fix on the LLVM side as it ties into larger problems with the way LLVM handles x87 registers holding `f32`/`f64`, combined with the general lack of developer interest in 32-bit x86 without SSE (although the ABI bug specifically also affects targets with SSE2 enabled). This problem has been tracked on the Rust side in rust-lang#115567, with rust-lang#123351 ensuring that Rustic ABIs avoid the x87 stack altogether and rust-lang#113053 adding notes to the platform support page detailing how the targets are non-complaint (including the tier 1 targets `i686-unknown-linux-gnu` and `i686-pc-windows-msvc`). This PR fixes rust-lang#115567 by manually converting NaNs to and from `x86_fp80` when a function returns `f32`/`f64` (this was briefly discussed in [#t-compiler > &rust-lang#96;x87_f80&rust-lang#96; is weird @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/.60x87_f80.60.20is.20weird/near/619651653)). r? @tgross35 cc @RalfJung try-job: i686-*
…uwer Rollup of 23 pull requests Successful merges: - #165928 (Replace most uses of `type_array` with `type_byte_array` in `rustc_codegen_llvm`) - #160618 (require a consistent codegen backend when -Zllvm-target-feature is present) - #161950 (Ensure floats are returned losslessly by the C ABI on 32-bit x86) - #162267 (Fix an edge within `std::fs::create_dir_all` for certain platforms that views trailing cur dir components as a separate missing directory components) - #164043 (alloc: Unstably add `Vec::repeat` and `Vec::repeat_in`) - #164808 (asm const ptr: move name generation to asm-specific logic) - #164823 (Use `DerefMut` and `IndexMut` before `&raw mut` autoborrow) - #165568 (rustdoc: Suggest corrections for reference label typos) - #165641 (Treat all proc macro dependencies as host dependencies in bootstrap) - #165924 (rustc_codegen_gcc subtree update 2026-11-13) - #165927 (Remove `const_struct` from `rustc_codegen_ssa`) - #163696 (Stabilize `int_from_ascii`) - #163882 (docs(vec): clarify into_boxed_slice allocation guarantee) - #164362 (Emit note for `non_exhaustive` within defining crate for `unreachable_patterns`) - #164465 (Under `guard_patterns` allow parenthesized guarded patterns in match arms) - #164997 (Fix as str suggestion in subpattern) - #165825 (make `Complex<x87_f80>` ABI-compatible with C) - #165868 (Small changes to dyn compatibility) - #165879 (Add `#[rustc_test_entrypoint_marker]` back !) - #165902 (Add allocator() to vec_deque::IntoIter and vec_deque::Drain) - #165903 (Fix borrowck ICE when a borrow outlives a local in a const block) - #165914 (Use `trim_prefix` in bootstrap) - #165919 (tests: accept better behavior on underaligned accesses) Failed merges: - #165893 (Prune un/under used methods of `ast::Attribute`)
…uwer Rollup of 23 pull requests Successful merges: - #165928 (Replace most uses of `type_array` with `type_byte_array` in `rustc_codegen_llvm`) - #160618 (require a consistent codegen backend when -Zllvm-target-feature is present) - #161950 (Ensure floats are returned losslessly by the C ABI on 32-bit x86) - #162267 (Fix an edge within `std::fs::create_dir_all` for certain platforms that views trailing cur dir components as a separate missing directory components) - #164043 (alloc: Unstably add `Vec::repeat` and `Vec::repeat_in`) - #164808 (asm const ptr: move name generation to asm-specific logic) - #164823 (Use `DerefMut` and `IndexMut` before `&raw mut` autoborrow) - #165568 (rustdoc: Suggest corrections for reference label typos) - #165641 (Treat all proc macro dependencies as host dependencies in bootstrap) - #165924 (rustc_codegen_gcc subtree update 2026-11-13) - #165927 (Remove `const_struct` from `rustc_codegen_ssa`) - #163696 (Stabilize `int_from_ascii`) - #163882 (docs(vec): clarify into_boxed_slice allocation guarantee) - #164362 (Emit note for `non_exhaustive` within defining crate for `unreachable_patterns`) - #164465 (Under `guard_patterns` allow parenthesized guarded patterns in match arms) - #164997 (Fix as str suggestion in subpattern) - #165825 (make `Complex<x87_f80>` ABI-compatible with C) - #165868 (Small changes to dyn compatibility) - #165879 (Add `#[rustc_test_entrypoint_marker]` back !) - #165902 (Add allocator() to vec_deque::IntoIter and vec_deque::Drain) - #165903 (Fix borrowck ICE when a borrow outlives a local in a const block) - #165914 (Use `trim_prefix` in bootstrap) - #165919 (tests: accept better behavior on underaligned accesses) Failed merges: - #165893 (Prune un/under used methods of `ast::Attribute`)
Rollup merge of #161950 - beetrees:x86-ret-snan-c, r=folkertdev Ensure floats are returned losslessly by the C ABI on 32-bit x86 The x86 C ABI returns `f32` and `f64` using the x87 extended precision floating point format (henceforth `x86_fp80`) on the x87 floating point stack. Currently, LLVM uses regular `fld`/`fstp` instructions to convert `f32`/`f64` to and from `x86_fp80` when returning from a function with the C ABI, however these instructions perform floating point format conversions that quieten signalling NaNs, which breaks Rust's (and LLVM's) guarantees. This is a long-standing LLVM bug (llvm/llvm-project#66803) but is difficult to fix on the LLVM side as it ties into larger problems with the way LLVM handles x87 registers holding `f32`/`f64`, combined with the general lack of developer interest in 32-bit x86 without SSE (although the ABI bug specifically also affects targets with SSE2 enabled). This problem has been tracked on the Rust side in #115567, with #123351 ensuring that Rustic ABIs avoid the x87 stack altogether and #113053 adding notes to the platform support page detailing how the targets are non-complaint (including the tier 1 targets `i686-unknown-linux-gnu` and `i686-pc-windows-msvc`). This PR fixes #115567 by manually converting NaNs to and from `x86_fp80` when a function returns `f32`/`f64` (this was briefly discussed in [#t-compiler > `x87_f80` is weird @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/.60x87_f80.60.20is.20weird/near/619651653)). r? @tgross35 cc @RalfJung try-job: i686-*
|
This is awesome, thanks a ton. :) On our non-SSE targets, this now means that the |
|
The When SSE is disabled, the mitigations will have little practical affect: while a few edge cases might theoretically benefit, in practice LLVM is extremely likely to load the |
I am not sure what you mean... the usual "Rust" ABI mitigation is that we use SSE registers to return floats, which obviously cannot be applied on non-SSE x86? |
|
Oh right I entirely forgot about that. :) |
View all comments
The x86 C ABI returns
f32andf64using the x87 extended precision floating point format (henceforthx86_fp80) on the x87 floating point stack. Currently, LLVM uses regularfld/fstpinstructions to convertf32/f64to and fromx86_fp80when returning from a function with the C ABI, however these instructions perform floating point format conversions that quieten signalling NaNs, which breaks Rust's (and LLVM's) guarantees. This is a long-standing LLVM bug (llvm/llvm-project#66803) but is difficult to fix on the LLVM side as it ties into larger problems with the way LLVM handles x87 registers holdingf32/f64, combined with the general lack of developer interest in 32-bit x86 without SSE (although the ABI bug specifically also affects targets with SSE2 enabled). This problem has been tracked on the Rust side in #115567, with #123351 ensuring that Rustic ABIs avoid the x87 stack altogether and #113053 adding notes to the platform support page detailing how the targets are non-complaint (including the tier 1 targetsi686-unknown-linux-gnuandi686-pc-windows-msvc).This PR fixes #115567 by manually converting NaNs to and from
x86_fp80when a function returnsf32/f64(this was briefly discussed in #t-compiler > `x87_f80` is weird @ 💬).r? @tgross35
cc @RalfJung
try-job: i686-*