forward the bootstrap `runner` to `run-make` #141856

folkertdev · 2025-06-01T13:33:05Z

The runner was already forwarded to compiletest, this just passes it on to run-make and uses it in the run functions.

The configuration can look like this

# in bootstrap.toml
[target.s390x-unknown-linux-gnu]
runner = "qemu-s390x -L /usr/s390x-linux-gnu"

Any C compilation automatically sets the correct target. Calls to rustc must use .target(target()). Then, a command like below will work by cross-compiling to the given target, and using the given runner for that target to execute the binary:

./x test tests/run-make/c-link-to-rust-va-list-fn --target s390x-unknown-linux-gnu

The runner can also be used for e.g. running with valgrind.

This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure?

r? @jieyouxu

try-job: test-various
try-job: armhf-gnu

rustbot · 2025-06-01T13:33:11Z

Some changes occurred in src/tools/compiletest

cc @jieyouxu

This PR modifies run-make tests.

cc @jieyouxu

The run-make-support library was changed

cc @jieyouxu

jieyouxu

Thanks, one nit

src/tools/run-make-support/src/run.rs

jieyouxu · 2025-06-01T13:35:09Z

tests/run-make/c-link-to-rust-va-list-fn/rmake.rs


 fn main() {
-    rustc().input("checkrust.rs").run();
+    rustc().input("checkrust.rs").target(target()).run();


Remark: this will not be needed if I finish auto-cross #139244

I removed target(target() bit now (yay!) and use the new needs-target-std in that test file.

jieyouxu · 2025-06-01T13:37:54Z

This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure?

Do you happen to know which CI jobs? armhf-gnu?

folkertdev · 2025-06-01T13:43:27Z

I really don't know what cross-compilation actually happens in our CI and would run this run-make test. I've locally run this test for powerpc and s390x. Testing it with a 32-bit arm sounds like a good idea at least.

jieyouxu · 2025-06-01T13:49:12Z

Let's give it a try.
@bors try

…r, r=<try> forward the bootstrap `runner` to `run-make` The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions. The configuration can look like this ```toml # in bootstrap.toml [target.s390x-unknown-linux-gnu] runner = "qemu-s390x -L /usr/s390x-linux-gnu" ``` Any C compilation automatically sets the correct target. Calls to rustc must use `.target(target())`. Then, a command like below will work by cross-compiling to the given target, and using the given runner for that target to execute the binary: ``` ./x test tests/run-make/c-link-to-rust-va-list-fn --target s390x-unknown-linux-gnu ``` The runner can also be used for e.g. running with `valgrind`. This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure? r? `@jieyouxu` try-job: test-various try-job: armhf-gnu

bors · 2025-06-01T13:50:23Z

⌛ Trying commit ab6646a with merge 1d3fe3c...

jieyouxu · 2025-06-01T13:51:28Z

Reminder (for myself): if try job fails, use @bors2 try to try out new bors

rust-bors · 2025-06-01T13:51:31Z

Unknown command "try`".

bors · 2025-06-01T14:26:38Z

💔 Test failed - checks-actions

jieyouxu · 2025-06-01T14:35:16Z

You probably need sth like //@ ignore-unknown (needs target std)

folkertdev · 2025-06-01T14:56:16Z

Hmm, that directive is never actually used, and also does not solve the issue for that target. It is a known target, just not one that has std (just core I assume). None of the directives in the list seem to really capture this? Maybe needs-threads is a decent proxy but that's not ideal.

jieyouxu · 2025-06-01T15:16:05Z

Yeah, a proper fix is sth like //@ needs-std, but for this PR, //@ ignore-none is a proxy for ignoring the *-unknown targets. I suppose //@ ignore-nvptx is better tho.

EDIT: opened #141863

jieyouxu · 2025-06-01T15:16:28Z

Hmm, that directive is never actually used, and also does not solve the issue for that target. It is a known target, just not one that has std (just core I assume). None of the directives in the list seem to really capture this? Maybe needs-threads is a decent proxy but that's not ideal.

(Sorry, I wrote //@ ignore-unknown but I meant //@ ignore-none as a temporary directive to unblock this PR)

folkertdev · 2025-06-01T15:51:34Z

Apparently you need to explicitly exluce nvptx (several other tests do this).

//@ ignore-none
// Reason: no-std is not supported
//@ ignore-nvptx64-nvidia-cuda
// Reason: can't find crate for 'std'

Implement `//@ needs-target-std` compiletest directive Closes rust-lang#141863. Needed to unblock rust-lang#139244 and rust-lang#141856. ### Summary This PR implements a `//@ needs-target-std` compiletest directive that gates test execution based on whether the target supports std or not. For some cases, this should be preferred over e.g. some combination of `//@ ignore-none`, `//@ ignore-nvptx` and more[^none-limit]. ### Implementation limitation Unfortunately, since there is currently [no reliable way to determine from metadata whether a given target supports std or not](rust-lang#142296), we have to resort to a hack. Bootstrap currently determines whether or not a target supports std by a naive target tuple substring comparison: a target supports std if its target tuple does *not* contain one of `["-none", "nvptx", "switch"]` substrings. This PR simply pulls that hack out into `build_helpers` to avoid reimplementing the same hack in compiletest, and uses that logic to inform `//@ needs-target-std`. ### Auxiliary changes This PR additionally changes a few run-make tests to use `//@ needs-target-std` over an inconsistent combination of target-based `ignore`s. This should help with rust-lang#139244. --- r? bootstrap [^none-limit]: Notably, `target_os = "none"` is **not** a sufficient condition for "target does not support std"

Rollup merge of #142297 - jieyouxu:needs-target-std, r=Kobzol Implement `//@ needs-target-std` compiletest directive Closes #141863. Needed to unblock #139244 and #141856. ### Summary This PR implements a `//@ needs-target-std` compiletest directive that gates test execution based on whether the target supports std or not. For some cases, this should be preferred over e.g. some combination of `//@ ignore-none`, `//@ ignore-nvptx` and more[^none-limit]. ### Implementation limitation Unfortunately, since there is currently [no reliable way to determine from metadata whether a given target supports std or not](#142296), we have to resort to a hack. Bootstrap currently determines whether or not a target supports std by a naive target tuple substring comparison: a target supports std if its target tuple does *not* contain one of `["-none", "nvptx", "switch"]` substrings. This PR simply pulls that hack out into `build_helpers` to avoid reimplementing the same hack in compiletest, and uses that logic to inform `//@ needs-target-std`. ### Auxiliary changes This PR additionally changes a few run-make tests to use `//@ needs-target-std` over an inconsistent combination of target-based `ignore`s. This should help with #139244. --- r? bootstrap [^none-limit]: Notably, `target_os = "none"` is **not** a sufficient condition for "target does not support std"

folkertdev · 2025-06-11T23:21:44Z

Would it help if I made a PR that uses needs-target-std for the tests that fail with nvptx here? It looks like that WIP PR is just accumulating merge conflicts, but adding that attribute doesn't really depend on anything and could happen now.

It's kind of tedious but with vim and some music it should be OK.

jieyouxu · 2025-06-11T23:28:24Z

Would it help if I made a PR that uses needs-target-std for the tests that fail with nvptx here? It looks like that WIP PR is just accumulating merge conflicts, but adding that attribute doesn't really depend on anything and could happen now.

That would definitely be appreciated. Let me rebase that PR later today, and I would like to break that PR up into smaller pieces (some e.g. needs-target-std tests could land in a separate PR). I was waiting for the needs-target-std PR to land first.

Implement `//@ needs-target-std` compiletest directive Closes rust-lang/rust#141863. Needed to unblock rust-lang/rust#139244 and rust-lang/rust#141856. ### Summary This PR implements a `//@ needs-target-std` compiletest directive that gates test execution based on whether the target supports std or not. For some cases, this should be preferred over e.g. some combination of `//@ ignore-none`, `//@ ignore-nvptx` and more[^none-limit]. ### Implementation limitation Unfortunately, since there is currently [no reliable way to determine from metadata whether a given target supports std or not](rust-lang/rust#142296), we have to resort to a hack. Bootstrap currently determines whether or not a target supports std by a naive target tuple substring comparison: a target supports std if its target tuple does *not* contain one of `["-none", "nvptx", "switch"]` substrings. This PR simply pulls that hack out into `build_helpers` to avoid reimplementing the same hack in compiletest, and uses that logic to inform `//@ needs-target-std`. ### Auxiliary changes This PR additionally changes a few run-make tests to use `//@ needs-target-std` over an inconsistent combination of target-based `ignore`s. This should help with rust-lang/rust#139244. --- r? bootstrap [^none-limit]: Notably, `target_os = "none"` is **not** a sufficient condition for "target does not support std"

ignore `run-make` tests that need `std` on targets without `std` In particular, anything that includes `none` in the target triple, and `nvptx64-nvidia-cuda`. Right now we don't cross-compile the `run-make` tests, but we want to in the future. This uses `//@ needs-target-std` introduced in #142297. Useful for #139244 and #141856. The modified files are based on running #141856 locally. It might be that #139244 uncovers some additional files, but that PR needs to be rebased (though actually I'd advice to rebase the non-test changes onto this PR, probably faster that way). r? `@jieyouxu` <details> <summary>vim notes for future me</summary> Make a file with lines like this ``` /home/folkertdev/rust/rust/tests/run-make/export/disambiguator/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/invalid-so/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/no-builtins-attribute/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/export/extern-opt/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/link-dedup/rmake.rs:1:1 ``` then ``` :set errorformat=%f:%l:%c :cfile /tmp/files-to-fix.txt ``` ``` :copen :cnext :cprev ``` are your friends </details> try-job: test-various try-job: armhf-gnu

…ieyouxu ignore `run-make` tests that need `std` on targets without `std` In particular, anything that includes `none` in the target triple, and `nvptx64-nvidia-cuda`. Right now we don't cross-compile the `run-make` tests, but we want to in the future. This uses `//@ needs-target-std` introduced in rust-lang#142297. Useful for rust-lang#139244 and rust-lang#141856. The modified files are based on running rust-lang#141856 locally. It might be that rust-lang#139244 uncovers some additional files, but that PR needs to be rebased (though actually I'd advice to rebase the non-test changes onto this PR, probably faster that way). r? `@jieyouxu` <details> <summary>vim notes for future me</summary> Make a file with lines like this ``` /home/folkertdev/rust/rust/tests/run-make/export/disambiguator/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/invalid-so/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/no-builtins-attribute/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/export/extern-opt/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/link-dedup/rmake.rs:1:1 ``` then ``` :set errorformat=%f:%l:%c :cfile /tmp/files-to-fix.txt ``` ``` :copen :cnext :cprev ``` are your friends </details>

[WIP] Enable automatic cross-compilation in run-make tests > [!CAUTION] > > Stacked on top of #142414, needs rebase. Supersedes #138066. Blocker for #141856. Based on #138066 with #139242 + #139239 cherry-picked in, plus `rustdoc()` cross-compile changes. r? `@ghost` try-job: armhf-gnu try-job: test-various

…ieyouxu ignore `run-make` tests that need `std` on targets without `std` In particular, anything that includes `none` in the target triple, and `nvptx64-nvidia-cuda`. Right now we don't cross-compile the `run-make` tests, but we want to in the future. This uses `//@ needs-target-std` introduced in rust-lang#142297. Useful for rust-lang#139244 and rust-lang#141856. The modified files are based on running rust-lang#141856 locally. It might be that rust-lang#139244 uncovers some additional files, but that PR needs to be rebased (though actually I'd advice to rebase the non-test changes onto this PR, probably faster that way). r? ``@jieyouxu`` <details> <summary>vim notes for future me</summary> Make a file with lines like this ``` /home/folkertdev/rust/rust/tests/run-make/export/disambiguator/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/invalid-so/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/no-builtins-attribute/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/export/extern-opt/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/link-dedup/rmake.rs:1:1 ``` then ``` :set errorformat=%f:%l:%c :cfile /tmp/files-to-fix.txt ``` ``` :copen :cnext :cprev ``` are your friends </details>

Rollup merge of #142414 - folkertdev:ignore-nostd-tests, r=jieyouxu ignore `run-make` tests that need `std` on targets without `std` In particular, anything that includes `none` in the target triple, and `nvptx64-nvidia-cuda`. Right now we don't cross-compile the `run-make` tests, but we want to in the future. This uses `//@ needs-target-std` introduced in #142297. Useful for #139244 and #141856. The modified files are based on running #141856 locally. It might be that #139244 uncovers some additional files, but that PR needs to be rebased (though actually I'd advice to rebase the non-test changes onto this PR, probably faster that way). r? ``@jieyouxu`` <details> <summary>vim notes for future me</summary> Make a file with lines like this ``` /home/folkertdev/rust/rust/tests/run-make/export/disambiguator/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/invalid-so/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/no-builtins-attribute/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/export/extern-opt/rmake.rs:1:1 /home/folkertdev/rust/rust/tests/run-make/link-dedup/rmake.rs:1:1 ``` then ``` :set errorformat=%f:%l:%c :cfile /tmp/files-to-fix.txt ``` ``` :copen :cnext :cprev ``` are your friends </details>

[WIP] Enable automatic cross-compilation in run-make tests > [!CAUTION] > > Stacked on top of #142414, needs rebase. Supersedes #138066. Blocker for #141856. Based on #138066 with #139242 + #139239 cherry-picked in, plus `rustdoc()` cross-compile changes. r? `@ghost` try-job: armhf-gnu try-job: test-various

[WIP] Enable automatic cross-compilation in run-make tests > [!CAUTION] > > Conflicts with #142563. Supersedes #138066. Blocker for #141856. Based on #138066 with #139242 + #139239 cherry-picked in, plus `rustdoc()` cross-compile changes. r? `@ghost` try-job: armhf-gnu try-job: test-various

Enable automatic cross-compilation in run-make tests Supersedes #138066. Blocker for #141856. Based on #138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: armhf-gnu try-job: test-various

Enable automatic cross-compilation in run-make tests Supersedes #138066. Blocker for #141856. Based on #138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: dist-various-1 try-job: dist-various-1

Enable automatic cross-compilation in run-make tests Supersedes #138066. Blocker for #141856. Based on #138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: dist-various-1

jieyouxu · 2025-06-19T09:38:24Z

@rustbot review

The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions.

rustbot assigned jieyouxu Jun 1, 2025

jieyouxu approved these changes Jun 1, 2025

View reviewed changes

folkertdev force-pushed the run-make-forward-compiletest-runner branch from 3f206ce to ab6646a Compare June 1, 2025 13:39

jieyouxu mentioned this pull request Jun 1, 2025

New bors looks within markdown code blocks rust-lang/bors#295

Closed

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 1, 2025

This comment has been minimized.

Sign in to view

jieyouxu mentioned this pull request Jun 1, 2025

Consider adding a //@ needs-std directive for target std support #141863

Closed

folkertdev force-pushed the run-make-forward-compiletest-runner branch from ab6646a to e3fee6a Compare June 1, 2025 15:52

folkertdev mentioned this pull request Jun 12, 2025

ignore run-make tests that need std on targets without std #142414

Merged

folkertdev force-pushed the run-make-forward-compiletest-runner branch from 272dfe8 to abbf4a9 Compare June 18, 2025 08:49

folkertdev force-pushed the run-make-forward-compiletest-runner branch from abbf4a9 to 8eef8cf Compare June 19, 2025 09:29

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Jun 19, 2025

forward the bootstrap runner to run-make

d023fe7

The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions.

folkertdev force-pushed the run-make-forward-compiletest-runner branch from 8eef8cf to d023fe7 Compare June 19, 2025 10:32

forward the bootstrap runner to run-make #141856

Are you sure you want to change the base?

forward the bootstrap runner to run-make #141856

Uh oh!

Conversation

folkertdev commented Jun 1, 2025 • edited by jieyouxu Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jun 1, 2025

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jieyouxu Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

folkertdev Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

folkertdev Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Jun 1, 2025

Uh oh!

folkertdev commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 1, 2025

Uh oh!

bors commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 1, 2025

Uh oh!

rust-bors bot commented Jun 1, 2025

Uh oh!

This comment has been minimized.

bors commented Jun 1, 2025

Uh oh!

This comment has been minimized.

jieyouxu commented Jun 1, 2025

Uh oh!

folkertdev commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jieyouxu commented Jun 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

folkertdev commented Jun 1, 2025

Uh oh!

folkertdev commented Jun 11, 2025

Uh oh!

jieyouxu commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jieyouxu commented Jun 19, 2025

Uh oh!

Uh oh!

forward the bootstrap `runner` to `run-make` #141856

forward the bootstrap `runner` to `run-make` #141856

folkertdev commented Jun 1, 2025 •

edited by jieyouxu

Loading

jieyouxu commented Jun 1, 2025 •

edited

Loading

jieyouxu commented Jun 1, 2025 •

edited

Loading

jieyouxu commented Jun 11, 2025 •

edited

Loading