Skip to content

x/benchmarks: cockroachdb failing after CL 564197 (runtime: only poll network from one P at a time in findRunnable) #73474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
prattmic opened this issue Apr 22, 2025 · 3 comments
Open

x/benchmarks: cockroachdb failing after CL 564197 (runtime: only poll network from one P at a time in findRunnable) #73474

prattmic opened this issue Apr 22, 2025 · 3 comments
Assignees
@cagedmantis
Labels
Builders x/build issues (builders, bots, dashboards) compiler/runtime Issues related to the Go compiler and/or runtime. Soon This needs action soon. (recent regressions, service outages, unusual time-sensitive situations)
Milestone
Unreleased

Comments

@prattmic
Copy link
Member

As of https://go.dev/cl/564197 (for #65064), all 4 perf builders are failing due to cockroachdb errors.

e.g., https://ci.chromium.org/ui/p/golang/builders/ci/gotip-linux-arm64_c4ah72-perf_vs_release/b8716897839307984241/overview

[sweet] error: run benchmark cockroachdb for config experiment: exit status 1
Tail of log (/home/swarming/.swarming/w/ir/x/t/go-sweet2521400662/results/cockroachdb/experiment.log):
external I/O path:   /home/swarming/.swarming/w/ir/x/t/go-sweet2521400662/work/cockroachdb/experiment/tmp/roach-node/extern
store[0]:            path=/home/swarming/.swarming/w/ir/x/t/go-sweet2521400662/work/cockroachdb/experiment/tmp/roach-node
storage engine:      pebble
clusterID:           2d2165f0-849e-4d6d-952c-6a7068f2b375
status:              initialized new cluster
nodeID:              1

=== Instance "roach-node" original stderr ===
I250422 11:36:15.334032 1 util/log/file_sync_buffer.go:242 ⋮ [config]   file created at: 2025/04/22 11:36:15
I250422 11:36:15.334043 1 util/log/file_sync_buffer.go:242 ⋮ [config]   running on machine: ‹golang-ciw-c4a-72-linux-arm64-bookworm-us-east4-a-0-149a›
I250422 11:36:15.334050 1 util/log/file_sync_buffer.go:242 ⋮ [config]   binary: CockroachDB CCL v24.2.0-alpha.00000000-dev (linux arm64, built , devel 352dd2d932c1c1c6dbc3e112fcdfface07d4fffb)
I250422 11:36:15.334056 1 util/log/file_sync_buffer.go:242 ⋮ [config]   arguments: [‹/home/swarming/.swarming/w/ir/x/t/go-sweet2521400662/work/cockroachdb/experiment/bin/cockroach› ‹start-single-node› ‹--insecure› ‹--listen-addr› ‹localhost:26257› ‹--http-addr› ‹localhost:26258› ‹--cache› ‹0.25› ‹--store› ‹/home/swarming/.swarming/w/ir/x/t/go-sweet2521400662/work/cockroachdb/experiment/tmp/roach-node› ‹--log-dir› ‹/home/swarming/.swarming/w/ir/x/t/go-sweet2521400662/work/cockroachdb/experiment/tmp/roach-node-log›]
I250422 11:36:15.334072 1 util/log/file_sync_buffer.go:242 ⋮ [config]   log format (utf8=✓): crdb-v2
I250422 11:36:15.334074 1 util/log/file_sync_buffer.go:242 ⋮ [config]   line format: [IWEF]yymmdd hh:mm:ss.uuuuuu goid [chan@]file:line redactionmark \[tags\] [counter] msg
I250422 11:36:15.333849 1 util/log/flags.go:222  [-] 1  stderr capture started

error: workload failed to become available within timeout: error: signal: killed: output:
I250422 11:36:40.865692 1 workload/cli/run.go:640  [-] 1  random seed: 6233835059417767123
I250422 11:36:40.865770 1 workload/cli/run.go:432  [-] 2  creating load generator...

At the moment, this CL is the most recent run of these builders, so more data will be useful to see if this is consistent. But given that all builders failed, it seems less likely to be a flake.
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Apr 22, 2025
@gopherbot gopherbot added this to the Unreleased milestone Apr 22, 2025
@prattmic prattmic added Soon This needs action soon. (recent regressions, service outages, unusual time-sensitive situations) and removed compiler/runtime Issues related to the Go compiler and/or runtime. labels Apr 22, 2025
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Apr 22, 2025
@gabyhelp
Copy link

Related Issues

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@gabyhelp gabyhelp added the Builders x/build issues (builders, bots, dashboards) label Apr 22, 2025
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/667336 mentions this issue: Revert "runtime: only poll network from one P at a time in findRunnable"

gopherbot pushed a commit that referenced this issue Apr 22, 2025
@cagedmantis @gopherbot
Revert "runtime: only poll network from one P at a time in findRunnable"
489917f 
This reverts commit 352dd2d.

Reason for revert: cockroachdb benchmark failing. Likely due to CL 564197.

For #73474

Change-Id: Id5d83cd8bb8fe9ee7fddb8dc01f1a01f2d40154e
Reviewed-on: https://go-review.googlesource.com/c/go/+/667336
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
@mknyszek mknyszek moved this from Todo to In Progress in Go Compiler / Runtime Apr 23, 2025
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/668735 mentions this issue: sweet/harnesses: update the version of Cockroachdb benchmarked

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cagedmantis cagedmantis

Labels
Builders x/build issues (builders, bots, dashboards) compiler/runtime Issues related to the Go compiler and/or runtime. Soon This needs action soon. (recent regressions, service outages, unusual time-sensitive situations)
Projects
Go Compiler / Runtime
Status: In Progress
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@cagedmantis @prattmic @gopherbot @gabyhelp