Out of memory

[arvid-norlander]

When running this on a closed source base (~30k C++ files, ~40k headers, ~2 GB source) I get a out of memory kill from Linux after about 4 minutes. This is on a computer with 64 GB RAM, running Ubuntu 24.04 and using Bazel 7.4.1.

However when I try to debug by running in a cgroup with limited memory (so it doesn't take down the entire system) it finishes after 5 minutes 30 seconds (generating a 916MB compile_commands.json). So maybe it somehow auto-adapts? Very strange!

The command I run is:

systemd-run --scope -p MemoryMax=17G -p MemoryHigh=16G \
    bazel run @hedron_compile_commands//:refresh_all \
    -- --config hermetic-clang-toolchain --disk_cache=/mnt/other_disk/bazel_disk_cache

The command that uses too much RAM is:

/home/USER/.cache/bazel/_bazel_USER/HASH_HERE/execroot/_main/bazel-out/k8-fastbuild/bin/external/hedron_compile_commands+/refresh_all.runfiles/rules_python++python+python_3_12_x86_64-unknown-linux-gnu/bin/python3 /home/USER/.cache/bazel/_bazel_USER/HASH_HERE/execroot/_main/bazel-out/k8-fastbuild/bin/external/hedron_compile_commands+/refresh_all.runfiles/_main/../hedron_compile_commands+/refresh_all.check_python_version.py --config hermetic-clang-toolchain --disk_cache=/mnt/other_disk/bazel_disk_cache

Also it seems that hedron invoke the compiler (which I understand it shouldn't), as I see invoke the compiler several times. This seems to happen on Qt related source files. Maybe somehow it gets confused by the moc-preprocessor of Qt?

It does print some errors that I don't see during normal build (normal build works):

some/path/QtWidgetThing.cpp:22:18: error: function-like macro 'QT_VERSION_CHECK' is not defined
   22 | #if QT_VERSION < QT_VERSION_CHECK(5, 0, 0)
      |                  ^

but also:

In file included from external/xz+/src/common/tuklib_cpucores.c:13:
In file included from external/xz+/src/common/tuklib_cpucores.h:16:
In file included from external/xz+/src/common/tuklib_common.h:18:
In file included from external/xz+/src/common/tuklib_config.h:4:
external/xz+/src/common/sysdefs.h:128:4: error: size_t is not 32-bit or 64-bit
  128 | #               error size_t is not 32-bit or 64-bit
      |                 ^
external/xz+/src/common/sysdefs.h:132:3: error: size_t is not 32-bit or 64-bit
  132 | #       error size_t is not 32-bit or 64-bit
      |         ^

Both of these build fine with bazel build --config hermetic-clang-toolchain --disk_cache=/mnt/other_disk/bazel_disk_cache ...

So there seems to be multiple possible issues, but I'm not entirely sure how to pick things apart and debug this.