Add json-log-path setting
#13003
Add json-log-path setting
#13003
Conversation
github-actions
bot
added
documentation
new-cli
edolstra
force-pushed
the
json-log-path
branch
3 times, most recently
from
b177d16 to
dff428c
Compare
github-actions
bot
added
the
with-tests
This setting specifies a path (which can be a regular file or Unix domain socket) that receives a copy of all Nix log messages (in JSON format).
|
cc @elikoga do you want to review this from an consumer perspective? |
|
I'm surprised #12647 is already merged, I thought it'd arrive with a PR like this, seems I did not look too closely two weeks ago Additionally, it seems like from the big DetSys PR in I
I see this PR useful for users of nix in a terminal that want to have additional information accessible after a nix command execution while seeing the normal human-readable output in the terminal. For that it works great. I use |
|
I believe One should maybe warn the user of the resulting filesize, if one decides to run many commands with the setting and write the logs to a normal plain file without any compression, then the disk may run full. One build produced more than 100 megabytes of logs for me. Useful feature in any case |
|
The design could suggest a new useful log message type like |
| Setting<Path> jsonLogPath{ | ||
| this, "", "json-log-path", | ||
| R"( | ||
| A path to which JSON records of Nix's log output will be |
There was a problem hiding this comment.
| A path to which JSON records of Nix's log output will be | |
| A file or unix socket to which JSON records of Nix's log output will be |
| R"( | ||
| A path to which JSON records of Nix's log output will be | ||
| written, in the same format as `--log-format internal-json` | ||
| (without the `@nix ` prefixes on each line). |
There was a problem hiding this comment.
Optional as suggested by the reviewer:
| (without the `@nix ` prefixes on each line). | |
| (without the `@nix ` prefixes on each line). | |
| When writing to a file be aware that the output can in the order of 100MBs per build. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
@edolstra only some small doc changes and I think this is good to go? |
| AutoCloseFD fd = | ||
| std::filesystem::is_socket(path) | ||
| ? connect(path) | ||
| : toDescriptor(open(path.c_str(), O_CREAT | O_APPEND | O_WRONLY, 0644)); |
There was a problem hiding this comment.
This will allow multiple Nix instances to append to the same file simultaneously, but the writing isn't synchronized between them, which will lead to interleaved (corrupted) entries, especially if some of the log lines are larger than the page size (e.g. long gcc invocation logs from make, but could be anything)
| /** | ||
| * Create a logger that sends log messages to `mainLogger` and the | ||
| * list of loggers in `extraLoggers`. Only `mainLogger` is used for | ||
| * writing to stdout and getting user input. | ||
| */ |
There was a problem hiding this comment.
thought, no action required:
Would be nice to split these concerns at some point. A UI is a logger, but a logger is not a UI, something like that.
Motivation
The
json-log-pathsetting specifies a path (which can be a regular file or Unix domain socket) that receives a copy of all Nix log messages (in the same format used by--log-format internal-jsonbut without the@nixprefixes). Internally this is implemented using the newTeeLoggerclass that dispatches log messages to multiple underlying loggers (i.e. the JSON logger and the regular progress bar).The main use case is to provide an easy way in CI to process all Nix log messages centrally to extract hash mismatches, build/substitution results, etc.
Context
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.