depfile dependency on dyndep output does not trigger rebuild

[dseomn]

In this build file, copy depends on other, but when other changes, copy is not rebuilt:

rule dyndep
  command = printf 'ninja_dyndep_version = 1\nbuild stamp | other: dyndep\n' > $out

rule build_with_dyndep
  command = touch $out other
  dyndep = $out.dd

rule depends_on_dyndep
  command = printf '%s: other\n' $out > $out.d && cp other $out
  depfile = $out.d

build stamp.dd: dyndep
build stamp: build_with_dyndep || stamp.dd
build copy: depends_on_dyndep || stamp

$ ninja --version
1.12.1
$ ninja -v
[1/3] printf 'ninja_dyndep_version = 1\nbuild stamp | other: dyndep\n' > stamp.dd
[2/3] touch stamp other
[3/3] printf '%s: other\n' copy > copy.d && cp other copy
$ # edit build.ninja to add some spaces to build_with_dyndep's command
$ ninja -v
[1/1] touch stamp     other
$ ninja -v
[1/1] printf '%s: other\n' copy > copy.d && cp other copy
$ ninja -v
ninja: no work to do.
$ ninja -t missingdeps
Processed 3 nodes.
No missing dependencies on generated files found.

[dseomn]

While trying to write a test case for my generator, I realized that if the rule for the dyndep file itself also runs, then so does the rule that depends on the dynamic output:

$ ninja -v
[1/3] printf 'ninja_dyndep_version = 1\nbuild stamp | other: dyndep\n' > stamp.dd
[2/3] touch stamp other
[3/3] printf '%s: other\n' copy > copy.d && cp other copy
$ # add spaces to both dyndep and build_with_dyndep
$ ninja -v
[1/2] printf     'ninja_dyndep_version = 1\nbuild stamp | other: dyndep\n' > stamp.dd
[2/3] touch stamp     other
[3/3] printf '%s: other\n' copy > copy.d && cp other copy
$ ninja -v
ninja: no work to do.

However, changing the last line to build copy: depends_on_dyndep || stamp stamp.dd (i.e., adding stamp.dd) does not work around the bug.

[digit-google]

Thanks for the report. Using ninja -v -d explain is useful to understand what's going on:

# Initial invocation
$ ninja -v -d explain
ninja explain: output stamp.dd doesn't exist
[1/3] printf 'ninja_dyndep_version = 1\nbuild stamp | other: dyndep\n' > stamp.dd
ninja explain: output stamp doesn't exist
[2/3] touch stamp other
ninja explain: depfile 'copy.d' is missing
[3/3] printf '%s: other\n' copy > copy.d && cp other copy

# Second one, after adding spaces to the build_with_dyndep command
$ ninja -v -d explain
ninja explain: command line changed for stamp
[1/1] touch stamp other

# Third invocation
$ $NINJA -v -d explain
ninja explain: output copy older than most recent input other (1754660242406683726 vs 1754660274551264634)
[1/1] printf '%s: other\n' copy > copy.d && cp other copy

So it looks like Ninja decided to rebuild stamp but failed to consider that this command modified other.
I wonder if this is consistent with the definition of out-of-order dependencies (from the manual, stamp being rebuilt does not trigger copy to be rebuilt per se unless another reason point to it).

I suspect a bug in Plan::DyndepsLoaded() or the way it is invoked.

[moritzx22]

The issue can be located here, it is in DependencyScan::RecomputeNodeDirty.
Only the current input is checked, this is fine if no dyndep did apply in one of the inputs. In case dyndep applied, all inputs needs to be checked, as Dyndep might have changed the graph.

See this commit for a proposed solution. This has been added to Pull Request#2675 as it is related to a similar issue #2653.
Here the most relevant code snippet.

Applying the proposed solution will report as following:

$ ninja -v -d explain
ninja explain: output stamp.dd doesn't exist
[1/3] printf 'ninja_dyndep_version = 1\nbuild stamp | other: dyndep\n' > stamp.dd
ninja explain: output stamp doesn't exist
ninja explain: output stamp doesn't exist
[2/3] touch  stamp other
ninja explain: depfile 'copy.d' is missing
ninja explain: output copy doesn't exist
[3/3] printf '%s: other\n' copy > copy.d && cp other copy

# Second one, after adding spaces to the build_with_dyndep command
$ ninja -v -d explain
ninja explain: command line changed for stamp
[1/2] touch  stamp other
ninja explain: stamp is dirty
[2/2] printf '%s: other\n' copy > copy.d && cp other copy