insufficient compiler arguments escape on windows with trailing `\` seperator on directories (regression since vibe-core update)

[the-horo]

This started happening since 102a8c1 and is pretty easy to hit. Take a blank single-file project:

$ cat a.d
/+ dub.sdl:
name "a"
+/
void foo() {}

With the master branch of dub, this project does not test with ldc2:

$ ../dub.exe test --single a.d --compiler=ldc2 -v
Note: Failed to determine version of package g at . Assuming ~master.
             Generating test runner configuration 'a-test-application' for 'application' (executable).
Get module name from path: C:\Users\toxic brat\Downloads\g\a.d
Configuring dependent a, deps:
    Starting Performing "unittest" build using ldc2 for x86_64.
Target 'C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-m7TrmtkXz7CAR8jHtYMqbg\a-test-application.exe' doesn't exist, need rebuild.
    Building a ~master: building configuration [a-test-application]
ldc2 -d-debug -g -unittest -w -d-version=Have_a "-IC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\code\a-test-application-unittest-HMZWcXqa5anv-2_t4Vr-9Q\" "C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\code\a-test-application-unittest-HMZWcXqa5anv-2_t4Vr-9Q\dub_test_root.d" a.d -c "-ofC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-m7TrmtkXz7CAR8jHtYMqbg\a-test-application.obj" -vcolumns
Error: unrecognized file extension obj
-vcolumns
FAIL C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-m7TrmtkXz7CAR8jHtYMqbg a-test-application executable
Error ldc2 failed with exit code 1.

but with older dubs it works:

$ dub.exe test --single a.d --compiler=ldc2 -v
Note: Failed to determine version of package g at .. Assuming ~master.
             Generating test runner configuration 'a-test-application' for 'application' (executable).
Get module name from path: C:/Users/toxic brat/Downloads/g/a.d
Configuring dependent a, deps:
    Starting Performing "unittest" build using ldc2 for x86_64.
Target 'C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-IEI8rg1R0VDXNXvM9rPFRg\a-test-application.exe' doesn't exist, need rebuild.
    Building a ~master: building configuration [a-test-application]
ldc2 -d-debug -g -unittest -w -d-version=Have_a "-IC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\code\a-test-application-unittest-HMZWcXqa5anv-2_t4Vr-9Q" "C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\code\a-test-application-unittest-HMZWcXqa5anv-2_t4Vr-9Q\dub_test_root.d" a.d -c "-ofC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-IEI8rg1R0VDXNXvM9rPFRg\a-test-application.obj" -vcolumns
     Linking a-test-application
ldc2 "-ofC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-IEI8rg1R0VDXNXvM9rPFRg\a-test-application.exe" "C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-IEI8rg1R0VDXNXvM9rPFRg\a-test-application.obj" -g
Copying target from C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-IEI8rg1R0VDXNXvM9rPFRg\a-test-application.exe to C:\Users\toxic brat\Downloads\g
Copying target from C:\Users\toxic brat\AppData\Local\dub\cache\a\~master\build\a-test-application-unittest-IEI8rg1R0VDXNXvM9rPFRg\a-test-application.pdb to C:\Users\toxic brat\Downloads\g
     Running a-test-application.exe
All unit tests have been run successfully.

This seems to be caused by the " characters dub adds around paths with spaces and the fact that those paths, since the vibe update, started receiving a trailing \ separator, which is now escaping the ":

"-IC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\code\a-test-application-unittest-HMZWcXqa5anv-2_t4Vr-9Q\"
vs
"-IC:\Users\toxic brat\AppData\Local\dub\cache\a\~master\code\a-test-application-unittest-HMZWcXqa5anv-2_t4Vr-9Q"

Weirdly enough, ldmd2 and dmd do work and I have no clue why.

[s-ludwig]

The question is how the response file format is defined for LDC (which is how the arguments are actually passed to the compiler, despite the verbose output suggesting they are being passed directly). If it uses the typical Posix shell style escape rules, then we need to adjust dub.compilers.ldc.escapeArgs accordingly. However, if that's the case, I wonder why we didn't encounter issues with paths like C:\newline\bell before. Probably it's using a simplified version that only supports \" as an escape sequence then.

[the-horo]

The response files are a little bit more strict with their escaping. The format is documented on https://dlang.org/phobos/dmd_root_response.html#.responseExpandFrom and the ldc implementation that handles escaping is https://github.com/ldc-developers/ldc/blob/46bbe8b47f7a69aff651e913d80dd846e1d6f613/driver/response.cpp#L30. Long story short, when ldc finds a " character it checks how many \ are before it, and it does the escaping there (\\ becomes \, \" becomes "). The issue didn't exist before because, like you guessed, a \ in the middle of the path wouldn't be treated specially and it would be kept.

I've also checked the response files produced by dub and they contain the same arguments as shown in the logs, each on its own line.

The argument which I showed at the end is the specific one that makes ldc fail. If it is changed to contain two \ before the " instead of one \ then ldc is happy.