Dependent libraries and static storage duration data

[suVrik]

Blink is amazing, but I can see a couple issues right now:

1. Linked libraries. Many projects are designed in a modular way: executable and a bunch of libraries linked to it. Currently blink works only with compilation units linked directly to the executable;

2. Static storage duration data. After the first hot reload addresses of static variables are changed, and these variables seem contain zero-initialized data. Would be nice if these variables keep the old addresses. Currently I have no idea how to resolve this issue;

3. Sometimes blink and executable, blink is connected to, hang with the following message in terminal:
   Finished compiling "c:\my_project\source\foo.obj" with code 0.
   I'm trying to find the cause, but no success yet.

I'm working on these when I have some free time, so if you have any ideas, please, let me know.

[crosire]

On 1: Jup (see also #8). This would require going through the import address table (which blink already does anyway) and load all the PDBs for imported libraries too, instead of just the application one (to extend the symbol list and source/object file name lists to contain information about the libraries too). The rest should work as it is.

On 2: The relevant code is here:

blink/source/blink_linker.cpp

Line 360 in 171d683

else if (strcmp(reinterpret_cast<const char *>(section.Name), ".bss") == 0 || strcmp(reinterpret_cast<const char *>(section.Name), ".data") == 0)

This is where blink connects addresses of old static variables with the new code. The .bss section contains zero-initialized variables and the .data section should contain all other global variables. So technically blink should already keep the old addresses and I remember this working in the past. Could you give an example of where the address is no longer the same after reloading?

On 3: This is when application::link is invoked. There is a lot happening in there. And it does suspend all threads in the application for the duration of the function to avoid race conditions while blink links in the new code:

blink/source/blink_linker.cpp

Line 125 in 171d683

thread_scope_guard _scope_guard_;

Depending on how long that takes, this will be noticeable with a hang. Adding some performance timers in application::link should help narrowing down which part takes the longest time. My guess is symbol resolving, but could also be file IO while the object file is read into memory.

[suVrik]

2. I can reproduce it with this code:

#include <stdio.h>
#include <windows.h>

void print_message() {
	static int test = 0;
	printf("%d %I64u\n", test++, reinterpret_cast<uintptr_t>(&test));
}

int main(int argc, char* argv[]) {
	while (true) {
		print_message();
		Sleep(100);
	}
	return 0;
}

Once I change the message I see:

100 140695734960484
101 140695734960484
102 140695734960484
0 140695735043080!
1 140695735043080!
2 140695735043080!
3 140695735043080!

3. I should have clarified that. I mean it hangs forever.

[suVrik]

So yea, 2 happens only the very first time because _symbols is filled from PDB file and it seems like static variables are not there (checked it with cvdump). Later on it is added to _symbols with the new value from updated OBJ file, which is not valid. It seems like we should process OBJ at the very beginning. (When I say "later on" I mean #13, haha)

[sheepmax]

My guess for 3. is that up to 12 bytes are changed at the beginning of a function call. This is enough to cause issues if the program happened to be inside that region OR if another function is returning into that region. It could be possible to check if this the issue by having a sleep at the start of the function, and as soon as this is entered, make a change to the source code. Chances are, the sleep will return into the overwritten code and cause chaos.