Virtual table parsing

[Ilyasml]

Hello

I have an interface which looks like

class IBaseInterface
{
public:
	virtual int Type() = 0;	
	IBaseInterface() = default;
	virtual ~IBaseInterface() = default;
};


class IString :public IBaseInterface
{
public:
	virtual const wchar_t* String() = 0;
	virtual HRESULT  SetStringOutProp(const wchar_t* Value) = 0;	
	IString() = default;
	virtual ~IString() = default;
	
};

and I have a real class which implements the IString interface:

class RealStringClass: public IString
{
public:

	virtual ~RealStringClass()
	{
	}

	std::wstring m_String;

	virtual const wchar_t*  String() override;
	virtual HRESULT  SetStringOutProp(const wchar_t* Value)  override;
	virtual int Type() override
	{
		return 0;
	}
};

There is a function which is used to create the IString interface:

IString*  CreateString()
{
	RealStringClass *ret = new RealStringClass();
	return ret;
}

All is great, I am able to use the interface, until I try to delete the created implementation using the function

void  DeleteInterface(void* ToDelete)
{
	delete (IBaseInterface *)ToDelete;
}

The problem is that I do not hit the RealStringClass::~RealStringClass() descructor. I have found that the vtable is being changed in CppSharp.Runtime.VTables SetupVTables function, where destructor entry is changed.

Could you advice parse option, if any, which changes this behavior for a particular class?

[tritao]

I don't think there is any particular parsing option that changes this behavior, it may be a bug.

Which platform is this, Linux or VS?

[Ilyasml]

It is Visual Studio, I can attach an entire project too.

[tritao]

Would be worth to check in Linux then, because the vtable code is different.

Also I wonder if the problem could be related to passing in a void* in DeleteInterface, could be worth checking with a pointer to concrete type and see if it makes a difference.

[Ilyasml]

I have tried to pass the parameter of the DeleteInterface function as IBaseInterface, but it did not help. Specification of the parameter as IString did not helped either.

Besides, please look at the picture, at the Watch panel. The ret variable is captured in the function CreateString(), the ToDelete variable in the function DeleteInterface. Please note, that the both pointers (ret and ToDelete) are equal (the red squares), however, the second entry in vptr table is different. For the ret variable the second entry is RealStringClass::~RealStringClass() destructor, and for the ToDelete variable it is an other value.

Unfortunately, I have no Linux machine, so I am not able to check the behavior.

[Ilyasml]

I am just getting into Linux with a task to port a Windows project

[Ilyasml]

Could you tell me if there is any advice of how to deal with the issue? Maybe there is a way of somehow rewrite the code?

[tritao]

I think first it needs to be diagnosed exactly where it's going wrong, supposedly CppSharp overrides the virtual table destructor entry, and after that it's going wrong. I think we need to figure out if that vtable override is being called, or not, or if it fails up later calling the original destructor.

[Ilyasml]

Do you mean the *.cs output file of the CppSharp? I can also attach the entire vs project too

[tritao]

Yeah, the .cs file

[Ilyasml]

Here is the gist: https://gist.github.com/Ilyasml/9323b88eb8c77013ee22dd048d554617

[Ilyasml]

Besides, I uploaded all the projects on Github: https://github.com/Ilyasml/ForLinuxTranslation.

The ServerStubProject is the C++ project which implements the interface mentioned above, the InterfaceStub is a WPF project which uses the CppSharp output and the dll of ServerStubProject . The cppsharptry is project which uses CppSharp directly.

Sorry for the absolute paths in projects, they due to the projects are trial.

[tritao]

One thing I noticed is that:

            var __result0 = global::ServerStubProject.dll.IString.__GetOrCreateInstance(___ret, true);

This line passes skipVTables = true, but you previously said the vtable was being overriden, did you confirm it by setting some breakpoints?

[tritao]

https://gist.github.com/Ilyasml/9323b88eb8c77013ee22dd048d554617#file-gistfile1-txt-L563

[Ilyasml]

I can place them as in this picture: #1890 (comment)

[Ilyasml]

https://gist.github.com/Ilyasml/9323b88eb8c77013ee22dd048d554617#file-gistfile1-txt-L563

OK, just a second

[Ilyasml]

Here is the screenshot, please tell if it helps, other screenshots can be done if necessary

[Ilyasml]

global::ServerStubProject.dll.IString.__GetOrCreateInstance(___ret, true);

The function has three parameters

internal static new IString __GetOrCreateInstance(__IntPtr native, bool saveInstance = false, bool skipVTables = false)
{
    if (native == __IntPtr.Zero)
        return null;
    if (__TryGetNativeToManagedMapping(native, out var managed))
        return (IString)managed;
    var result = __CreateInstance(native, skipVTables);
    if (saveInstance)
        __RecordNativeToManagedMapping(native, result);
    return result;
}

and is called

 var __result0 = global::ServerStubProject.dll.IString.__GetOrCreateInstance(___ret, true);

so the third parameter is taken by default as false, please refer to the screenshot:

[tritao]

Try and compile with MSVC/cl command line, https://godbolt.org/z/93zE6KEYq, and try to dump vtable layout: https://gist.github.com/GavinRay97/700ff1631d7e5ac460efd0780759c908

[Ilyasml]

And check if any of the functions like:

        private static void _dtorDelegateHook(__IntPtr __instance, int delete)

are being called.

https://gist.github.com/Ilyasml/9323b88eb8c77013ee22dd048d554617#file-gistfile1-txt-L936

One of those is being called

[Ilyasml]

Try and compile with MSVC/cl command line, https://godbolt.org/z/93zE6KEYq, and try to dump vtable layout: https://gist.github.com/GavinRay97/700ff1631d7e5ac460efd0780759c908

OK, I am trying to do it

[tritao]

And check if any of the functions like:

        private static void _dtorDelegateHook(__IntPtr __instance, int delete)

are being called.
gist.github.com/Ilyasml/9323b88eb8c77013ee22dd048d554617#file-gistfile1-txt-L936

One of those is being called

Ok, in that case the vtable hook is being successful, and just the calling of the original destructor fails.

[tritao]

I am not seeing callNativeDtor being referenced anywhere in the Dispose implementation, so maybe this is the issue.

[tritao]

The relevant code is

CppSharp/src/Generator/Generators/CSharp/CSharpSources.cs

Line 2320 in b9685f0

var dtor = @class.Destructors.FirstOrDefault();

[tritao]

You should add some breakpoints in the generator in

CppSharp/src/Generator/Generators/CSharp/CSharpSources.cs

Line 2320 in b9685f0

var dtor = @class.Destructors.FirstOrDefault();

and figure out why the code branch that generates the call to the native dtor is not getting hit.

[Ilyasml]

I am trying to debug the package, however I run into a strange error.

I downloaded the main branch, build it accordingly to the instructions, then replaced the files of the nuget package (.nuget\packages\cppsharp\1.1.5.3168) by the build output to make the debugging possible. However, when I try to run the library I got the error like

System.IO.FileNotFoundException: 'Could not load file or assembly 'CppSharp.Parser.CLI, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.'

I placed the CppSharp.Parser.CLI.dll file from the build to every possible path (,\bin\x64\Debug\net9.0, etc) however the error persists.

Could you give me some glues where to look for the solution?

[Ilyasml]

As for some disasembly of the nuget package suggested by Visual studio, here is a screenshot of the suggested line.

It seems that HasNonTrivialDestructor should be set

[tritao]

According to:

Trivial destructor
The destructor for class T is trivial if all of the following is true:

• The destructor is not user-provided (meaning, it is either implicitly declared, or explicitly defined as defaulted on its first declaration).
• The destructor is not virtual (that is, the base class destructor is not virtual).
• All direct base classes have trivial destructors.
• All non-static data members of class type (or array of class type) have trivial destructors.
• A trivial destructor is a destructor that performs no action. Objects with trivial destructors don't require a delete-expression and may be disposed of by simply deallocating their storage. All data types compatible with the C language (POD types) are trivially destructible.

Then I think since the destructor is virtual, it should not be trivial, and that should be set indeed.

[Ilyasml]

Thank you for your reply, it is very useful, it is a part of the solution, the second part is that the class is an abstract one. Therefore condition !@class.IsAbstract does not allow to get into the if statement. I have rewrote the classes to make then non-abstract and the destructor started to be called. However, there is another question: there is a memory leak of 48 bytes each time the create/destroy code listed below is called

IString stringNative = CppLibraryInterfaceDesk.CreateString();   
CppLibraryInterfaceDesk.DeleteInterface(stringNative);

The leak looks like

Detected memory leaks!
Dumping objects ->
{222} normal block at 0x00000232E02AB650, 48 bytes long.
 Data: <                > C8 12 9A BE FE 7F 00 00 00 00 00 00 00 00 00 00 
{220} normal block at 0x00000232E02AD2C0, 48 bytes long.
 Data: <                > C8 12 9A BE FE 7F 00 00 00 00 00 00 00 00 00 00 
{218} normal block at 0x00000232E02AD250, 48 bytes long.
 Data: <                > C8 12 9A BE FE 7F 00 00 00 00 00 00 00 00 00 00 
{216} normal block at 0x00000232E02AB730, 48 bytes long.
 Data: <                > C8 12 9A BE FE 7F 00 00 00 00 00 00 00 00 00 00 
{214} normal block at 0x00000232E02AB5E0, 48 bytes long.
 Data: <                > C8 12 9A BE FE 7F 00 00 00 00 00 00 00 00 00 00 

It seems that leaks something like relative addresses, because data of block does not change from one application run to another.

The leaks in the listing are made by five consecutive calls of create/destroy sequence

[Ilyasml]

It seems that regardless the call of the destructor the memory allocated for the object itself leaks. I have changed the implementation of the class to another and the meak changed from 48 bytes to 16.
The first implementation contained std::wstring and was like code below.

class RealStringClass: public IString
{
public:

	virtual ~RealStringClass()
	{
		int i = 0;
		i++;
	}

	std::wstring m_String;

	virtual const wchar_t*  String() override;
	virtual HRESULT  SetStringOutProp(const wchar_t* Value)  override;
	virtual int Type() override
	{
		return 0;
	}
};

The second implementation has std::wstring replaced by a pointer, therefore the size of the class and size of the leak changed.

class RealStringClass: public IString
{
public:

	virtual ~RealStringClass()
	{
		int i = 0;
		i++;
		delete m_String;
	}

	std::wstring *m_String;

	virtual const wchar_t*  String() override;
	virtual HRESULT  SetStringOutProp(const wchar_t* Value)  override;
	virtual int Type() override
	{
		return 0;
	}
	RealStringClass()
	{
		m_String = nullptr;
	}
};

[tritao]

Can you confirm the value of the pointer on creation, and the value passed to delete? Either it's different or there's another allocation in C# not being properly accounted for.

[Ilyasml]

The pointers are the same, please refer to the screenshot attached

[tritao]

In that case, it's probably this allocation: https://gist.github.com/Ilyasml/9323b88eb8c77013ee22dd048d554617#file-gistfile1-txt-L1088

[tritao]

I think the issue here is that we can't know the lifetime of the returned pointer, maybe that's why we end up copying it, but then we should also be freeing it. It's been a few years since I really touched this, so I'd have to think carefully what's going on and what needs to be done to fix it.

[Ilyasml]

Could you tell me where does the pointer is being copied? I navigated through the code (the new gist is https://gist.github.com/Ilyasml/a89364cc8c0b135d010acaf28e9aa7b2) and there were no copy of allocated memory, just copying of returned native pointer value.
The call stack to the downmost constructor is below:

	InterfaceStub.dll!ServerStubProject.dll.IBaseInterface.IBaseInterface(void* native, bool skipVTables) Line 252	C#
 	InterfaceStub.dll!ServerStubProject.dll.IString.IString(void* native, bool skipVTables) Line 466	C#
 	InterfaceStub.dll!ServerStubProject.dll.IString.__CreateInstance(nint native, bool skipVTables) Line 421	C#
 	InterfaceStub.dll!ServerStubProject.dll.IString.__GetOrCreateInstance(nint native, bool saveInstance, bool skipVTables) Line 430	C#
 	InterfaceStub.dll!ServerStubProject.dll.CppLibraryInterfaceDesk.CreateString() Line 1230	C#
 	InterfaceStub.dll!InterfaceStub.MainWindow.Button_Click_1(object sender, System.Windows.RoutedEventArgs e) Line 65	C#

[tritao]

Could you tell me where does the pointer is being copied? I navigated through the code (the new gist is gist.github.com/Ilyasml/a89364cc8c0b135d010acaf28e9aa7b2) and there were no copy of allocated memory, just copying of returned native pointer value. The call stack to the downmost constructor is below:

	InterfaceStub.dll!ServerStubProject.dll.IBaseInterface.IBaseInterface(void* native, bool skipVTables) Line 252	C#
 	InterfaceStub.dll!ServerStubProject.dll.IString.IString(void* native, bool skipVTables) Line 466	C#
 	InterfaceStub.dll!ServerStubProject.dll.IString.__CreateInstance(nint native, bool skipVTables) Line 421	C#
 	InterfaceStub.dll!ServerStubProject.dll.IString.__GetOrCreateInstance(nint native, bool saveInstance, bool skipVTables) Line 430	C#
 	InterfaceStub.dll!ServerStubProject.dll.CppLibraryInterfaceDesk.CreateString() Line 1230	C#
 	InterfaceStub.dll!InterfaceStub.MainWindow.Button_Click_1(object sender, System.Windows.RoutedEventArgs e) Line 65	C#

Then I guess my previous analysis was not correct as it was referencing the old code.

The only other allocation I can see is this: https://github.com/mono/CppSharp/blob/main/src/Runtime/VTables.cs#L53

But then would its size change by changing the class fields as you noted before? Hmm..

[Ilyasml]

I can say that for sure it is a memory of the object itself. If create and delete functions are rewritten as below, there is no leak.

IString*  CreateString()
{
	byte* mem = new byte[sizeof(RealStringClass)];
	RealStringClass *ret = new(mem) RealStringClass();
	return ret;
}

void  DeleteInterface(IBaseInterface* ToDelete)
{
	delete (IBaseInterface *)ToDelete;
	delete[](byte*)ToDelete;
}

I suppose it is something in chain of destructors is being broken, however, to be honest, I am not familiar with the field. Would you mind if I create an issue request on the topic?

[tritao]

Sure