Physical layout

[helviett]

Great library in so many ways but the only thing I'm missing to pick it as math library to use in every project is physical layout of the matrices. Logical layout allows you to create proper matrices (e.g. translation(...), orthographics(...), etc.) to multiply by vector on the right (mul(m, v)) on CPU but you have to transpose matrix if shader compiler assumes column major layout of the matrix.

[redorav]

Hi @helviett thanks for wrting. Can you clarify for me what you are trying to do with some code? Give me the matrix, the vector and the expected result, and how you want them to be laid out in memory?

I can do this currently

float4x4 leMatrix = float4x4
{
	1, 0, 0, 1,
	0, 1, 0, 2,
	0, 0, 1, 3,
	0, 0, 0, 1
};

float4 leVector = float4(1, 2, 3, 1);

float4 leProductVector = mul(leMatrix, leVector);

and this ends up producing a vector 2, 4, 6 and the translation in the matrix is on "the right side" of the matrix. A bit more detail would help to see what you're unable to achieve

[helviett]

In premake:

  defines({
    "HLSLPP_FEATURE_TRANSFORM",
    "HLSLPP_LOGICAL_LAYOUT=HLSLPP_LOGICAL_LAYOUT_COL_MAJOR",
    "HLSLPP_COORDINATES=HLSLPP_COORDINATES_RIGHT_HANDED",
  })

In cpp:

float4x4 m = float4x4::translation(1, 2, 3);
/* same as
float4x4 m = float4x4
{
  1, 0, 0, 1,
  0, 1, 0, 2,
  0, 0, 1, 3,
  0, 0, 0, 1
};
*/
float4 v = float4(1, 2, 3, 1);
float4 result = mul(m, v);

In HLSL:

struct VsOut
{
  float4 position : SV_Position;
};

VsOut
vs(VsIn input)
{
  VsOut out;
  out.position = mul(m, input.position);
  return out;
}

HLSLPP_LOGICAL_LAYOUT doesn't affect the way matrix is stored. Even though translation is generated for pre multiply (mul(M, v)) matrix is stored as row major. In HLSL matrix layout is column major, so it interprets m as:

{
  1, 0, 0, 0,
  0, 1, 0, 0
  0, 0, 1, 0
  1, 2, 3, 1
}

Which is incorrect matrix to be used as translation matrix in pre multiplied form.

[redorav]

Hi @helviett I think now I see what you mean, you want a matrix that treats physical rows as if they were columns. I think that is a bit more intrusive than just changing the logical layout of the matrix because it now involves going into the matrix multiplication functions and flipping the order of operations (an effective transpose) based on a define. The functions are already written so that's not an issue but dealing with the define branches and tests, etc isn't something I was planning to do in the near future.

Would you like to contribute such a change? You can test it properly as well as your use case needs it and I can clean it up after. If you can't I'm happy to keep it open until I get to it.

[redorav]

Of course as a workaround you can tag your hlsl matrices as row_major but I'm sure you know that already, just a suggestion in any case

[helviett]

Yeah, we know about the workarounds. But for some reason it's something that is not trivial to support for all our platforms (PS4, PS5, Metal, Vulkan, D3D12(X)). There are known bugs with reading/writing matrices when dealing with DXC -> SPIRV. Adding tags or pragmas probably multiplies number of bugs.

By the way, just out of curiosity tried to run this benchmark for hlslpp. Its on par with DXM and even beats it in vector operations.

[redorav]

Is DXM what you are trying to replace? I didn't know about this benchmark, I'll make a note to test, sounds more interesting than the small microbenchmarks I have.

In terms of the feature, I figured changing the shader wasn't going to be that simple and in an established engine probably less so.

In hlsl++, I was thinking whether return types need to change for non-square matrices (?) and other bits of complexity that might be a bit annoying, although some matrices (e.g. 3x4 and 4x3) don't really have a physical layout to change (they optimize for space) so it makes it extra complicated. As in, 4x3 and 3x4 matrices are just 3 SIMD vectors, a 2x2 matrix is a single SIMD vector as well, things just don't map cleanly to shaders.

I'll give it some thought, if in the meantime you'd like to contribute something I'm open like I said. I'm on holiday as well so I have more limited access and time.

[jlaumon]

Hey, just chiming in to say that I started using your library and hit the same issue (and was confused for a bit 😅). Currently I'm transposing all the matrices before sending them to the GPU (which works, but isn't ideal).

Specifically for sharing data with the GPU, I think only the layout of float4x4 matters. All the other matrices have weird packing rules depending on the API and type of buffers, so I'm not sure it's worth offering flexibility there (and people are used to packing them manually). Documenting what the memory layout is could be useful though 🙂

Edit: okay maybe also float2x2 🙂

[redorav]

Hi @jlaumon, I can definitely document the physical layout better in the readme, I'll do it when I have a bit of time. The problem wit changing the physical layout is that it changes the multiplication logic for the rest of the library. As an alternative, one thing people have been asking for is interop classes that seamlessly translate/copy to the target format. I'm taking my sweet time with it due to life in general but I have a small prototype locally that looks kind of like this:

namespace interop
{
	struct float4
	{
		float4() = default;
		float4(hlslpp::float4 f) { hlslpp::store(f, &x); }

		float x, y, z, w;
	};
}

In the same vein, for matrices it could look like this:

namespace interop
{
	struct float3x4
	{
		float3x4() = default;
		float3x4(hlslpp::float3x4 m) { hlslpp::store(m, &m00); }
		float m00, m01, m02, m03,
		      m10, m11, m12, m13,
		      m20, m21, m22, m23;
	};

	struct float4x4
	{
		float4x4() = default;
		float4x4(hlslpp::float4x4 m) { hlslpp::store(m, &m00); }
		float m00, m01, m02, m03,
		      m10, m11, m12, m13,
		      m20, m21, m22, m23,
		      m30, m31, m32, m33;
	};
}

Within those interop classes a define that specifies what the physical layout of the target is going to be could happen, and there we could have more optimized routines that maybe transpose straight into the copied memory? With AVX it could even swizzle the components into place instead of transposing, then copying. The interop classes could then interact however you want (e.g. the float2x2 case which is a little silly in hlsl in my opinion) Would that work for you?

[jlaumon]

That sounds fine to me 👍
Thanks a lot!

[redorav]

@jlaumon Have you encountered this website? https://maraneshi.github.io/HLSL-ConstantBufferLayoutVisualizer

I wasn't aware just how cursed trying to match the C++ and HLSL layout was, I've been giving it much thought and I don't think you really can match any existing HLSL struct to C++ and correctly match padding and alignment, at least not without alignas() magic that would have to come through some reflection system external to hlsl++. Are there any in particular you care about? Otherwise I'll start off with the vectors and the 3x4 and 4x4 matrices. Anything else is bound to be confusing and likely wrong. The other thing is that anything hlsl++ provides needs to take structured buffers into account too

[jlaumon]

Indeed it's cursed and indeed I don't think you an make a perfect match 🙂

As far as I'm concerned, I only care about the column major part for float4x4. I can do the padding manually where needed (and I know to avoid 3-components types 🙂)