tweak(extra-natives/rdr3): Improve Unproject function to avoid shakin…

[Identity-labs]

Goal of this PR

This PR enhances the accuracy and stability of the GetWorldCoordFromScreenCoord function in RedM by refining the mathematical transformations involved in converting screen-space coordinates to world-space coordinates. The improvements aim to address issues related to precision errors, resulting in more reliable world-coordinate calculations.

How is this PR achieving the goal

The changes introduced in this PR focus on improving the mathematical computations within the Unproject function, which is responsible for converting screen-space coordinates to world-space coordinates. Key modifications include:

Normalization of View Matrix Axes: The view matrix (worldView) axes are explicitly normalized to ensure they remain orthonormal. This step reduces accumulated drift and maintains the integrity of the transformation.

Separate Inversion of Projection and View Matrices: Instead of inverting the combined view-projection matrix, the projection and view matrices are inverted separately. This approach is numerically more stable and enhances the accuracy of the resulting world coordinates.

Screen-Space to NDC Conversion: The screen-space coordinates are first converted to Normalized Device Coordinates (NDC) by mapping the screen's (0..1) range to NDC's (-1..1) range. This step ensures consistency across different screen resolutions and aspect ratios.

Transformation from NDC to View Space: The NDC coordinates are transformed into view space using the inverted projection matrix. The resulting vector is then divided by its w-component to account for perspective division.

Transformation from View Space to World Space: The view-space coordinates are transformed into world space using the inverted view matrix. Similar to the previous step, the resulting vector is divided by its w-component to obtain the final world coordinates.

These improvements collectively enhance the precision and reliability of the GetWorldCoordFromScreenCoord function, leading to more accurate world-coordinate calculations.

This PR applies to the following area(s)

RedM

Natives

Successfully tested on

Game builds: Latest RedM build
Platforms: Windows

Checklist

[x] Code compiles and has been tested successfully.

[x] Code explains itself well and/or is documented.

[x] My commit message explains what the changes do and what they are for.

[x] No extra compilation warnings are added by these changes.

Fixes issues

Improves accuracy of the GetWorldCoordFromScreenCoord native for RedM.

Before : https://streamable.com/723vjd
After : https://streamable.com/kokvbp

Thanks to https://github.com/ktos93 for the help :)

[manzarek123]

Could this be also done in FiveM? Because the same thing is happening

[Identity-labs]

This is easy change, no need extra checks, it will allow usage of this native because currently is not usable.
I have already send multiple videos to see the benefits of it

[prikolium-cfx]

Thanks for your contribution