simGetCameraInfo.proj_mat return an unexpected form of matrix

[Joyymoonh]

Bug Report

• AirSim Version/#commit: 1.8.1
• UE/Unity version: 4.27.2
• Autopilot version: N/A
• OS Version: Windows 11

What's the issue you encountered?

The projection matrix (proj_mat) returned by the simGetCameraInfo API does not match the expected structure of a standard perspective projection matrix.

The returned projection matrix is as follows:

<ProjectionMatrix> {   
  'matrix': [   
    [0.0, 1.0016834735870361, 0.0, 0.0],
    [0.0, 0.0, -1.7807706594467163, 0.0],
    [0.0, 0.0, 0.0, 10.0],
    [-1.0, 0.0, 0.0, 0.0]
  ]
}

This matrix does not conform to the standard pinhole camera projection matrix structure (which typically includes parameters such as focal lengths fx/fy and principal points cx/cy), making it unusable for common computer vision tasks such as 3D-to-2D coordinate transformation.

Settings

Custom settings.json configuration is used as follows:

{
    "SettingsVersion": 1.2,
    "SimMode": "",
    "CameraDefaults": {
        "CaptureSettings": [
            {
                "ImageType": 0,
                "Width": 1920,
                "Height": 1080
            },
            {
                "ImageType": 1,
                "Width": 1920,
                "Height": 1080
            },
            {
                "ImageType": 3,
                "Width": 1920,
                "Height": 1080
            },
            {
                "ImageType": 5,
                "Width": 1920,
                "Height": 1080
            }
        ]
    },
    "ClockSpeed": 1.0,
    "ViewMode": "",
    "Vehicles": {
        "test1_car": {
            "VehicleType": "PhysXCar",
            "X": 0,
            "Y": 0,
            "Z": 0,
            "ApiServerPort": 41461,
            "Cameras": {
                "front_center": {
                    "X": 0.0, "Y": 0.0, "Z": 0.0,
                    "Pitch": 0, "Roll": 0, "Yaw":0
                }
            }
        },
        "test2_drone": {
            "VehicleType": "SimpleFlight",
            "X": -5,
            "Y": 0,
            "Z": 0,
            "ApiServerPort": 41471,
            "Cameras": {
                "bottom_center": {
                    "X": 0.0, "Y": 0.0, "Z": 0.0,
                    "Pitch": -90, "Roll": 0, "Yaw": 0
                }
            }
        }
    },
    "WindowWidth": 1920,
    "WindowHeight": 1080,
    "ClockType": "ScalableClock"
}

How can the issue be reproduced?

1. Launch the AirSim simulation environment based on UE 4.27.2 on Windows 11 with the above settings.json configuration.
2. Run the following Python code to connect to the vehicle and retrieve camera information:

   import airsim
   car_client = airsim.CarClient(port=41461)
   car_client.confirmConnection()
   camera_info = car_client.simGetCameraInfo("0", vehicle_name="test1_car")
   print(camera_info.proj_mat)

3. Observe the structure of the returned proj_mat.matrix in the output.

Include full error message in text form

No explicit error message is displayed, but the returned projection matrix has an abnormal structure. The specific content is as follows:

ProjectionMatrix(matrix=[[0.0, 1.0016834735870361, 0.0, 0.0], [0.0, 0.0, -1.7807706594467163, 0.0], [0.0, 0.0, 0.0, 10.0], [-1.0, 0.0, 0.0, 0.0]])

[zhicongsun]

omg I meet the same issue. any update?

[Joyymoonh]

天哪，我遇到了同样的问题。有什么更新吗？

I didn't solve it, pity