Repository navigation
Open
Description
Bug Report
- AirSim Version/#commit: 1.8.1
- UE/Unity version: 4.27.2
- Autopilot version: N/A
- OS Version: Windows 11
What's the issue you encountered?
The projection matrix (proj_mat) returned by the simGetCameraInfo API does not match the expected structure of a standard perspective projection matrix.
The returned projection matrix is as follows:
<ProjectionMatrix> {
'matrix': [
[0.0, 1.0016834735870361, 0.0, 0.0],
[0.0, 0.0, -1.7807706594467163, 0.0],
[0.0, 0.0, 0.0, 10.0],
[-1.0, 0.0, 0.0, 0.0]
]
}
This matrix does not conform to the standard pinhole camera projection matrix structure (which typically includes parameters such as focal lengths fx/fy and principal points cx/cy), making it unusable for common computer vision tasks such as 3D-to-2D coordinate transformation.
Settings
Custom settings.json configuration is used as follows:
{
"SettingsVersion": 1.2,
"SimMode": "",
"CameraDefaults": {
"CaptureSettings": [
{
"ImageType": 0,
"Width": 1920,
"Height": 1080
},
{
"ImageType": 1,
"Width": 1920,
"Height": 1080
},
{
"ImageType": 3,
"Width": 1920,
"Height": 1080
},
{
"ImageType": 5,
"Width": 1920,
"Height": 1080
}
]
},
"ClockSpeed": 1.0,
"ViewMode": "",
"Vehicles": {
"test1_car": {
"VehicleType": "PhysXCar",
"X": 0,
"Y": 0,
"Z": 0,
"ApiServerPort": 41461,
"Cameras": {
"front_center": {
"X": 0.0, "Y": 0.0, "Z": 0.0,
"Pitch": 0, "Roll": 0, "Yaw":0
}
}
},
"test2_drone": {
"VehicleType": "SimpleFlight",
"X": -5,
"Y": 0,
"Z": 0,
"ApiServerPort": 41471,
"Cameras": {
"bottom_center": {
"X": 0.0, "Y": 0.0, "Z": 0.0,
"Pitch": -90, "Roll": 0, "Yaw": 0
}
}
}
},
"WindowWidth": 1920,
"WindowHeight": 1080,
"ClockType": "ScalableClock"
}How can the issue be reproduced?
- Launch the AirSim simulation environment based on UE 4.27.2 on Windows 11 with the above
settings.jsonconfiguration. - Run the following Python code to connect to the vehicle and retrieve camera information:
import airsim car_client = airsim.CarClient(port=41461) car_client.confirmConnection() camera_info = car_client.simGetCameraInfo("0", vehicle_name="test1_car") print(camera_info.proj_mat)
- Observe the structure of the returned
proj_mat.matrixin the output.
Include full error message in text form
No explicit error message is displayed, but the returned projection matrix has an abnormal structure. The specific content is as follows:
ProjectionMatrix(matrix=[[0.0, 1.0016834735870361, 0.0, 0.0], [0.0, 0.0, -1.7807706594467163, 0.0], [0.0, 0.0, 0.0, 10.0], [-1.0, 0.0, 0.0, 0.0]])
Metadata
Metadata
Assignees
Labels
No labels
Activity
zhicongsun commentedon Aug 16, 2025
omg I meet the same issue. any update?
Joyymoonh commentedon Aug 30, 2025
I didn't solve it, pity
deansereigns commentedon Jan 28, 2026
Camera Model and Projection in AirSim
AirSim uses a pinhole camera model, implemented through Unreal Engine (UE4), to render images.
Key Points
About
CameraInfo.proj_matCameraInfo.proj_matis not a pinhole projection matrix of the form:P = K [R | t]
It represents Unreal Engine’s clip-space projection matrix, intended for rendering.
It includes internal engine transformations (coordinate conversions, perspective divide, near/far planes).
Because of this, intrinsic parameters cannot be extracted or decomposed from
proj_mat.Correct Workflow
Treat the AirSim camera as a pinhole camera
Compute the intrinsic matrix K analytically from FOV and image resolution
Compute extrinsics R, t from
CameraInfo.poseConstruct the full projection explicitly as:
P = K [R | t]
Important Note
The camera pose is different from the vehicle/drone pose.
Camera offsets and rotations relative to the vehicle must be taken into account when computing transformations.
Reference
somveeryada289-lgtm commentedon Jun 28, 2026
Somveer yadav