LiDAR–Camera Projection

The driver reports centroids in the LiDAR frame. Classification reads image pixels. Projection is the map between them.

For a homogeneous LiDAR point $\tilde{x} = (x,y,z,1)^\top$ and $3\times4$ matrix $P$:

$$p = P\tilde{x}, \qquad u = p_x / p_z, \qquad v = p_y / p_z.$$

$P$ is projection_matrix_l or projection_matrix_r. On the feature-branch calibrator, $P = K_{\text{runtime}}[R \mid t]$; see projection math. Prod may still use a free DLT $P$ (legacy DLT).

StateManager::transform_points applies $P$, divides by projected $z$, and stores planar LiDAR range for YOLO box-height heuristics / HSV ROI sizing. Cone index order is preserved through left/right classify, merge, and ConeArray construction.

Parameters

• projection_matrix_l/r (and ll/rr on some ZED builds)
• Compile-time cam_width_px / cam_height_px
• SeeCam intrinsic YAMLs for undistortion before classification (must match the pixel space $P$ was built for)

Constant offset at rest → extrinsics / wrong YAML. Edge stretch → intrinsics / size ≠ 1920×1200. Procedure: physical calibration.