Calibration Procedure

Math behind the solves: intrinsics, extrinsics, projection.

Intrinsics

Capture stills from raw V4L2 (default /dev/video4; override with --device):

cd $DRIVERLESS/driverless_ws/src/point_to_pixel/scripts/undistortion
python gather_rect_calibration_images.py --camera l --split train
python gather_rect_calibration_images.py --camera l --split validation
# repeat with --camera r

SPACE saves only when the frame is 1920×1200 and the board is detected. q exits. Images go to data/calibration_images/{l|r}/{split}/.

python run_intrinsic_calibration.py --camera both -y

Manual path: calibrate_rect.py then validate_rect_calibration.py. The solver wants enough detections with FOV coverage (boards not only in the image center). Validation target in the scripts: mean reprojection ≤ 1.0 px.

Extrinsics

Prerequisites:

ROS 2 sourced and overlayed
sensor_msgs/PointCloud2 on /lidar_points (vendor Hesai driver or a bridge). Live cone topics /cones and /lidar/raw_points use interfaces/msg/Points and do not feed these scripts.
Runtime-rectified camera images (already undistorted; right camera already rotated 180°). The extrinsic solver does not undistort or rotate again.

Live gather:

cd $DRIVERLESS/driverless_ws/src/point_to_pixel/scripts
python gather_extrinsic_poses.py --camera l --square-m 0.025 \
  --board-width-m 0.3 --board-height-m 0.2
# right: --camera r

SPACE records a pose when $|t_{\text{lidar}} - t_{\text{cam}}| \le 50\,\text{ms}$ (script sync gate). c forces a solve; q exits. After enough valid poses, a passing solve writes projection_matrix_{cam} into params.yaml.

Poses need varied board orientation; parallel normals fail observability (script defaults: plane-normal angle ≥15°, centroid std ≥0.15 m). See extrinsics math.

Offline: saved pose_XXX/ directories (image_*.png, cloud.npy) into calibrate_extrinsics.py.

Write gate in the scripts: mean corner error ≤ 2.0 px, p95 ≤ 4.0 px. --write-anyway writes despite a failing gate.

dlt_proj_adjuster.py applies small $\Delta R$ / $\Delta u,\Delta v$ after a solve. See legacy DLT.

Non-spatial params in the same YAML

color_calibrate.py: HSV low/high bounds. Used on HSV builds only.
imu_to_lidar: hand-set lever arm (car vs cart comments). Used when integrating /filter/twist.

Runtime use

Camera managers prefer seecam_calibration_{l|r}.yaml, fall back to legacy seecam_calibration.yaml, build undistort maps, and rotate the right camera 180° each frame. Point-to-pixel loads projection_matrix_l/r and maps each centroid to a pixel. See projection math and runtime projection.

Script gates

Stage	Typical script criterion
Intrinsic gather	Board detected; 1920×1200
Intrinsic validate	Mean reprojection ≤ 1.0 px
Extrinsic poses	≥3 poses with chessboard and board-sized LiDAR plane (≥40 inliers)
Observability	Normal angle ≥15°; centroid std ≥0.15 m
Extrinsic write	Mean ≤ 2 px; p95 ≤ 4 px
Live sync	$\lvert t_{\text{lidar}} - t_{\text{cam}}\rvert \le 50\,\text{ms}$

Common failures: wrong resolution, poor FOV coverage, board not elevated (no LiDAR plane), similar poses, sync rejects, stale /lidar_points, unset DRIVERLESS.