Oak-D pro stereo depth and other possible sensors of the camera

[AncientGrey]

Robot Model

Turtlebot4 Standard

ROS distro

Humble

Networking Configuration

Simple Discovery

OS

Ubuntu 22.04

Built from source or installed?

Built from Source

Package version

ros-humble-turtlebot4-turtlebot4_robot

Type of issue

Camera

Expected behaviour

Hi, is it possible to provide a camera (oak-d pro) config .yaml file, where every Sensor and Actuator is included and activated?
That way it is easier to get an overview of all the possible uses of the camera and the turtlebot.

Actual behaviour

I have been trying to get the depth image to work, but i have troubles with the camera config.

Error messages

No response

To Reproduce

edit oakd_pro.yaml
run oakd_bringup

Other notes

No response

[RustyCPR]

Hi @AncientGrey
Here are some example launch files: https://github.com/luxonis/depthai-ros/tree/humble/depthai_examples/launch

Please see this ticket reply related to additional depth camera topics: #277 (comment)

Regards,
Rusty

[AncientGrey]

Hi @RustyCPR ,
would it be possible to compile all options in one .yaml file, with all options for the oakd_pro.yaml?

[RustyCPR]

Hi @AncientGrey
Not clear what your request is. Can you please provide a more in-depth description of what you are looking for?

[AncientGrey]

I was looking for a full config for the Oak-D Pro camera. If i change the pipeline type to anything apart from 'RGB' the Launch fails with an Error.
I would like to use the depth data of the camera to approach objects.
Following are my config paranmeters of the oakd:pro.yaml file and the Error of the launch.

/oakd:
  ros__parameters:
    camera:
      i_calibration_dump: false
      i_enable_imu: true
      i_enable_ir: true
      i_external_calibration_path: ''
      i_floodlight_brightness: 400
      i_ip: ''
      i_laser_dot_brightness: 800
      i_mx_id: ''
      i_nn_type: none
      i_pipeline_dump: true
      i_pipeline_type: RGBD
      i_publish_tf_from_calibration: false
      i_restart_on_diagnostics_error: false
      # i_tf_base_frame: oak
      # i_tf_cam_pitch: '0.0'
      # i_tf_cam_pos_x: '0.0'
      # i_tf_cam_pos_y: '0.0'
      # i_tf_cam_pos_z: '0.0'
      # i_tf_cam_roll: '0.0'
      # i_tf_cam_yaw: '0.0'
      # i_tf_camera_model: ''
      # i_tf_camera_name: oak
      # i_tf_custom_urdf_location: ''
      # i_tf_custom_xacro_args: ''
      i_tf_imu_from_descr: 'true'
      # i_tf_parent_frame: oak-d-base-frame
      # i_usb_port_id: ''
      i_usb_speed: SUPER_PLUS
    rgb:
      i_add_exposure_offset: false
      i_board_socket_id: 0
      i_calibration_file: ''
      i_disable_node: false
      i_enable_feature_tracker: false
      i_enable_lazy_publisher: true
      i_enable_nn: false
      i_enable_preview: false
      i_exposure_offset: 0
      i_fps: 30.0
      i_fsync_continuous: false
      i_fsync_trigger: false
      i_get_base_device_timestamp: false
      i_height: 720
      i_interleaved: false
      i_isp_den: 3
      i_isp_num: 2
      i_keep_preview_aspect_ratio: true
      i_low_bandwidth: false
      i_low_bandwidth_quality: 50
      i_max_q_size: 30
      i_output_isp: true
      i_preview_height: 300
      i_preview_size: 300
      i_preview_width: 300
      i_publish_topic: true
      i_resolution: 1080P
      i_reverse_stereo_socket_order: false
      i_sensor_img_orientation: AUTO
      i_set_isp3a_fps: false
      i_set_isp_scale: true
      i_simulate_from_topic: false
      i_simulated_topic_name: ''
      i_update_ros_base_time_on_ros_msg: false
      i_width: 1280
      r_exposure: 20000
      r_focus: 1
      r_iso: 800
      r_set_man_exposure: false
      r_set_man_focus: false
      r_set_man_whitebalance: false
      r_whitebalance: 3300
    imu:
      i_acc_cov: 0.0
      i_acc_freq: 400
      i_batch_report_threshold: 5
      i_enable_rotation: false
      i_get_base_device_timestamp: false
      i_gyro_cov: 0.0
      i_gyro_freq: 400
      i_mag_cov: 0.0
      i_max_batch_reports: 10
      i_message_type: IMU
      i_rot_cov: -1.0
      i_sync_method: LINEAR_INTERPOLATE_ACCEL
      i_update_ros_base_time_on_ros_msg: false
    left:
      i_add_exposure_offset: false
      i_board_socket_id: 1
      i_calibration_file: ''
      i_disable_node: false
      i_enable_feature_tracker: false
      i_enable_lazy_publisher: true
      i_enable_nn: false
      i_exposure_offset: 0
      i_fps: 30.0
      i_fsync_continuous: false
      i_fsync_trigger: false
      i_get_base_device_timestamp: false
      i_height: 720
      i_low_bandwidth: false
      i_low_bandwidth_quality: 50
      i_max_q_size: 30
      i_publish_topic: true
      i_resolution: 720P
      i_reverse_stereo_socket_order: false
      i_sensor_img_orientation: AUTO
      i_set_isp3a_fps: false
      i_simulate_from_topic: true
      i_simulated_topic_name: '/text/left'
      i_update_ros_base_time_on_ros_msg: false
      i_width: 1280
      r_exposure: 1000
      r_iso: 800
      r_set_man_exposure: false
    right:
      i_add_exposure_offset: false
      i_board_socket_id: 2
      i_calibration_file: ''
      i_disable_node: false
      i_enable_feature_tracker: false
      i_enable_lazy_publisher: true
      i_enable_nn: false
      i_exposure_offset: 0
      i_fps: 30.0
      i_fsync_continuous: false
      i_fsync_trigger: false
      i_get_base_device_timestamp: false
      i_height: 720
      i_low_bandwidth: false
      i_low_bandwidth_quality: 50
      i_max_q_size: 30
      i_publish_topic: true
      i_resolution: 720P
      i_reverse_stereo_socket_order: false
      i_sensor_img_orientation: AUTO
      i_set_isp3a_fps: false
      i_simulate_from_topic: false
      i_simulated_topic_name: ''
      i_update_ros_base_time_on_ros_msg: false
      i_width: 1280
      r_exposure: 1000
      r_iso: 800
      r_set_man_exposure: false
    stereo:
      i_add_exposure_offset: false
      i_align_depth: true
      i_bilateral_sigma: 0
      i_board_socket_id: 0
      i_depth_filter_size: 5
      i_depth_preset: HIGH_ACCURACY
      i_disparity_width: DISPARITY_96
      i_enable_alpha_scaling: false
      i_enable_brightness_filter: false
      i_enable_companding: false
      i_enable_decimation_filter: false
      i_enable_disparity_shift: false
      i_enable_distortion_correction: false
      i_enable_lazy_publisher: true
      i_enable_spatial_filter: false
      i_enable_spatial_nn: false
      i_enable_speckle_filter: false
      i_enable_temporal_filter: false
      i_enable_threshold_filter: false
      i_exposure_offset: 0
      i_extended_disp: false
      i_get_base_device_timestamp: false
      i_height: 720
      i_left_rect_add_exposure_offset: false
      i_left_rect_enable_feature_tracker: false
      i_left_rect_exposure_offset: 0
      i_left_rect_low_bandwidth: false
      i_left_rect_low_bandwidth_quality: 50
      i_left_socket_id: 1
      i_low_bandwidth: false
      i_low_bandwidth_quality: 50
      i_lr_check: true
      i_lrc_threshold: 10
      i_max_q_size: 30
      i_output_disparity: false
      i_publish_left_rect: false
      i_publish_right_rect: false
      i_publish_synced_rect_pair: false
      i_publish_topic: true
      i_rectify_edge_fill_color: 0
      i_reverse_stereo_socket_order: false
      i_right_rect_add_exposure_offset: false
      i_right_rect_enable_feature_tracker: false
      i_right_rect_exposure_offset: 0
      i_right_rect_low_bandwidth: false
      i_right_rect_low_bandwidth_quality: 50
      i_right_socket_id: 2
      i_set_disparity_to_depth_use_spec_translation: false
      i_set_input_size: false
      i_socket_name: rgb
      i_spatial_nn_source: right
      i_stereo_conf_threshold: 240
      i_subpixel: false
      i_update_ros_base_time_on_ros_msg: false
      i_width: 1280
    use_sim_time: false

Nov 21 21:00:09 ubuntu turtlebot4-start[1112]: [joint_state_publisher-8] [INFO] [1700600409.710459042] [blastoise.joint_state_publisher]: Waiting for robot_description to be published on the robot_description topic...
Nov 21 21:00:12 ubuntu turtlebot4-start[1112]: [component_container-6] [INFO] [1700600412.662308717] [blastoise.oakd]: Camera with MXID: 18443010B162BC0800 and Name: 1.1.2 connected!
Nov 21 21:00:12 ubuntu turtlebot4-start[1112]: [component_container-6] [INFO] [1700600412.663868408] [blastoise.oakd]: USB SPEED: SUPER
Nov 21 21:00:12 ubuntu turtlebot4-start[1112]: [component_container-6] [INFO] [1700600412.695560891] [blastoise.oakd]: Device type: OAK-D-PRO
Nov 21 21:00:12 ubuntu turtlebot4-start[1112]: [component_container-6] [INFO] [1700600412.700948703] [blastoise.oakd]: Pipeline type: RGBD
Nov 21 21:00:13 ubuntu turtlebot4-start[1112]: [component_container-6] [INFO] [1700600413.174712718] [blastoise.oakd]: Finished setting up pipeline.
Nov 21 21:00:13 ubuntu turtlebot4-start[1112]: [component_container-6] [INFO] [1700600413.178229506] [blastoise.oakd]: Saving pipeline schema to: /tmp/18443010B162BC0800_pipeline.json
Nov 21 21:00:17 ubuntu turtlebot4-start[1112]: [component_container-6] [18443010B162BC0800] [1.1.2] [1700600417.479] [host] [warning] Device crashed, but no crash dump could be extracted.
Nov 21 21:00:19 ubuntu turtlebot4-start[1112]: [component_container-6] [ERROR] [1700600419.071206213] [blastoise.oakd_container]: Component constructor threw an exception: Device already closed or disconnected: Input/output error
Nov 21 21:00:19 ubuntu turtlebot4-start[1112]: [ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'oakd' of type 'depthai_ros_driver::Camera' in container '/blastoise/oakd_container': Component constructor threw an exception: Device already closed or disc>