Perception Autonomy Cleanup #450
Conversation
| * @param newPixelFormat the PixelFormat of the new image. | ||
| * @return A new {@link RawImage} with the same intrinsics & metadata, but with a different image. | ||
| */ | ||
| public RawImage replaceImage(Mat newCpuImageMat, PixelFormat newPixelFormat) |
There was a problem hiding this comment.
Realized this was missing
| @@ -222,22 +246,12 @@ public Instant getAcquisitionTime() | |||
|
|
|||
| public int getWidth() | |||
| { | |||
| if (hasCpuImage()) | |||
There was a problem hiding this comment.
These checks became unnecessary when CameraIntrinsics were added
| * @param depthImage 16UC1 depth image, used to get points of detected objects | ||
| */ | ||
| public void runYOLODetection(YOLOv8ObjectDetector yoloDetector, RawImage colorImage, RawImage depthImage) | ||
| { | ||
| if (yoloDetector.isReady() && !yoloExecutorService.isShutdown()) | ||
| { | ||
| // Acquire the images | ||
| if (colorImage.get() == null || depthImage.get() == null) |
There was a problem hiding this comment.
Moved calls to get here so it guarantees you "get" the images before they're deallocated.
| if (yoloDetectionResults.containsKey(lastRunDetectorIndex)) | ||
| yoloDetectionResults.remove(lastRunDetectorIndex).destroy(); | ||
| yoloDetectionResults.put(lastRunDetectorIndex, yoloResults); | ||
| synchronized (yoloDetectionResults) |
There was a problem hiding this comment.
I think the detection results may have been getting deallocated while the annotated image thread was using them, so I put think synchronized block for now.
Really, this class needs an overhaul.
| @@ -23,7 +23,7 @@ public class YOLOv8DetectionResults | |||
| private final MatVector outputBlobs; | |||
| private final RawImage detectionImage; | |||
| private final FloatIndexer outputMasksIndexer; | |||
| private final Map<YOLOv8DetectionOutput, Mat> objectMasks = new HashMap<>(); | |||
| private final Map<YOLOv8DetectionOutput, RawImage> objectMasks = new HashMap<>(); | |||
There was a problem hiding this comment.
Using RawImage instead of Mat just to be extra safe about memory allocation/deallocation here
| @@ -127,19 +127,22 @@ public boolean encodeNextFrame(Consumer<AVPacket> packetConsumer) | |||
| return error != AVERROR_EOF(); | |||
| } | |||
|
|
|||
| public boolean flush(Consumer<AVPacket> packetConsumer) | |||
There was a problem hiding this comment.
Turns out we should be flushing out the encoders once all the frames are sent (a few packets might've stayed in the encoder, waiting to be grouped together)
| @@ -139,6 +136,10 @@ private String getDeviceInfo(rs2_device rs2Device, int deviceInfoEnum) | |||
|
|
|||
| private int updateDeviceCount() | |||
| { | |||
| realsense2.rs2_delete_device_list(devices); | |||
| devices = realsense2.rs2_query_devices(context, error); | |||
There was a problem hiding this comment.
Feels like this is correct (performing another query when we want to update the device count). Seems to have solved an issue I was having
| import java.time.Instant; | ||
| import java.util.function.Supplier; | ||
|
|
||
| public class RealsenseImageSensor extends ImageSensor |
There was a problem hiding this comment.
Tempted to name it RealSenseImageSensor to be consistent with the brand name
TomaszTB
force-pushed
the
feature/perception-autonomy-cleanup-2
branch
2 times, most recently
from
79a1355 to
188efc7
Compare
TomaszTB
force-pushed
the
feature/perception-autonomy-cleanup-2
branch
from
188efc7 to
0f4c80a
Compare
TomaszTB
force-pushed
the
feature/perception-autonomy-cleanup-2
branch
2 times, most recently
from
b2763e2 to
4eb90b2
Compare
TomaszTB
force-pushed
the
feature/perception-autonomy-cleanup-2
branch
from
4eb90b2 to
061aaba
Compare
Added abstract ImageSensor + new NadiaPerceptionAutonomyProcess which can be used with real or simulated sensors.
Requires #439 to be merged first.