Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft PR that targets the work done towards TrackMate v9. #263

Closed
wants to merge 264 commits into from
Closed

Conversation

tinevez
Copy link
Member

@tinevez tinevez commented May 10, 2023

Support 3D segmentation by adding spots with triangular meshes.

A 3D viewer for TrackMate based on the BVV.

#280

image

A tool to configure a sub-selection of feature analyzers.

#261

There is now a new 'plugin' that launches a configuration panel for the TrackMate feature analyzers.

It can be launched with the menu item: Edit > Options > Configure TrackMate feature analyzers.... and show this dialog:

Screenshot 2023-05-10 at 17 05 43

It allows selecting or deselecting feature analyzers.
Upon pressing the Save to user defaults button, a JSON file is created:

❯ pwd
/Users/tinevez/.trackmate
❯ cat featureselection.json
{
  "allAnalyzers": {
    "spots": {
      "Manual spot color": true,
      "Spot 2D shape descriptors": true,
      "Spot 3D shape descriptors": true,
      "Spot contrast and SNR": false,
      "Spot fit 2D ellipse": true,
      "Spot fit 3D ellipsoid": true,
      "Spot intensity": true
    },
    "edges": {
      "Directional change": true,
      "Edge location": true,
      "Edge speed": true,
      "Edge target": true,
      "Manual edge color": true
    },
    "tracks": {
      "Branching analyzer": true,
      "Track duration": true,
      "Track index": true,
      "Track location": true,
      "Track motility analysis": true,
      "Track quality": true,
      "Track speed": true
    }
  }
}%                         

This selection will be used next time the user launches the TrackMate GUI.
This allows skipping the computation of analyzers that the users do not need. This will become useful in particular for 3D analysis with segmentation, where iterating through all the pixels of all meshes might be very long.

However it brings a risk: because TrackMate does not manage dependencies, the user might remove an analyzer that is needed by another one (example: spot SNR requires spot intensity). This risk is not mitigated yet.

Spot is now an interface, with 3 derived class.

Spot -> the main interface, used by default in trackers. Define basic methods to get and store feature values.
SpotBase -> Plain spots, like for TrackMate v<7
SpotRoi -> spot has a polygon as a contour in 2D
SpotMesh -> spot has a 3D mesh

More elegant and extensible to app consuming TrackMate trackers with special objects.

The detection preview is cancelable.

Of course it only actually cancels the preview if the detector that is used is itself cancelable.
Screenshot 2024-03-28 at 20 21 43

@tinevez tinevez changed the title Support 3D segmentation in TrackMate. Draft PR that targets the work done towards TrackMate v8. Sep 12, 2023
tinevez and others added 23 commits July 7, 2024 19:36
And remove debug code.
I am not so sure it is a good idea for the color...
and use the white LUT when the imp is not displayed as a Composite.
Otherwise we get crashes when we have more than 4k labels. Which is not
what Labkit is optimized for but we will see that in a second time.
In case we change our minds on the backing integer type, right now
the labkit launcher and importer classes are generic.
The re-importing of labels from Tabkit to TrackMate could fail for
2D images and labels with a large index. For instance, it failed
consistently when trying to re-import labels with an index larger
than 65643.

This problem roots in the getSpots() method of LabkitImporter. It
relies on a trick: We get the new label image, and create spots from
this label image. But we want the new spots to keep track of the index
in the label image they were generated from.

For this, in 2D, we use the MaskUtils.fromLabelingWithRoi()
method. These methods accept an image as last argument used to
read a value in the label image within the spot, that is normally
used for the quality value of the new spot.

But the SpotRoiUtils.from2DLabelingWithRoi() method converted the
extra image to ImagePlus (because I was lazy). So the label image
was effectively cast on ushort for an IntegerType image, hence
the problem with the max label being 65453.

The solution is to rewrite the from2DLabelingWithRoi() so that
it does not rely on converting to ImagePlus, but on good old
iteration with imglib2.
Provided that the detector that is called is cancelable.
Make sure the detection preview panel is slanted at the bottom of its display.
The bug was causing weird issues with unedited spots being deleted,
unedited spots being duplicated etc.

It took me really long to understand the cause. It was hidden in the
step where we go from a label image to a collection of spots. Because
spots are polygons, with simplified contours, there might be some pixels
on the edges of the object that are not strictly inside the label.

In this importer, we read the label value in one go, by storing it
in the QUALITY value of the spot, in the MaskUtils class. But since
the spots have simplified contours, and since the QUALITY value is
the maximal value iterated over, our approach might fail on border cases:
- when the contout is approximated and include pixels from another
object
- and when this object has a label value higher than the lael of
the spot.

This commit include a temporary fix: we reiterate over the spot
but takes the median value iterated over, to make sure we read
the correct value for the label.

Other attempts will follow, for reference. But a true fix involves
making a method that returns a map from label value to spot.
This time we fix it by creating spots that do not have a simplified
contours. In that case we strictly iterate over the pixels inside
label and get the correct value.
However the created spots have a pixelated aspect (contours are
not simplified), which might not be what the user wants. We should
let them choose.

Still not the perfect solution, as mentionned in the previous
commmit.
tinevez and others added 20 commits July 22, 2024 15:41
The re-importing of labels from Tabkit to TrackMate could fail for
2D images and labels with a large index. For instance, it failed
consistently when trying to re-import labels with an index larger
than 65643.

This problem roots in the getSpots() method of LabkitImporter. It
relies on a trick: We get the new label image, and create spots from
this label image. But we want the new spots to keep track of the index
in the label image they were generated from.

For this, in 2D, we use the SpotRoiUtils.from2DLabelingWithRoi()
method. These methods accept an image as last argument used to
read a value in the label image within the spot, that is normally
used for the quality value of the new spot.

But the SpotRoiUtils.from2DLabelingWithRoi() method converted the
extra image to ImagePlus (because I was lazy). So the label image
was effectively cast on ushort for an IntegerType image, hence
the problem with the max label being 65453.

The solution is to rewrite the from2DLabelingWithRoi() so that
it does not rely on converting to ImagePlus, but on good old
iteration with imglib2.
Moving the plugin implementation out of the test class and
removing the legacy injector make the test pass in maven.
Instead of the interval, so that this is consistent across
TrackMate.
This works around a test failure in TrackMatePluginTest caused by the
original ImageJ classes being loaded too soon. There are other ways of
working around this problem, such as creating a SciJava context more
eagerly, but it's a distraction from the business of testing TrackMate,
and anyway we don't need imagej-legacy on the classpath.

The only reason labkit-ui has imagej-legacy as a dependency is for one
line of code, which can be refactored to avoid it, so hopefully the
exclusion will be able to disappear later after labkit-ui is updated.
* Avoid SNAPSHOT versions.
* Factor out version pins to properties.
* Avoid jogamp *-main uber-JARs.
We want to retrieve spots as a map feom labels to corresponding
spots.
When editing the whole movie, if the label of a new spot was using
the label of an existing spot in another time-point, the existing
one was removed. Because the new spot was identified as a modification
of the existing one.
The solution is to pass to the importer only the list of existing
spots in the current time-frame.
tinevez and others added 5 commits July 23, 2024 11:17
Counterpart to the same one in SpotRoiUtils.
Nota: the methods signature order should be harmonized between the
two utility classes.
There is one discrepancy with the 2D part, linked to TrackMate:
In 2D, one connected components will give one spot, even if several
connected components belong to the same label.
In 3D, it is possible to have one spot made of several disconnected
components. Will have to document this.
Not implemented yet, but at least symmetric with the 2D case.
@tinevez tinevez closed this Nov 7, 2024
@tinevez tinevez deleted the v8 branch November 7, 2024 09:56
@tinevez tinevez changed the title Draft PR that targets the work done towards TrackMate v8. Draft PR that targets the work done towards TrackMate v9. Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

3 participants