Feature: Use Perlin ROI to create noise on to the object of interest #1061

leonardloh · 2023-04-28T09:56:39Z

Description

Provide a summary of the modification as well as the issue that has been resolved. List any dependencies that this modification necessitates.
Our aim is to generate fake anomalous images for the validation set, and then use Anomalib's existing adaptive thresholding mechanism. But the current Perlin Noise augmentation will cover the whole image instead of the object of interest.
To automatically segment out the ROI Mask of the object given an image and then implement Perlin Noise onto the object only.

Code changes:

Subclass Augmenter to create another class call PerlinROIAugmenter
Create PerlinROI methods in the PerlinROIAugmenter
Enable Dataset to access synthetic type (default: Perlin) from config.yaml

Changes

Bug fix (non-breaking change which fixes an issue)
Refactor (non-breaking change which refactors the code base)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist

My code follows the pre-commit style and check guidelines of this project.
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing tests pass locally with my changes
I have added a summary of my changes to the CHANGELOG (not for minor changes, docs and tests).

…nomalib into feature/augment_roi

ashwinvaidya17

This is a great addition! Thanks. The only thing I am concerned about is the keys in config. It might require some design. But it is something we will have to discuss internally.

ashwinvaidya17 · 2023-04-28T11:26:50Z

src/anomalib/models/padim/config.yaml

@@ -13,9 +13,11 @@ dataset:
 transform_config:
 train: null
 eval: null
- test_split_mode: from_dir # options: [from_dir, synthetic]
+ test_split_mode: synthetic # options: [from_dir, synthetic]
+ test_synthetic_type: perlin_roi


Maybe these should live under a separate key. Since synthetic is optional test_synthetic_mode probably shouldn't be at the same level as other keys.
How about?

test_split: mode: synthetic/from_dir optional_key1: ... optional_key2: ... ...

Hi Ashwin, thanks for the suggestion. We will work on the changes and keep you updated.

Indeed, the flexibility to choose various generation methods (test set & validation set) might require further processing during setup, perhaps in a separate discussion/PR.

hi @ashwinvaidya17 can we proceed with this design? We would like your approval before we make any changes as many codes will be impacted.

test_split: mode: synthetic # options: [from_dir, synthetic] type: perlin_roi ratio: 0.2

Since @djdameln designed this, he might have better insight regarding the design. We can wait for his feedback.

ashwinvaidya17 · 2023-04-28T11:28:06Z

tests/pre_merge/datasets/test_synthetic_data.py

 return synthetic_dataset

 return make


 @pytest.fixture(autouse=True)
-def synthetic_dataset_from_samples():
+def make_perlinroi_synthetic_dataset():


Since both the methods are very similar, we can use mark.parameterize https://docs.pytest.org/en/6.2.x/parametrize.html#pytest-mark-parametrize-parametrizing-test-functions

Noted, we will work on the changes. Thanks again Ashwin

ashwinvaidya17 · 2023-04-28T11:37:50Z

src/anomalib/data/utils/augmenter.py

+ )
+ perturbation, mask = self.generate_perturbation(height, width, anomaly_source_path)
+ perturbations_list.append(Tensor(perturbation).permute((2, 0, 1)))
+ masks_list.append((Tensor(mask).permute((2, 0, 1)) * thresh_batch))


Except for thresh_batch here this is similar to the base method. Maybe we can just add an if statement in the base class and add this line there.

good idea, added this and removed the child method

ashwinvaidya17 · 2023-04-28T11:46:22Z

src/anomalib/data/utils/augmenter.py

+
+ def gaussian_blur(self, image: Tensor, transform: A.Compose) -> tuple[Tensor, Tensor]:
+ """Generate Gaussian Blur.
+


Can you also describe the intention behind performing gaussian blur and finding out the thresholds based on it.

we renamed the function generate_roi as it does more than gaussian blur. The intention is to generate the region of interest.

ashwinvaidya17 · 2023-04-28T11:51:36Z

src/anomalib/data/utils/augmenter.py

+
+ cv2.bitwise_not(thresh)
+
+ if thresh[0, 0] & thresh[-1, -1] & thresh[-1, 0] & thresh[0, -1] == 255:


Can you explain this line? From what I understand this inverts the image if the background is white

similar to your understanding: if all four corners of the image are white, we invert the mask.
we added this as a comment in the code base as well

ashwinvaidya17 · 2023-04-28T11:53:19Z

src/anomalib/data/utils/augmenter.py

+
+
+class PerlinROIAugmenter(Augmenter):
+ def augment_batch(self, batch: Tensor, thresh_batch: Tensor | None = None) -> tuple[Tensor, Tensor]:


Can you add a short description on how this works. Something like, extracts foreground object based on binary thresholding and limits perlin noise augmentation to that region.

added as the docstring for this class

djdameln

Thanks. This could be a useful addition to our Perlin noise mechanism. Overall I have the same comments as Ashwin, in that some more explanation about the methods used to obtain the final noise masks could be provided in the docstrings.

djdameln · 2023-05-19T11:12:45Z

src/anomalib/data/folder_3d.py

@@ -292,8 +294,10 @@ class Folder3D(AnomalibDataModule):
 during validation.
 Defaults to None.
 test_split_mode (TestSplitMode): Setting that determines how the testing subset is obtained.
+ test_synthetic_type (TestSyntheticType): Method for generating synthetic test data.


I don't know if it makes sense to expose these parameters to the 3d datasets. The anomalous augmentations are only applied to the images, not to the 3d point cloud data.

thanks @djdameln, we removed it from this dataset. please let us know if there are more.

djdameln · 2023-05-19T11:22:21Z

src/anomalib/data/utils/augmenter.py

 """Generate anomalous augmentations for a batch of input images.

 Args:
 batch (Tensor): Batch of input images
+ thresh_batch (Tensor | None, optional): Batch of


this parameter description is incomplete

added comment and refactored the class based on @ashwinvaidya17's suggestions

rlskit13 and others added 20 commits April 27, 2023 15:30

WIP commit

4a54ada

Add PerlinROIAugmenter import

0bb4f5a

Added image thresholding function

503bf58

Code refactored

eaa29dc

add synthetic method enums

126f4ab

Fixes to pass precommit checks

71c6cd1

Merge branch 'feature/augment_roi' of https://github.com/leonardloh/a…

bd3490d

…nomalib into feature/augment_roi

Fixed precommit checks

eaf27be

insert hook between datamodule and dataset

4915d58

clean up & documentation

56bc7d6

Code refactored

2f25f85

propogate synthetic to all datamodules

d3ebf10

add val_synthetic to config

1bf92a6

remove synthetic from videos

2ff4b93

Merge branch 'openvinotoolkit:main' into feature/augment_roi

0dee755

update documentation for augmenter class

634a9e9

add default to from_dataset fn

62e9caf

add defaults to synthetic class

249a0e5

add test cases for synthetic data

eb3814e

cosmetic changes

61c8722

leonardloh requested review from samet-akcay, ashwinvaidya17 and djdameln as code owners April 28, 2023 09:56

github-actions bot added the Tests label Apr 28, 2023

leonardloh changed the title ~~Feature/augment roi~~ Feature: Use Perlin ROI to create noise on to the object of interest Apr 28, 2023

samet-akcay added T2 Tests and removed Tests labels Apr 28, 2023

Merge branch 'main' into feature/augment_roi

026f9e3

ashwinvaidya17 requested changes Apr 28, 2023

View reviewed changes

samet-akcay added the Hackathon label May 5, 2023

djdameln reviewed May 19, 2023

View reviewed changes

kxtan added 2 commits May 22, 2023 18:47

remove synthetic from folder_3d

51bd477

improve clarity of augmenter classes

b8a4154

samet-akcay removed the Feature label Aug 4, 2023

samet-akcay removed the Hackathon label Jan 2, 2024

samet-akcay removed the T2 label Apr 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: Use Perlin ROI to create noise on to the object of interest #1061

Feature: Use Perlin ROI to create noise on to the object of interest #1061

leonardloh commented Apr 28, 2023

ashwinvaidya17 left a comment

ashwinvaidya17 Apr 28, 2023

kxtan May 1, 2023 •

edited

leonardloh May 23, 2023

ashwinvaidya17 May 25, 2023

ashwinvaidya17 Apr 28, 2023

kxtan May 1, 2023

ashwinvaidya17 Apr 28, 2023

kxtan May 23, 2023

ashwinvaidya17 Apr 28, 2023

kxtan May 23, 2023

ashwinvaidya17 Apr 28, 2023

kxtan May 23, 2023

ashwinvaidya17 Apr 28, 2023

kxtan May 23, 2023

djdameln left a comment

djdameln May 19, 2023

kxtan May 23, 2023

djdameln May 19, 2023

kxtan May 23, 2023


		def gaussian_blur(self, image: Tensor, transform: A.Compose) -> tuple[Tensor, Tensor]:
		"""Generate Gaussian Blur.


		cv2.bitwise_not(thresh)

		if thresh[0, 0] & thresh[-1, -1] & thresh[-1, 0] & thresh[0, -1] == 255:



		class PerlinROIAugmenter(Augmenter):
		def augment_batch(self, batch: Tensor, thresh_batch: Tensor \| None = None) -> tuple[Tensor, Tensor]:

Feature: Use Perlin ROI to create noise on to the object of interest #1061

Are you sure you want to change the base?

Feature: Use Perlin ROI to create noise on to the object of interest #1061

Conversation

leonardloh commented Apr 28, 2023

Description

Changes

Checklist

ashwinvaidya17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kxtan May 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djdameln left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kxtan May 1, 2023 •

edited