rastervision semantic segmentation pipeline for five classes

[mustafa-ec]

How to use semantic segmentation pipeline for five classes? What modifications need to be done in the existing code available in
https://docs.rastervision.io/en/stable/framework/quickstart.html ??

[AdeelH]

To change the classes, you will need to modify the ClassConfig. Also take a look at the ISPRS Potsdam example, which is a more advanced example than the one in the Quickstart.

[AdeelH]

Good question! To use GeoJSON labels, take a look at how the label_source is created in this example:

raster-vision/rastervision_pytorch_backend/rastervision/pytorch_backend/examples/semantic_segmentation/spacenet_vegas.py

Lines 117 to 129 in 0cac03c

	vector_source = GeoJSONVectorSourceConfig(
	uris=label_uri,
	transformers=[
	ClassInferenceTransformerConfig(default_class_id=0),
	BufferTransformerConfig(
	geom_type='LineString', class_bufs={0: 15}),
	BufferTransformerConfig(geom_type='Point'),
	])
	label_source = SemanticSegmentationLabelSourceConfig(
	raster_source=RasterizedSourceConfig(
	vector_source=vector_source,
	rasterizer_config=RasterizerConfig(background_class_id=1)))

You can ignore the BufferTransformerConfigs, but you will probably still need to keep the ClassInferenceTransformerConfig. ClassInferenceTransformerConfig is for specifying how to assign classes to polygons in the GeoJSON file (see docs for more details).

Let me know if you have any more questions.

[mustafa-ec]

I was running the ISPRS Potsdam example (https://github.com/azavea/raster-vision/blob/master/rastervision_pytorch_backend/rastervision/pytorch_backend/examples/semantic_segmentation/isprs_potsdam.py) on ISPRS_Potsdam data but it is giving below error.

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 271, in <module>
    _main()
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 267, in _main
    main()
  File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 256, in run_command
    _run_command(
INFO:botocore.credentials:Found credentials in environment variables.
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 238, in _run_command
    command_fn()
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/rv_pipeline.py", line 128, in train
    backend.train(source_bundle_uri=self.config.source_bundle_uri)
  File "/opt/src/rastervision_pytorch_backend/rastervision/pytorch_backend/pytorch_learner_backend.py", line 128, in train
    learner.main()
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 398, in main
    self._main()
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 411, in _main
    self.train()
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 448, in train
    self._train(start_epoch, end_epoch)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 452, in _train
WARNING:rasterio._env:CPLE_AppDefined in /opt/data/tmp/cache/s3/graham-tx-data/Patched_Files/potsdam/4_Ortho_RGBIR/top_potsdam_6_12_RGBIR.tif: TIFFReadDirectory:Sum of Photometric type-related color channels and ExtraSamples doesn't match SamplesPerPixel. Defining non-color channels as ExtraSamples.
    self.on_train_start()
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 562, in on_train_start
    self.plot_dataloaders(self.cfg.data.preview_batch_limit)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 1368, in plot_dataloaders
    self.plot_dataloader(
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 1358, in plot_dataloader
    x, y = next(iter(dl))
           ^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 630, in __next__
INFO:botocore.credentials:Found credentials in environment variables.
    data = self._next_data()
           ^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1344, in _next_data
    return self._process_data(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1370, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.11/site-packages/torch/_utils.py", line 706, in reraise
    raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
            ~~~~~~~~~~~~^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/utils/data/dataset.py", line 350, in __getitem__
    return self.datasets[dataset_idx][sample_idx]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/dataset.py", line 270, in __getitem__
    out = super().__getitem__(window)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/dataset.py", line 68, in __getitem__
    val = self.orig_dataset[key]
          ~~~~~~~~~~~~~~~~~^^^^^
  File "/opt/src/rastervision_core/rastervision/core/data/scene.py", line 74, in __getitem__
    y = self.label_source[key]
        ~~~~~~~~~~~~~~~~~^^^^^
  File "/opt/src/rastervision_core/rastervision/core/data/label_source/semantic_segmentation_label_source.py", line 101, in __getitem__
    return self.get_label_arr(key)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/src/rastervision_core/rastervision/core/data/label_source/semantic_segmentation_label_source.py", line 79, in get_label_arr
    label_arr = self.raster_source.get_chip(window)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/src/rastervision_core/rastervision/core/data/raster_source/rasterio_source.py", line 172, in get_chip
    chip = self._get_chip(window, out_shape=out_shape, bands=bands)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/src/rastervision_core/rastervision/core/data/raster_source/rasterio_source.py", line 147, in _get_chip
    chip = fill_overflow(self.bbox, window, chip)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/src/rastervision_core/rastervision/core/data/utils/raster.py", line 35, in fill_overflow
    chip[..., :, xmax:, :] = fill_value
    ~~~~^^^^^^^^^^^^^^^^^^
TypeError: slice indices must be integers or None or have an __index__ method

[AdeelH]

Can you share the full logs from running the example? Were there any warnings before the error?

[AdeelH]

I suspect this is happening because the label TIFFs in the original Potsdam dataset are not georeferenced. Raster Vision recently dropped support for pairing non-georeferenced label files with georefereced image files, but the documentation for the example was not updated.

I think if you do something like this for all the files (using this gdalcopyproj.py), the error will be fixed:

python gdalcopyproj.py 4_Ortho_RGBIR/top_potsdam_2_10_RGBIR.tif 5_Labels_for_participants/top_potsdam_2_10_label.tif

[AdeelH]

On top of that, I believe this issue was also at play here: #2259