ArcGIS Interoperability #10040

tdipisa · 2024-03-07T11:01:48Z

Description

It is requested to improve on the interoperability to support better ArcGIS services in MapStore. This issue involves two main activities for which the following task is in common:

subtasks
Support of tokens for private layers - MapStore token support

ArcGIS WMS Server interoperability for GetFeatureInfo requests

Thanks to #9048, the serverType property has been recently added in MapStore to define which type of server is providing the WMS service.
It should be possible to use this option to exclude the exceptions parameter not supported in an ArcGIS WMS service.
During the development phase it is requested to evaluate if it is necessary an additional serverType or the NO_VENDOR type used also by other services is enough.
Possibly also hide some functionalities not supported by WMS endpoint that does not support GetFeatureInfo application/json responses with geometry, to do so it is necessary also to store the information related to the list of supported get feature info responses.

Some notes about findings from the preliminary investigation:

It seems that the DescribeLayer request used by MapStore to add the search parameter with WFS functionality is not supported in the ArcGIS WMS service. This means is not possible to associate a WFS service with a WMS programmatically in addition some tests done on agsnotfed2/rest/services/Test/country_4m/MapServer show that the WMS/WFS capabilities endpoint used two different name for the layer (0/country_4m:apgispaudw.common_data.country_4m) and this makes even difficult to associate them manually (but maybe this is only a configuration problem).

Involved sub tasks are:

subtasks
Review WMS get feature info request based on server type
Hide the highlight feature icon of get feature info plugin if GeoJSON response is not supported. This include work related to the storing of additional information in the layer config or alternative strategy

ArcGIS MapServer Catalog and Layer support

The work for this task focus on the support of ArcGIS MapServer endpoints inside MapStore. The task should include following improvement inside MapStore:

New catalog type for ArcGIS MapServer endpoint with the possibility to list layers and add them to the map
New ArcGIS MapServer layer type implementations for OpenLayers, Cesium and Leaflet with the default lifecycle
Support of base layer settings: opacity, visibility, title, description, group and visibility limits
Support of get feature info using the /query endpoint

Some notes about findings from the preliminary investigation:

A MapServer endpoint could provide a single layer or multiple layer that could compose a layer group. It is requested to evaluate in the development phase if the information about subLayerIds and parentLayerId have effect on the visualization in map
It shoud lbe possible to use maxScale and minScale properties of a layer to set the visibility limits. In general we should take a look to all the property available in the layer object
It is possible to send a request to /MapServer/{layerId} to get additional information about the specific layer
It is possible to use the query endpoint /MapServer/{layerId}/query (POST) for Get Feature Info

Involved sub tasks are:

subtasks
Support of ArcGIS MapServer endpoint in MapStore: catalog, layer visualization, get feature info and base layer settings

Acceptance criteria

It will be possible to successfully perform GetFeatureInfo requests according to the server type and show Identify results to end users
It will be possible to support layers from ArcGIS MapServer added to the map through the Catalog tool
The MS developer guide will be updated accordingly by including all related and necessary info

Other useful information

Here a branch where a base implementation sa been tested during the preliminary evaluation of this work. That should be completely refactored to be accepted in MapStore, take this just as a prototype.

The text was updated successfully, but these errors were encountered:

Igi-ID · 2024-04-30T14:05:58Z

These two public services have been used during development of the arcgis support so far:

More specifically:

These can also be used for easier testing when development is concluded.

Igi-ID · 2024-05-02T11:28:47Z

The current state of development is the following (updating this post as progress is made):

subtasks
Support of tokens for private layers - MapStore token support

Not yet implemented

WMS interoperability and GFI support

subtasks
Review WMS get feature info request based on server type
Hide the highlight feature icon of get feature info plugin if GeoJSON response is not supported. This include work related to the storing of additional information in the layer config or alternative strategy

GetFeatureInfo requests are supported by previous work done in Add MIME type support for info_format=application/geo+json for getFeatureInfo #9564, mainly due to esri servers using the geo+json mime type for these requests rather than json.
As a corollary, the MS features which aid GFI requests, such as the highlight feature, should work as expected.

ArcGIS MapServer Catalog and Layer support

subtasks
Support of ArcGIS MapServer endpoint in MapStore: catalog, layer visualization, get feature info and base layer settings

Service support (catalog) is available (*minor issues still present)
OL layer visualization is available (*minor issues still present)
Cesium layer visualization is not availble
Leaflet layer visualization is not available
GFI requests are available but fairly unstable (see minor issues below).
Base layer settings are available.

Specific issues:

allyoucanmap · 2024-05-06T11:07:32Z

@tdipisa @Igi-ID Some feedback:

Cesium layer visualization is not availble
Leaflet layer visualization is not available

just pushed some initial code to support arcgis mapserver layer on Cesium and Leaflet see 92eadf0 on dev_10040 so please push the code locally. The Leaflet implementation is not currently supporting tiling but for the moment we could skip this but we should ensure the Cesium implementation is working properly.

Catalog search tool (which filters available records) not implemented yet.

If the mapserver does not support pagination we should use a similar approach to the WMS one where we simulate the pagination and filter client side

raster sets with an empty layers property are not currently handled (due to data structure, at the base in fetchData). These are usually background images, orthophotos and the like.

Do you have an url example? Probably this case need to use a sort of TMS approach, looking at the Cesium code it is possible to request cached tiles stored in the tileInfo property. We should verify if it's the case.

the format of the identify feature is not handled (currently hardcoded), meaning you cannot change the format from the global settings, nor layer settings (menu unavailable).

I think it's fine for the moment to prevent the selection of the identify but I would like to understand where it has been hardcoded, could you please share the hardcoded part?

identify feature intersects tile with geometry (bbox?) rather than feature geometry with point.
As a consequence, you can click even outside geometries, but inside the bounding box of a tile and get a successful request, highlighting all geometries within that tile. Metadata is aggregated in the identify panel.

Did you try to use a point geometry without reconstruct a small bbox based on scale level?

I'm seeing this implementation with an implementation similar to WMS but probably it's not needed

maybe we could use lat and long from center property and use the supported geometry point available in MapServer query

Identify feature may occasionally fail by not registering the layer at all, especially in the case of spatially overlapping layers.

Could you provide steps to reproduce this?

Updating OL layers is not handled to its full extent and there is currently a critical failure related to reprojection.

Could you double check if this has been solved with my new commit?

I think this is not needed because we are not rendering features client side, the source is based on image tiles and not vector

Bounding box as well as few other relevant information points are not part of the layer object.

I think it's missing an additionl request to the layer id endpoint to get layer metadata, we can place this request in the addLayerAndDescribeEpic. This action is triggered when the layer is added from catalog so we can add an if statement for arcgis with the request to layer.url/layer.name. This endpoint should contain all the needed info for bounding box and probably also if the layer is queryable or not, at the moment we expect at minimum the following information for the layer:

{
    type: 'arcgis',
    url: '/rest/MapServer',
    name: '0',
    visibility: true,
    opacity: 1,
    bbox: { ... }
}

Note: please does not pass the id of the layer so mapstore is able to create a uuid

Zoom to feature (identify panel) is not working.

Zoom to feature is not working because is missing the bbox property in the layer options

Add layer feature in layer properties panel is not working.

Could you explain this issue with replicable steps?

Igi-ID · 2024-05-07T15:04:27Z

I'll update this post when some answers are better formed.

@tdipisa @Igi-ID Some feedback:

Cesium layer visualization is not availble
Leaflet layer visualization is not available

just pushed some initial code to support arcgis mapserver layer on Cesium and Leaflet see 92eadf0 on dev_10040 so please push the code locally. The Leaflet implementation is not currently supporting tiling but for the moment we could skip this but we should ensure the Cesium implementation is working properly.

Catalog search tool (which filters available records) not implemented yet.

If the mapserver does not support pagination we should use a similar approach to the WMS one where we simulate the pagination and filter client side

Noted on both points above. Thank you for bringing up server side pagination, I'll re-check that and proceed accordingly.

raster sets with an empty layers property are not currently handled (due to data structure, at the base in fetchData). These are usually background images, orthophotos and the like.

Do you have an url example? Probably this case need to use a sort of TMS approach, looking at the Cesium code it is possible to request cached tiles stored in the tileInfo property. We should verify if it's the case.

http://www.pcn.minambiente.it/arcgis/rest/services/immagini/IGM_25000/MapServer

the format of the identify feature is not handled (currently hardcoded), meaning you cannot change the format from the global settings, nor layer settings (menu unavailable).

I think it's fine for the moment to prevent the selection of the identify but I would like to understand where it has been hardcoded, could you please share the hardcoded part?

MapStore2/web/client/utils/mapinfo/arcgis.js

Line 54 in 92eadf0

f: 'geojson',

identify feature intersects tile with geometry (bbox?) rather than feature geometry with point.
As a consequence, you can click even outside geometries, but inside the bounding box of a tile and get a successful request, highlighting all geometries within that tile. Metadata is aggregated in the identify panel.

Did you try to use a point geometry without reconstruct a small bbox based on scale level?

Good pointer, will retest

I'm seeing this implementation with an implementation similar to WMS but probably it's not needed

maybe we could use lat and long from center property and use the supported geometry point available in MapServer query

Identify feature may occasionally fail by not registering the layer at all, especially in the case of spatially overlapping layers.

Could you provide steps to reproduce this?

My report on the issue is incomplete and badly phrased. It's essentially this reported bug: #10227

namely, when we put layers into a group and select the group in the panel, GFI will resolve with 'No active queryable layer'.

Updating OL layers is not handled to its full extent and there is currently a critical failure related to reprojection.

Could you double check if this has been solved with my new commit?

Solved indeed.

I think this is not needed because we are not rendering features client side, the source is based on image tiles and not vector

Bounding box as well as few other relevant information points are not part of the layer object.

I think it's missing an additionl request to the layer id endpoint to get layer metadata, we can place this request in the addLayerAndDescribeEpic. This action is triggered when the layer is added from catalog so we can add an if statement for arcgis with the request to layer.url/layer.name. This endpoint should contain all the needed info for bounding box and probably also if the layer is queryable or not, at the moment we expect at minimum the following information for the layer:
{
    type: 'arcgis',
    url: '/rest/MapServer',
    name: '0',
    visibility: true,
    opacity: 1,
    bbox: { ... }
}
Note: please does not pass the id of the layer so mapstore is able to create a uuid

Great pointer, making the adjustments.

Zoom to feature (identify panel) is not working.

Zoom to feature is not working because is missing the bbox property in the layer options

So related to the point above as expected, will retest when bbox is available

Add layer feature in layer properties panel is not working.

Could you explain this issue with replicable steps?

This is also a case of bad phrasing, the issue is related to the add layer to selected group action and the scenario is the following:

create a group, open toc, select group
click ``add layer to selected group` button, pick a layer from the catalog and add it to the map.
the layer is not added to the group
Initial investigation showed that this action works only for WMS. Video below.

add.layer.to.selected.group.mp4

…upport (#10330) --------- Co-authored-by: Igor Dimov <[email protected]>

tdipisa · 2024-05-23T09:35:02Z

These two public services have been used during development of the arcgis support so far:

http://www.pcn.minambiente.it/arcgis/rest/services

https://hazards.fema.gov/arcgis/rest/services

More specifically:

https://hazards.fema.gov/arcgis/rest/services/public/NFHL/MapServer

https://hazards.fema.gov/arcgis/rest/services/Studies/StudiesTracking/MapServer

http://www.pcn.minambiente.it/arcgis/rest/services/ADB/MapServer

http://www.pcn.minambiente.it/arcgis/rest/services/qu_lidar_0202/MapServer

http://www.pcn.minambiente.it/arcgis/rest/services/immagini/IGM_25000/MapServer

These can also be used for easier testing when development is concluded.

This also is another one available for testing:
https://sampleserver6.arcgisonline.com/arcgis/rest/services

tdipisa added the New Feature used for new functionalities label Mar 7, 2024

tdipisa added this to the 2024.02.00 milestone Mar 7, 2024

tdipisa assigned Igi-ID Mar 7, 2024

tdipisa added the C130-TOTAL-2024-GeoNode-DEV label Mar 7, 2024

Igi-ID mentioned this issue Apr 30, 2024

Init ArcGIS Support #10265

Merged

6 tasks

Igi-ID linked a pull request Apr 30, 2024 that will close this issue

Init ArcGIS Support #10265

Merged

6 tasks

tdipisa linked a pull request May 2, 2024 that will close this issue

update dev #10274

Merged

tdipisa assigned allyoucanmap and unassigned Igi-ID May 14, 2024

allyoucanmap mentioned this issue May 16, 2024

#10040 ArcGIS Interoperability - ArcGIS MapServer Catalog and Layer support #10330

Merged

5 tasks

allyoucanmap linked a pull request May 17, 2024 that will close this issue

#10040 ArcGIS Interoperability - ArcGIS MapServer Catalog and Layer support #10330

Merged

5 tasks

tdipisa added the ArcGIS label May 17, 2024

allyoucanmap closed this as completed in #10330 May 21, 2024

allyoucanmap added a commit that referenced this issue May 21, 2024

#10040 ArcGIS Interoperability - ArcGIS MapServer Catalog and Layer s…

9baf447

…upport (#10330) --------- Co-authored-by: Igor Dimov <[email protected]>

ElenaGallo self-assigned this May 22, 2024

allyoucanmap added a commit to allyoucanmap/MapStore2 that referenced this issue May 22, 2024

Fix geosolutions-it#10040 print arcgis layer with different projections

e8c8e9b

allyoucanmap mentioned this issue May 22, 2024

Fix #10040 print arcgis layer with different projections #10354

Merged

5 tasks

allyoucanmap linked a pull request May 22, 2024 that will close this issue

Fix #10040 print arcgis layer with different projections #10354

Merged

5 tasks

offtherailz pushed a commit that referenced this issue Jun 11, 2024

Fix #10040 print arcgis layer with different projections (#10354)

cc4d494

ElenaGallo mentioned this issue Jun 11, 2024

Use Guide - ArcGIS Interoperability #10423

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ArcGIS Interoperability #10040

ArcGIS Interoperability #10040

tdipisa commented Mar 7, 2024 •

edited

Igi-ID commented Apr 30, 2024

Igi-ID commented May 2, 2024 •

edited

allyoucanmap commented May 6, 2024

Igi-ID commented May 7, 2024 •

edited

tdipisa commented May 23, 2024

ArcGIS Interoperability #10040

ArcGIS Interoperability #10040

Comments

tdipisa commented Mar 7, 2024 • edited

Description

ArcGIS WMS Server interoperability for GetFeatureInfo requests

ArcGIS MapServer Catalog and Layer support

Acceptance criteria

Other useful information

Igi-ID commented Apr 30, 2024

Igi-ID commented May 2, 2024 • edited

allyoucanmap commented May 6, 2024

Igi-ID commented May 7, 2024 • edited

tdipisa commented May 23, 2024

tdipisa commented Mar 7, 2024 •

edited

Igi-ID commented May 2, 2024 •

edited

Igi-ID commented May 7, 2024 •

edited