Memory issues when exporting large tracks as shapefile

[bpross-52n]

I tried to export a track with a length of ca. 340 km. This made the Tomcat crash due to a PermGen error.

[nuest]

What is the Tomcat configuration? The standard configures only very little RAM...

[matthesrieke]

This is probably a problematic issue. Even if we raise the memory (did so, 768 PermGen max now, was 256 before) this still can lead to memory issues. Long track with higher sampling rates may still shoot over the top.

Besides that, we actually do not have control on the shapefile generation. It is within geotools and probably will not support streaming output. Suggestion: we could limit the shapefile generation to tracks that have a defined maximum of measurements. Or provide one track in multiple shapefile to allow Java to do GC on the geotools encoders.

[nuest]

Any other suggestion to solve this?

Imho a limitation on the size of the tracks is fine for now, we should include a text in the error message to get in touch with us if people encounter this problem! Then we can wait and see if this is actually a real issue for the currently ongoing work/research.
(Only if that is the case then we explore the idea of splitting up the shapefile generation. The question then is if geotools runs into the same issues when we try to compine the shapefiles.)

Offering a downloadable tool to do the conversion (a .jar file called from the command line) will probably just transfer the same permgen issue over to the user's computer, correct?

[matthesrieke]

+1 for error reporting in the first place.

a downloadable tool would require some additional work as the current implementation is based on internal entities and not JSON decoding.