Datafeeder : Allow import of csv files #4137
Conversation
datafeeder/src/main/java/org/georchestra/datafeeder/service/DatasetsService.java
Show resolved
Hide resolved
pmauduit
force-pushed
the
DF/csv-in-datafeeder
branch
from
d27b863
to
28af16e
Compare
pmauduit
force-pushed
the
DF/csv-in-datafeeder
branch
2 times, most recently
from
10ac6da
to
2b31eb7
Compare
...feeder/src/main/java/org/georchestra/datafeeder/batch/service/DataUploadAnalysisService.java
Outdated
Show resolved
Hide resolved
CSV support - adding extra analysis in case of CSV file df - srs not mandatory in case of CSV format df - adding CSV support Tests: fixing ITs, runtime tested, still some tests to add in the testsuite. df - adding jacoco to have an idea of the missing coverages df - updating api.yml type: map is not valid, using an object instead df - some refactoring df - adding some tests for CSV support feat bump geotools to 29 (not 30 cause of package change) feat: update analysis to remove univocity and conditional display of ogc btn
mvn clean verify OK locally.
If the sec-user base64-encode json header is received with already prefixed roles, one want to avoid prefixing another time, ending up with a list of roles containing "ROLE_ROLE_..." named roles. tests: adding an UT.
f-necas
force-pushed
the
DF/csv-in-datafeeder
branch
from
f727c0f
to
e83d2aa
Compare
|
Force pushed squashed commits. See branch |
f-necas
force-pushed
the
DF/csv-in-datafeeder
branch
from
5557112
to
21e3e71
Compare
datafeeder/pom.xml
Outdated
| @@ -19,7 +19,7 @@ | |||
| <java.version>1.8</java.version> | |||
There was a problem hiding this comment.
do we need this? or shall it inherit the java 11 config from the root pom
There was a problem hiding this comment.
If I remove it, I need to explicitly add 1.8 to maven-compiler-plugin :
//pom.xml:540
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
<annotationProcessorPaths>
<path>
<groupId>org.projectlombok</groupId>
Don't know what's the best.
There was a problem hiding this comment.
If I remove it, I need to explicitly add 1.8 to maven-compiler-plugin
Is it something related to your IDE ? If so, I'll leave the pom as it is originally, and trust maven instead.
There was a problem hiding this comment.
or shall it inherit the java 11 config from the root pom
It won't, the datafeeder maven project is currently disconnected from the georchestra root pom (parent being commented out):
https://github.com/georchestra/georchestra/blob/master/datafeeder/pom.xml#L5-L9
relying on the commit messages, it was due to incompatible spring boot versions provided by the root pom.
I just checked and in the current state, the DF is still targeting java8 (meaning the class/bytecode being produced is java1.8 / bytecode v52).
There was a problem hiding this comment.
I'd suggest to set the java.version to 11 in the DF pom.
f-necas
force-pushed
the
DF/csv-in-datafeeder
branch
2 times, most recently
from
7d3fb65
to
fa0ae63
Compare
f-necas
force-pushed
the
DF/csv-in-datafeeder
branch
from
fa0ae63
to
b81410a
Compare
|
The PR should include a line in the release notes to explain that the dataset name is now generated from the dataset title rather than the original file name. This is an important change. |
CSV in datafeeder
Allows to import CSV geo and non geo data in datafeeder.
For the front part, see PR : georchestra/geonetwork-ui#9
Screenshot