Help running scan with FossID #8591

luismartinpolo22 · 2024-04-29T11:04:32Z

luismartinpolo22
Apr 29, 2024

Hi everyone,

I'm trying to run the scan with our FOSSID instance and I was able to configure the scan in the config-yml file, it looks like the connection works but I get the following error message. Can somebody help me to understand what's going on?

Many thanks!

12:59:09.301 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
12:59:09.309 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is .
12:59:09.310 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is .
12:59:09.330 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
12:59:09.418 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by retrofit2.Platform (file:/home/luismartin/Software/ort/cli/build/install/ort/lib/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
12:59:09.703 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server is running version 2023.1.0.
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is .
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is .
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
12:59:09.704 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
Scanning projects with:
        FossId (version 2023.1.0)
Scanning packages with:
        FossId (version 2023.1.0)
Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `org.ossreviewtoolkit.clients.fossid.model.Project` from Array value (token `JsonToken.START_ARRAY`)
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 61] (through reference chain: org.ossreviewtoolkit.clients.fossid.EntityResponseBody["data"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1767)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1541)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1488)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeFromArray(BeanDeserializer.java:654)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:211)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:440)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1533)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:33)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:23)
	at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:243)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:153)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

sschuberth · 2024-04-29T11:12:46Z

sschuberth
Apr 29, 2024
Maintainer

This looks like a case of #8462. The work-around (or actually the fix) is to resolve the underlying access problem.

0 replies

nnobelis · 2024-04-29T11:19:07Z

nnobelis
Apr 29, 2024
Collaborator

@sschuberth I am not so sure as the error happens now in getProject....

I am a bit concerned that there are no output Parameterizing the name with pattern 'XXX'.

Could you please post your config-yml, without the sensitive data ?

0 replies

nnobelis · 2024-04-29T11:28:09Z

nnobelis
Apr 29, 2024
Collaborator

12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is .
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is .

I think the problem is that you have no naming pattern in the configuration.

Try to add the following options to your config.yml:

ort:
  scanner:
    config:
      FossId:
        options:
          namingProjectPattern: "$FossidProjectName"
          namingScanPattern: "scan_#projectName_#branch_#currentTimestamp_#deltaTag"
          namingVariableFossidProjectName: '<put your project name here>'

0 replies

luismartinpolo22 · 2024-04-29T11:44:25Z

luismartinpolo22
Apr 29, 2024
Author

Hi @nnobelis ,

many thanks for the quick answer. This is an extract of my config now, after the changes:

  FossId:
    options:
      serverUrl: 'http://localhost'

      namingProjectPattern: '$FossidProjectName'
      namingScanPattern: 'scan_#projectName_#branch_#currentTimestamp_#deltaTag'
      namingVariableFossidProjectName: 'BH34ORT'

      waitForResult: true

      keepFailedScans: false
      deltaScans: true
      deltaScanLimit: 10

      detectLicenseDeclarations: true
      detectCopyrightStatements: true

      timeout: 60

      urlMappingExample: "https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}"

    secrets:
      user: xxx
      apiKey: xxx

And the error:

13:38:29.814 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
13:38:29.823 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is $FossidProjectName.
13:38:29.824 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is scan_#projectName_#branch_#currentTimestamp_#deltaTag.
13:38:29.844 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
13:38:29.927 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by retrofit2.Platform (file:/home/luismartin/Software/ort/cli/build/install/ort/lib/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
13:38:30.286 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server is running version 2023.1.0.
13:38:30.287 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
13:38:30.287 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is $FossidProjectName.
13:38:30.287 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is scan_#projectName_#branch_#currentTimestamp_#deltaTag.
13:38:30.288 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
13:38:30.288 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
13:38:30.296 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server is running version 2023.1.0.
Scanning projects with:
        FossId (version 2023.1.0)
Scanning packages with:
        FossId (version 2023.1.0)
13:38:31.582 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossId - Found project name 'gaia-bhthree' in URL https://github.com/esa/gaia-bhthree.git.
Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `org.ossreviewtoolkit.clients.fossid.model.Project` from Array value (token `JsonToken.START_ARRAY`)
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 61] (through reference chain: org.ossreviewtoolkit.clients.fossid.EntityResponseBody["data"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1767)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1541)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1488)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeFromArray(BeanDeserializer.java:654)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:211)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:440)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1533)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:33)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:23)
	at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:243)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:153)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Maybe there is some info missing in the analyser result?

0 replies

nnobelis · 2024-04-29T12:28:25Z

nnobelis
Apr 29, 2024
Collaborator

No I think it's an error in the communication format with FossID.

Could you please parametrize the following curl request, run it and post us the response ?

curl --request POST \
  --url <url of your fossid instance>/api.php \
  --header 'Content-Type: application/json' \
  --data '{
	"action" : "get_information",
	"group" : "projects",
	
	"data" : {
		"username":"<username>",
		"key":"<api key>",
		"project_code": "gaia-bhthree"
	}
}

0 replies

luismartinpolo22 · 2024-04-29T14:55:47Z

luismartinpolo22
Apr 29, 2024
Author

This is all I can get:
{"operation":"projects_get_information","status":"0","data":null,"error":"An error occurred. Please check WebApp logs.","message":"An error occurred. Please check WebApp logs.","message_parameters":[]}

I was trying with the web user/password but was always getting an authentication error. When I changed to the fossid database user/password, then it returns this error. When I check the nginx logs, nothing really useful appears there.

FYI, I am using php 8.2 with fossid, not sure of this might be causing the problem.

1 reply

nnobelis Apr 30, 2024
Collaborator

The apikey is not the Web user's password.
You need to generate one in the UI:

luismartinpolo22 · 2024-04-30T09:09:10Z

luismartinpolo22
Apr 30, 2024
Author

Indeed, that was the issue. Everything works now, thank you for you help!

0 replies

nnobelis · 2024-04-30T09:23:50Z

nnobelis
Apr 30, 2024
Collaborator

Glad to be of help.
I created #8605 to improve the error reporting.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Help running scan with FossID #8591

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 8 comments 1 reply

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Help running scan with FossID #8591

luismartinpolo22 Apr 29, 2024

Replies: 8 comments · 1 reply

sschuberth Apr 29, 2024 Maintainer

nnobelis Apr 29, 2024 Collaborator

nnobelis Apr 29, 2024 Collaborator

luismartinpolo22 Apr 29, 2024 Author

nnobelis Apr 29, 2024 Collaborator

luismartinpolo22 Apr 29, 2024 Author

nnobelis Apr 30, 2024 Collaborator

luismartinpolo22 Apr 30, 2024 Author

nnobelis Apr 30, 2024 Collaborator

luismartinpolo22
Apr 29, 2024

Replies: 8 comments 1 reply

sschuberth
Apr 29, 2024
Maintainer

nnobelis
Apr 29, 2024
Collaborator

nnobelis
Apr 29, 2024
Collaborator

luismartinpolo22
Apr 29, 2024
Author

nnobelis
Apr 29, 2024
Collaborator

luismartinpolo22
Apr 29, 2024
Author

nnobelis Apr 30, 2024
Collaborator

luismartinpolo22
Apr 30, 2024
Author

nnobelis
Apr 30, 2024
Collaborator