Experiences ORT x SW360

[ohneb]

Hello everybody,

I didn't know which is the best category for this kind of discussion so I just chose this one. Hope this is okay.

I wondered if there are some people who are using ORT in combination with Eclipse SW360 and might want to share their experiences and workflows?
I am currently looking into tooling to support our OSS Clearing activities and quite like the concept and well-desgined-ness of ORT. Also I like the idea of SW360 as a repository of all my OSS components and relations to my own projects. The thing is that I ran into a couple of problems when trying to upload ORT results to SW360. I gotta say that I am currently running SW360 locally via the projects docker-compose so that may have to do with it but not sure.
Anyways, if someone wants to share their insights or discuss I would be really happy!

Thanks a lot to the ORT-Team for building such a great tool, btw!

Greetings,
Benjamin

[sschuberth]

I didn't know which is the best category for this kind of discussion so I just chose this one. Hope this is okay.

Sure it's okay! We're also open to introducing more discussion categories if required. Maybe something like "Interoperability" or so?

and quite like the concept and well-desgined-ness of ORT.

Thank you ❤️

The thing is that I ran into a couple of problems when trying to upload ORT results to SW360.

What problems exactly? Please report each as an issue to document the problems, and eventually get them fixed.

TBH, I believe non of the core ORT devs used the SW360 integrations recently, so I'm not sure which state they're in.

Anyways, if someone wants to share their insights or discuss I would be really happy!

Maybe @heliocastro can share something, also WRT the SW360 client library that we use, which IIUC is going to be deprecated, or?

[ohneb]

Hey,
thank you for your replies!

Interesting to know that the client library will be deprecated and the main interface will the the REST API.

When trying to upload my results to SW360 vie the CLI helper I've so far encountered two problems:

1. I get an Invalid prefix or suffix exception which results from ORT trying to create a tempdir where it stores the downloaded sources when called with the -a option. I was able to fix this locally so it seemed to work. Since I am not familiar with Kotlin and the codebase in general I did not yet consider creating a PR for this. I will look into it though.

Details

19:56:27.016 [main] INFO  org.ossreviewtoolkit.plugins.commands.api.utils.ExtensionsKt - Read ORT result from 'advisor-result.yml' (1.27 MiB) in 1.077085820s.
Exception in thread "main" java.lang.IllegalArgumentException: Invalid prefix or suffix
        at java.base/java.nio.file.TempFileHelper.generatePath(TempFileHelper.java:60)
        at java.base/java.nio.file.TempFileHelper.create(TempFileHelper.java:125)
        at java.base/java.nio.file.TempFileHelper.createTempDirectory(TempFileHelper.java:171)
        at java.base/java.nio.file.Files.createTempDirectory(Files.java:1017)
        at org.ossreviewtoolkit.utils.ort.ExtensionsKt.createOrtTempDir(Extensions.kt:29)
        at org.ossreviewtoolkit.utils.ort.ExtensionsKt.createOrtTempDir(Extensions.kt:36)
        at org.ossreviewtoolkit.plugins.commands.uploadresulttosw360.UploadResultToSw360Command.run(UploadResultToSw360Command.kt:97)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:279)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:292)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:41)
        at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:457)
        at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:454)
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:474)
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:481)
        at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:85)

2. I am not sure if the second problem is a problem of ort or the SW360 client. The component '@types/validator' has already been created previously but it doesn't have a release. So I would expect it to create the release for the component and upload the source. But probably it doesn't handle the case that there is no such release for this component correctly?

Details

20:07:30.070 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Downloaded source code for 'NPM:@types:validator:13.11.9' from RemoteArtifact(url=https://registry.npmjs.org/@types/validator/-/validator-13.11.9.tgz, hash=Hash(value=adfe96520b437a0eaa798a475877bf2f75ee402d, algorithm=SHA-1)) in 88.664196ms.
Exception in thread "main" org.eclipse.sw360.clients.utils.SW360ClientException: Asynchronous call failed.
        at org.eclipse.sw360.clients.utils.FutureUtils.block(FutureUtils.java:63)
        at org.eclipse.sw360.clients.adapter.SyncClientAdapterHandler.invoke(SyncClientAdapterHandler.java:100)
        at jdk.proxy2/jdk.proxy2.$Proxy33.uploadAttachments(Unknown Source)
        at org.ossreviewtoolkit.plugins.commands.uploadresulttosw360.UploadResultToSw360Command.run(UploadResultToSw360Command.kt:118)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:279)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:292)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:41)
        at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:457)
        at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:454)
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:474)
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:481)
        at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:85)
Caused by: java.io.IOException: java.lang.NullPointerException: Cannot invoke "org.eclipse.sw360.clients.rest.resource.releases.SW360Release.getEmbedded()" because "release" is null
        at org.eclipse.sw360.http.utils.HttpUtils.wrapInIOException(HttpUtils.java:396)
        at org.eclipse.sw360.http.utils.HttpUtils.waitFor(HttpUtils.java:101)
        at org.eclipse.sw360.clients.utils.FutureUtils.block(FutureUtils.java:58)
        ... 11 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.eclipse.sw360.clients.rest.resource.releases.SW360Release.getEmbedded()" because "release" is null
        at org.eclipse.sw360.clients.adapter.SW360ReleaseClientAdapterAsyncImpl.lambda$uploadAttachments$4(SW360ReleaseClientAdapterAsyncImpl.java:119)
        at org.eclipse.sw360.clients.adapter.SW360AttachmentUtils.lambda$uploadAttachments$1(SW360AttachmentUtils.java:85)
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
        at org.eclipse.sw360.clients.adapter.SW360AttachmentUtils.uploadAttachments(SW360AttachmentUtils.java:84)
        at org.eclipse.sw360.clients.adapter.SW360ReleaseClientAdapterAsyncImpl.uploadAttachments(SW360ReleaseClientAdapterAsyncImpl.java:118)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.eclipse.sw360.clients.adapter.SyncClientAdapterHandler.invoke(SyncClientAdapterHandler.java:97)
        ... 10 more

I am not sure if this is any help. I can also open actual issues if that makes more sense to you? I was also starting to trying to debug the second issue myself but didn't come far yet.

Thank you very much for your help!

[sschuberth]

I was able to fix this locally so it seemed to work.

What was your fix? Could you try whether #8279 fixes your issue as well?

I am not sure if the second problem is a problem of ort or the SW360 client.

Me neither, I'll leave it to @heliocastro to answer 😉

[ohneb]

What was your fix? Could you try whether #8279 fixes your issue as well?

My fix was to replace all / with - in createOrtTempDir in Extensions.kt in the ort utils package. As I said I did not check this for other unwanted sideeffects.
#8279 does seem to work fine though, thank you! Now my only remaining problem is with the release null as per my second point in my previous response.

I will look into the second issue likely in the evening today or tomorrow again as I am a bit swamped with other stuff.

[heliocastro]

@ohneb, for the second issue, Is actually both sides problem, unfortunately.
Since component is created on SW360, it can't be overwritten, only delete, this is not an usual operations, or update, so you got that error due to attempt to overwrite the component
On the side of Ort, it doesn't know about this, is assuming that is creating a new component/version that do not exists, which is untrue, and even if knew ahead, it doesn't know how to do an update operation.

So, to understand a little the semantics that is assigned to this specific issue, the entire idea behind it is prevent that someone try to overwrite a component with another data that invalidates the original, poisoning the results.
It can be a case that the information was really wrong and a mistake, but most of the times we will not change the single component/version

[JSurf]

This might be related: eclipse-sw360/sw360#2330

We are also evaluating the combination of ORT and SW360. SW360 shows a lot of potential and has a lot of powerful features, but currently we are not quite happy with software quality, experiencing lots of bugs on the way.