-
-
Notifications
You must be signed in to change notification settings - Fork 221
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Possible to publish WIP to maven-central? #85
Comments
@cwensel I would also be happy to help add github-actions worflows for CI on PR and CD for wip branches to make it easier to integrate |
this is a good request. I could publish to maven snapshot repo, but unsure of the rules and accessibility (re requiring creds). unfortunately the builds take many hours, and I haven't tried to convince github actions to perform the builds yet (not even sure I can get a durable build number for the wip name). the other solution is to simply release more often. there are only a few things I would have liked to get into 4.1/4.5, but unsure I have time to tackling them and probably should defer them. is this currently a blocker? |
Not a blocker, but nice to have (I have a local workaround for now). I can comment on the github actions: the modern timeout is quite generous (and could be expanded to forever it seems like): https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes and if sharded by module should finish in time.
I've implemented a github-actions on scalding their builds are also quite large and it seems to work fine. See twitter/scalding#1963 I can throw a draft for github actions if you would like |
The way I understand is that github-actions runs in the context of a user-name. So if I fork cascading and open a PR against the repo, the build will be provisioned/attributed under my github account. Build minutes for open source should be unlimited currently. |
for the versioning this is how it is typically handled:
|
fwiw, I have a full build server (on my network). everything is automated already, only I can see it for now (it was open in the past, but got bricked somehow, so I closed off the ports). that said, automating PRs would be cool. so the real question is where to publish the artifacts. so, we could publish them where suggested above, just need to confirm it's allowed and results in an improvement. also, there isn't any reason to not proxy the github artifacts elsewhere. for example, https://conjars.wensel.net is read-only clone of https://www.conjars.org. Maybe we could create an action that pushes the github artifacts to the s3 bucket we use behind https://conjars.wensel.net. sadly that will cost dollars serve out, so I'd rather find public service. do you have an nexus/artifactory server you can proxy through? |
personally I am not too familiar with the packaging ecosystem on java. I could look into the rules around maven-central but for now I don't see an issue with just publishing WIP there as well as normal releases anyways, I think it's good to start with just a github-actions for running tests on PR and then move from there. |
ok, let's solve the first problem, getting WIPs from behind a credential wall. I'll research the rules on Maven central, but snapshot builds (which semantically WIPs are) aren't allowed. But they do have a snapshot repo I need to look into. if that's a dead end, i'll look at syncing the github packages with another location. the challenge is that it costs dollars to ship bytes out of s3. even 404 response errors add up, so I don't see this as a long term solution. many corporate teams use artifactory/nexus as an internal proxy to mitigate bit supply chain security issues. this makes a case to get the artifacts published elsewhere (it's a pita to give those tools creds to access bits) as well as offers a solution (you don't need to distribute creds to get at bits). |
side note: I've seen a decent amount of usage for sonatype https://central.sonatype.org/publish/ looks like no specific snapshot rules and seems free for OSS:
|
it's a bit confusing because they call themselves "Central" and use maven in the domain name but:
|
so there isn't any confusion, Cascading final release are published to Maven Central (sontatype) WIP release are not, because they are simply If I simply make a final release, the artifacts will show up in Maven Central. This is already automated. |
refreshed my memory a bit on this subject. Distributing snapshot (or intermediate builds) isn't the intended purpose of Maven Central. There is a managed repository for That said, wip releases are intermediate releases for testing, once verified, we do an actual release to Maven Central. Going from wip to release for a maintenance version is quick, I usually just want the features verified in a build by the community before the wip is promoted to an actual release. For major/minor releases, this is how we test new features in flight before making the full release. we can see 100 wip builds over a period of time as features are developed. This is the case now. 4.1/4.5 isn't ready and some apis are still in flight. As it stands, I don't see a way forward exactly. Other than make a 4.1/4.5 release, which means I need to find time to finish any outstanding work. The good news, the bits are available as a wip, in github packages, but just not super convenient due to the creds requirement. My recommendation for that is to proxy/cache the wip releases in a local caching repository that manages the creds to github. As someone who as hosted a maven repository out of his own pocket for 10 years or so, setting up a caching proxy for maven artifacts is a good thing for reliability on the build side, and lowering costs on the serving side. |
thanks for all the info. I agree it's not really a big deal though I might take up some experimenting with the SNAPSHOT idea if I have spare time. The way I would approach it is per branch. So for any branch on
For PR test automation I have #86 open |
The Github Artifacts require authentication for read. This makes them hard to use seamlessly in other open source projects. Would it be possible to cross-publish to maven central or another maven-compatible repo with read enabled without auth?
The text was updated successfully, but these errors were encountered: