make option --arch optional

[rickysarraf]

Having it optional makes it convenient to be used in other routines

Signed-off-by: Ritesh Raj Sarraf [email protected]

[rickysarraf]

My intent with this PR is to make the download-binaries --arch option to be optional as a first step. My ultimate goal (follow-up) is to be able to invoke download-binaries routine for our certain use case.

[sjoerdsimons]

If it's optional make it an Option

[sjoerdsimons]

Please document what you're trying to achieve; I can't really judge this PR by itself without it ; "for our certain use case" is less then helpful :)

[refi64]

this doesn't actually work, a bunch of commands here do actually need --arch so the tests fail:

122 tests run: 26 passed, 96 failed, 0 skipped

[rickysarraf]

Thank you. I tried cargo test but the tests got hung for me.

... snipped ...

test upload::tests::test_create_list_package ... ok
test handler::tests::test_handler_flow::dput_test_4::build_success_1::log_test_3::download_binaries_1::prune_only_if_job_unsuccessful_2 ... FAILED
test retry::tests::test_retry_on_nested_non_client_errors ... ok
test retry::tests::test_retry_on_non_client_errors ... ok
test handler::tests::test_handler_flow::dput_test_4::build_success_1::log_test_1::download_binaries_1::prune_only_if_job_unsuccessful_1 ... FAILED
test handler::tests::test_handler_flow::dput_test_4::build_success_1::log_test_2::download_binaries_1::prune_only_if_job_unsuccessful_1 ... FAILED

test monitor::tests::test_fails_after_repeated_duplicate_endtimes ... FAILED
test monitor::tests::test_handles_old_build_status has been running for over 60 seconds
test prune::tests::test_prune has been running for over 60 seconds
test retry::tests::test_no_retry_on_client_errors has been running for over 60 seconds
test retry::tests::test_no_retry_on_nested_client_errors has been running for over 60 seconds


^C

[rickysarraf]

Please document what you're trying to achieve; I can't really judge this PR by itself without it ; "for our certain use case" is less then helpful :)

So we have the use case where we have a Parent Pipeline which generates child pipelines to do the actual work. The work primarily being building deb packages and other housekeeping around it.

Currently, the artifacts generated in the child pipeline aren't easily propagatable to the parent. Please see issue: https://gitlab.com/gitlab-org/gitlab/-/issues/285100#note_1610649891

So, right now, we do download the binaries via the download-binaries routine. This results in those artifacts being captured at 2 locations, in the child pipeline where it is generated, and in the parent pipeline where it is pulled in.

So the above description is the problem statement.

To overcome it:

• We tell generate-monitor command to not download the built artifacts.
• This PR; where the intent is to make download-binaries work without the --arch option. Essentially so that we can overcome same restrictions on the osc getbinaries PROJECT PACKAGE REPO ARCH by replacing its call with download-binaries; albeit without the --arch option

[rickysarraf]

To overcome it:

• We tell generate-monitor command to not download the built artifacts.
• This PR; where the intent is to make download-binaries work without the --arch option. Essentially so that we can overcome same restrictions on the osc getbinaries PROJECT PACKAGE REPO ARCH by replacing its call with download-binaries; albeit without the --arch option

So to have this, I was thinking of 2 things.

1. Continue further with making --arch option for download-binaries be an optional option.
2. Build download-binaries as a binary file, so that we can invoke it independently.

@refi64 Would this be the right approach to proceed with ? I need your guidance here 🙏🏽