Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

builderv2: support fetching sources from git (and making tarball out of it for packaging purposes) #9088

Closed
5 tasks
marmarek opened this issue Apr 4, 2024 · 0 comments · Fixed by QubesOS/qubes-builderv2#133
Closed
5 tasks

builderv2: support fetching sources from git (and making tarball out of it for packaging purposes) #9088

marmarek opened this issue Apr 4, 2024 · 0 comments · Fixed by QubesOS/qubes-builderv2#133
Labels
C: builder Qubes Builder P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. pr submitted A pull request has been submitted for this issue. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality.

Comments

@marmarek
Copy link
Member

marmarek commented Apr 4, 2024

How to file a helpful issue

The problem you're addressing (if any)

Not all projects publish tarballs, some publish only a git repository. When on github.com, one can get github-generated tarball (and in some places we do), but that isn't perfect. If the release tag is signed, we could use that for integrity verification, even if tarballs are not.

The solution you'd like

Support setting git repository as a source in .qubesbuilder file. This should support setting:

  • repository url (enforce https?)
  • commit hash or a tag name (in case of tag name, @VERSION@ should be recognized)
  • public key to verify tag signature (required if tag is used, not needed in case of explicit commit id)
  • (optional?) name of the output tarball - if not given, use the name normally used for component repo tarball

If fetch and verify succeeds, it should produce a tarball that can be consumed by rpm/dpkg for package building.

Note: this would be the first(?) feature in .qubesbuilder that doesn't have matching legacy builder equivalent. Since R4.2 should be compatible with both qubes-builder versions, this new feature needs to either be used only for R4.3+ packages, or it needs to be supported with legacy builder too (most likely via some canonical snippet for the get-sources target in Makefile).

The value to a user, and who that user might be

Easier packaging of software distributed as git repository only. And in some places, bypass manually generated tarballs that could be a place where backdoor is inserted.

Completion criteria checklist

  • Decide on option names for .qubesbuilder entries
  • Implement the feature
  • Prepare canonical Makefile snippet for legacy builder (if decided to go this route)
  • Add tests
  • Document in README
@marmarek marmarek added T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality. C: builder Qubes Builder P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. labels Apr 4, 2024
@marmarek marmarek mentioned this issue Jul 8, 2024
Merged
2 tasks
marmarek added a commit to marmarek/qubes-builderv2 that referenced this issue Jul 13, 2024
@marmarek
Add support for 'git-url' in 'files' section of .qubesbuilder
f2abbca 
This allows fetching upstream sources from a git repository directly.
Not all projects produce release tarballs, some use only
github-generated tarballs (which aren't always reproducible).
Additionally, as xz-utils story shows, manual release tarballs is
additional attack vector in the supply chain.

Right now support two modes:
1. Specific commit id, for example:
```
files:
- git-url: https://github.com/some/repository
  commit-id: (full sha of the commit)
  git-basename: something-@Version@
```

2. Signed tag, for example:
```
files:
- git-url: https://github.com/some/repository
  tag: v@VERSION@
  pubkeys:
  - pubkey.asc
```

Fixes QubesOS/qubes-issues#9088
marmarek added a commit to marmarek/qubes-builderv2 that referenced this issue Jul 13, 2024
@marmarek
Add support for 'git-url' in 'files' section of .qubesbuilder
2dede4c 
This allows fetching upstream sources from a git repository directly.
Not all projects produce release tarballs, some use only
github-generated tarballs (which aren't always reproducible).
Additionally, as xz-utils story shows, manual release tarballs is
additional attack vector in the supply chain.

Right now support two modes:
1. Specific commit id, for example:
```
files:
- git-url: https://github.com/some/repository
  commit-id: (full sha of the commit)
  git-basename: something-@Version@
```

2. Signed tag, for example:
```
files:
- git-url: https://github.com/some/repository
  tag: v@VERSION@
  pubkeys:
  - pubkey.asc
```

Fixes QubesOS/qubes-issues#9088
marmarek added a commit to marmarek/qubes-builderv2 that referenced this issue Jul 13, 2024
@marmarek
Add support for 'git-url' in 'files' section of .qubesbuilder
c4ab9f6 
This allows fetching upstream sources from a git repository directly.
Not all projects produce release tarballs, some use only
github-generated tarballs (which aren't always reproducible).
Additionally, as xz-utils story shows, manual release tarballs is
additional attack vector in the supply chain.

Right now support two modes:
1. Specific commit id, for example:
```
files:
- git-url: https://github.com/some/repository
  commit-id: (full sha of the commit)
  git-basename: something-@Version@
```

2. Signed tag, for example:
```
files:
- git-url: https://github.com/some/repository
  tag: v@VERSION@
  pubkeys:
  - pubkey.asc
```

Fixes QubesOS/qubes-issues#9088
@andrewdavidwong andrewdavidwong added the pr submitted A pull request has been submitted for this issue. label Jul 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C: builder Qubes Builder P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. pr submitted A pull request has been submitted for this issue. T: enhancement Type: enhancement. A new feature that does not yet exist or improvement of existing functionality.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix having a submodule as a primary source marmarek/qubes-builderv2
2 participants
@marmarek @andrewdavidwong