Fix brew-update util, and add CI #207

danielbayley · 2024-12-23T18:53:54Z

Fixes util/brew-update to work cross-platform
Add CI workflow

YAMLScript

Ideally, the util/brew-update script would be replaced with brewbump-formula-pr, but this doesn't currently seem to work properly with formulae containing multiple urls and sha256sums. Fixing that will be yet another PR, over on Homebrew…

CI

The workflow will run on every release, and uses brewlivecheck to only update if the release SemVer tag is newer than the latest formula. You can preview the CI in action on my fork…
The workflow just requires an organisation ACCESS_TOKEN…

See also #205 and #197 (comment).

ingydotnet

@danielbayley Thanks so much for this effort. Like so many things with YS I have wanted to (find the time to) encode more automations with GHA.
Looking at your workflow YAML it is obvious that you know what you are doing at multiple levels: GHA, YAML and shell commands. (I'm very particular about YAML and shell so good to see!)

I don't see anything to comment on in the workflow yaml.
I have a few comments for the brew-update ys script...

Also, let me know if you are up for helping with the overall release process.

util/brew-update

ingydotnet · 2024-12-24T14:28:25Z

util/brew-update


 defn main(version):
  check-version: version
  download-releases: version
  update-brew: version

+defn sha256sum(file):


I love to see you refactoring with YS code!

util/brew-update

ingydotnet · 2024-12-24T14:46:59Z

util/brew-update

+        res =: sh("curl --silent --location $url --output $file")
        when res.exit:N.?:
          die: "Download of '$url' failed:\n$(res.err)"


YS has std/curl so you could also replace this with

spit file: curl(url)

I wrote this script pretty quickly so didn't think of that before.

YS has std/curl so you could also replace this with

spit file: curl(url)

Cool. The thing I didn't know about was spit, which is not the most intuative method name… So I missed it in the docs, searching instead for write/file or similar :)

The spit function comes from straight from clojure.core

$ ys -pe spit #object[clojure.core$spit 0x2d4edc62 "clojure.core$spit@2d4edc62"]

it's supposed to be the opposite of slurp. I really don't like the name spit for that. read and write might be better. Any ideas?

it's supposed to be the opposite of slurp. I really don't like the name spit for that. read and write might be better. Any ideas?

I see, makes more sense in that case. Well since they're in Clojure core, they should keep that naming for users used to that convention, but also maybe add read and write aliases for everyone else…

ingydotnet · 2024-12-24T14:49:27Z

util/brew-update

@@ -62,7 +69,7 @@ defn releases(version)::
  MAC_INT:: "ys-${version}-macos-x64.tar.xz"

 defn check-version(version):
-  when version !~ /^0\.1\.\d+$/:
+  when version !~ /^\d(\.\d+){2}$/:


Let's not generalize this quite yet. I purposefully coded it that way, because I wanted to remember to circle back here when I got to a major release.

Maybe I need to add more comments in general :)

Ok, reverted…

Note
For future reference, the general pattern here is missing a +, it should actually be /^\d+(\.\d+){2}$/.

danielbayley · 2024-12-26T15:15:51Z

I'm very particular about YAML and shell

I'm also very particular, about everything!

@ingydotnet As you mentioned in #205…

I think the big thing that YS needs first is a language server.

If/once you have an LSP up and running, it should be straightforward enough to build a linter to enforce these kind of formatting concerns on top of it…

ingydotnet · 2025-01-17T05:40:29Z

@danielbayley Sorry about the delay.
I put out 0.1.88 yesterday.
Just now I started with your PR. I rebased in my devel branch, and retargeted the PR at that.
I redid the commits a little and force pushed back to you.
I used your brew-update to push out the 0.1.88 release by hand.
I noticed that you changed the remote url to use https rather that git@ ssh.
Is that for the CI integration?

I'll try to find time this weekend to get to the CI part.
Cheers

danielbayley · 2025-01-18T15:48:12Z

@danielbayley Sorry about the delay. I put out 0.1.88 yesterday. Just now I started with your PR. I rebased in my devel branch, and retargeted the PR at that. I redid the commits a little and force pushed back to you. I used your brew-update to push out the 0.1.88 release by hand.

I'll try to find time this weekend to get to the CI part. Cheers

No worries.

I noticed that you changed the remote url to use https rather that git@ ssh. Is that for the CI integration?

Yeah, it was broken otherwise. Maybe there is a way around it, if you want to insist on ssh… But that is what worked for me.

ingydotnet · 2025-01-19T21:49:53Z

@danielbayley

I have some questions about brew.

My current process is to run my release process (run make yamlscript-release ... by hand on 4 machines which updates https://github.com/yaml/yamlscript/releases)

Then I run util/brew-update 0.1.88 by hand which clones the https://github.com/yaml/homebrew-yamlscript repo and downloads the new tarballs, gets their sha digests and updates the repo. Then I commit and push.

I'd like to automate it all eventually.

I don't know brew very well, but I have used it over the years. When I set this up I learned about using brew install yaml/yamlscript/ys instead of brew install yamlscript which lets you install from a repo instead of going through the process of making a new formulae for yamlscript. I did this because it was easier at the time, but maybe we should do it right. Have you looked into registering a formulae for yamlscript yet?

I did the 0.1.88 update using your changes to from this PR but I just tested this on a mac and I get this:

$ brew search yamlscript                  
==> Formulae
yaml/yamlscript/ys                    yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]
yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]             yaml/yamlscript/[email protected]
$ brew install yaml/yamlscript/[email protected]
==> Fetching yaml/yamlscript/[email protected]
==> Downloading https://github.com/yaml/yamlscript/releases/download/0.1.86/ys-0.1.86-macos-aarch64.tar.xz
Already downloaded: /Users/ingy/Library/Caches/Homebrew/downloads/138e0c4b24e211b921bb5f727b9a7bce4f5bac8fb4a726d2a2c379e0d6e10075--ys-0.1.86-macos-aarch64.tar.xz
==> Installing [email protected] from yaml/yamlscript
Warning: A newer Command Line Tools release is available.
Update them from Software Update in System Settings.

If that doesn't show you any updates, run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/all/.
You should download the Command Line Tools for Xcode 15.4.

🍺  /opt/homebrew/Cellar/[email protected]/0.1.86: 6 files, 56.9MB, built in 1 second
==> Running `brew cleanup [email protected]`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
$ brew install yaml/yamlscript/[email protected]
==> Fetching yaml/yamlscript/[email protected]
==> Downloading https://github.com/yaml/yamlscript/releases/download/0.1.88/ys-0.1.88-macos-aarch64.tar.xz
Already downloaded: /Users/ingy/Library/Caches/Homebrew/downloads/1600118db096fad28699fe169256108e330596adc6e452421f06f2468cd11688--ys-0.1.88-macos-aarch64.tar.xz
Error: [email protected]: SHA256 mismatch
Expected: eddcb16690054007204d4cf207ceffc9ebd87f851b97d0e6714ff2e2d795286f
  Actual: 28b89c3d28bb81b776852640b81693a3812408059798ba52d26f5462449650db
    File: /Users/ingy/Library/Caches/Homebrew/downloads/1600118db096fad28699fe169256108e330596adc6e452421f06f2468cd11688--ys-0.1.88-macos-aarch64.tar.xz
To retry an incomplete download, remove the file above.

Can you figure out what happened there?

Also open to suggestions about how to proceed.

Also I have another way to install things that I want to ask you about.
I'll make another issue for that and mention this issue so they link.

ingydotnet · 2025-01-19T21:54:01Z

@danielbayley This is also interesting...

ys has a --upgrade option that uses https://yamlscript.org/install script to install the latest version in the same dir as ys. Looks like it has an issue with symlinks.

I'd like things to work such that once people have a ys of any version installed by any method, they can use ys to upgrade and also install/manage YS libraries etc.

We'll have to think about how this works with brew and other pkg sources...

$ brew install yaml/yamlscript/[email protected] 
==> Auto-updating Homebrew...
Adjust how often this is run with HOMEBREW_AUTO_UPDATE_SECS or disable with
HOMEBREW_NO_AUTO_UPDATE. Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> Fetching yaml/yamlscript/[email protected]
==> Downloading https://github.com/yaml/yamlscript/releases/download/0.1.86/ys-0.1.86-macos-aarch64.tar.xz
Already downloaded: /Users/ingy/Library/Caches/Homebrew/downloads/138e0c4b24e211b921bb5f727b9a7bce4f5bac8fb4a726d2a2c379e0d6e10075--ys-0.1.86-macos-aarch64.tar.xz
==> Installing [email protected] from yaml/yamlscript
Warning: A newer Command Line Tools release is available.
Update them from Software Update in System Settings.

If that doesn't show you any updates, run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/all/.
You should download the Command Line Tools for Xcode 15.4.

🍺  /opt/homebrew/Cellar/[email protected]/0.1.86: 6 files, 56.9MB, built in 1 second
==> Running `brew cleanup [email protected]`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
$ which ys
/opt/homebrew/bin/ys
$ ys --upgrade
Exception in thread "main" java.lang.RuntimeException: Path /opt/homebrew/bin/ys-sh-0.1.86 does not point to executable file
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.PosixProcessPropertiesSupport.exec(PosixProcessPropertiesSupport.java:133)
	at org.graalvm.nativeimage/org.graalvm.nativeimage.ProcessProperties.exec(ProcessProperties.java:172)
	at babashka.process$exec.invokeStatic(process.cljc:611)
	at yamlscript.cli$do_upgrade.invokeStatic(cli.clj:195)
	at yamlscript.cli$do_main.invokeStatic(cli.clj:505)
	at yamlscript.cli$_main.invokeStatic(cli.clj:568)
	at yamlscript.cli$_main.doInvoke(cli.clj:520)
	at clojure.lang.RestFn.applyTo(RestFn.java:140)
	at yamlscript.cli.main(Unknown Source)
	at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
$

This was referenced Dec 23, 2024

Refactor formulae, update 0.1.87, and add CI yaml/homebrew-yamlscript#3

Open

Chained merges <<: don’t work #197

Closed

danielbayley force-pushed the main branch from 8e78513 to 4bcbf1e Compare December 24, 2024 00:01

ingydotnet requested changes Dec 24, 2024

View reviewed changes

danielbayley requested a review from ingydotnet December 26, 2024 15:16

ingydotnet and others added 6 commits January 15, 2025 17:57

util: Change A to atom

f8354ab

release: Revert recent changes to release scripts

cc33acc

util: Fix brew-update cross-platform

ed5a5a7

util: Fix brew-url

c96623b

util: Refactor brew-update cross-platform

51a9e87

ci: Add Homebrew update

d0f44c3

ingydotnet force-pushed the main branch from e6e51c1 to d0f44c3 Compare January 17, 2025 05:35

ingydotnet changed the base branch from main to devel January 17, 2025 05:36

ingydotnet mentioned this pull request Jan 19, 2025

YS binary and pkg installation plan #213

Open

ingydotnet force-pushed the devel branch from ec01583 to 5278776 Compare January 21, 2025 16:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix brew-update util, and add CI #207

Fix brew-update util, and add CI #207

danielbayley commented Dec 23, 2024 •

edited

Loading

ingydotnet left a comment •

edited

Loading

ingydotnet Dec 24, 2024

ingydotnet Dec 24, 2024

danielbayley Dec 26, 2024 •

edited

Loading

ingydotnet Jan 17, 2025

danielbayley Jan 18, 2025

ingydotnet Dec 24, 2024

ingydotnet Dec 24, 2024

danielbayley Dec 26, 2024

danielbayley commented Dec 26, 2024

ingydotnet commented Jan 17, 2025

danielbayley commented Jan 18, 2025

ingydotnet commented Jan 19, 2025

ingydotnet commented Jan 19, 2025 •

edited

Loading

Fix brew-update util, and add CI #207

Are you sure you want to change the base?

Fix brew-update util, and add CI #207

Conversation

danielbayley commented Dec 23, 2024 • edited Loading

YAMLScript

CI

ingydotnet left a comment • edited Loading

Choose a reason for hiding this comment

ingydotnet Dec 24, 2024

Choose a reason for hiding this comment

ingydotnet Dec 24, 2024

Choose a reason for hiding this comment

danielbayley Dec 26, 2024 • edited Loading

Choose a reason for hiding this comment

ingydotnet Jan 17, 2025

Choose a reason for hiding this comment

danielbayley Jan 18, 2025

Choose a reason for hiding this comment

ingydotnet Dec 24, 2024

Choose a reason for hiding this comment

ingydotnet Dec 24, 2024

Choose a reason for hiding this comment

danielbayley Dec 26, 2024

Choose a reason for hiding this comment

danielbayley commented Dec 26, 2024

ingydotnet commented Jan 17, 2025

danielbayley commented Jan 18, 2025

ingydotnet commented Jan 19, 2025

ingydotnet commented Jan 19, 2025 • edited Loading

danielbayley commented Dec 23, 2024 •

edited

Loading

ingydotnet left a comment •

edited

Loading

danielbayley Dec 26, 2024 •

edited

Loading

ingydotnet commented Jan 19, 2025 •

edited

Loading