Merge pull request #978 from rubygems/bundler-2-4-docs

Add Bundler 2.4 blog post and release notes

source/blog/2023-01-31-bundler-v2-4.html.markdown

@@ -0,0 +1,167 @@
+---
+title: "Bundler v2.4: new resolver, gems with Rust extensions, and more"
+date: 2023-01-31 14:43 UTC
+tags:
+author: David Rodríguez
+author_url: https://github.com/deivid-rodriguez
+category: release
+---
+
+2022 has been a busy year for the Bundler team, and we're glad to present
+several improvements that we hope will make our users happy :)
+
+## A better, PubGrub based, resolver
+
+Bundler now uses the most advanced algorithm to resolve versions, PubGrub.
+Kudos to Natalie Weizenbaum for [inventing
+it](https://nex3.medium.com/pubgrub-2fb6470504f) and to John Hawthorn for
+[porting it to Ruby](https://github.com/jhawthorn/pub_grub)!
+
+Our previous resolver, [Molinillo](https://github.com/CocoaPods/Molinillo), worked pretty well, but it really got in the
+middle when it didn't.
+
+This may sound familiar for some:
+
+~~~
+$ bundle
+Fetching gem metadata from https://rubygems.org/............
+Resolving dependencies....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C
+$ # Ok, that was enough waiting
+~~~
+
+Our new resolver, PubGrub, is usually much faster, because it learns from the
+resolution conflicts it finds during the resolution process to avoid redoing the
+same work over and over again. You can find more about this "conflict-driven
+clause learning" techniques in its [presentation blog
+post](https://nex3.medium.com/pubgrub-2fb6470504f) back from 2018.
+
+Molinillo sometimes took too long to resolve because it would try the same
+things, backtrack, and run into the same conflicts over and over again, having
+to traverse very inefficiently a huge search space. But that was a relatively
+rare case in the real world.
+
+What's probably more common is specifying version requirements in your Gemfile,
+that can't be all satisfied at the same time. This is when the version solving
+problem does not have a solution, and when it becomes crucial to explain to users
+_why_, so that they can fix their set of version requirements to become
+solvable.
+
+Molinillo run into trouble here, and in cases with many moving parts, like
+upgrading Rails for example, it could end up printing a lot of conflicts, not
+easy to understand and solve. This is an old example from a public ticket:
+
+~~~
+Bundler could not find compatible versions for gem "actionpack":
+  In Gemfile:
+    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
+      actionpack (>= 3.2, < 5)
+
+    rails (= 4.2.0) was resolved to 4.2.0, which depends on
+      actionpack (= 4.2.0)
+
+Bundler could not find compatible versions for gem "activesupport":
+  In Gemfile:
+    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
+      has_scope (~> 0.6.0.rc) was resolved to 0.6.0, which depends on
+        activesupport (>= 3.2, < 5)
+
+    rails (= 4.2.0) was resolved to 4.2.0, which depends on
+      activesupport (= 4.2.0)
+
+Bundler could not find compatible versions for gem "railties":
+  In Gemfile:
+    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
+      railties (>= 3.2, < 5)
+
+    rails (= 4.2.0) was resolved to 4.2.0, which depends on
+      railties (= 4.2.0)
+
+    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
+      responders was resolved to 1.1.2, which depends on
+        railties (>= 3.2, < 4.2)
+~~~
+
+Not easy to know what to do about it.
+
+With PubGrub, you should now get human-readable explanations of failures. The
+most complex cases may are still, well... complex. But explanations should
+always make sense and point to the root cause of resolution failures.
+
+Here's an example from our test suite:
+
+~~~
+Because every version of c depends on a < 1
+  and every version of b depends on a >= 2,
+  every version of c is incompatible with b >= 0.
+So, because Gemfile depends on b >= 0
+  and Gemfile depends on c >= 0,
+  version solving has failed.
+~~~
+
+We tried to make this migration as backwards-compatible as possible, but
+there's a chance of experiencing some different solutions to the ones found by
+Molinillo, since the version solving problem does not have unique solutions.
+Please report any issues you find with the new resolver.
+
+
+### Easily generate gems with Rust extensions using `bundle gem`
+
+It's now easier than ever to get started using Rust inside your gems. Check out
+[this blog post](rust-gem-skeleton.html) to learn how to generate a gem with all
+the boilerplate necessary with just a few commands.
+
+### Faster git sources
+
+In the Bundler world, it's common to point to git repositories when there's
+no version released to rubygems.org that includes the changes that you need.
+This works fine, but it can get slow and use a lot of disk space when dealing
+with very big repositories.
+
+We have improved the way we clone these repositories to be faster and use less
+disk space. For example, something like
+
+```ruby
+gem "rails", github: "rails/rails"
+```
+
+in your Gemfile could previously take ~30s and use up to 1Gb of disk space,
+because we would clone the full Rails repository, which has a large history.
+
+Now we just clone what's strictly necessary for Bundler to work, resulting in
+big disk space savings, and much faster bundling.
+
+### New CLI features
+
+We added a few small CLI features, such as a new `--pre` flag to `bundle update`
+and `bundle lock` to explicitly opt-in to prereleases of selected (of all) gems
+without having to explictly change your Gemfile with pre-release requirements
+such as `>= 7.1.0.beta`.
+
+### Some minor breaking changes
+
+We took new year's release to move on and get rid of some stuff that was causing
+maintenance burden for us:
+
+* Ruby 2.3, 2.4, and 2.5 are no longer supported.
+* RubyGems 2.5, 2.6, and 2.7 are no longer supported.
+
+In general, this support drop should not break anything because RubyGems should
+be able to choose the latest supported Bundler on the Ruby version that you're
+using. But there are still some old RubyGems out there that don't have this
+feature, and the `gem install bundler` command could break there. We have
+warned using Bundler on those old rubies for a year now, so we believe it's time
+to move on.
+
+We also completely removed a controversial (mis-)feature from the Bundler code
+base, where Bundler would automatically acquire sudo permissions when not having
+the proper access rights. A great majority of users considered this feature harmful and
+hardly useful, so we decided to get rid of it.
+
+### And bug fixes
+
+As always, we continue to smooth the experience of using Bundler, so that it
+gets the job done and does not get in the middle other than that. And we're also
+shipping a bunch of bug fixes to keep moving towards that goal.
+
+That's all from the Bundler team. Have a happy new year, and enjoy using Bundler
+2.4! 🎉

source/blog/2023-01-31-rust-gem-skeleton.html.markdown

@@ -0,0 +1,158 @@
+---
+title: "Generate gem skeleton with Rust extension"
+date: 2023-01-31 14:42 UTC
+tags:
+author: Josef Šimánek
+author_url: https://github.com/simi
+category: feature
+---
+
+Do you think [dynamically typed](https://en.wikipedia.org/wiki/Dynamic_programming_language) [interpreted](https://en.wikipedia.org/wiki/Interpreter_(computing)) [Ruby language](https://www.ruby-lang.org/) and [statically typed](https://en.wikipedia.org/wiki/Type_system#Static_type_checking) [compiled](https://en.wikipedia.org/wiki/Compiled_language) [Rust language](https://www.rust-lang.org/) could be friends? Yes, they can! And actually, they are!
+
+Officially it all started when [YJIT](https://github.com/ruby/ruby/blob/d5635dfe36588b04d3dd6065ab4e422f51629b11/doc/yjit/yjit.md) was [ported to Rust](https://bugs.ruby-lang.org/issues/18481) and [Ruby codebase](https://github.com/ruby/ruby) has officially [onboarded Rust code](https://github.com/ruby/ruby/tree/master/yjit/src). This friendship matured when RubyGems [3.3.11](https://rubygems.org/gems/rubygems-update/versions/3.3.11) (with a new [*Add cargo builder for rust extensions*](https://github.com/rubygems/rubygems/pull/5175) feature) [was released](https://blog.rubygems.org/2022/04/07/3.3.11-released.html) capable of compiling Rust-based extensions during gem installation process (similar to well-known C-based gem extensions like nokogiri, pg or puma).
+
+And now, with Bundler 2.4, `bundle gem` skeleton generator can provide all the glue you need to start using Rust inside your gems thanks to the new `--ext=rust` parameter!
+
+## What's new?
+
+Thanks to new parameter it is possible to generate simple Rust-based gem extension.
+
+*Make sure to use RubyGems 3.4.6 or higher for the best experience.*
+
+*Notice I already have `bundle gem` command configured. Your output can differ. When running `bundle gem` for the first time, it will interactively ask you few questions.*
+
+~~~
+$ bundle gem --ext=rust hello_rust
+Creating gem 'hello_rust'...
+MIT License enabled in config
+Initializing git repo in /home/retro/code/hello_rust
+      create  hello_rust/Gemfile
+      create  hello_rust/lib/hello_rust.rb
+      create  hello_rust/lib/hello_rust/version.rb
+      create  hello_rust/sig/hello_rust.rbs
+      create  hello_rust/hello_rust.gemspec
+      create  hello_rust/Rakefile
+      create  hello_rust/README.md
+      create  hello_rust/bin/console
+      create  hello_rust/bin/setup
+      create  hello_rust/.gitignore
+      create  hello_rust/test/test_helper.rb
+      create  hello_rust/test/test_hello_rust.rb
+      create  hello_rust/LICENSE.txt
+      create  hello_rust/Cargo.toml
+      create  hello_rust/ext/hello_rust/Cargo.toml
+      create  hello_rust/ext/hello_rust/extconf.rb
+      create  hello_rust/ext/hello_rust/src/lib.rs
+Gem 'hello_rust' was successfully created. For more information on making a RubyGem visit https://bundler.io/guides/creating_gem.html
+~~~
+
+For Rust-based extension last 4 entries are interesting.
+
+- `hello_rust/Cargo.toml`
+    - Top-level `Cargo.toml` is just pointing to "nested" `Cargo.toml` in `ext` folder.
+    - It is useful to be able to run all `cargo` commands in top-level directory (next to `bundle`, `gem`, ...).
+    - It is also useful for your IDE to be able to recognize there is Rust code in this folder, but not in standard path for Rust crate.
+- `hello_rust/ext/hello_rust/Cargo.toml`
+    - Actual `Cargo.toml` as known from Rust crates. It includes package metadata, configuration and dependencies. You can think of this file as a "gemspec for Rust packages".
+- `hello_rust/ext/hello_rust/extconf.rb`
+    - Config file responsible for configuration of compilation of your Rust code in Ruby world (for example during gem installation).
+    - Currently based on [rb_sys gem](https://github.com/oxidize-rb/rb-sys/tree/main/gem#the-rb_sys-gem). Check [project README](https://github.com/oxidize-rb/rb-sys/tree/main/gem#create_rust_makefile) for more info.
+- `hello_rust/ext/hello_rust/src/lib.rs`
+    - Yes, the holy grail of Rust-based extension - the Rust code!
+
+## Hello from Rust!
+
+Generated `hello_rust/ext/hello_rust/src/lib.rs` contains hello world example method defined at base class of extension. In my case it is `HelloRust#hello` with 1 string argument returning string as well. It is using [magnus](https://github.com/matsadler/magnus) Rust bindings to Ruby for super smooth developer experience.
+
+~~~rust
+# hello_rust/ext/hello_rust/src/lib.rs
+use magnus::{define_module, function, prelude::*, Error};
+
+fn hello(subject: String) -> String {
+    format!("Hello from Rust, {}!", subject)
+}
+
+#[magnus::init]
+fn init() -> Result<(), Error> {
+    let module = define_module("HelloRust")?;
+    module.define_singleton_method("hello", function!(hello, 1))?;
+    Ok(())
+}
+~~~
+
+That is equivalent to following Ruby code, including some boilerplate code, to enable Rust extension to communicate with Ruby.
+
+~~~ruby
+module HelloRust
+  def self.hello(subject)
+    "Hello from Rust, #{subject}!"
+  end
+end
+~~~
+
+## Let's compile and run some Rust!
+
+To be able to test this boilerplate code, you need to run `bundle install` first (to install all Ruby dependencies) followed by `bundle exec rake compile` compiling Rust code.
+
+*Notice generated gemspec is not valid by default and running `bundle install` can break. In that case it is needed to update gemspec first and replace all TODO values with some real ones.*
+
+~~~
+$ bundle install
+Fetching gem metadata from https://rubygems.org/.
+Resolving dependencies...
+Using rake 13.0.6
+Using bundler 2.4.0
+Using hello_rust 0.1.0 from source at `.`
+Using minitest 5.16.3
+Using rake-compiler 1.2.1
+Using rb_sys 0.9.52
+Bundle complete! 5 Gemfile dependencies, 6 gems now installed.
+Use `bundle info [gemname]` to see where a bundled gem is installed.
+~~~
+
+At this stage, everything is ready to compile Rust code and glue it with Ruby.
+
+*You need to have Rust already installed on your system. See [rustup](https://rustup.rs/) for a simple installation experience.*
+
+~~~
+$ bundle exec rake compile
+mkdir -p tmp/x86_64-linux/hello_rust/3.1.2                                                                                                                                             
+cd tmp/x86_64-linux/hello_rust/3.1.2                                                                                                                                                   
+/home/retro/.rubies/ruby-3.1.2/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/hello_rust/extconf.rb             
+cd -                                           
+cd tmp/x86_64-linux/hello_rust/3.1.2
+/usr/bin/gmake
+generating target/release/libhello_rust.so (release)
+cargo rustc --target-dir target --manifest-path ../../../../ext/hello_rust/Cargo.toml --lib --release -- -C linker=gcc -L native=/home/retro/.rubies/ruby-3.1.2/lib -C link-arg=-lm
+    Updating crates.io index
+... shortened
+   Compiling magnus-macros v0.2.0
+   Compiling rb-sys-build v0.9.52
+   Compiling rb-sys v0.9.52
+   Compiling hello_rust v0.1.0 (/home/retro/code/hello_rust/ext/hello_rust)
+    Finished release [optimized] target(s) in 1m 03s
+cd -
+mkdir -p tmp/x86_64-linux/stage/lib/hello_rust
+/usr/bin/gmake install target_prefix=
+generating target/release/libhello_rust.so (release)
+cargo rustc --target-dir target --manifest-path ../../../../ext/hello_rust/Cargo.toml --lib --release -- -C linker=gcc -L native=/home/retro/.rubies/ruby-3.1.2/lib -C link-arg=-lm
+    Finished release [optimized] target(s) in 0.09s
+installing hello_rust.so to /home/retro/code/hello_rust/lib/hello_rust
+/usr/bin/install -c -m 0755 hello_rust.so /home/retro/code//hello_rust/lib/hello_rust
+cp tmp/x86_64-linux/hello_rust/3.1.2/hello_rust.so tmp/x86_64-linux/stage/lib/hello_rust/hello_rust.so
+~~~
+
+And finally, it is possible to call `hello` method defined in Rust returning a string and printing it to the console.
+
+~~~
+$ bundle exec ruby -rhello_rust -e 'puts HelloRust.hello("Josef")'
+"Hello from Rust, Josef!"
+~~~
+
+*Feel free to try to break this extension. For example you can try to pass different types of argument (like number or symbol). [magnus](https://github.com/matsadler/magnus) is doing a great job [automatically converting](https://github.com/matsadler/magnus#defining-methods) all those mistakes with friendly error messages.*
+
+## Summary
+
+Starting Bundler 2.4, you can generate gem skeleton with all boilerplate code needed to start using Rust. But it is not only about your custom Rust code you can easily integrate into gems now. Thanks to integration with [cargo](https://doc.rust-lang.org/cargo/) (Rust package manager) you can use any of [Rust crates](https://crates.io/) available. Rust ecosystem is well known for highly optimized and memory safe libraries. Thanks to [magnus](https://github.com/matsadler/magnus) and `bundle gem` command, it is possible to glue those Rust libraries into Ruby world smoothly. Sky is the limit ;-)
+
+*To see real-life example how powerful could be Rust for data processing, I recommend to check [kirby project](https://github.com/rubytogether/kirby) parsing logs for [rubygems.org](http://rubygems.org/).*

source/v2.4/whats_new.html.md

@@ -1,4 +1,35 @@
 # What's New in v2.4
 
-As always, a detailed list of every change is provided in
+The [Bundler 2.4 announcement](/blog/2022/12/22/bundler-v2-4.html)
+includes context and a more detailed explanation of the changes in this version. This is a summary of the biggest changes. As always, a detailed list of every change is provided in
 [the changelog](https://github.com/rubygems/rubygems/blob/3.4/bundler/CHANGELOG.md).
+
+### A new PubGrub based resolver
+
+Bundler now uses [pub_grub](https://github.com/jhawthorn/pub_grub) under the
+hood to resolve versions. The most advance algorithm to approach the version
+solving problem! 💪
+
+### Generate gems with Rust extensions
+
+`bundle gem` now supports Rust! Pass the `--ext=rust` flag to generate a gem
+with a Rust extension.
+
+### Faster git sources in Gemfile
+
+Git sources in Gemfile now work faster and use less disk space.
+
+### Old Ruby and RubyGems no longer supported
+
+Support for Ruby 2.3, 2.4, and 2.5, and RubyGems 2.5, 2.6, and 2.7 has been dropped.
+
+### No more auto-sudo
+
+Bundler no longer tries to use `sudo` to upgrade privileges under any circumstances.
+
+### Other improvements
+
+Bundler 2.4 also includes other improvements like a new `--pre` flag
+to `bundle lock` and `bundle update` to explicitly opt-in to prereleases.
+
+<a href="https://github.com/rubygems/rubygems/blob/3.4/bundler/CHANGELOG.md" class="btn btn-primary">Full 2.4 changelog</a>