update known_plugins, and put into site

closes #505

also update the rake task. use RubyGems itself to fetch data from
rubygems.org, and actually filter gems by ones that have a proper
Bundler plugins.rb

data/known_plugins.yml

@@ -1,21 +1,81 @@
 ---
+- :name: bootboot
+  :summary: Dualbooting your ruby app made easy.
+  :uri: https://github.com/shopify/bootboot
+- :name: bundler-as_of
+  :summary: Resolve gem dependencies as-of a date in the past.
+  :uri: https://github.com/flavorjones/bundler-as_of
 - :name: bundler-changelogs
+  :summary: A bundler plugin that shows changelogs of your gem dependencies that specify
+    changelog urls [not yet filtered to git version updates].
   :uri: https://github.com/jdar/bundler-changelogs
-- :name: bundler-audit
-  :uri: https://github.com/rubysec/bundler-audit
+- :name: bundler-commentate
+  :summary: Bundler plugin to add gem summaries to a Gemfile
+  :uri: https://gitlab.com/fjc/bundler-commentate
+- :name: bundler-console
+  :summary: A bundler plugin that starts a console session with your gem dependencies.
+  :uri: https://github.com/kddnewton/bundler-console
+- :name: bundler-ctags_generator
+  :summary: Provides a hook after gem installation to generate ctags
+  :uri: https://github.com/okuramasafumi/bundler-ctags_generator
 - :name: bundler-dependency_graph
+  :summary: Generate a visual representation of your gem dependencies.
   :uri: https://github.com/kerrizor/bundler-dependency_graph
+- :name: bundler-download
+  :summary: bundler-download is a Bundler plugin for auto-downloading specified extra
+    files on `bundle install`
+  :uri: http://github.com/AndyObtiva/bundler-download
 - :name: bundler-ecology
+  :summary: Bundler plugin to avoid installing unwanted gems
   :uri: https://github.com/eco-rb/bundler-ecology
 - :name: bundler-explain
+  :summary: Explains bundle update conflicts
   :uri: https://github.com/jhawthorn/bundler-explain
+- :name: bundler-fast_git
+  :summary: Make bundler git source faster
+  :uri: https://github.com/mtsmfm/bundler-fast_git
+- :name: bundler-graph
+  :summary: Generates a visual dependency graph for your Gemfile
+  :uri: https://github.com/rubygems/bundler-graph
 - :name: bundler-inject
+  :summary: A bundler plugin that allows extension of a project with personal and
+    overridden gems
   :uri: https://github.com/ManageIQ/bundler-inject
+- :name: bundler-install_dash_docs
+  :summary: Bundler plugin to install gem documentation into the macOS documentation
+    browser Dash https://kapeli.com/dash
+  :uri: https://github.com/e28eta/bundler-install_dash_docs
 - :name: bundler-licensed
+  :summary: A bundler hook for https://github.com/github/licensed
   :uri: https://github.com/sergey-alekseev/bundler-licensed
+- :name: bundler-mac
+  :summary: exclude your bundle from Time Machine and Spotlight on macOS
+  :uri: https://github.com/indirect/bundler-mac
+- :name: bundler-multilock
+  :summary: Support Multiple Lockfiles
+  :uri: https://github.com/instructure/bundler-multilock
+- :name: bundler-override
+  :summary: This bundler plugin allows to change dependencies for a gem. It can be
+    helpful in situation when a developer needs to use some other dependency than
+    default for the gem.
+  :uri: https://github.com/tarnowsc/bundler-override
 - :name: bundler-private_install
+  :summary: A Bundler plugin that installs gems in an additional private Gemfile after
+    bundle install.
   :uri: https://github.com/fphilipe/bundler-private_install
+- :name: bundler-shellsplit-plugin
+  :summary: Plugin to fix shell splitting of bundle config arguments in Bundler 1.17.3
+    to Bundler 2.0.2, fixed in Bundler 2.1.0.
+  :uri: https://github.com/wovnio/bundler-shellsplit-plugin
 - :name: bundler-source-aws-s3
+  :summary: Add aws-s3 source to bundler via plugin.
   :uri: https://github.com/eki/bundler-source-aws-s3
 - :name: bundler-symlink
+  :summary: Post-install hook for bundler to symlink to all gems from a local directory
   :uri: https://github.com/petekinnecom/bundler-symlink
+- :name: bundler-why
+  :summary: Explains the presence of a dependency.
+  :uri: https://github.com/jaredbeck/bundler-why
+- :name: extended_bundler-errors
+  :summary: Extended Errors for Bundler
+  :uri: http://github.com/jules2689/extended_bundler-errors

lib/tasks/regenerate_known_plugins_yml.rake

@@ -1,91 +1,42 @@
-desc "Recreate data/known_plugins.yml from RubyGems.org data. " \
-  "Look for gems with bundler- prefix names, and get their information, " \
-  "skipping invalid gems. Run with RUBYOPT=-W1 enabled to see diagnostic " \
-  "output about data not found. Uses curl, grep."
+desc "Recreate data/known_plugins.yml from RubyGems.org data."
 task :regenerate_known_plugins_yml do
+  require "json"
+  require "rubygems/package"
+  require "rubygems/remote_fetcher"
   require "yaml"
 
-  names_content = `curl --silent https://index.rubygems.org/names | grep '^bundler-'`
-  all_plugin_names = names_content.lines.map(&:chomp)
-
-  skipped_gem_names = %w[
-    bundler-add functionality-exists-in-bundler-now
-    bundler-auto-update
-    bundler-bootstrap empty
-    bundler-budit not-authoritative
-    bundler-changelog empty
-    bundler-fastupdate
-    bundler-fu
-    bundler-gem-hg
-    bundler-geminabox
-    bundler-gem_version_tasks
-    bundler-github functionality-exists-in-bundler-now
-    bundler-bouncer
+  known_plugins = %w[
+    bootboot
+    extended_bundler-errors
+  ]
+  skipped_gems = %w[
     bundler-interactive source-does-not-exist yanked-all-but-last
-    bundler-maglev-
     bundler-next
-    bundler-norelease
-    bundler-pgs
-    bundler-prehistoric
-    bundler-sass
-    bundler-turbo
-    bundler-updater
-  ]
-  not_plugins = %w[
-    bundler-audit
-    bundler-audited_update
-    bundler-audit-ng
-    bundler-advise
-    bundler-bower
-    bundler-commentator
-    bundler-dependencies
-    bundler-diff
-    bundler-fixture
-    bundler-grep
-    bundler-gtags
-    bundler-leak
-    bundler-native-gems
-    bundler-organization_audit
-    bundler-patch
-    bundler-reorganizer
     bundler-security
-    bundler-squash
-    bundler-stats
-    bundler-update_stdout
-    bundler-talks
-    bundler-unload
-    bundler-verbose
   ]
-  # Suspected:
-  # bundler-fastupdate
-  plugin_names = all_plugin_names - skipped_gem_names - not_plugins
 
-  # https://guides.rubygems.org/rubygems-org-api/#gem-methods
-  # /api/v1/gems/[GEM NAME].(json|yaml)
-  # Example: https://rubygems.org/api/v1/gems/rails.json
-  plugins = plugin_names.map do |gem_name|
-    json_string = `curl --silent https://rubygems.org/api/v1/gems/#{gem_name}.json`
-    next "<#{gem_name.inspect} is unknown by the API>" if json_string == "This rubygem could not be found."
+  rubygems = Gem::Source.new("https://rubygems.org")
+  known_plugins = rubygems.load_specs(:latest).filter_map do |name_tuple|
+    next unless name_tuple.name.start_with?("bundler-") ||
+                known_plugins.include?(name_tuple.name)
+    next if skipped_gems.include?(name_tuple.name)
 
-    gem_info = JSON.parse(json_string)
-
-    uri = if gem_info["homepage_uri"].to_s == ""
-      gem_info["project_uri"]
-    else
-      gem_info["homepage_uri"]
-    end
+    spec = rubygems.fetch_spec(name_tuple)
+    path = rubygems.download(spec, Gem.dir)
+    gem = Gem::Package.new(path)
+    # make sure it's actually a Bundler plugin
+    next unless gem.contents.include?("plugins.rb")
+    next unless spec.homepage
+    next unless spec.summary
 
     {
-      name: gem_name,
-      uri: uri
+      name: spec.name,
+      summary: spec.summary,
+      uri: spec.homepage
     }
   end
-  # RUBYOPT=-W1 will display this diagnostic output
-  warn plugins.reject { |e| e.is_a?(Hash) }.inspect
-
-  valid_plugins = plugins.select { |e| e.is_a?(Hash) }
 
-  File.write(File.expand_path("../../data/known_plugins.yml", __dir__), YAML.dump(valid_plugins))
-  puts "Saved #{valid_plugins.size} plugins as data/known_plugins.yml"
+  File.write(File.expand_path("../../data/known_plugins.yml", __dir__), YAML.dump(known_plugins))
+  puts "Saved #{known_plugins.size} plugins as data/known_plugins.yml"
   puts "Done."
 end

source/guides/bundler_plugins.html.md

@@ -182,11 +182,13 @@ Although plugins can be installed from a git branch, it's recommended to install
 Here are a few plugins that you can use as examples and inspiration:
 
 - For a plugin that adds a command, take a look at
-[rubysec/bundler-audit](https://github.com/rubysec/bundler-audit)
+[kddnewton/bundler-console](https://github.com/kddnewton/bundler-console)
 - For a plugin that makes use of hooks, take a look at
 [jules2689/extended_bundler-errors](https://github.com/jules2689/extended_bundler-errors)
 - For an example of source plugin, take a look at Bundler's implementations for
 [the RubyGems source,](https://github.com/rubygems/rubygems/blob/master/bundler/lib/bundler/source/rubygems.rb)
 [the git source,](https://github.com/rubygems/rubygems/blob/master/bundler/lib/bundler/source/git.rb)
 and
 [the path source](https://github.com/rubygems/rubygems/blob/master/bundler/lib/bundler/source/path.rb)
+
+You can also look at the [full list of bundler plugins](./plugins.html).

source/guides/plugins.html.haml

@@ -0,0 +1,13 @@
+---
+title: Known Plugins
+---
+
+
+.container
+  .row
+    .col
+      - data.known_plugins.each do |c|
+        %li.contributor
+          = link_to c[:name], c[:uri], title: c[:name]
+          = " - "
+          = c[:summary]