Bundler: Questions related to implementing dependency confusion check on downstream Bundler.

[junaruga]

This is just a question on the source code level in Bundler. Is there a good way to get only indirect dependency gem names from only non global (scoped) rubygems sources in bundler/lib/bundler/definition.rb on the latest master on this repository?

For example in the reproducer script https://github.com/junaruga/report-bundler-dependency-confusion I shared on #5029 (comment) , I only want to get the a-0.0.1 gem.

Maybe I need to use sources.non_global_rubygems_sources (non global sources) and dependencies objects? Is there a good method for that?

Thanks.

[junaruga]

@deivid-rodriguez do you have a suggestion for this?

[deivid-rodriguez]

It's hard to help you without knowing what are you trying to accomplish. But I guess you want something like Bundler.definition.specs.select {|s| Bundler.definition.send(:sources).non_global_rubygems_sources.include?(s.source) }. Completely untested so I'm sure it's wrong, but maybe gives you some ideas.

[junaruga]

@deivid-rodriguez Thanks! That's a good clue! I couldn't find which object has the info. Related to the PR: #5029 , I am trying to implement a logic to raise an error only if there is a scoped repository in a Gemfile and there is indirect dependency gem in the scoped repository into the Bundler 1 bundled in Fedora/RHEL Ruby RPM package. This can be the downstream Ruby package specific feature.

[junaruga]

Here is the updated logic. I see above logic includes the gems by gem list (whose class is StubSpecification) in a timing. So, I excluded. it seems the indirect dependencies I want is the gems of EndpointSpecification class.

    def indirect_dependency_names_in_non_global_rubygems_soruces
      # All the dependency gem names (direct + indirect).
      dep_names = [] 
      # Direct dependency gem names
      direct_dep_names = dependencies.map {|d| d.name }

      sources.non_global_rubygems_sources.each do |s|
        # If the non dependency API source is used, the `dependency_names`
        # returns gems not only used in the `Gemfile`, but also existing
        # in the repository here. This method shouldn't be used with the non
        # dependency API sources.
        s.specs.dependency_names.each do |dep_name|
          # Exclude direct dependency gems.
          next if direct_dep_names.include?(dep_name)

          s.specs.local_search(dep_name).each do |spec|
            # Debug gems with unexpected `spec.class`.
            Bundler.ui.debug "Found dependency gem #{dep_name} (#{spec.class}) in scoped sources."
            # StubSpecification extending RemoteSpecification: the gems by
            #   `gem list`. Exclude the gems.
            # EndpointSpecification: gems returned by dependency API such as
            #   geminabox
            # RemoteSpecification: gems returned by non dependency API such as
            #   gem server. This method cannot be executed with the non
            #   dependency API sources.
            dep_names << dep_name if spec.class == EndpointSpecification
          end
        end
      end  

      dep_names.sort.uniq
    end

Seeing the method index.rb#local_search, I am not sure if I also need to exclude the gems with Gem::Specification, RemoteSpecification, or LazySpecification.

rubygems/bundler/lib/bundler/index.rb

Lines 92 to 101 in 6e55de6

	def local_search(query, base = nil)
	case query
	when Gem::Specification, RemoteSpecification, LazySpecification, EndpointSpecification then search_by_spec(query)
	when String then specs_by_name(query)
	when Gem::Dependency then search_by_dependency(query, base)
	when DepProxy then search_by_dependency(query.dep, base)
	else
	raise "You can't search for a #{query.inspect}."
	end
	end

[junaruga]

It's hard to help you without knowing what are you trying to accomplish.

Here is my work junaruga@cdfe149 on the wip/dependency-confusion-on-bundler-1.17.3 branch on my forked repository to raise an error or warn on the dependency confusion cases. Though I think you might not want to see the code. This is what I want to accomplish.

Now I have another question related to this topic. I want to know how to check if a gem exists on a global source repository in definition.rb. It seems that the sources.default_source.specs object of the class Bundler::Index could be used. Because when the indirect dependency gems are found on a scoped (non global) source, if the gems don't exist on the global source, I want to exclude the gems from raising the error. Do you have a suggestion for the way? Thanks.

Edited: the final version of the commit is updated here on this comment.

[junaruga]

Now I have another question related to this topic. I want to know how to check if a gem exists on a global source repository in definition.rb.

I could find the ways to check. Later I would update this page with the way.

[deivid-rodriguez]

@junaruga Of course you're free to proceed as you please, but I don't really support that direction. In my opinion, efforts should be focused on upgrading bundler, not on patching an unmaintained version. So since I don't support that direction I won't be dedicating any effort to this. Sorry 🙏.

[junaruga]

It's okay. I understand your preference and interest. Thanks. However we have to choose the way to apply the patch to fix the security issue in the downstream old Bundlers used with Ruby 2.6, 2.5, and Ruby 2.0.0 (!). We (= people in the team) have to support Bundler on Ruby 2.0.0. If we upgrade Bundler to the latest version, it's a high chance to see additional issues on Ruby 2.0.0. I think upgrading the old Bundler 1 to the latest version of Bundler 2 is risky and costly for users who already use the downstream old Bundler in terms of compatibility.

[junaruga]

I am sharing my code by following the policy that I should share the code even when it is actually not used on the upstream. Someone may pick up a part of my code.

[deivid-rodriguez]

I understand the situation. It must be hard to keep ruby 2.0.0 "secure" these days, given it's been more than 5 years without official security fixes. I hope you drop support for it soon and encourage your users to upgrade! In any case, keep up the good work, thanks for packaging ruby and opening it to a wider audience ❤️.