spec/acceptance/puppet/cups_issue_4766_spec.rb fails on centos7

[tequeter]

Given the current master 35efb59 and current Vagrant box bento/centos-7.6 (virtualbox, 201907.24.0).

When I run the acceptance testsuite for centos7

PUPPET_INSTALL_TYPE=agent BEAKER_IS_PE=no BEAKER_PUPPET_COLLECTION=puppet6 BEAKER_debug=true BEAKER_destroy=no bundle exec rake beaker:centos-7.6-x64

In order to get some patches ready for review.

Then I get the error message:

Failures:
                                                                                                                                                                                                      
  1) Circumventing CUPS issue #4766 when ensuring a queue when is shared via IPP by some remote host when 'shared' is set to true fails to apply changes                                              
     Failure/Error: command = apply_manifest(manifest, expect_failures: true)                                                                                                                         
     Beaker::Host::CommandFailure:
       Host 'centos-7-x64' exited with 2 running:
        puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.qn2oBl                                                                                                                     
       Last 10 lines of output were:                                                                                                                                                                  
        Info: Loading facts
        Info: Loading facts 
        Notice: Compiled catalog for centos-7-x64.fr.dgs.group in environment production in 0.05 seconds                                                                                              
        Info: Applying configuration version '1581696287'
        Notice: /Stage[main]/Main/Cups_queue[Office]/ensure: created a printer
        Notice: Applied catalog in 0.10 seconds
       
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/host.rb:395:in `exec'
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/helpers/host_helpers.rb:83:in `block in on'                                                                                       
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/shared/host_manager.rb:130:in `run_block_on'                                                                                          
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/helpers/host_helpers.rb:63:in `on'                                                                                                
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-puppet-1.18.11/lib/beaker-puppet/helpers/puppet_helpers.rb:529:in `block in apply_manifest_on'                                                          
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/shared/host_manager.rb:130:in `run_block_on'                                                                                          
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-puppet-1.18.11/lib/beaker-puppet/helpers/puppet_helpers.rb:457:in `apply_manifest_on'                                                                   
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-puppet-1.18.11/lib/beaker-puppet/helpers/puppet_helpers.rb:536:in `apply_manifest'                                                                      
     # ./spec/acceptance/puppet/cups_issue_4766_spec.rb:52:in `block (5 levels) in <top (required)>'                                                                                                  

  2) Circumventing CUPS issue #4766 when ensuring a queue when is shared via IPP by some remote host when 'shared' is set to false fails to apply changes                                             
     Failure/Error: command = apply_manifest(manifest, expect_failures: true)
     Beaker::Host::CommandFailure:
       Host 'centos-7-x64' exited with 2 running:
        puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.YIK9KE
       Last 10 lines of output were:
        Info: Loading facts
        Info: Loading facts
        Notice: Compiled catalog for centos-7-x64.fr.dgs.group in environment production in 0.05 seconds                                                                                              
        Info: Applying configuration version '1581696290'
        Notice: /Stage[main]/Main/Cups_queue[Office]/ensure: created a printer
        Notice: Applied catalog in 7.24 seconds
       
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/host.rb:395:in `exec'
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/helpers/host_helpers.rb:83:in `block in on'                                                                                       
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/shared/host_manager.rb:130:in `run_block_on'                                                                                          
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/helpers/host_helpers.rb:63:in `on'                                                                                                
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-puppet-1.18.11/lib/beaker-puppet/helpers/puppet_helpers.rb:529:in `block in apply_manifest_on'                                                          
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/shared/host_manager.rb:130:in `run_block_on'                                                                                          
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-4.14.1/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-puppet-1.18.11/lib/beaker-puppet/helpers/puppet_helpers.rb:457:in `apply_manifest_on'                                                                   
     # ./vendor/bundle/ruby/2.5.0/gems/beaker-puppet-1.18.11/lib/beaker-puppet/helpers/puppet_helpers.rb:536:in `apply_manifest'                                                                      
     # ./spec/acceptance/puppet/cups_issue_4766_spec.rb:74:in `block (5 levels) in <top (required)>'                                                                                                  

Finished in 8 minutes 17 seconds (files took 1 minute 16.87 seconds to load)
147 examples, 2 failures

Failed examples:

rspec ./spec/acceptance/puppet/cups_issue_4766_spec.rb:51 # Circumventing CUPS issue #4766 when ensuring a queue when is shared via IPP by some remote host when 'shared' is set to true fails to apply changes
rspec ./spec/acceptance/puppet/cups_issue_4766_spec.rb:73 # Circumventing CUPS issue #4766 when ensuring a queue when is shared via IPP by some remote host when 'shared' is set to false fails to apply changes

Indeed, on that Vagrant box, it does not seem to be an error to share an IPP queue:

[root@centos-7-x64 ~]# lpadmin -p test -v ipp://127.0.0.1/printers/MyPrinter
[root@centos-7-x64 ~]# lpadmin -E -p test -o printer-is-shared=false
[root@centos-7-x64 ~]# lpadmin -E -p test -o printer-is-shared=true
[root@centos-7-x64 ~]# rpm -q cups
cups-1.6.3-40.el7.x86_64

Would it make sense to retain the first example in that file (most important AFAICT) and remove the two that fail, as they are testing a varying behavior of CUPS rather than the module?

[leoarnold]

This spec is meant to test graceful handling of a bug in early versions of CUPS 2:

puppet-cups/spec/acceptance/puppet/cups_issue_4766_spec.rb

Lines 5 to 6 in c30842e

	# See also: https://github.com/leoarnold/puppet-cups/issues/28
	RSpec.describe 'Circumventing CUPS issue #4766' do

Since you are not using such a version of CUPS, the spec does not need to pass.

[tequeter]

IMHO it's bad practice to accept failing tests, because real failures (aka. bugs) do not stand out anymore.

I think that the last two examples are not validating that puppet-cups behaves correctly: they only test that calling puppet-cups in a certain way triggers a certain error message from CUPS. That's not a behavior you are trying to enforce in your code or that you would document as a feature.

If you still think they are useful, what about skipping them on platforms other than Ubuntu 16.04?