500 error on Checking MD5

[shrabok-surge]

Hello,
I've been working on an install of snort with pulled pork on 4 different systems.
Currently Centos 6, Centos 7, Ubuntu 14, Ubuntu 16.

Everything works on each system except Centos 7.

Here is the system details:
CentOS Linux release 7.2.1511 (Core)
Perl v5.16.3
Pulledpork 0.7.3
Snort 2.9.7.6
Daq 2.0.6

When I run:
/usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -P
Note: has replaced the real oinkcode
the output is:

Checking latest MD5 for snortrules-snapshot-2976.tar.gz....
        Error 500 when fetching https://www.snort.org/reg-rules/snortrules-snapshot-2976.tar.gz.md5 at /usr/local/bin/pulledpork.pl line 534.
        main::md5file('<oinkcode>', 'snortrules-snapshot-2976.tar.gz', '/tmp/', 'https://www.snort.org/reg-rules/') called at /usr/loc
al/bin/pulledpork.pl line 2006

when run with verbose

Config File Variable Debug /etc/snort/pulledpork.conf
        snort_path = /usr/local/bin/snort
        black_list = /etc/snort/rules/iplists/default.blacklist
        IPRVersion = /etc/snort/rules/iplists
        rule_path = /etc/snort/rules/snort.rules
        ignore = deleted.rules,experimental.rules,local.rules
        snort_control = /usr/local/bin/snort_control
        rule_url = ARRAY(0x2c04040)
        sid_msg_version = 1
        sid_changelog = /var/log/sid_changes.log
        sid_msg = /etc/snort/sid-msg.map
        config_path = /etc/snort/snort.conf
        temp_path = /tmp
        distro = RHEL-6-0
        sorule_path = /usr/local/lib/snort_dynamicrules/
        version = 0.7.3
        local_rules = /etc/snort/rules/local.rules
MISC (CLI and Autovar) Variable Debug:
MISC (CLI and Autovar) Variable Debug:
        Process flag specified!
        arch Def is: x86-64
        Operating System is: linux
        CA Certificate File is: OS Default
        Config Path is: /etc/snort/pulledpork.conf
        Distro Def is: RHEL-6-0
        Disabled policy specified
        local.rules path is: /etc/snort/rules/local.rules
        Rules file is: /etc/snort/rules/snort.rules
        sid changes will be logged to: /var/log/sid_changes.log
        sid-msg.map Output Path is: /etc/snort/sid-msg.map
        Snort Version is: 2.9.7.6
        Snort Config File: /etc/snort/snort.conf
        Snort Path is: /usr/local/bin/snort
        SO Output Path is: /usr/local/lib/snort_dynamicrules/
        Will process SO rules
        Verbose Flag is Set
        File(s) to ignore = deleted.rules,experimental.rules,local.rules
        Base URL is: https://www.snort.org/reg-rules/|snortrules-snapshot.tar.gz|<oinkcode> http://talosintelligence.com/feeds/ip-filt
er.blf|IPBLACKLIST|open
Checking latest MD5 for snortrules-snapshot-2976.tar.gz....
        Fetching md5sum for: snortrules-snapshot-2976.tar.gz.md5
** GET https://www.snort.org/reg-rules/snortrules-snapshot-2976.tar.gz.md5/<oinkcode> ==> 500 addr is not a string (1s)
        Error 500 when fetching https://www.snort.org/reg-rules/snortrules-snapshot-2976.tar.gz.md5 at /usr/local/bin/pulledpork.pl line 534.
        main::md5file('<oinkcode>', 'snortrules-snapshot-2976.tar.gz', '/tmp/', 'https://www.snort.org/reg-rules/') called at /usr/loc
al/bin/pulledpork.pl line 2006

So the 500 addr is not a string seems perl related... but not clear on why?
Also why it works on every system but Centos 7 is confusing as well.

Any help would be appreciated

[shirkdog]

Are there any issues with your oinkcode or did you put it into your pulledpork.conf?

Can you login to www.snort.org and download the signatures by hand? If so, make sure your oinkcode is set correctly,

[DigiAngel]

That md5 file is MIA:

--2016-11-18 14:17:45--  https://www.snort.org/reg-rules/snortrules-snapshot-2976.tar.gz.md5
Resolving www.snort.org (www.snort.org)... 104.16.62.75, 104.16.66.75, 104.16.64.75, ...
Connecting to www.snort.org (www.snort.org)|104.16.62.75|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2016-11-18 14:17:46 ERROR 404: Not Found.

[shrabok-surge]

So I copied the url posted from the error: https://www.snort.org/reg-rules/snortrules-snapshot-2976.tar.gz.md5/<oinkcode> containing my oinkcode and it downloaded the md5.
And it works on every other system just fine.
I am currently testing some stuff against 4 Vagrant boxes and each one works except this one.
I thought it might be the version of Perl being different, but they are not consistent at all.
Centos 7: perl 5.16.3
Centos 6: perl 5.10.1
Ubuntu 16: perl 5.22.1
Ubuntu 14: perl 5.18.2

[JonZeolla]

Wasn't this fixed via 9d4e9e5?

[shrabok-surge]

Is still experience the issue and did some additional testing. If I run the command skipping md5 check (Again only occuring on Centos 7)
perl /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -P -d -vv

I get this response:

Rules tarball download of snortrules-snapshot-2983.tar.gz....
        Fetching rules file: snortrules-snapshot-2983.tar.gz
But not verifying MD5
** GET https://www.snort.org/reg-rules/snortrules-snapshot-2983.tar.gz/<oinkcode> ==> 500 addr is not a string (1s)
        A 500 error occurred, please verify that you have recently updated your root certificates!

Edit - I also tried with the -d -w options and still get the same response A 500 error occurred, please verify that you have recently updated your root certificates!

[marcindulak]

I've packaged pulledpork 0.7.2 for CentOS7 http://koji.fedoraproject.org/koji/buildinfo?buildID=816715, and have no problems when downloading rules.

# grep '^rule_url' /etc/pulledpork/pulledpork.conf
rule_url=https://www.snort.org/reg-rules/|snortrules-snapshot.tar.gz|<oinkcode> # replace your oinkcode

Downloading snortrules-snapshot-2983.tar.gz works for me with both pulledpork 0.7.2 and
ddd67eb

Do you have the latest:

# rpm -qa | grep ca-certificates
ca-certificates-2015.2.6-70.1.el7_2.noarch

[marcindulak]

As a test:

# yum downgrade -y http://vault.centos.org/7.0.1406/os/x86_64/Packages/ca-certificates-2013.1.95-71.el7.noarch.rpm
# rpm -qa | grep ca-certificates
# ca-certificates-2013.1.95-71.el7.noarch

and pulledpork still has no problems downloading snortrules-snapshot-2983.tar.gz.

I can trigger the 500 error if the following module is missing:

# yum -y install perl-IO-Socket-SSL

and 501 error with this missing (see #221):

# yum -y install perl-LWP-Protocol-https

[shrabok-surge]

Thanks for all the info. I tried everything you provided and there was no change.
these were the packages I installed on build

    - git
    - perl-libwww-perl
    - perl-LWP-Protocol-https
    - perl-Crypt-SSLeay
    - perl-Sys-Syslog
    - perl-Archive-Tar

All the ca-certs and packages were in place. So I tried installing perlbrew and set it to the latest stable perl v5.24.0 and installed the corresponding cpan modules.

After that everything worked. So it seems something is wrong with either Perl v5.16.3 or the modules in yum being used with that version. Hopefully the packages get their updates soon. But too bad its not an easier fix.

[marcindulak]

Can you try the following Vagrantfile, this downloads community-rules.tar.gz. If this works you can vagrant ssh and modify /etc/pulledpork/pulledpork.conf to pull snortrules-snapshot.tar.gz

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  config.vm.define 'centos7' do |machine|
    machine.vm.box = 'centos/7'
    machine.vm.box_url = 'centos/7'
    machine.vm.synced_folder '.', '/vagrant', disabled: true
    machine.vm.provider 'virtualbox' do |v|
       v.memory = 256
      v.cpus = 1
    end
  end
  config.vm.define 'centos7' do |machine|
    machine.vm.provision :shell, :inline => 'hostname centos7', run: 'always'
    machine.vm.provision :shell, :inline => 'yum clean all'
    machine.vm.provision :shell, :inline => 'yum -y install https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-8.noarch.rpm'
    machine.vm.provision :shell, :inline => 'yum -y update'
    machine.vm.provision :shell, :inline => 'yum -y install https://www.snort.org/downloads/snort/snort-2.9.8.3-1.centos7.x86_64.rpm'
    machine.vm.provision :shell, :inline => 'yum -y install pulledpork'
    machine.vm.provision :shell, :inline => 'sed -i "s/0.7.3/0.7.2/" /etc/pulledpork/pulledpork.conf'  #    pulledpork.conf error fixed in the latest EPEL build https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-8d899b0fc2
    machine.vm.provision :shell, :inline => 'mkdir /etc/snort/rules/iplists'
    machine.vm.provision :shell, :inline => 'pulledpork -PE -c /etc/pulledpork/pulledpork.conf'
  end
end

[sgoodliff]

Hi,

I have a similar issue, lwp-download fails for me connecting to snort.org so its underneath pulledpork. curl / wget all work fine. we are stuck behind a mcafee web proxy so for us its the interaction between perl lwp and our proxy but havent got to the bottom of this yet.

lwp-download https://www.snort.org
lwp-download: 500 handshakefailed

[shirkdog]

Will need to look at this with 0.7.3 released, as there have been issues with various proxy setups, and the addition of some configuration options must be ironed out to take care of all issues.

[jw-upmi]

I have this issue on Manjaro and have a solution. Running PulledPork v0.7.4 (pulledpork AUR version 3.0.0.4-1). The issue is not with your Perl code but an advanced release that does not yet have rules specifically built for it on the Snort website.

Running: snort -V:
,,_ -> Snort++ <-
o" )~ Version 3.1.22.0
'''' By Martin Roesch & The Snort Team
http://snort.org/contact#team
Copyright (C) 2014-2022 Cisco and/or its affiliates. All rights reserved.
Copyright (C) 1998-2013 Sourcefire, Inc., et al.
Using DAQ version 3.0.6
Using LuaJIT version 2.1.0-beta3
Using OpenSSL 1.1.1m 14 Dec 2021
Using libpcap version 1.10.1 (with TPACKET_V3)
Using PCRE version 8.45 2021-06-15
Using ZLIB version 1.2.11
Using FlatBuffers 2.0.5
Using Hyperscan version 5.4.0 2022-01-19
Using LZMA version 5.2.5

NOTE: The highest iteration of Snort rules is curently 3.1.21.0 but my Snort version is 3.1.22.0. My Snort application was built through pamac AUR from https://github.com/snort3/snort3/archive/refs/tags/3.1.22.0.tar.gz, meaning the version number is correct.

I received this error by running: pulledpork.pl -c /etc/pulledpork/pulledpork.conf -Pw

https://github.com/shirkdog/pulledpork
  _____ ____
 `----,\    )
  `--==\\  /    PulledPork v0.7.4 - Helping you protect your bitcoin wallet!
   `--==\\/
 .-~~~~-.Y|\\_  Copyright (C) 2009-2020 JJ Cummings, Michael Shirk

@_/ / 66_ and the PulledPork Team!
| \ \ _(")
\ /-| ||'--' Rules give me wings!
_\ _\

Checking latest MD5 for snortrules-snapshot-31220.tar.gz....
Error downloading https://www.snort.org/rules/snortrules-snapshot-31220.tar.gz.md5?oinkcode=_HIDDEN_: 422 Unprocessable Entity [ 422 ]

I corrected this issue by running:  **pulledpork.pl -c /etc/pulledpork/pulledpork.conf -Pw -S 3.1.21.0**