SNMP process table missing in 24.1.7

[geotekberlin]

Describe the bug

In 24.1.7 it is no longer possible to query running processes, because hrSWRunTable (OID .1.3.6.1.2.1.25.4.2) is empty. This table was always correctly populated in versions up to and including 24.1.6.

Monitoring running processes is important because it allows to deal with processes that tend to die often, such as suricata.

To Reproduce

Enable Net-SNMP plugin and set a SNMP Community string
Browse to SNMP OID .1.3.6.1.2.1.25.4.2 (hrSWRunTable) with snmpwalk or any external MIB Browser using SNMP V.2
The query returns only one process name: snmpd itself, all other services are missing.

Expected behavior

The returned table should be populated with the names of all running tasks.

Describe alternatives you considered

I added the "Add AbentX Support" and "Layer 3 Visibility" checkboxes and restarted the snmp daemon, but to no avail. Updated another Firewall from 24.1.6 to 24.1.7 and the hrSWRunTable dissapeared there as well.

Screenshots

not applicable

Relevant log files

not applicable, net-snmp service starts without errors

Additional context

not applicable

Environment

OPNsense 24.1.7-amd64
QEMU Virtual CPU version 2.5+ and
Intel(R) Xeon(R) D-2123IT CPU @ 2.20GHz (4 cores, 8 threads)

[OPNsense-bot]

Thank you for creating an issue.
Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.

For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

The easiest option to gain traction is to close this ticket and open a new one using one of our templates.

[AdSchellevis]

looks like an upstream issue, should be fixed in the next build if I'm reading it correctly freebsd/freebsd-ports@52fe068 , the snmpd process is just not allowed to see the other processes anymore.

Local test:

snmpwalk -Os -c public -v 1 localhost hrSWRunTable

(as user root):

killall snmpd
snmpd

and repeat the snmpwalk.

Easiest action for now is to revert the old package and wait for the next update.

opnsense-revert -r 24.1.6 net-snmp

[geotekberlin]

Thanks AdSchellevis, reverting to the previous net-smd recovers the lost snmp functionality.

[geotekberlin]

The 24.1.7_1 patch had solved the issue, but with the 24.1.7_4-amd64 release, the problem has returned.

[fichtner]

what's a 24.1.7_1 patch? oO anytime you apply an update local modifications are removed / replaced with newer code. Also for net-snmp plugin so you should lock the package from the firmware GUI to prevent that...

[geotekberlin]

Just want to report that the issue is still present in 24.1.10_8-amd64.

[fichtner]

I don't see FreeBSD ports addressing the issue. After having confirmed that 24.7 has the same behaviour I would recommend raising your concern here https://bugs.freebsd.org instead.

[geotekberlin]

I can confirm that 24.7 has the same issue. Even worse: reverting to 24.1.6 net-snmp fails with the message:

Fetching net-snmp.pkg: ..[fetch: https://pkg.opnsense.org/FreeBSD:14:amd64/24.7/ MINT/24.1.6/latest/Latest/net-snmp.pkg.sig: Not Found] failed

Testing with a MIB Browser I can see that without reverting net-snmp the hrSWRunTable is empty, on 24.1 (reverted) and older firewalls, the hrSWRunTable is filled correctly with all processes.

What I don't understand is that snmpd --version reports exactly the same version on firewalls with working snmp and those with failing snmp:

NET-SNMP version:  5.9.4.pre2
Web:               http://www.net-snmp.org/
Email:             [email protected]

So how can I report this bug to FreeBSD Bugzilla if I can't even tell them which version fails? Isn't it more likely a OPNsense compilation problem?

With no workaround available, this bug is now critical for us, because it blocks us completely from updating our firewalls to 24.7.

[fichtner]

24.7/MINT/24.1.6/latest <-- I think you borked the firmware flavour setting ;)

[fichtner]

Sorry I misread. You cannot revert crossing major version boundaries.

Triaging which net-snmp version broke this should be trivial on 24.1? The test on 24.7 made sure nothing was broken due to FreeBSD 13.2 being EoL.

That being said you can probably rebuild the last “good” version using the ports tree. I’ll even sponsor you a package for 27.4 if you tell me which 24.1.x worked, but I cannot solve whatever net-snmp project or FreeBSD ports changed about the software. It needs to be reported and discussed there.

Cheers,
Franco

[geotekberlin]

net-snmp worked until 24.1.7. Reverting to net-snmp from 24.1.6 solved the issue on all of our 50+ firewalls. I would be willing to report this issue to FreeBSD, but since I have no FreeBSD testbed, don't know how exactly net-snmp is compiled and can't even say which version is broken (remember: the failing and the working net-snmp versions show the same version number) this won't be effective. I have also browsed their bug tracker and found no mention of this failure.

Anyway, if you could provide us with a workaround that works with 24.7 that would be really great :-)

[fichtner]

Preface: please note that what I am currently doing is business grade support on community time. Consider donating to the project if you think this is worth the effort.

Between 24.1.6 and 24.1.7 the net-snmp process appears to have gained the ability to drop privileges which is also the default behaviour now. It can be disabled at service start so let's just try that on a stock 24.7... 623837e7

# opnsense-patch -c plugins 623837e7

Reapply settings in the GUI to force a full restart to see if it works then...

Cheers,
Franco

[geotekberlin]

Fantastic!!! This patch resolves the issue on a 24.7_9 firewall after a complete reboot, stopping and starting the snmp daemon alone doesn't seem to suffice.

Thanks a lot, I will indeed consider a donation.

Is there any hope that this will be fixed in upcoming versions?

[fichtner]

Ok, nice. I will merge this into 24.7.1.

Cheers,
Franco