WWSympa crashing on "Malformed UTF-8 character" triggered by malicious hits on do_arcsearch()

[adam12b1]

Version

Sympa 6.2.72

Installation method

FreeBSD port

Expected behavior

WWSympa should filter out malicious traffic so it doesn't cause problems

Actual behavior

WWSympa is crashing, 6 times within 1 hour today, as a result of some kind of malicious probe hitting the archive search page. The last line logged is:

Aug 29 13:52:42 npogroups wwsympa[41899]: err main::#1569 > main::do_arcsearch#9116 > Sympa::WWW::Marc::Search::match_this#488 > (eval)#1 DIED: Malformed UTF-8 character (fatal) at (eval 70918) line 1.

Steps to reproduce

I wish I knew. :( But since this is just some awful malicious probe for vulnerabilities, and the process dies without logging the input string, we have no way of knowing. The best we can do is provide the log lines from some of the other things they're trying, which are getting correctly filtered or blocked and logged, as shown below.

Additional information

Here is the logging for some of the hits before the hit that brings WWSympa down, whatever it might be.

Incident 1:

Aug 29 12:55:47 npogroups wwsympa[79845]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 162.158.114.176] [list relt]
Aug 29 12:55:48 npogroups wwsympa[79845]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.218.178] Syntax error for parameter case value "0'XOR(if(now()=sysdate(),sleep(15),0))XOR'Z" not conform to regexp:[\w\-\.]+
Aug 29 12:55:48 npogroups wwsympa[79845]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.218.178] [list relt]
Aug 29 12:55:48 npogroups wwsympa[79845]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.214.31] Syntax error for parameter case value "if(now()=sysdate(),sleep(15),0)" not conform to regexp:[\w\-\.]+
Aug 29 12:55:48 npogroups wwsympa[79845]: info main::do_arcsearch_form(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.214.31] [list relt]
Aug 29 12:55:49 npogroups wwsympa[79845]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 162.158.179.142] Syntax error for parameter age value "X51cQti3'" not conform to regexp:[\w\-\.]+
Aug 29 12:55:49 npogroups wwsympa[79845]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 162.158.179.142] [list relt]
Aug 29 12:55:50 npogroups wwsympa[79845]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.214.241] Syntax error for parameter csrftoken value "79wfHLig')) OR 26=(SELECT 26 FROM PG_SLEEP(15))--" not conform to regexp:[\w\-\.]+
Aug 29 12:55:51 npogroups wwsympa[79845]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.210.91] [list relt]
Aug 29 12:55:52 npogroups wwsympa[79845]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.210.152] Syntax error for parameter case value "LyHlZsAR') OR 857=(SELECT 857 FROM PG_SLEEP(15))--" not conform to regexp:[\w\-\.]+
Aug 29 12:55:53 npogroups wwsympa[79845]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.68.225.179] [list relt]
Aug 29 12:55:53 npogroups wwsympa[79845]: err main::#1569 > main::do_arcsearch#9116 > Sympa::WWW::Marc::Search::match_this#488 > (eval)#1 DIED: Malformed UTF-8 charact
er (fatal) at (eval 4697494) line 1.

Incident 2:

Aug 29 13:02:13 npogroups wwsympa[67237]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 162.158.178.73] [list relt]
Aug 29 13:02:14 npogroups wwsympa[67237]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.214.234] Syntax error for parameter how value "0"XOR(if(now()=sysdate(),sleep(15),0))XOR"Z" not conform to regexp:[\w\-\.]+
Aug 29 13:02:14 npogroups wwsympa[67237]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.214.234] [list relt]
Aug 29 13:02:14 npogroups wwsympa[67237]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.214.234] Syntax error for parameter how value "(select(0)from(select(sleep(15)))v)/*'+(select(0)from(select(sleep(15)))v)+'"+(select(0)from(select(sleep(15)))v)+"*/" not conform to regexp:[\w\-\.]+
Aug 29 13:02:15 npogroups wwsympa[67237]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.210.152] [list relt]
Aug 29 13:02:15 npogroups wwsympa[67237]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.214.25] Syntax error for parameter date value "17b9gRtT'" not conform to regexp:[\w\-\.]+
Aug 29 13:02:15 npogroups wwsympa[67237]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.210.168] [list relt]
Aug 29 13:02:15 npogroups wwsympa[67237]: err main::#1569 > main::do_arcsearch#9116 > Sympa::WWW::Marc::Search::match_this#488 > (eval)#1 DIED: Malformed UTF-8 character (fatal) at (eval 5873) line 1.

Incident 3:

Aug 29 13:07:49 npogroups wwsympa[70232]: info main::do_arcsearch_form(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.219.55] [list relt]
Aug 29 13:07:50 npogroups wwsympa[70232]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.68.225.178] Syntax error for parameter subj value "5ELj4mFp') OR 858=(SELECT 858 FROM PG_SLEEP(15))--" not conform to regexp:[\w\-\.]+
Aug 29 13:07:50 npogroups wwsympa[70232]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.68.225.178] [list relt]
Aug 29 13:07:50 npogroups wwsympa[70232]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 172.71.214.241] Syntax error for parameter match value "rb0Cfa4n') OR 386=(SELECT 386 FROM PG_SLEEP(15))--" not conform to regexp:[\w\-\.]+
Aug 29 13:07:51 npogroups wwsympa[70232]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.210.152] [list relt]
Aug 29 13:07:52 npogroups wwsympa[70232]: err main::#1266 > main::get_parameters#2133 [robot lists.nnedv.org] [client 162.158.179.142] Syntax error for parameter subj value "if(now()=sysdate(),sleep(15),0)" not conform to regexp:[\w\-\.]+
Aug 29 13:07:53 npogroups wwsympa[70232]: info main::do_arcsearch(relt) [robot lists.nnedv.org] [session 48207448006353] [client 172.71.210.16] [list relt]
Aug 29 13:07:53 npogroups wwsympa[70232]: err main::#1569 > main::do_arcsearch#9116 > Sympa::WWW::Marc::Search::match_this#488 > (eval)#1 DIED: Malformed UTF-8 character (fatal) at (eval 8019) line 1.

[ikedas]

Hi @adam12b1 ,
Could you please apply this patch and check if the problem will be solved?

[ikedas]

Additional Note:
This PR requires the modules Unicode-Normalize and Unicode-UTF8 to be mandatory.
FYI for FreeBSD: Currently the ports collection has not contained Unicode-UTF8 and it could be installed manually.

[adam12b1]

Additional Note: This PR requires the modules...

Ah yes, thank you, I ran into that when attempting to test the patch, and got stuck trying to find a FreeBSD package... so I guess we'll need to go with CPAN instead. Will test as soon as I can.

[bboyle262]

On our service I can also see a lot of syntax errors in the logs, similar to the above, likely malicious. However, it doesn't seem to have resulted in any crashes. Still recommend installing the above patch?

We are running RHEL9. Thanks

[adam12b1]

@bboyle262 we haven't seen any more crashes since that first day, but we assume this kind of injection attack will come back at some point in the future, and this protection will be useful then.

and @ikedas we have now applied this patch, after installing Unicode::UTF8 via CPAN (we already had Unicode::Normalize as part of the core perl5 package), and WWSympa.fcgi still runs fine. We can't actually tell you if it fixes the problem because we don't know exactly what incoming string was triggering it, but we trust that you have improved the stability of WWSympa, thank you!

[ikedas]

@adam12b1 ok, I'll try to merge the patch above in the next some days. But please let us know here if you see the effect of the patch on your end. Thanks!