[SPARK-41794][SQL] Add try_remainder function and re-enable column tests

[grundprinzip]

What changes were proposed in this pull request?

As part of re-enabling the ANSI mode tests for Spark Connect, we discovered that we don't have an equivalent for try_* for the remainder of operations. This patch adds the try_remainder function in Scala, Python, and Spark Connect and adds the required testing.

Why are the changes needed?

ANSI and Spark 4

Does this PR introduce any user-facing change?

Yes, it adds the try_remainder function that behaves according to ANSI for division by zero.

How was this patch tested?

Added new UT and E2E tests.

Was this patch authored or co-authored using generative AI tooling?

No

[HyukjinKwon]

cc @gengliangwang

[dongjoon-hyun]

Could you fix Python linter failure, @grundprinzip ?

./python/pyspark/sql/tests/connect/test_connect_column.py:1029:5: F401 'os' imported but unused
    import os
    ^
1     F401 'os' imported but unused

[grundprinzip]

yes, done.

[dongjoon-hyun]

Thank you.

BTW, the recent test failure is relevant? For me, it looks irrelevant. Could you take a look at?

[info] *** 1 TEST FAILED ***
[error] Failed: Total 10578, Failed 1, Errors 0, Passed 10577, Ignored 29
[error] Failed tests:
[error] 	org.apache.spark.sql.execution.python.PythonStreamingDataSourceSuite
[error] (sql / Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 3020 s (50:20), completed May 8, 2024, 1:48:01 PM

[gengliangwang]

@grundprinzip thanks for the work!
Let's also mention the new function in https://spark.apache.org/docs/latest/sql-ref-ansi-compliance.html#useful-functions-for-ansi-mode

[grundprinzip]

@gengliangwang @dongjoon-hyun I addressed all comments and added the additional test, PTAL. Thanks!

[HyukjinKwon]

Suggested change

	logWarning(
	"StreamingQueryListenerBus Handler thread received exception, all client" +
	" side listeners are removed and handler thread is terminated.",
	e)
	logWarning("StreamingQueryListenerBus Handler thread received exception, all client" +
	" side listeners are removed and handler thread is terminated.", e)

[grundprinzip]

I did the spark connect auto format and it produced these changes. I'm ok with reverting them in the worst case but at the same time your suggestions introduce na manual style adjustment.

LMK

[HyukjinKwon]

Suggested change

	"org.apache.spark.sql.streaming.StreamingQueryListenerBus"),
	"org.apache.spark.sql.streaming.StreamingQueryListenerBus"
	),

[grundprinzip]

Will fix the wrong version typo in two places.

[HyukjinKwon]

Looks fine but I defer to @gengliangwang who drove this.

[gengliangwang]

Thanks for adding the new function! Merging to master.

[gengliangwang]

@grundprinzip Shall we create a new jira for this instead of using https://issues.apache.org/jira/browse/SPARK-41794?

[zhengruifeng]

add it to python API references in #46566