Explore kTLS for kernel TLS termination

[RaasAhsan]

One of the main reasons why sendfile(2) and other zero-copy mechanisms (SOCKMAP via eBPF, splice(2), vmsplice(2)) don't work for edge services is the requirement of SSL/TLS.

SSL/TLS encryption forces the program to copy data into userland to have userland encryption applied to it before being sent back to the kernel.

In addition to disabling zero-copy mechanisms, userland crypto also prevents kernel primitives from introspecting data in transit, severely limiting the capabilities of eBPF programs.

To solve this, the Linux kernel has implemented kernel space TLS via a system called kTLS.

A 7% performance boost has been observed by using kTLS to re-enable sendfile(2). See the original kTLS paper for benchmark results: https://netdevconf.info/1.2/papers/ktls.pdf

I imagine that since kTLS, io_uring and even eBPF are newer additions to Linux, older kernel versions don't support it. Do you think it's worthwhile (or if it's even possible) to write code that falls back to other primitives for older kernels?

I think it's definitely possible in the case of kTLS and io_uring. This is effectively what glibc does. The worthwhile part is probably more of a case-by-case basis thing.

Doing a fallback for a kernel that doesn't have eBPF enabled seems way more challenging of a thing and might not be possible in certain eBPF use cases.