From 9685e8ee4a359ada08e5ed7d3bb2dab083065270 Mon Sep 17 00:00:00 2001 From: Avi Kivity Date: Fri, 9 Feb 2024 20:10:27 +0200 Subject: [PATCH] reactor: io_uring: enable some optimization flags Enable some optimization flags in an attempt to improve performance with io_uring: IORING_SETUP_COOP_TASKRUN - prevents a completion from interrupting the reactor if it is running. Requires that the reactor issue an io_uring_enter system call in a timely fashion, but thanks to the task quota timer, we do. IORING_SETUP_TASKRUN_FLAG - sets up a flag that notifies the reactor that the kernel has pending completions that it did not process. This allows the reactor to issue an io_uring_enter even if it has no pending submission queue entries or completion queue entries (e.g. it indicates a third queue, in the kernel, is not empty). IORING_SETUP_SINGLE_ISSUER - elides some locking by guaranteeing that only a single thread plays with the ring; this happens to be true for us. IORING_SETUP_DEFER_TASKRUN - batches up completion processing in an attempt to get some more performance. This flags bump up the dependencies to Linux 6.1 and liburing 2.2. This seems worthwhile as right now io-uring lags behind linux-aio (which processes completions from interrupt context and therefore doesn't need all these optimizations). After this exercise, io_uring is still slower than linux-aio. --- cmake/SeastarDependencies.cmake | 2 +- src/core/reactor_backend.cc | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/cmake/SeastarDependencies.cmake b/cmake/SeastarDependencies.cmake index 6c80d0fac41..9aff230b2ce 100644 --- a/cmake/SeastarDependencies.cmake +++ b/cmake/SeastarDependencies.cmake @@ -133,7 +133,7 @@ macro (seastar_find_dependencies) seastar_set_dep_args (Protobuf REQUIRED VERSION 2.5.0) seastar_set_dep_args (LibUring - VERSION 2.0 + VERSION 2.2 OPTION ${Seastar_IO_URING}) seastar_set_dep_args (StdAtomic REQUIRED) seastar_set_dep_args (hwloc diff --git a/src/core/reactor_backend.cc b/src/core/reactor_backend.cc index ff098591c4d..912e09aadbd 100644 --- a/src/core/reactor_backend.cc +++ b/src/core/reactor_backend.cc @@ -1213,13 +1213,21 @@ try_create_uring(unsigned queue_len, bool throw_on_error) { IORING_OP_SEND, IORING_OP_RECV, }; + auto required_flags = + IORING_SETUP_COOP_TASKRUN // linux 5.19; required for performance + | IORING_SETUP_TASKRUN_FLAG // linux 5.19; avoids a syscall if no event is ready + | IORING_SETUP_SINGLE_ISSUER // linux 6.0; helps with performance + | IORING_SETUP_DEFER_TASKRUN // linux 6.1; avoids context switches + ; auto maybe_throw = [&] (auto exception) { if (throw_on_error) { throw exception; } }; - auto params = ::io_uring_params{}; + auto params = ::io_uring_params{ + .flags = required_flags, + }; ::io_uring ring; auto err = ::io_uring_queue_init_params(queue_len, &ring, ¶ms); if (err != 0) {