Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add perf test for http client #2442

Merged
merged 1 commit into from
Oct 22, 2024

Conversation

xemul
Copy link
Contributor

@xemul xemul commented Sep 18, 2024

The test opens a server socket that reads socket up until double CRLF and then responds back with "HTTP/1.1 200 OK host: test" line in a loop. Then runs http client against it measuring time it takes to perform the given amount of requests. It uses in-memory loopback sockets (seastar wrapper around seastar::queue).

tests/perf/CMakeLists.txt Outdated Show resolved Hide resolved
@xemul
Copy link
Contributor Author

xemul commented Sep 19, 2024

upd:

  • fixed trash in CMakefile
  • removed unused capture on lambda
  • added total mallocs stats

@xemul
Copy link
Contributor Author

xemul commented Sep 19, 2024

@tchaikov , CI fails compiling a file that's not touched by this PR:

[208/360] Building CXX object tests/unit/CMakeFiles/test_unit_rpc.dir/rpc_test.cc.o
FAILED: tests/unit/CMakeFiles/test_unit_rpc.dir/rpc_test.cc.o 
ccache /usr/bin/g++ -DBOOST_ALL_DYN_LINK -DFMT_SHARED -DSEASTAR_API_LEVEL=7 -DSEASTAR_BUILD_SHARED_LIBS -DSEASTAR_DEFERRED_ACTION_REQUIRE_NOEXCEPT -DSEASTAR_DEPRECATED_OSTREAM_FORMATTERS -DSEASTAR_ENABLE_ALLOC_FAILURE_INJECTION -DSEASTAR_HAS_MEMBARRIER -DSEASTAR_HAVE_ASAN_FIBER_SUPPORT -DSEASTAR_HAVE_HWLOC -DSEASTAR_HAVE_NUMA -DSEASTAR_HAVE_SYSTEMTAP_SDT -DSEASTAR_HAVE_URING -DSEASTAR_LOGGER_COMPILE_TIME_FMT -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SSTRING -DSEASTAR_TESTING_MAIN -DSEASTAR_TESTING_WITH_NETWORKING=1 -DSEASTAR_TYPE_ERASE_MORE -I/__w/seastar/seastar/tests/unit -I/__w/seastar/seastar/src -I/__w/seastar/seastar/include -I/__w/seastar/seastar/build/dev/gen/include -I/__w/seastar/seastar/build/dev/gen/src -O1 -std=gnu++23 -Wno-maybe-uninitialized -Wno-error=unused-result -fstack-clash-protection -UNDEBUG -Wall -Werror -Wimplicit-fallthrough -Wdeprecated -Wno-error=deprecated -Wno-error=stringop-overflow -Wno-error=array-bounds -Wdeprecated-declarations -Wno-error=deprecated-declarations -ftls-model=initial-exec -gz -MD -MT tests/unit/CMakeFiles/test_unit_rpc.dir/rpc_test.cc.o -MF tests/unit/CMakeFiles/test_unit_rpc.dir/rpc_test.cc.o.d -o tests/unit/CMakeFiles/test_unit_rpc.dir/rpc_test.cc.o -c /__w/seastar/seastar/tests/unit/rpc_test.cc
/__w/seastar/seastar/tests/unit/rpc_test.cc: In lambda function:
/__w/seastar/seastar/tests/unit/rpc_test.cc:1366:89: error: class template argument deduction failed:
 1366 |             return make_ready_future<rpc::tuple<int, long>>(rpc::tuple(1, 0x7'0000'0000L));
      |                                                                                         ^
/__w/seastar/seastar/tests/unit/rpc_test.cc:1366:89: error: call of overloaded 'tuple(int, long int)' is ambiguous
In file included from /usr/include/c++/14/bits/uses_allocator_args.h:39,
                 from /usr/include/c++/14/bits/memory_resource.h:41,
                 from /usr/include/c++/14/string:68,
                 from /usr/include/c++/14/stdexcept:39,
                 from /usr/include/c++/14/system_error:43,
                 from /__w/seastar/seastar/tests/unit/loopback_socket.hh:24,
                 from /__w/seastar/seastar/tests/unit/rpc_test.cc:22:
/usr/include/c++/14/tuple:1970:5: note: candidate: 'std::tuple(_Ts ...)-> seastar::rpc::tuple<T ...> [with T = {int, long int}]'
...

@tchaikov
Copy link
Contributor

lemme check.

@tchaikov
Copy link
Contributor

lemme check.

i can reproduce this with GCC-14 on fedora 40. seems it's related to ADL.

@tchaikov
Copy link
Contributor

tchaikov commented Sep 19, 2024

@xemul fixed by #2446

@xemul
Copy link
Contributor Author

xemul commented Sep 19, 2024

Thanks! Rebased the branch

tests/perf/http_client_perf.cc Show resolved Hide resolved
tests/perf/http_client_perf.cc Outdated Show resolved Hide resolved
tests/perf/http_client_perf.cc Outdated Show resolved Hide resolved
@xemul
Copy link
Contributor Author

xemul commented Sep 23, 2024

upd:

  • added top-level comment
  • renamed server::do_echo -> server::run_serve_loop
  • coroutinized server and client core loops

@xemul xemul requested a review from nyh September 24, 2024 04:42
@@ -88,3 +88,7 @@ seastar_add_test (allocator

seastar_add_test (container
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It uses in-memory loopback sockets (seastar wrapper around seastar::queue).

But I already prepared mentally to write a comment about it

co_return;
}

_req += sstring(buf.get(), buf.size());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quadratic, but I expect n=1, so n²=1 too

});
return when_all(std::move(sf), std::move(cf)).discard_result();
});
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coroutine or thread?

Would be nicer to report insn/op and allocs/op.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread looks nicer indeed

Report usec/op and allocs/op

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not insn/op (in addition)? It's more stable with todays frequency agile processors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... Where to look for an example that collects instructions for a perf test?

Copy link
Contributor

@tchaikov tchaikov Oct 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we don't plan to use PERF_TEST because it's complicated to pass command line arguments to the individual tests. probably we can still use linux_perf_event::user_instructions_retired() directly ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perf-simple-query in scylla

@xemul
Copy link
Contributor Author

xemul commented Sep 27, 2024

upd:

  • close server-side streams
  • report per-op results
  • use async() in main app lambda

@xemul
Copy link
Contributor Author

xemul commented Oct 2, 2024

@avikivity , @nyh , reminder ping

The test opens a server socket that reads socket up until double CRLF
and then responds back with "HTTP/1.1 200 OK host: test" line in a loop.
Then runs http client against it measuring time it takes to perform the
given amount of requests. It uses in-memory loopback sockets (seastar
wrapper around seastar::queue).

Signed-off-by: Pavel Emelyanov <[email protected]>
@xemul
Copy link
Contributor Author

xemul commented Oct 22, 2024

upd:

  • added insns/op, cpu_cycles/op and tasks/op stats

@xemul xemul requested a review from avikivity October 22, 2024 08:08
@avikivity avikivity merged commit 2a4426d into scylladb:master Oct 22, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants