Skip to content

Commit

Permalink
Add string benchmark results
Browse files Browse the repository at this point in the history
  • Loading branch information
eliaskosunen committed Oct 28, 2023
1 parent 586e093 commit aeedf41
Show file tree
Hide file tree
Showing 3 changed files with 97 additions and 10 deletions.
28 changes: 19 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,19 +199,27 @@ Lower is better.

![Integer result, chart](benchmark/runtime/results/int.png)

| Test | `scn::scan` | `scn::scan_value` | `std::stringstream` | `sscanf` | `strtol` | `std::from_chars` |
|:-------|--------------:| ----------------: | ------------------: |---------:|----------|------------------:|
| Test 1 | 55.9 | 50.9 | 142 | 81.6 | 21.2 | 13.3 |
| Test 2 | 56.3 | 51.8 | 59.1 | 512 | 49.7 | 13.9 |
| Test | `scn::scan` | `scn::scan_value` | `stringstream` | `sscanf` | `strtol` | `from_chars` |
|:-------|--------------:| ----------------: | -------------: |---------:|----------|-------------:|
| Test 1 | 55.9 | 50.9 | 142 | 81.6 | 21.2 | 13.3 |
| Test 2 | 56.3 | 51.8 | 59.1 | 512 | 49.7 | 13.9 |

#### Floating-point number parsing (`double`)

![Float result, chart](benchmark/runtime/results/float.png)

| Test | `scn::scan` | `scn::scan_value` | `std::stringstream` | `sscanf` | `strtod` | `std::from_chars` |
|:-------| ----------: | ----------------: | ------------------: |---------:|-----------|------------------:|
| Test 1 | 83.6 | 76.6 | 403 | 206 | 50.1 | 28.7 |
| Test 2 | 79.8 | 73.9 | 268 | 716 | 87.2 | 30.4 |
| Test | `scn::scan` | `scn::scan_value` | `stringstream` | `sscanf` | `strtod` | `from_chars` |
|:-------| ----------: | ----------------: | -------------: |---------:|-----------|-------------:|
| Test 1 | 83.6 | 76.6 | 403 | 206 | 50.1 | 28.7 |
| Test 2 | 79.8 | 73.9 | 268 | 716 | 87.2 | 30.4 |

#### String "word" (whitespace-separated character sequence) parsing (`string` and `string_view`)

![String result, chart](benchmark/runtime/results/string.png)

| `scn::scan<string>` | `scn::scan<string_view>` | `scn::scan_value<string>` | `scn::scan_value<string_view>` | `stringstream` | `sscanf` |
| ------------------: | -----------------------: | ------------------------: | -----------------------------: | -------------: | -------: |
| 35.6 | 35.4 | 27.8 | 26.4 | 173 | 69.5 |

#### Conclusions

Expand All @@ -231,6 +239,7 @@ Above,
which contains multiple values in their text representations, separated by spaces.
The time used for creating any state needed for the scanner is not included.
This test is called `"repeated"` in the benchmark sources.
* The string test is an exception: strings are read one after another from a sample of Lorem Ipsum.

The difference between "Test 1" and "Test 2" is most pronounced when using a `stringstream`,
which is relatively expensive to construct, and seems to be adding around ~100ns of runtime.
Expand All @@ -242,7 +251,8 @@ These benchmarks were run on a Fedora 37 machine, running Linux kernel version 6
with an AMD Ryzen 7 5700X processor, and compiled with gcc version 12.3.1,
with `-O3 -DNDEBUG -march=haswell` and LTO enabled.
C++20 was used, with the standard library (libstdc++) `<ranges>` implementation.
These benchmarks were run on 2023-08-24 (commit fdefc09).
These benchmarks were run on 2023-08-24 (commit fdefc09), except for the string-benchmark,
which was run on 2023-10-28 (commit 586e093).

The source code for these benchmarks can be found in the `benchmark` directory.
You can run these benchmarks yourself by enabling the CMake variable `SCN_BENCHMARKS`.
Expand Down
Binary file added benchmark/runtime/results/string.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
79 changes: 78 additions & 1 deletion benchmark/runtime/string/string_bench.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,39 @@ BENCHMARK(bench_string_scn<wchar_t, std::wstring, unicode_tag>);
BENCHMARK(bench_string_scn<wchar_t, std::wstring_view, lipsum_tag>);
BENCHMARK(bench_string_scn<wchar_t, std::wstring_view, unicode_tag>);

template <typename SourceCharT, typename DestStringT, typename Tag>
static void bench_string_scn_value(benchmark::State& state)
{
auto input = get_benchmark_input<SourceCharT, Tag>();
auto subr = scn::ranges::subrange{input};
for (auto _ : state) {
if (auto result = scn::scan_value<DestStringT>(subr)) {
benchmark::DoNotOptimize(result->value());
subr = result->range();
}
else if (result.error() == scn::scan_error::end_of_range) {
subr = scn::ranges::subrange{input};
}
else {
state.SkipWithError("Failed scan");
break;
}
}
}

BENCHMARK(bench_string_scn_value<char, std::string_view, lipsum_tag>);
BENCHMARK(bench_string_scn_value<char, std::string_view, unicode_tag>);
BENCHMARK(bench_string_scn_value<char, std::string, lipsum_tag>);
BENCHMARK(bench_string_scn_value<char, std::string, unicode_tag>);
BENCHMARK(bench_string_scn_value<char, std::wstring, lipsum_tag>);
BENCHMARK(bench_string_scn_value<char, std::wstring, unicode_tag>);
BENCHMARK(bench_string_scn_value<wchar_t, std::string, lipsum_tag>);
BENCHMARK(bench_string_scn_value<wchar_t, std::string, unicode_tag>);
BENCHMARK(bench_string_scn_value<wchar_t, std::wstring, lipsum_tag>);
BENCHMARK(bench_string_scn_value<wchar_t, std::wstring, unicode_tag>);
BENCHMARK(bench_string_scn_value<wchar_t, std::wstring_view, lipsum_tag>);
BENCHMARK(bench_string_scn_value<wchar_t, std::wstring_view, unicode_tag>);

template <typename CharT, typename Tag>
static void bench_string_sstream(benchmark::State& state)
{
Expand All @@ -89,4 +122,48 @@ BENCHMARK(bench_string_sstream<char, unicode_tag>);
BENCHMARK(bench_string_sstream<wchar_t, lipsum_tag>);
BENCHMARK(bench_string_sstream<wchar_t, unicode_tag>);

BENCHMARK_MAIN();
namespace {
const char* sscanf_impl(const char* input, std::string& value)
{
auto r = std::sscanf(input, " %255s", value.data());
if (r != 1) {
return nullptr;
}
auto len = std::strlen(value.c_str());
value.resize(len);
return input + len;
}
const wchar_t* sscanf_impl(const wchar_t* input, std::wstring& value)
{
auto r = std::swscanf(input, L" %255s", value.data());
if (r != 1) {
return nullptr;
}
auto len = std::wcslen(value.c_str());
value.resize(len);
return input + len;
}
} // namespace

template <typename CharT, typename Tag>
static void bench_string_scanf(benchmark::State& state)
{
auto input = get_benchmark_input<CharT, Tag>();
const CharT* begin = input.data();
for (auto _ : state) {
std::basic_string<CharT> val{};
val.resize(256);
auto p = sscanf_impl(begin, val);
if (!p) {
begin = input.data();
continue;
}
begin = p;
benchmark::DoNotOptimize(val);
}
}

BENCHMARK(bench_string_scanf<char, lipsum_tag>);
BENCHMARK(bench_string_scanf<char, unicode_tag>);
BENCHMARK(bench_string_scanf<wchar_t, lipsum_tag>);
BENCHMARK(bench_string_scanf<wchar_t, unicode_tag>);

0 comments on commit aeedf41

Please sign in to comment.