Benchmark #37
Closed
Benchmark #37
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ne-sachirou
force-pushed
the
benchmark
branch
3 times, most recently
from
47881c8
to
7078a6f
Compare
% mix benchmark
Benchmark.Cases.DataStoreKind
Not all of your protocols have been consolidated. In order to achieve the
best possible accuracy for benchmarks, please ensure protocol
consolidation is enabled in your benchmarking environment.
Operating System: macOS
CPU Information: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Number of Available Cores: 16
Available memory: 16 GB
Elixir 1.10.2
Erlang 22.3.2
Benchmark suite executing with the following configuration:
warmup: 2 s
time: 10 s
memory time: 2 s
parallel: 1
inputs: 100, 100_000, 10_000, 1_000
Estimated total run time: 5.60 min
Benchmarking empty with input 100...
Benchmarking empty with input 100_000...
Benchmarking empty with input 10_000...
Benchmarking empty with input 1_000...
Benchmarking ets with input 100...
Benchmarking ets with input 100_000...
Benchmarking ets with input 10_000...
Benchmarking ets with input 1_000...
Benchmarking fast_global with input 100...
Benchmarking fast_global with input 100_000...
Benchmarking fast_global with input 10_000...
Benchmarking fast_global with input 1_000...
Benchmarking gen_server with input 100...
Benchmarking gen_server with input 100_000...
Benchmarking gen_server with input 10_000...
Benchmarking gen_server with input 1_000...
Benchmarking heap with input 100...
Benchmarking heap with input 100_000...
Benchmarking heap with input 10_000...
Benchmarking heap with input 1_000...
Benchmarking persistent_term with input 100...
Benchmarking persistent_term with input 100_000...
Benchmarking persistent_term with input 10_000...
Benchmarking persistent_term with input 1_000...
##### With input 100 #####
Name ips average deviation median 99th %
empty 11.23 89.01 ms ±16.11% 87.91 ms 135.71 ms
persistent_term 9.96 100.37 ms ±10.65% 100.34 ms 159.07 ms
heap 9.36 106.89 ms ±9.87% 107.06 ms 142.37 ms
ets 1.89 530.22 ms ±22.47% 546.90 ms 712.95 ms
gen_server 0.41 2414.44 ms ±3.19% 2438.51 ms 2514.60 ms
fast_global 0.28 3544.83 ms ±25.04% 3457.03 ms 4473.17 ms
Comparison:
empty 11.23
persistent_term 9.96 - 1.13x slower +11.36 ms
heap 9.36 - 1.20x slower +17.88 ms
ets 1.89 - 5.96x slower +441.21 ms
gen_server 0.41 - 27.13x slower +2325.43 ms
fast_global 0.28 - 39.83x slower +3455.82 ms
Memory usage statistics:
Name average deviation median 99th %
empty 97.05 KB ±369.38% 14.78 KB 1577.43 KB
persistent_term 275.26 KB ±292.07% 14.78 KB 3140.08 KB
heap 14.64 KB ±0.50% 14.66 KB 14.73 KB
ets 14.88 KB ±0.99% 14.82 KB 15.09 KB
gen_server 14.94 KB ±0.00% 14.94 KB 14.94 KB
fast_global 14.78 KB ±0.00% 14.78 KB 14.78 KB
Comparison:
empty 14.78 KB
persistent_term 275.26 KB - 2.84x memory usage +178.21 KB
heap 14.64 KB - 0.15x memory usage -82.40995 KB
ets 14.88 KB - 0.15x memory usage -82.17167 KB
gen_server 14.94 KB - 0.15x memory usage -82.11308 KB
fast_global 14.78 KB - 0.15x memory usage -82.26933 KB
##### With input 100_000 #####
Name ips average deviation median 99th %
empty 11.14 89.75 ms ±10.12% 90.13 ms 108.58 ms
persistent_term 7.23 138.23 ms ±11.46% 137.70 ms 187.99 ms
heap 3.00 332.81 ms ±3.89% 333.17 ms 354.21 ms
ets 1.61 621.23 ms ±18.77% 620.75 ms 794.38 ms
fast_global 0.33 3043.92 ms ±7.05% 3082.18 ms 3259.32 ms
gen_server 0.30 3326.28 ms ±2.52% 3369.48 ms 3379.58 ms
Comparison:
empty 11.14
persistent_term 7.23 - 1.54x slower +48.48 ms
heap 3.00 - 3.71x slower +243.06 ms
ets 1.61 - 6.92x slower +531.48 ms
fast_global 0.33 - 33.91x slower +2954.17 ms
gen_server 0.30 - 37.06x slower +3236.53 ms
Memory usage statistics:
Name average deviation median 99th %
empty 249.19 KB ±229.73% 14.78 KB 1577.43 KB
persistent_term 14.74 KB ±0.45% 14.70 KB 14.86 KB
heap 8609.43 KB ±29.82% 8609.39 KB 12515.97 KB
ets 14.99 KB ±1.97% 14.86 KB 15.33 KB
fast_global 14.63 KB ±0.00% 14.63 KB 14.63 KB
gen_server 14.78 KB ±0.00% 14.78 KB 14.78 KB
Comparison:
empty 14.78 KB
persistent_term 14.74 KB - 0.06x memory usage -234.44414 KB
heap 8609.43 KB - 34.55x memory usage +8360.24 KB
ets 14.99 KB - 0.06x memory usage -234.19674 KB
fast_global 14.63 KB - 0.06x memory usage -234.56133 KB
gen_server 14.78 KB - 0.06x memory usage -234.40508 KB
##### With input 10_000 #####
Name ips average deviation median 99th %
empty 10.26 97.44 ms ±13.34% 98.10 ms 137.32 ms
persistent_term 9.12 109.69 ms ±9.72% 108.53 ms 136.85 ms
heap 8.07 123.96 ms ±8.51% 123.55 ms 162.74 ms
ets 1.90 525.46 ms ±22.58% 508.56 ms 715.56 ms
gen_server 0.39 2589.56 ms ±0.67% 2591.22 ms 2605.30 ms
fast_global 0.32 3103.88 ms ±8.61% 3108.77 ms 3348.83 ms
Comparison:
empty 10.26
persistent_term 9.12 - 1.13x slower +12.25 ms
heap 8.07 - 1.27x slower +26.52 ms
ets 1.90 - 5.39x slower +428.02 ms
gen_server 0.39 - 26.58x slower +2492.12 ms
fast_global 0.32 - 31.86x slower +3006.44 ms
Memory usage statistics:
Name average deviation median 99th %
empty 14.80 KB ±0.62% 14.78 KB 15.09 KB
persistent_term 112.45 KB ±347.42% 14.78 KB 1577.43 KB
heap 1019.35 KB ±97.08% 1577.27 KB 3140.16 KB
ets 14.94 KB ±1.28% 14.94 KB 15.17 KB
gen_server 14.86 KB ±0.00% 14.86 KB 14.86 KB
fast_global 14.70 KB ±0.00% 14.70 KB 14.70 KB
Comparison:
empty 14.78 KB
persistent_term 112.45 KB - 7.60x memory usage +97.64 KB
heap 1019.35 KB - 68.86x memory usage +1004.55 KB
ets 14.94 KB - 1.01x memory usage +0.135 KB
gen_server 14.86 KB - 1.00x memory usage +0.0564 KB
fast_global 14.70 KB - 0.99x memory usage -0.09983 KB
##### With input 1_000 #####
Name ips average deviation median 99th %
empty 10.47 95.51 ms ±11.92% 95.93 ms 134.55 ms
heap 9.03 110.79 ms ±12.41% 108.16 ms 169.12 ms
persistent_term 8.71 114.81 ms ±20.19% 109.88 ms 278.29 ms
ets 1.80 554.76 ms ±21.89% 565.71 ms 744.95 ms
gen_server 0.40 2472.88 ms ±1.92% 2496.32 ms 2518.39 ms
fast_global 0.32 3122.38 ms ±4.73% 3079.61 ms 3335.38 ms
Comparison:
empty 10.47
heap 9.03 - 1.16x slower +15.28 ms
persistent_term 8.71 - 1.20x slower +19.30 ms
ets 1.80 - 5.81x slower +459.25 ms
gen_server 0.40 - 25.89x slower +2377.37 ms
fast_global 0.32 - 32.69x slower +3026.87 ms
Memory usage statistics:
Name average deviation median 99th %
empty 343.75 KB ±243.35% 14.78 KB 3140.16 KB
heap 112.47 KB ±347.37% 14.78 KB 1577.51 KB
persistent_term 119.05 KB ±338.92% 14.86 KB 1577.51 KB
ets 15.09 KB ±2.74% 15.02 KB 15.64 KB
gen_server 3140.08 KB ±0.00% 3140.08 KB 3140.08 KB
fast_global 14.70 KB ±0.00% 14.70 KB 14.70 KB
Comparison:
empty 14.78 KB
heap 112.47 KB - 0.33x memory usage -231.28534 KB
persistent_term 119.05 KB - 0.35x memory usage -224.70529 KB
ets 15.09 KB - 0.04x memory usage -328.65789 KB
gen_server 3140.08 KB - 9.13x memory usage +2796.33 KB
fast_global 14.70 KB - 0.04x memory usage -329.04852 KB |
|
Good. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs #18