Completed replacement of Python's enum internals.
Pre-releaseThis release completes the reimplementation of Python's Enum and IntFlag types to make them much faster.
General rough benchmarks I got from my PC with general test data were as follows. You can view the full benchmark script by checking scripts/*_benchmark.py
in this repository.
Operation | Pure Python | Hikari Implementation | Performance Gain |
---|---|---|---|
Profiling SomeEnum(x) 1 million times |
2,000,003 function calls in 0.716s | 1,000,003 function calls in 0.263s | +50% // +63% |
--- | --- | --- | --- |
SomeEnum(x) |
0.445µs | 0.143µs | +68% |
SomeEnum._value2member_map_[x] * |
0.107µs | 0.046µs | +57% |
SomeEnum[x] |
0.172µs | 0.103µs | +40% |
--- | --- | --- | --- |
SomeFlag(x) where x is a power of 2 named member |
4.061µs | 1.209µs | +70% |
SomeFlag(x) where x is a non-power of 2 value that has never been declared before** |
18.283µs | 1.179µs | +94% |
SomeFlag(x) where x is a non-power of 2 value that has been declared before |
0.418µs | 0.295µs | +29% |
* the Hikari implementation is named _value_to_member_map_
instead of _value2member_map_
.
** both Python's IntFlag and Hikari's Flag cache combinations of flags that are created, this significantly decreases the time needed to transform, construct, and validate a composite flag value later on.
Making highest priority, os.SCHED_RR
sys.getswitchinterval 0.5
sched_getscheduler 2
sched_getparam posix.sched_param(sched_priority=1)
sched_getaffinity {7}
sched_getprioritymax 0
sched_getprioritymin 0
sched_rr_getinterval 0.09999999000000001
- CPython 3.8.5 x86_64 in non-optimised mode.
- Linux pc 5.8.11-1-MANJARO #1 SMP PREEMPT Wed Sep 23 14:35:40 UTC 2020 x86_64 GNU/Linux
- CPU
- model name : Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
- microcode: 0xd6
- 6802.13 bogomips
- hyper-threaded: 4 physical cores, 8 logical cores.
- cache_alignment: 64
- No swap
Unlike Python's IntFlag, Hikari now limits the cache size for each enum to a maximum of 2^10 combinations. This means that for a long running bot that received a hypothetically large range of possibilities for permissions on a role via events, you would only ever see the cache grow to 8192 bytes in size (8 KiB), rather than the unbound value that Python allows which would currently be larger than
34,359,738,368 bytes (32 GiB). This means you get a potentially MASSIVE decrease in the memory footprint for your application over time, as values are no longer "leaked" in such a potentially devastating way.
In addition, these changes make a large difference to how long your application will block for when parsing medium-to-large sized guilds on start up during the Guild Create events. Enum parsing was identified as one of the biggest bottlenecks at the time, so you should notice a performance increase there.
New features
- Flag types have new methods! You can now use these on things like intents and permissions and guild flags.
difference
- set difference, equiv to-
intersection
- equiv to&
invert
- equiv to~
is_disjoint
is_subset
- equiv toin
is_superset
- equiv to reversedin
symmetric_difference
- equiv to^
union
- equiv to|
__len__
and__bool__
and__iter__
, which now makes this conform to an AbstractSet structural supertype!- Operators now guaranteed to work in reverse and with raw integers.
Breaking changes
- Flag methods renamed
has_all
->all
has_any
->any
has_none
->none
+
operator for flags has been removed, and will default to the int implementation.- You should now compare flag members by the
==
operator, rather than theis
operator due to internal optimisations.
Other changes
- Documentation is less bad now, as methods on enums and flags should be documented correctly.
- Anchors now work correctly on functions and methods.