-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow parsers #5
Comments
cd05f1f brought in serde+csv. This, from an API perspective is much cleaner — users can just specify structs and use serde's Deserialize derive attribute to handle parsing. However, comparing against here appears to be a performance hit. Here is f144ab4, but with the
Here are two runs on
So far, it looks like serde's deserialization lead to speed ups, but the serialization (or maybe Making windows is a fast operation in absolute terms, so this matters little... but one can see the cost of serde deserialize versus the old |
Updates: this is on f991e23 which may disappear as it's squashed etc.
with
So parsing is slow, but something in particular isn't scaling well. |
GRange's parsers are slow-ish. I got a 20-25% gain in speed from using serde + csv. But, I think
String
types are killing us, compared to raw bytes. But to materialize those benefits a new ASCII or raw byte vector type is need through out (I think noodles uses a similar approach?).The text was updated successfully, but these errors were encountered: