New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize snapshot reading when deriving tables #1823

Open

segfault-magnet opened this issue Apr 11, 2024 · 0 comments · May be fixed by #1838

Assignees

Labels

Contributor

segfault-magnet commented Apr 11, 2024

Currently if we're deriving a table we're re-reading the snapshot to get the data. This adds IO/encoding overhead.

The ImportTable interface can be adapted to handle both importing the original data and generate and import any derived data as well.

segfault-magnet added the enhancement label

segfault-magnet self-assigned this

segfault-magnet linked a pull request

that will close this issue

perf: single-pass snapshot reading #1838

Draft

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment