Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support long-lived iteration #24

Open
ecton opened this issue Oct 28, 2021 · 0 comments
Open

Support long-lived iteration #24

ecton opened this issue Oct 28, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@ecton
Copy link
Member

ecton commented Oct 28, 2021

Originally I skipped building an iterator interface for scanning, but I realized there is no reason it can't be done. Ultimately the iterator is about the values produced, not about the other callbacks.

There are two possible ways to go about this: switch to storing the scanning state on a manually managed stack or by using a bounded channel interface.

The bounded channel interface may sound like a hack, but it has real merits: It keeps the file locked for read, preventing rename from unlinking the file contents during a compaction. This means that the iterator will be surprisingly resilient. The downside is that it will necessarily require a thread, meaning it would be a feature unavailable to no_std if we approached it that way.

The alternate requires opening the file for each step in the scan operation -- our FileManager layer will cache it, but that still is wasteful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Backburner
Development

No branches or pull requests

1 participant