Add CRCs to the transaction log #35
Labels
enhancement
New feature or request
Comments
ecton
added a commit
that referenced
this issue
Feb 9, 2022
Was reviewing coverage and saw a block of test code that never invoked. Turns out a good chunk of testing was being skipped. The testing encryption vault now adds a header to be able to detect an unencrypted payload. I've added #35 to give us more validation inside of transaction log scanning, but for now, the test has been flipped to try using a vault on an unencrypted payload instead of the other direction. Functionally the same, but the CRCs would have allowed it to work either way.
ecton
added a commit
that referenced
this issue
Jul 10, 2022
This is still a little ugly, but all unit tests are passing. Compaction is currently not implemented, and currently Sediment acts as an append-only file because Roots doesn't ever checkpoint it. This commit replaces the append-only transaction format with a commit-log powered transaction log using Sediment. When committing multiple trees, the transaction log thread will send the trees to the thread pool to be committed, while it commits the transaction log itself. Once the transaction log is fully synced, the log manager thread waits for all other trees to finish syncing, then continues to work like the previous implementation. The benefit of using Sediment is that this format now supports checkpointing the transaction log, and it also implements several other todos in the process. Closes #42. Closes #35. Closes #40.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Was just fixing a unit test that wasn't working correctly and realized that there's no true validation happening on the transaction log. We should consider adding a CRC to each entry to help detect bitrot/disk errors.
The text was updated successfully, but these errors were encountered: