Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Epic] Split Omni specific implementation from Bitcoin core #1305

Merged
merged 18 commits into from
Nov 10, 2023
Merged

[Epic] Split Omni specific implementation from Bitcoin core #1305

merged 18 commits into from
Nov 10, 2023

Conversation

bvbfan
Copy link

@bvbfan bvbfan commented Aug 18, 2023

  • Switch Omni protocol validation to event driven using validation interface
  • Switch Omni specific data storage out of bitcoin core databases
  • Move global coins cache to local one
  • Simple and efficient mempool and transaction storage
  • Remove txindex hard requirements
  • Rework all database records to use binary prefix instead of string data
  • Remove usage of core mutexes to allow validation to work in its own thread
  • Omnicore could be started as prune node, a lightweight client

Todo's:

  • Remove Omni specific coin selection from Bitcoin core
  • Remove Omni specific functions from Bitcoin wallet interface

Pending:

  • Full resync to measure time and storage

Fixes: #1293 #1294

@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch from f4e7db6 to 3869278 Compare August 21, 2023 06:58
bvbfan added 10 commits August 21, 2023 16:20
@bvbfan
Prepare moving omni related code to its own path
954f716 
* Move experimental btc balance records to its own database
* Split mempool functionality
* Switch to transaction and mempool events
* Switch coins cache to local storage

Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Database refactoring
e43b0cf 
* Remove depends to block index
* DB / persistent storage simplifications

Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Move away from txindex requirement
5b0b72e 
* Migrate db records to prefix binary format
* Remove depends to internal core mutexes
* TxIndex isn't mandatory

Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Code/Tests revamps
7564581 
* Full txindex removal
* Code refinition, simplification
* All tests pass

Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
DB keys rework, fix partial key
3edb61b 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Optimize storage
f14be2d 
* Optimize protocol layer transaction storage
* Simplify leveldb batch
* Allow transaction existance check

Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Simplify coin selection
6c0853a 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Transaction tracking fixes
624b0e0 
* Fix omni mempool transaction tracking
* Keep spent transaction inputs 100 blocks in case of reorg

Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Don't allow RPC method to access db(s) at block processing time
fbe5a84 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Keep wallet context in different way
39f6448 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch from 3869278 to 9ef64ae Compare August 21, 2023 13:57
@bvbfan
Fix lint
b4e5007 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch from 9ef64ae to 6b12e8c Compare August 21, 2023 16:25
@bvbfan
Rework log initialization
ccfd268 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch from 6b12e8c to ccfd268 Compare August 22, 2023 04:51
bvbfan added 3 commits August 29, 2023 08:17
@bvbfan
Performance: Reuse undo in case of reorg
9c24828 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Refactor serialization
e390885 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan
Better handling reorg recovering
1699e23 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch from 8d4bfda to 1699e23 Compare September 13, 2023 05:36
@dexX7
Copy link
Member

dexX7 commented Sep 13, 2023

Very nice! Any chance we get testing green? What's currently blocking it?

@bvbfan
Copy link
Author

bvbfan commented Sep 13, 2023 •
edited
Loading

Tests are generally green but sometimes CI environment is slow
AssertionError: Mempool sync timed out after 2400s:
omnij tests could be rerun.
It should be workable, no blockers.
./omnicored -reindex -prune=2048
To run full reindex, prune node with 2GiB block data

  • Todo: Reuse bitcoin core coin db to safe more space and increase performance.

@bvbfan
Cache chain tip
dad689d 
Signed-off-by: Anthony Fieroni <[email protected]>
@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch 4 times, most recently from 5906912 to a86091d Compare October 4, 2023 05:45
@bvbfan bvbfan force-pushed the split_omni_from_bitcoin branch from a86091d to c8fcc2f Compare October 9, 2023 05:50
@bvbfan
Optimize leveldb iterator lookups
ea2f801 
Signed-off-by: Anthony Fieroni <[email protected]>
@dexX7 dexX7 merged commit a2c114c into OmniLayer:develop Nov 10, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DerionFaleneva DerionFaleneva DerionFaleneva approved these changes

Assignees
No one assigned
Labels
consensus status: ready
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bvbfan @dexX7 @DerionFaleneva