[5/5]: lnwallet: extensions to aux leaf store to integrate custom channels #8641

guggero · 2024-04-11T13:00:08Z

Depends on #8632.

Extends the aux leaf store with more required parameters.

Link to all PRs in the saga:

1/5: [1/?]: multi: add ability to fund+use musig2 channels that commit to a tapscript root #8683
2/5: [2/?]: lnwallet+channeldb: add new AuxLeafStore for dynamic aux leaves #8684
3/5: [3/5]: multi: add new AuxFundingController for custom external funding flows #8622
4/5: [4/5]: lnwallet: add new AuxSigner interface to mirror SigPool #8632
5/5: [5/5]: lnwallet: extensions to aux leaf store to integrate custom channels #8641
6/5: [6/5]: invoice+rpc: add exit hop InvoiceAcceptor sub-systems and RPC calls #8771

coderabbitai · 2024-04-11T13:00:14Z

Important

Review Skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

github-actions · 2024-04-11T13:00:25Z

Pull reviewers stats

Stats of the last 30 days for lnd:

User	Total reviews	Time to review	Total comments
guggero 🥇	24 ▀▀▀	2d 13h 46m	52 ▀▀▀
ellemouton 🥈	12 ▀	5d 14h 33m ▀	21 ▀
bhandras 🥉	9 ▀	30m	0
Roasbeef	7 ▀	7d 8h 43m ▀	20 ▀
yyforyongyu	6 ▀	3h 18m	0
ziggie1984	5 ▀	3d 1h 5m	14 ▀
ProofOfKeags	4	4d 6h 3m	42 ▀▀
GeorgeTsagk	4	2d 20h 31m	10 ▀
bitromortac	2	13d 3h 49m ▀	6
saubyk	2	5h 24m	0
Chinwendu20	2	18d 5h 38m ▀▀	9
positiveblue	1	4d 21h 57m ▀	0
AndySchroder	1	17d 9h 5m ▀▀	4
ffranr	1	14h 10m	4
Crypt-iQ	1	7d 14h 36m ▀	4
dstadulis	1	1d 22h 10m	4
morehouse	1	2d 2h 39m	1
carlaKC	1	12h 2m	0
mohamedawnallah	1	1d 4h 44m	0

Roasbeef · 2024-04-11T23:26:23Z

lnwallet/commitment.go

@@ -654,25 +654,29 @@ type AuxLeafStore interface {
 // FetchLeavesFromView attempts to fetch the auxiliary leaves that
 // correspond to the passed aux blob, and pending fully evaluated HTLC
 // view.
- FetchLeavesFromView(prevBlob tlv.Blob, view *HtlcView,
+ FetchLeavesFromView(chanState *channeldb.OpenChannel, prevBlob tlv.Blob,


Alternatively could just be that special funding blob (the one in the open channel struct), we could also use a type alias here. Unsure though if we do need all that other information or not.

We need the local/remote channel config, custom blob, thaw height, initiator and channel type. So it's easiest to just pass in the whole channel state. Not sure what a type alias would give us? Other than perhaps being a bit shorter.

Roasbeef · 2024-04-11T23:29:18Z

lnwallet/commitment.go

+ // of the pkScript and CLTV timeout, and the value is the index the
+ // output should be placed at. The CLTV timeout is encoded as a big
+ // endian before hashing and is assumed to be 0 for non-HTLC outputs.
+ // sha256(pkScript || bigEndian(cltvDelta)) -> outputIndex


IIUC, this also incudes the value as well.

Ah yes, good catch. Fixed.

Roasbeef · 2024-04-11T23:31:03Z

lnwallet/commitment.go

+ // lets us skip sending the entire transaction over, instead we'll just
+ // send signatures.
+ if auxLeaves.IsSome() &&
+ auxLeaves.UnsafeFromSome().CustomOutputOrder != nil {


If we want to mask this a bit more, then we can have a functional option to pass in a new custom sorting interface/type. Default is the existing one, then we know to override it at the config level for these special channels.

Good point. I'm meeting you half way: Instead of passing in a function, we now require a sort function to be returned. But that way at least the sorting logic is not in the lnd code base anymore.

input/script_utils.go

Roasbeef · 2024-04-11T23:32:30Z

lnwallet/commitment.go

@@ -660,7 +660,8 @@ type AuxLeafStore interface {
 // correspond to the passed aux blob, and pending original (unfiltered)
 // HTLC view.
 FetchLeavesFromView(chanState *channeldb.OpenChannel, prevBlob tlv.Blob,
- unfilteredView *HtlcView,
+ unfilteredView *HtlcView, isOurCommit bool, ourBalance,
+ theirBalance lnwire.MilliSatoshi,


Balances needed for the new sorting algo?

Yes, for sorting, making sure we have non-dust BTC amounts for any asset amounts and for determining whether we need commitment anchor outputs (which we need to create exclusion proofs for).

guggero · 2024-05-17T13:26:52Z

This PR should now contain everything from the PoC branch and is ready for review.

This commit adds an optional data parser that can inspect and in-place format custom data of certain RPC messages.

ffranr

I appreciate that each commit pretty much does one thing 👍

ffranr · 2024-05-24T11:30:43Z

.golangci.yml

- # Allow fmt.Printf() in lncli.
- - path: cmd/lncli/*
+ # Allow fmt.Printf() in commands.
+ - path: cmd/commands/*


This refactoring commit seems unnecessary in this PR. Would be simpler to do this sort of change once the saga is merged into master perhaps.

The commit body should explain why this change is necessary.

ffranr · 2024-05-24T11:39:16Z

lnwallet/channel.go

+// AddHeight returns a pointer to the height at which the HTLC was added to the
+// commitment chain. The height is returned based on the chain the HTLC is
+// being added to (local or remote chain).
+func AddHeight(htlc *PaymentDescriptor, remoteChain bool) *uint64 {
+ if remoteChain {
+ return &htlc.addCommitHeightRemote
+ }
+
+ return &htlc.addCommitHeightLocal
+}
+
+// RemoveHeight returns a pointer to the height at which the HTLC was removed
+// from the commitment chain. The height is returned based on the chain the HTLC
+// is being removed from (local or remote chain).
+func RemoveHeight(htlc *PaymentDescriptor, remoteChain bool) *uint64 {


Why do these functions return pointer types?

ffranr · 2024-05-24T11:46:58Z

rpcserver.go

+type AuxDataParser interface {
+ // InlineParseCustomData replaces any custom data binary blob in the
+ // given RPC message with its corresponding JSON formatted data. This
+ // transforms the binary (likely TLV encoded) data to a human-readable
+ // JSON representation (still as byte slice).
+ InlineParseCustomData(msg proto.Message) error
+}


This commit adds an interface without an implementation. Was that intentional? I think we should clarify the commit message if it was intentional. Otherwise the implementation is unexpectedly missing.

ffranr · 2024-05-24T11:50:17Z

lnwallet/channel.go

+ // When the HTLC was decoded from the wire, the ExtraData field was
+ // parsed and split into the BlindingPoint and CustomRecords. Which
+ // means we don't have to copy over the ExtraData itself into the
+ // PaymentDescriptor, just the encoded CustomRecords.


I'm not sure if this comment is true. The ExtraData field still might contain data for a field which is lower in type than the custom records type range but not a defined field on the HTLC.

To avoid sorting issues with identical HTLCs (equal size, equal payment hash, equal CLTV), we need to also use the HTLC index to be able to distinguish between them.

guggero added taproot taproot chans custom chans labels Apr 11, 2024

Roasbeef reviewed Apr 11, 2024

View reviewed changes

guggero force-pushed the custom-channels-integration branch 5 times, most recently from d288b6a to f7f875b Compare April 16, 2024 11:23

guggero changed the base branch from aux-leaf-fetcher to aux-leaf-signing April 16, 2024 11:23

guggero force-pushed the custom-channels-integration branch 7 times, most recently from cb514a1 to f35aead Compare April 23, 2024 20:19

guggero changed the base branch from aux-leaf-signing to 0-19-staging April 23, 2024 20:21

guggero force-pushed the custom-channels-integration branch from f35aead to cf965f0 Compare April 24, 2024 15:04

guggero changed the base branch from 0-19-staging to aux-leaf-signing April 24, 2024 15:07

guggero changed the title ~~lnwallet: extensions to aux leaf store to integrate custom channels~~ [5/?]: lnwallet: extensions to aux leaf store to integrate custom channels Apr 24, 2024

guggero force-pushed the aux-leaf-signing branch from 1858e19 to ffa19c0 Compare April 25, 2024 17:06

guggero force-pushed the custom-channels-integration branch from cf965f0 to 0e6a9dd Compare April 25, 2024 17:06

dstadulis assigned guggero Apr 27, 2024

guggero marked this pull request as ready for review May 17, 2024 13:26

guggero requested review from Roasbeef, ffranr and GeorgeTsagk May 17, 2024 13:30

guggero force-pushed the custom-channels-integration branch 2 times, most recently from 5a834c1 to 7b9e627 Compare May 20, 2024 18:07

guggero force-pushed the aux-leaf-signing branch 2 times, most recently from c0b5902 to 6123a1c Compare May 21, 2024 07:27

guggero force-pushed the custom-channels-integration branch 2 times, most recently from 3b45569 to 12d4694 Compare May 21, 2024 07:46

guggero force-pushed the aux-leaf-signing branch from 6123a1c to e258b87 Compare May 21, 2024 07:46

guggero mentioned this pull request May 22, 2024

[6/5]: invoice+rpc: add exit hop InvoiceAcceptor sub-systems and RPC calls #8771

Open

guggero force-pushed the aux-leaf-signing branch from e258b87 to 380ba26 Compare May 22, 2024 16:25

guggero force-pushed the custom-channels-integration branch from 12d4694 to 44fe836 Compare May 22, 2024 16:25

guggero added 11 commits May 23, 2024 13:30

cmd/lncli: move commands and export

1f0fc78

lnwallet: add AddHeight and RemoveHeight funcs

f6615b1

lnwallet: export AnchorSize

982f3e8

lnwallet: export GenTaprootHtlcScript

974d39f

lnwallet: add Tree() method, fix formatting

95bfd73

cmd/commands: export StripPrefix

e4c7d26

lnrpc+rpcserver: add and populate custom channel data

cd8a989

lnd: add aux data parser

0b8fffd

This commit adds an optional data parser that can inspect and in-place format custom data of certain RPC messages.

lntest: fix comments

dc51928

server: fix logging of pubkey

85bec32

channeldb+lnwallet: fix CustomRecord decoding

5401a54

guggero force-pushed the aux-leaf-signing branch from 380ba26 to adfb663 Compare May 23, 2024 11:38

guggero force-pushed the custom-channels-integration branch from 44fe836 to 5401a54 Compare May 23, 2024 11:38

ffranr requested changes May 24, 2024

View reviewed changes

lnwallet: add HTLC index to commitment sorting function

d492303

To avoid sorting issues with identical HTLCs (equal size, equal payment hash, equal CLTV), we need to also use the HTLC index to be able to distinguish between them.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[5/5]: lnwallet: extensions to aux leaf store to integrate custom channels #8641

[5/5]: lnwallet: extensions to aux leaf store to integrate custom channels #8641

guggero commented Apr 11, 2024 •

edited

coderabbitai bot commented Apr 11, 2024 •

edited

Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

Documentation and Community

github-actions bot commented Apr 11, 2024

Roasbeef Apr 11, 2024

guggero Apr 12, 2024

Roasbeef Apr 11, 2024

guggero Apr 12, 2024

Roasbeef Apr 11, 2024

guggero Apr 12, 2024 •

edited

Roasbeef Apr 11, 2024

guggero Apr 12, 2024

guggero commented May 17, 2024

ffranr left a comment

ffranr May 24, 2024

ffranr May 24, 2024

ffranr May 24, 2024

ffranr May 24, 2024

[5/5]: lnwallet: extensions to aux leaf store to integrate custom channels #8641

Are you sure you want to change the base?

[5/5]: lnwallet: extensions to aux leaf store to integrate custom channels #8641

Conversation

guggero commented Apr 11, 2024 • edited

coderabbitai bot commented Apr 11, 2024 • edited

Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

github-actions bot commented Apr 11, 2024

Pull reviewers stats

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero Apr 12, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero commented May 17, 2024

ffranr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero commented Apr 11, 2024 •

edited

coderabbitai bot commented Apr 11, 2024 •

edited

CodeRabbit Configration File (`.coderabbit.yaml`)

guggero Apr 12, 2024 •

edited