Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chanbackup, server, rpcserver: put close unsigned tx, remote signature and commit height to SCB #8183

Merged
merged 13 commits into from
Oct 14, 2024

Conversation

starius
Copy link
Collaborator

@starius starius commented Nov 16, 2023

Change Description

In this PR I'm implementing the proposal from #7658 (comment)

I changed chanbackup to add CloseTxInputs optional field to chanbackup.Single structure. When the field is present, the version byte is XORed with closeTxVersion = 128. When this bit is present in the version byte, the field is properly extracted when decoding packaged single backup.

I added chantools scbforceclose command to extract and sign closing tx. lightninglabs/chantools#95

Implemented:

  • add global and maybe also lncli flags enabling close tx in SCB (now it is always on)
    • documentation for the flag and the feature
  • tests for new features of chanbackup
  • integration tests
  • lines wrap at 80
  • commit structure
  • when LND shuts down, update channel.backup with latest close tx inputs

Steps to Test

I tested the following scenario on testnet:

  • lncli openchannel
  • lncli exportchanbackup --chan_point xxx:x --output_file single.backup
  • lncli stop
  • chantools --testnet scbforceclose --single_backup xxx
  • Manually broadcasted the tx
  • Started LND from scratch on another machine
  • lncli create. Use existing seed, recover onchain funds.
  • lncli restorechanbackup --single_file single.backup
  • find the channel in lncli pendingchannels, total_limbo_balance is equal to channel balance minus fees

Pull Request Checklist

Testing

  • Your PR passes all CI checks.
  • Tests covering the positive and negative (error paths) are included.
  • Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

📝 Please see our Contribution Guidelines for further guidance.

Summary by CodeRabbit

  • New Features
    • Enhanced backup configuration flexibility with BackupOption parameters.
    • Added support for force close transaction data in backups.
    • Updated backup commands to include an option for force close transaction inputs.
    • Introduced a new option in LND for including data for unilateral channel closure in an SCB file.
    • Added a new parameter for controlling backup behavior in channel restoration scenarios.
    • Included an option to specify inclusion of inputs of the latest force close transaction in channel backups.
  • Bug Fixes
    • Fixed a typo in the log message within the SetupStandbyNodes function.
  • Refactor
    • Refined backup creation and serialization/deserialization logic to accommodate new backup options.
    • Updated LightningChannel to use a more modular approach for signing commitment transactions.
  • Documentation
    • Improved explanations on commitment transaction sweeping in various scenarios.

Copy link
Collaborator

@guggero guggero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a first look. I think the general idea is nice, this should help in some extreme edge cases.
Though the way we currently update those channels the commitment TX within the backup would be outdated very quickly for normal channels. So as a safety precaution we should not include a signature to make it hard for people to accidentally breach themselves.

chanbackup/single.go Outdated Show resolved Hide resolved
chanbackup/single.go Outdated Show resolved Hide resolved
contractcourt/commit_sweep_resolver.go Outdated Show resolved Hide resolved
chanbackup/backup.go Outdated Show resolved Hide resolved
chanbackup/backup.go Outdated Show resolved Hide resolved
chanbackup/backup.go Outdated Show resolved Hide resolved
starius added a commit to starius/chantools that referenced this pull request Dec 30, 2023
starius added a commit to starius/chantools that referenced this pull request Dec 31, 2023
@starius starius force-pushed the close-tx-in-static-backup branch 2 times, most recently from eebf352 to e7bc2ea Compare December 31, 2023 18:56
starius added a commit to starius/chantools that referenced this pull request Dec 31, 2023
@starius starius changed the title [WIP] chanbackup, server, rpcserver: put close tx to SCB [WIP] chanbackup, server, rpcserver: put close unsigned tx, remote signature and commit height to SCB Dec 31, 2023
@starius
Copy link
Collaborator Author

starius commented Dec 31, 2023

@guggero Thank you for feedback! Happy New Year!

I updated both PRs and also their descriptions. SCB file now contains unsigned CommitTx, CommitSig of remote party, and for taproot channels also CommitHeight. I kept unresolved the comments where further acctention may be needed.

Please take another look!

@starius starius requested a review from guggero December 31, 2023 20:27
@starius starius force-pushed the close-tx-in-static-backup branch from e7bc2ea to a1b38f8 Compare January 2, 2024 02:34
starius added a commit to starius/chantools that referenced this pull request Jan 2, 2024
@starius
Copy link
Collaborator Author

starius commented Jan 2, 2024

I fixed few bugs in this PR.

  1. Integration test failed. I reproduced the failure using the following command (bitcoind binary is needed in PATH):
make itest icase=channel_backup_restore_basic backend=bitcoind

I found the root cause in log file itest/.logs-tranche0/4-channel_backup_restore_basic-dave-021bb9a5.log .

panic stack trace
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x21ec98]

goroutine 1 [running]:
github.com/btcsuite/btcd/wire.(*MsgTx).BtcEncode(0x0, {0x14691a0, 0x4001c3f500?}, 0x26a2832?, 0x2)
/root/go/pkg/mod/github.com/btcsuite/[email protected]/wire/msgtx.go:736 +0x38
github.com/btcsuite/btcd/wire.(*MsgTx).Serialize(...)
/root/go/pkg/mod/github.com/btcsuite/[email protected]/wire/msgtx.go:828
github.com/lightningnetwork/lnd/chanbackup.(*Single).Serialize(0x40004141a0, {0x14691a0?, 0x4001c3f470})
/root/lnd/chanbackup/single.go:363 +0x560
github.com/lightningnetwork/lnd/chanbackup.Multi.PackToWriter({0xa8?, {0x4000414000?, 0x400000d110?, 0x4000169f88?}}, {0x14691a0, 0x4001c3f440}, {0xffff6c95df30, 0x400000d110})
/root/lnd/chanbackup/multi.go:84 +0x240
github.com/lightningnetwork/lnd/chanbackup.(*SubSwapper).updateBackupFile(0x40001e40e0, {0x0, 0x0, 0x0?})
/root/lnd/chanbackup/pubsub.go:238 +0x48c
github.com/lightningnetwork/lnd/chanbackup.(*SubSwapper).Start.func1()
/root/lnd/chanbackup/pubsub.go:157 +0x6c
sync.(*Once).doSlow(0x4000716c60?, 0x0?)
/usr/local/go/src/sync/once.go:74 +0x100
sync.(*Once).Do(...)
/usr/local/go/src/sync/once.go:65
github.com/lightningnetwork/lnd/chanbackup.(*SubSwapper).Start(0x400049a420?)
/root/lnd/chanbackup/pubsub.go:151 +0x50
github.com/lightningnetwork/lnd.(*server).Start.func1()
/root/lnd/server.go:2072 +0x2120
sync.(*Once).doSlow(0x4?, 0x11b931e?)
/usr/local/go/src/sync/once.go:74 +0x100
sync.(*Once).Do(...)
/usr/local/go/src/sync/once.go:65
github.com/lightningnetwork/lnd.(*server).Start(0x1bf?)
/root/lnd/server.go:1864 +0x6c
github.com/lightningnetwork/lnd.Main(0x400002ab00, {{0x0?, 0x40001822a0?, 0x4000182300?}}, 0x400010f080, {0x40001a2480, 0x40001822a0, 0x4000182300, 0x40001823c0, {0x0}})
/root/lnd/lnd.go:684 +0x3164
main.main()
/root/lnd/cmd/lnd/main.go:38 +0x198

LND panicked trying to serialize a backup with nil CommitTx inside non-nil CloseTxInputs. This is possible when DLP is active and this is exactly what is tested in the failing test (channel_backup_restore_basic).

I changed buildCloseTxInputs to return nil *CloseTxInputs producing classic SCB file in this case.

  1. Also I found a mistake in error handling in Single.Serialize, returning nil instead of error. Fixed it.

  2. Also I figured out that wire.MsgTx can be serialized directly to io.Writer, without a temp buffer. Putting length of its serialization is redundant. I changed it to read and write tx directly.

@Roasbeef
Copy link
Member

@coderabbitai review

Copy link
Contributor

coderabbitai bot commented Feb 14, 2024

Important

Review skipped

Auto reviews are limited to specific labels.

🏷️ Labels to auto review (1)
  • llm-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

This update introduces a new feature allowing the inclusion of close transaction inputs in channel backups, enhancing data recovery options. It involves adding a BackupConfig structure, modifying functions to accept backup options, and updating serialization/deserialization logic to handle the new CloseTxInputs data. Additionally, it refactors signing logic in LightningChannel for better modularity and clarity, and corrects minor issues across the codebase.

Changes

Files Change Summary
chanbackup/backup.go, chanbackup/pubsub.go, rpcserver.go, server.go Introduced BackupConfig and BackupOption for configurable backup creation; added support for including close transaction inputs.
chanbackup/single.go, chanbackup/single_test.go Added CloseTxInputs struct and updated serialization/deserialization logic; added tests for new backup version.
cmd/lncli/commands.go Changed ChanBackup field type and modified value assignment.
contractcourt/commit_sweep_resolver.go Enhanced explanations on commitment sweeping scenarios.
docs/recovery.md Added information on --backupclosetxinputs option for SCB files and fund recovery.
lnrpc/lightning.proto Added with_close_tx_inputs field to specify force close transaction inclusion.
lnwallet/channel.go Refactored signing logic for commitment transactions and added taproot channel support.

🐇✨
In the land of code and byte,
A rabbit hopped with sheer delight.
"New backups," it squeaked, "are here to stay,
With close tx inputs, hooray, hooray!
Through refactor and tweak, we hop along,
Ensuring our data's robust and strong."
🌟🐾

Possibly related issues


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 4

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 4d8fa34 and a1b38f8.
Files selected for processing (10)
  • chanbackup/backup.go (3 hunks)
  • chanbackup/pubsub.go (3 hunks)
  • chanbackup/single.go (6 hunks)
  • chanbackup/single_test.go (4 hunks)
  • cmd/lncli/commands.go (1 hunks)
  • contractcourt/commit_sweep_resolver.go (1 hunks)
  • lntest/harness.go (1 hunks)
  • lnwallet/channel.go (4 hunks)
  • rpcserver.go (3 hunks)
  • server.go (1 hunks)
Files not reviewed due to errors (1)
  • server.go (Error: unable to parse review)
Files skipped from review due to trivial changes (1)
  • lntest/harness.go
Additional comments: 21
chanbackup/backup.go (3)
  • 89-98: The WithCloseTxInputs function correctly implements the BackupOption type to modify BackupConfig. Verify that all calls to this function correctly reflect the user's intention regarding the inclusion of CloseTxInputs.
  • 53-112: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [104-130]

In FetchBackupForChan, the handling of includeCloseTxInputs option is correct. Ensure that the buildCloseTxInputs function is robust and handles all edge cases, especially since it directly affects the backup's content based on this option.

  • 124-145: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [137-164]

In FetchStaticChanBackups, the handling of includeCloseTxInputs option is consistent with FetchBackupForChan. Double-check that the iteration over channels and the conditional inclusion of CloseTxInputs are correctly implemented and tested.

chanbackup/pubsub.go (3)
  • 101-103: The addition of the includeCloseTxInputs field in the SubSwapper struct is consistent with the overall feature addition. Ensure that this field is used appropriately in all relevant backup operations within SubSwapper.
  • 97-120: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [111-144]

The NewSubSwapper function correctly processes BackupOption parameters to configure the SubSwapper instance. Confirm that all instances of SubSwapper creation throughout the codebase are updated to pass the necessary BackupOption parameters.

  • 280-289: The conditional inclusion of CloseTxInputs based on the includeCloseTxInputs field within the backupUpdater method is correctly implemented. Verify that the buildCloseTxInputs function correctly handles all channel types and scenarios.
chanbackup/single_test.go (3)
  • 5-5: The addition of the encoding/hex import is necessary for the new test cases involving hex encoding. Confirm that it's used appropriately in all test cases.
  • 99-132: The modifications to assertSingleEqual to include checks for CloseTxInputs are correct. Ensure that all relevant test cases now properly account for CloseTxInputs when comparing Single instances.
  • 292-375: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [248-366]

The new test cases in TestSinglePackUnpack for different versions with CloseTxInputs are well-structured. Verify that these cases cover all scenarios, including edge cases for CloseTxInputs handling.

contractcourt/commit_sweep_resolver.go (1)
  • 28-35: The added comments provide clear explanations of the logic for handling CSV delays and CLTV expiration. This enhances code readability and maintainability.
chanbackup/single.go (5)
  • 55-58: Introduced closeTxVersionMask to indicate the presence of CloseTxInputs in backups. This approach allows backward compatibility and future extensibility.
  • 145-152: Added CloseTxInputs struct to Single for storing force close transaction data. This is crucial for enhancing the robustness of SCBs by including necessary data for force-closing channels.
  • 358-383: In Serialize, the conditional inclusion of CloseTxInputs based on its presence and version-specific handling for CommitHeight is correctly implemented. Ensure that future versions also consider compatibility with this feature.
  • 485-487: In Deserialize, correctly handling the version mask to determine the presence of CloseTxInputs. This ensures backward compatibility and correct parsing of new backup formats.
  • 600-628: The deserialization logic for CloseTxInputs correctly handles the conditional parsing based on the version and the presence of CloseTxInputs. This maintains the integrity of the backup data structure.
cmd/lncli/commands.go (1)
  • 2418-2421: The change from []byte to string for the ChanBackup field and the use of hex.EncodeToString for chanBackup.ChanBackup is observed. Ensure that all related serialization, deserialization, and cryptographic operations are correctly updated to handle the string type without loss of data or functionality.
lnwallet/channel.go (5)
  • 6224-6251: The definition of SignedCommitTxInputs and TaprootSignedCommitTxInputs structures is clear and well-documented. However, ensure that all fields, especially those involving cryptographic operations (OurKey, TheirKey, SignDesc), are securely handled throughout their lifecycle to prevent any potential security vulnerabilities.
  • 6265-6285: In GetSignedCommitTx, the handling of taproot channels involves re-deriving the local nonce using channeldb.NewMusigVerificationNonce. Ensure that error handling for this operation is robust, as failure in nonce derivation could lead to critical issues in transaction signing.
  • 6295-6301: The creation of the musig session with NewPartialMusigSession and decoding of the remote signature with remoteSig.Decode are critical operations. Verify that the error handling for Decode is comprehensive and that the musig session creation is securely managed, especially in terms of key handling and nonce generation.
  • 6347-6365: For non-taproot channels, the generation of the witness stack in default case uses ecdsa.ParseDERSignature and signer.SignOutputRaw. Ensure that the cryptographic operations are secure and that the generated witness stack correctly adheres to the expected format for multi-sig spends. Additionally, verify that SignOutputRaw is called with the correct parameters, particularly SignDesc, to prevent any issues in signature generation.
  • 6374-6397: The refactoring of getSignedCommitTx to use SignedCommitTxInputs and potentially TaprootSignedCommitTxInputs is a significant change. Ensure that this refactoring does not introduce any regressions, especially in the handling of taproot and non-taproot channels. Verify that the GetSignedCommitTx function is called with the correct inputs and that the channel's state is correctly used to populate these inputs.

rpcserver.go Outdated Show resolved Hide resolved
rpcserver.go Outdated Show resolved Hide resolved
rpcserver.go Outdated Show resolved Hide resolved
chanbackup/backup.go Outdated Show resolved Hide resolved
@starius starius force-pushed the close-tx-in-static-backup branch from a1b38f8 to 4163908 Compare March 28, 2024 03:09
@starius
Copy link
Collaborator Author

starius commented Mar 28, 2024

I addressed feedback from @coderabbitai

  • added options to API requests and LND option to explicitly enable inclusion of CloseTxInputs into backups
    • when LND shuts down, update channel.backup with latest close tx inputs - DONE
  • documented new options
  • changed itest/lnd_channel_backup_test to run the tests in both modes (with the feature enabled/disabled via backupclosetxinputs flag of LND)

Copy link
Contributor

coderabbitai bot commented Mar 28, 2024

@starius, thank you for addressing the feedback and making those updates. It's great to see the inclusion of options for enabling the CloseTxInputs feature in backups, the documentation of these new options, and the adjustment of integration tests to cover both enabled and disabled states of the feature.

I'll review the changes to ensure everything aligns with best practices and the objectives of the PR. If there are any specific areas you'd like me to focus on or if there's additional context that might help with the review, please let me know.

Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Contributor

coderabbitai bot commented Mar 28, 2024

Tip

For best results, initiate chat on the files or code changes.

@starius, it's great to hear that you've made these updates based on the feedback. Adding options for enabling the CloseTxInputs feature, updating the channel.backup file upon LND shutdown, documenting the new options, and adjusting the integration tests to cover both enabled and disabled states of the feature are significant improvements.

Given the detailed updates you've provided, I'll proceed with reviewing the changes based on the descriptions you've shared. If there are specific files or sections of code you'd like me to focus on, please let me know, and I'll be happy to take a closer look.

Keep up the excellent work, and feel free to reach out if you have any questions or need further assistance.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 10

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between b76f733 and 4163908b7ff4b6f5a77a874649f94a297ac67393.
Files ignored due to path filters (3)
  • lnrpc/lightning.pb.go is excluded by !**/*.pb.go
  • lnrpc/lightning.pb.gw.go is excluded by !**/*.pb.gw.go
  • lnrpc/lightning.swagger.json is excluded by !**/*.json
Files selected for processing (13)
  • chanbackup/backup.go (3 hunks)
  • chanbackup/pubsub.go (3 hunks)
  • chanbackup/single.go (6 hunks)
  • chanbackup/single_test.go (4 hunks)
  • cmd/lncli/commands.go (7 hunks)
  • config.go (1 hunks)
  • contractcourt/commit_sweep_resolver.go (1 hunks)
  • docs/recovery.md (2 hunks)
  • itest/lnd_channel_backup_test.go (10 hunks)
  • lnrpc/lightning.proto (3 hunks)
  • lnwallet/channel.go (4 hunks)
  • rpcserver.go (4 hunks)
  • server.go (3 hunks)
Learnings (1)
rpcserver.go (1)
Added flags `lncli exportchanbackup --with_close_tx_inputs` and `lnd --backupclosetxinputs` enable inclusion of `CloseTxInputs` in backups. Documentation updated in docs/recovery.md.
Additional comments (36)
chanbackup/backup.go (2)
  • 82-87: The introduction of the BackupConfig struct with the includeCloseTxInputs field is a good addition for configuring backup creation. This allows for flexibility in including or excluding close transaction inputs in backups, which can be useful depending on the user's needs or storage constraints.
  • 89-98: The WithCloseTxInputs function is a well-implemented BackupOption that enables users to specify whether SCBs should contain inputs needed for force-closing channels. This is a valuable feature for enhancing the robustness of SCBs, especially in scenarios where unilateral channel closure is required.
chanbackup/pubsub.go (3)
  • 101-103: Adding the includeCloseTxInputs field to SubSwapper is a logical extension of the backup configuration options introduced in backup.go. This field allows the backup subsystem to be aware of whether close transaction inputs should be included in backups, which is crucial for the new chantools scbforceclose functionality.
  • 97-120: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [111-144]

The modification of NewSubSwapper to handle BackupOption variadic parameters is a good design choice, as it aligns with the changes made in backup.go and allows for consistent backup configuration across different parts of the system. The initialization of includeCloseTxInputs based on BackupConfig ensures that the backup subsystem's behavior is configurable and flexible.

  • 281-291: Updating backupUpdater to conditionally include CloseTxInputs in Single creation based on the includeCloseTxInputs field is a necessary change to support the new backup configuration options. This ensures that the backup subsystem can create backups that are consistent with the user's configuration preferences.
chanbackup/single_test.go (2)
  • 99-132: Adding checks and comparisons for CloseTxInputs fields in assertSingleEqual is crucial for ensuring that the new functionality works as expected. This update ensures that tests will accurately verify the presence and correctness of CloseTxInputs in SCBs, which is essential for the reliability of channel backups.
  • 292-375: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [248-366]

The addition of test cases with CloseTxInputs in versionTestCases is a thorough approach to testing the new SCB functionality. These cases cover various scenarios, including different versions and the presence of CloseTxInputs, ensuring comprehensive test coverage for the changes introduced in this PR.

contractcourt/commit_sweep_resolver.go (1)
  • 28-36: The detailed explanations provided in the commitSweepResolver about the conditions under which commitments can be swept are valuable for understanding the logic behind the resolution process. It's important to ensure that these conditions are accurately represented in the code and that the resolver behaves as expected under various scenarios, including CSV delays and channel leases.
Verification successful

To further verify the handling of CSV delays and channel leases in the commitSweepResolver, we need to inspect the methods within this resolver that implement the logic described in the comments. Specifically, we should look for methods that calculate delays or check conditions before sweeping commitments.


To accurately verify the handling of CSV delays and channel leases in the commitSweepResolver, we need to inspect the implementation details of the getCommitTxConfHeight and hasCLTV methods. These methods appear to be directly related to the conditions described in the comments.


The analysis of the getCommitTxConfHeight and hasCLTV methods confirms that the commitSweepResolver implements logic that matches the detailed explanations provided in the comments regarding handling CSV delays and channel leases. The getCommitTxConfHeight method is used to wait for the necessary block confirmations before proceeding, and the hasCLTV method checks for the presence of a CLTV condition for channel leases. This verifies that the resolver's implementation accurately represents the conditions under which commitments can be swept, as described.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify that the logic for handling CSV delays and channel leases in the
# commitSweepResolver matches the detailed explanations.
ast-grep --lang go --pattern $'commitSweepResolver { $$$ }'

Length of output: 1398



Script:

#!/bin/bash
# Search for methods within commitSweepResolver that handle CSV delays and channel leases
rg --type go "func \(c \*commitSweepResolver\) " ./contractcourt/commit_sweep_resolver.go

Length of output: 658



Script:

#!/bin/bash
# Inspect the getCommitTxConfHeight and hasCLTV methods for handling CSV delays and channel leases
rg --type go -A 10 "func \(c \*commitSweepResolver\) getCommitTxConfHeight\(" ./contractcourt/commit_sweep_resolver.go
echo "-----"
rg --type go -A 5 "func \(c \*commitSweepResolver\) hasCLTV\(" ./contractcourt/commit_sweep_resolver.go

Length of output: 785

chanbackup/single.go (6)
  • 56-58: The addition of the closeTxVersionMask constant is correctly implemented and follows a common pattern for encoding additional information within a byte. This approach allows for backward compatibility and future extensibility.
  • 155-175: The CloseTxInputs struct is well-designed and aligns with the objectives of enhancing SCB functionality. The inclusion of CommitHeight specifically for taproot channels is a thoughtful consideration for future channel types.
  • 145-152: The addition of the CloseTxInputs field to the Single struct is well-executed, with clear documentation and consideration for backward compatibility. This change is crucial for the PR's objectives and is correctly integrated into the existing structure.
  • 358-382: The updates to the Serialize method to handle CloseTxInputs are mostly correct. However, consider refining the error handling for the CommitTx serialization to ensure that any serialization failure is appropriately managed.
  • 597-632: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [485-628]

The updates to the Deserialize method to handle CloseTxInputs are correctly implemented. The method appropriately uses the closeTxVersionMask to detect the presence of CloseTxInputs and handles their deserialization efficiently, ensuring backward compatibility.

  • 142-178: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [1-629]

Overall, the changes to the chanbackup/single.go file are well-implemented, with clear documentation and adherence to coding standards. The introduction of CloseTxInputs and the corresponding updates to serialization and deserialization logic are crucial for enhancing SCB functionality and are executed with consideration for backward compatibility and future extensibility.

itest/lnd_channel_backup_test.go (7)
  • 81-90: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [59-87]

The introduction of the backupclosetxinputs flag to control backup behavior is well-implemented. However, it's crucial to ensure that this flag's effect is thoroughly tested across different scenarios within this test suite, especially in the context of SCBs and force-closing channels.

  • 420-434: The loop to test both states of the backupclosetxinputs flag (true and false) is a good approach to ensure that the new functionality works as expected under different configurations. This pattern is consistently applied throughout the test file, which is excellent for coverage.
  • 444-449: The newChanRestoreScenario function correctly incorporates the backupclosetxinputs flag into the node arguments. This ensures that the nodes being tested are initialized with the intended backup behavior, which is crucial for the validity of the tests.
  • 470-497: Testing the channel backup and restore functionality for unconfirmed channels with both states of the backupclosetxinputs flag is a comprehensive approach. It's important to verify that the system behaves correctly in scenarios where channels might not be fully established before a backup is needed.
  • 627-641: The loop to test different commitment types and zero-conf channels with both states of the backupclosetxinputs flag is thorough. It ensures that the backup and restore functionalities are compatible with various channel configurations, which is essential for the robustness of the feature.
  • 1178-1185: The testDataLossProtection function's approach to testing data loss protection with both states of the backupclosetxinputs flag is commendable. It's crucial to ensure that the system can handle scenarios where a node loses state, especially in the context of the new backup functionality.
  • 1206-1210: The conditional inclusion of the --backupclosetxinputs flag based on the test parameter is correctly implemented. This ensures that the node Dave is tested under both configurations, which is necessary for verifying the feature's impact on data loss protection scenarios.
cmd/lncli/commands.go (1)
  • 2412-2421: > 📝 NOTE

This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [2393-2436]

The addition of the --with_close_tx_inputs flag to the exportChanBackupCommand is correctly implemented. The flag's description clearly explains its purpose, which aligns with the PR objectives to enhance SCB functionality by including essential data for force-closing channels. The logic to check if the flag is set and pass this information to the ExportChannelBackup and ExportAllChannelBackups RPC calls is correctly implemented. This change should enable users to include the necessary data for force-closing channels in their SCB exports, improving the robustness of channel backups.

config.go (5)
  • 354-355: The addition of the BackupCloseTxInputs boolean field to the Config struct introduces a new configuration option for including inputs in the backup to produce the latest force close transaction. This change aligns with the PR's objective to enhance SCB functionality. Ensure that documentation and user guides are updated to reflect this new option and its implications for channel backup and recovery processes.
  • 351-358: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [2024-2048]

The supplyEnvValue function introduces a mechanism to handle environment variables within configuration values, supporting various formats including default values. This enhancement improves flexibility in configuration management. However, ensure that the documentation clearly explains how to use this feature, including examples of supported formats, to help users correctly configure their environment variables.

  • 351-358: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [2024-2048]

The supplyEnvValue function includes logic to redact sensitive information such as passwords when logging configuration values. This is a good security practice to prevent accidental exposure of sensitive data in logs. Continue to apply this approach consistently across all logging of configuration values to enhance security.

  • 351-358: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [2024-2048]

The use of regular expressions in functions like extractBtcdRPCParams and extractBitcoindRPCParams for parsing configuration files is appropriate for the use case. Given that these operations are performed once at startup, the impact on performance is minimal. However, consider precompiling regular expressions that are used multiple times or in loops to optimize performance in future enhancements.

  • 351-358: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [2024-2048]

The configuration parsing and validation logic is well-structured, with clear separation of concerns and comprehensive error handling. The use of reflective programming to flatten the configuration struct into a map for easier processing of deprecated options is a clever approach. Ensure that error messages are clear and informative, providing users with actionable guidance on resolving configuration issues.

lnrpc/lightning.proto (3)
  • 4469-4472: The addition of the with_close_tx_inputs field to ExportChannelBackupRequest is consistent with the PR's objective to enhance SCBs by including additional data for force-closing channels. This field allows users to specify whether they want the backup to include inputs of the latest force close transaction, which can be crucial for recovering funds without counterparty cooperation.
  • 4506-4508: The addition of the with_close_tx_inputs field to ChanBackupExportRequest aligns with the enhancement of SCBs. This change enables the inclusion of close transaction inputs in the backup, providing more comprehensive data for channel recovery processes.
  • 4550-4552: Including the with_close_tx_inputs field in ChannelBackupSubscription is a valuable addition for subscribing to channel backup updates. This enables subscribers to receive backups that contain the inputs of the latest force close transaction, enhancing the robustness of the backup data.
rpcserver.go (4)
  • 7267-7267: The addition of chanbackup.WithCloseTxInputs(in.WithCloseTxInputs) in the ExportChannelBackup function correctly implements the feature to include close transaction inputs in the backup. This aligns with the PR's objective to enhance SCB functionality.
  • 7438-7438: Similarly, the inclusion of chanbackup.WithCloseTxInputs(in.WithCloseTxInputs) in the ExportAllChannelBackups function is consistent with the PR's goal to improve backup robustness by allowing the inclusion of close transaction inputs.
  • 7538-7539: The introduction of the closeTx variable in the SubscribeChannelBackups function to handle close transaction inputs is a thoughtful addition. It ensures that the functionality to include close transaction inputs is efficiently managed within the subscription logic.
  • 7570-7570: The use of chanbackup.WithCloseTxInputs(closeTx) in the SubscribeChannelBackups function's internal call to FetchStaticChanBackups is appropriate and ensures that the close transaction inputs are included in the backups as per the user's request. This is a crucial part of implementing the new feature as described in the PR.
lnwallet/channel.go (2)
  • 6461-6488: The introduction of the SignedCommitTxInputs struct is a positive change for code modularity and clarity. It encapsulates all necessary inputs for signing a commitment transaction, which aligns well with the principles of clean code and separation of concerns. This change should make the codebase more maintainable and easier to understand for new contributors.
  • 6490-6500: The TaprootSignedCommitTxInputs struct is well-defined and serves a specific purpose for taproot channels, which is a good practice in terms of code organization and clarity. However, it's important to ensure that all necessary fields are included and accurately documented for future reference and maintenance.

lnwallet/channel.go Outdated Show resolved Hide resolved
lnwallet/channel.go Outdated Show resolved Hide resolved
lnwallet/channel.go Show resolved Hide resolved
lnwallet/channel.go Show resolved Hide resolved
lnwallet/channel.go Outdated Show resolved Hide resolved
docs/recovery.md Outdated Show resolved Hide resolved
docs/recovery.md Outdated Show resolved Hide resolved
chanbackup/backup.go Outdated Show resolved Hide resolved
chanbackup/backup.go Outdated Show resolved Hide resolved
server.go Outdated Show resolved Hide resolved
@starius
Copy link
Collaborator Author

starius commented Mar 28, 2024

There are no tests that the backup with CloseTxInputs can be used to produce a commit transaction. To add such tests, the code signing commit tx should be available to such a test. Currently the code is in lightninglabs/chantools#95 in file cmd/chantools/scbforceclose.go in function signCloseTx. One approach is to move the function to LND and test it here, unit tests and itest. Another approach is to make an optional itest depending on chantools which runs chantools scbforceclose command. Any ideas on should this be tested in the first place and which of the options is preferred?

@starius starius force-pushed the close-tx-in-static-backup branch from 5854317 to a8b55e2 Compare March 28, 2024 04:04
@guggero
Copy link
Collaborator

guggero commented Apr 22, 2024

@saubyk this is a nice security/recovery feature. Should we prioritize it for v0.19?

@starius, removing my request for review until prioritized (same for lightninglabs/chantools#95).

@guggero guggero removed their request for review April 22, 2024 12:35
Copy link
Collaborator

@ziggie1984 ziggie1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. ✅, successfully verified that also custom-channels are backuped correctly, see testdata
  2. dumpchanbackup is handled here: chantools scbforceclose: extract close tx from SCB and sign it lightninglabs/chantools#95 => must be merged when this PR is merged.
  3. @starius please open a tracking issue related to the timestamp.

starius added a commit to starius/lnd that referenced this pull request Oct 2, 2024
Previous to this change taproot assets channels and simple taproot channels were
considered the same in the context of chanbackup package, since they stored the
same data. In the following commits we are adding the data needed to produce a
signed commitment transaction from a SCB file and in order to do that we need to
add more fields and a custom channel gets one additional field (TapscriptRoot)
compared to a simple taproot channel. So now we have to distinguish these kinds
of channels in chanbackup package.

See PR lightningnetwork#8183 for more details.
@starius starius force-pushed the close-tx-in-static-backup branch from fb44fd6 to 621a963 Compare October 2, 2024 02:20
@starius
Copy link
Collaborator Author

starius commented Oct 2, 2024

I added small commit "itest: test channel.backup changes upon shutdown" in which I add a check in SCB itest that channel.backup file content changes upon node shutdown. This is needed to make sure that this feature actually works. Just few lines of code in itest, no new dependencies.

@starius
Copy link
Collaborator Author

starius commented Oct 2, 2024

@ziggie1984
timestamps issue opened: #9149

// Shutdown to keep its directory including channel.backup file.
ht.SuspendNode(dave)

// Read Dave's channel.backup file again to make sure it was updated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add in the comment why it needs to be updated, it will only update if the state changes, otherwise the closeInput info also remains the same ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

        // Read Dave's channel.backup file again to make sure it was updated
        // upon Dave's shutdown. In case LND state is lost and DLP protocol
        // fails, the channel.backup file and the commit tx in it are the
        // measure of last resort to recover funds from the channel. The file
        // is updated upon LND server shutdown to update the commit tx just in
        // case it is used this way. If an outdated commit tx is broadcasted,
        // the funds may be lost in a justice transaction. The file is encrypted
        // and we can't decrypt it here, so we just check that the content of
        // the file has changed.
        multi2, err := os.ReadFile(backupFilePath)
        require.NoError(ht, err)
        require.NotEqual(ht, multi, multi2)

it will only update if the state changes, otherwise the closeInput info also remains the same

The file is encrypted and it changes even if the content is the same.

We can't decrypt it in itest to check closeInputs (this would bring hdkeyring.go dependency which we don't want in this PR).

So I just checked that the file content has changed -- this means ManualUpdate was at least triggered.

@starius starius force-pushed the close-tx-in-static-backup branch from 621a963 to 42b0cbb Compare October 2, 2024 14:29
@lightninglabs-deploy
Copy link

@yyforyongyu: review reminder
@starius, remember to re-request review from reviewers when ready

Copy link
Member

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a few typo/line wrapping issues, otherwise LGTM🙏

// output paying to us, in the case that the remote party broadcasts their
// version of the commitment transaction. We can sweep this output immediately,
// as it doesn't have a time-lock delay.
// output paying to us (local channel balance). In the case that the local
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

case inputs.Taproot.IsSome():
// Extract Taproot from fn.Option. It is safe to call
// UnsafeFromSome because we just checked that it is some.
taproot := inputs.Taproot.UnsafeFromSome()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking: we prolly need to move this method to a _dev.go file and only build it when it's in a test env, otherwise it's very tempting to use this API cc @ProofOfKeags

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to have Unwrap() (T, bool) method, then this code could be rewritten as:

if taproot, has := inputs.Taproot.Unwrap(); has {
  ...

// in loss of funds! This may happen if an outdated channel backup is attempted
// to be used to force close the channel.
func buildCloseTxInputs(targetChan *channeldb.OpenChannel,
) fn.Option[CloseTxInputs] {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line wrapping needs to be fixed, reference

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

return encoded
}

// Decode decodes the encoding of the version from a channel backup.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decode -> DecodeVersion

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

if err != nil {
return
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could do !s.Version.IsTaproot() to save some indentations here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Also added a comment to explain what happens after the check:

                if !s.Version.IsTaproot() {
                        return
                }

                // Write fields needed for taproot channels.
                err = lnwire.WriteElements(
                        &singleBytes, inputs.CommitHeight,
                )
                if err != nil {
                        return
                }
                ...

@@ -543,6 +685,47 @@ func (s *Single) Deserialize(r io.Reader) error {
}
}

if hasCloseTx {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: same here, could do,

if !hasCloseTx {
    return nil
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Also added a comment:

        if !hasCloseTx {
                return nil
        }

        // Deserialize CloseTxInputs if it is present in serialized data.
        commitTx := &wire.MsgTx{}
        if err := commitTx.Deserialize(r); err != nil {
                return err
        }
        ...

It used to be base64, which is not compatible with verifychanbackup,
expecting hex.
It is used for sweeping time-locked outputs as well as non time-locked outputs.
This pure function creates signed commit transaction, using various
inputs passed as struct TaprootSignedCommitTxInputs and a signer.

This is needed to be able to store the inputs without a signature
in SCB and sign the transaction in chantools scbforceclose.

See https://github.com/lightningnetwork/lnd/pull/8183/files#r1423959791
Previous to this change taproot assets channels and simple taproot channels were
considered the same in the context of chanbackup package, since they stored the
same data. In the following commits we are adding the data needed to produce a
signed commitment transaction from a SCB file and in order to do that we need to
add more fields and a custom channel gets one additional field (TapscriptRoot)
compared to a simple taproot channel. So now we have to distinguish these kinds
of channels in chanbackup package.

See PR lightningnetwork#8183 for more details.
@starius starius force-pushed the close-tx-in-static-backup branch from 42b0cbb to 35a11cd Compare October 11, 2024 13:02
@starius
Copy link
Collaborator Author

starius commented Oct 11, 2024

@yyforyongyu Thanks for review! I addressed the feedback.
Please take another look!

Copy link
Member

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🙏

Copy link
Collaborator

@guggero guggero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-ACK.


// Serialize CloseTxInputs if it is provided. Fill err if it fails.
var err error
s.CloseTxInputs.WhenSome(func(inputs CloseTxInputs) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We could use fn.MapOptionZ() here to allow returning an error, which would follow the intended "no mutations within callbacks" paradigm of the fn library:

	// Serialize CloseTxInputs if it is provided. Fill err if it fails.
	err := fn.MapOptionZ(s.CloseTxInputs, func(inputs CloseTxInputs) error {
		err := inputs.CommitTx.Serialize(&singleBytes)
		if err != nil {
			return err
		}

		err = lnwire.WriteElements(
			&singleBytes,
			uint16(len(inputs.CommitSig)), inputs.CommitSig,
		)
		if err != nil {
			return err
		}

		if !s.Version.IsTaproot() {
			return nil
		}

		// Write fields needed for taproot channels.
		err = lnwire.WriteElements(
			&singleBytes, inputs.CommitHeight,
		)
		if err != nil {
			return err
		}

		if s.Version.HasTapscriptRoot() {
			opt := inputs.TapscriptRoot
			var tapscriptRoot chainhash.Hash
			tapscriptRoot, err = opt.UnwrapOrErr(
				errEmptyTapscriptRoot,
			)
			if err != nil {
				return err
			}

			err = lnwire.WriteElements(
				&singleBytes, tapscriptRoot[:],
			)
			if err != nil {
				return err
			}
		}

		return nil
	})
	if err != nil {
		return fmt.Errorf("failed to encode CloseTxInputs: %w", err)
	}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied the suggested fix. Thanks!

The field is optional. It stores inputs needed to produce signed commit tx using
chantools scbforceclose, which calls function GetSignedCommitTx. New backups
have this field filled if commit tx is available (for all cases except when DLP
is active). If a backup has this data, the field is filled from it, otherwise it
is kept empty.

Modified test function genRandomOpenChannelShell to cover new types of channels
(simple taproot channel and custom channel) and to cover combinations of bits.
Make sure that TapscriptRoot field is properly packed and unpacked.
This method inserts channel updates and waits for them to be processed. It will
be used to update channel.backup upon LND shutdown.
This is needed to keep channel.backup up-to-date if the node is stopped.
@starius starius force-pushed the close-tx-in-static-backup branch from 35a11cd to 65df996 Compare October 14, 2024 12:45
@guggero guggero merged commit 1080230 into lightningnetwork:master Oct 14, 2024
22 of 28 checks passed
@starius starius deleted the close-tx-in-static-backup branch October 16, 2024 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backups SCB Related to static channel backup
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

7 participants