Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IF: Update finalizer safety information and rules for how a finalizer signs #2135

Merged
merged 90 commits into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
bc2549f
Start implementing finalizer safety - wip.
greg7mdp Jan 18, 2024
e06936c
Finish initial implementation of `finalizer::decide_vote()`.
greg7mdp Jan 19, 2024
a02c3b9
Minor fixes.
greg7mdp Jan 19, 2024
f142d65
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 20, 2024
de92000
remove duplicate definition from merge.
greg7mdp Jan 20, 2024
8674aaa
moved finalizer.cpp into `hotstuff` directory.
greg7mdp Jan 22, 2024
21fdd7c
Update `decide_vote`, store `proposal_ref` including timestamp.
greg7mdp Jan 22, 2024
ead4ae2
Add `finalizer_set` struct and use it.
greg7mdp Jan 22, 2024
75860ff
remone unneeded `is_last_vote_strong`.
greg7mdp Jan 22, 2024
8e27842
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 22, 2024
83db966
Resolve merge conflicts.
greg7mdp Jan 22, 2024
62c0741
Update `decide_vote` according to discussion from 1/22/24 meeting
greg7mdp Jan 22, 2024
82a7327
Fix mistake (comparison operator) in my previous checkin.
greg7mdp Jan 23, 2024
fccbb93
Add support for the case where we don't find a previous qc in `assemb…
greg7mdp Jan 23, 2024
01ac2c1
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 23, 2024
b920ddc
Use `decide_vote`
greg7mdp Jan 23, 2024
441ee6d
Avoid storing duplicate timestamp in finalizer::safety_information.
greg7mdp Jan 23, 2024
a4417ee
Add persistence for finalizer dafety information.
greg7mdp Jan 23, 2024
459d20c
Small cleanups.
greg7mdp Jan 23, 2024
4153370
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 24, 2024
ce67f45
Move `qc_data_t` definition into `controller.cpp`.
greg7mdp Jan 24, 2024
b8dfd6f
Comment indentation change.
greg7mdp Jan 24, 2024
5422463
Remove outdated todo comments.
greg7mdp Jan 25, 2024
2fac071
Making `decide_vote` work - work in progress
greg7mdp Jan 25, 2024
ffecd2c
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 30, 2024
1b62f01
Fix a couple merge issues.
greg7mdp Jan 30, 2024
882852f
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 30, 2024
0e6473e
Vote strong for transition block.
greg7mdp Jan 30, 2024
0b8bcff
lib moves forward with a `strong` initial vote, and storing `fsi` info.
greg7mdp Jan 30, 2024
f62eabf
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 30, 2024
0fe4d7c
Implement correct signature for weak votes.
greg7mdp Jan 30, 2024
e584305
Add load/save of finalizer safety info (also provide `data_dir` to co…
greg7mdp Jan 31, 2024
13c1d65
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 31, 2024
c5efd8a
Remove safety file if loading it throws.
greg7mdp Jan 31, 2024
11fc7e9
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 31, 2024
16fc872
Fix compilation issue.
greg7mdp Jan 31, 2024
3eb0a8d
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Jan 31, 2024
e3a8a1a
Check if the correct `decide_vote` is causing the tests to fail.
greg7mdp Jan 31, 2024
2a0b9c4
Minor changes.
greg7mdp Feb 1, 2024
8a87bdd
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 1, 2024
6bb808c
Update libfc bls and pending_qc APIs to take `std::span` instead of v…
greg7mdp Feb 1, 2024
21aa913
Correct support for weak digest.
greg7mdp Feb 1, 2024
b0f9dba
Update load/save of finalizer safety file so we keep all existing saf…
greg7mdp Feb 1, 2024
6eca9d3
provide `node_startup_time` and some formatting changes.
greg7mdp Feb 1, 2024
d35d0da
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 2, 2024
c027ebd
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 2, 2024
908f66a
Notify `finalizer_set` of transition. also use `constexpr std::array`.
greg7mdp Feb 2, 2024
a79aecc
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 2, 2024
9cf5ba5
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 2, 2024
30cfbf1
comment out test which keeps failing.
greg7mdp Feb 5, 2024
fe9bbf9
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 5, 2024
2f3a1ff
Comment out test correctly!
greg7mdp Feb 5, 2024
14dccd9
Support transition as spec'ed [here](https://hackmd.io/JKIz2TWNTq-xcW…
greg7mdp Feb 5, 2024
7a9622c
add comment
greg7mdp Feb 6, 2024
2c022c6
update setting of finalizers safety info.
greg7mdp Feb 6, 2024
77e4eed
Initialize the finalizer safety information at Leap startup if needed.
greg7mdp Feb 6, 2024
b6ae78b
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 6, 2024
9d4eac2
comment out assert
greg7mdp Feb 6, 2024
12953aa
Shouldn't throw an exception when fsi file doesn't exist.
greg7mdp Feb 6, 2024
d86a33a
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 7, 2024
c1f0a1c
Fix finalizer safety file i/o and add first test.
greg7mdp Feb 8, 2024
ffbd2be
Update finalizer tests for safety file.
greg7mdp Feb 9, 2024
c7fa6e0
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 9, 2024
f6aaad4
Fix issue with preserving safety data for inactive finalizers.
greg7mdp Feb 9, 2024
9295f86
Add nicer output in large-lib-test.py
greg7mdp Feb 9, 2024
bba1e46
comment out larger_lib_if_test.
greg7mdp Feb 9, 2024
9d6962f
Remove unneeded destructor.
greg7mdp Feb 9, 2024
2751834
Address some comments from the PR review.
greg7mdp Feb 13, 2024
74998fb
More changes from PR review.
greg7mdp Feb 13, 2024
bb06782
More changes according to PR review feedback.
greg7mdp Feb 13, 2024
f073ee7
More changes according to PR review feedback.
greg7mdp Feb 13, 2024
1400f53
cleanup `get_qc` function
greg7mdp Feb 13, 2024
f5327b9
Move `extends` to fork_database.
greg7mdp Feb 13, 2024
6216370
Use new `extends` API from fork_database.
greg7mdp Feb 13, 2024
a120592
Use variables instead of `fsi_map::value_type` to avoid a cast.
greg7mdp Feb 13, 2024
c1d3038
Undo move of `extends` to fork_database.
greg7mdp Feb 13, 2024
5761b52
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 13, 2024
b78a572
Update criteria for advancing lock in finalizer safety information.
greg7mdp Feb 14, 2024
f196668
Add `last_qc_block_timestamp` to `qc_claim_t` and `quorum_certificate`.
greg7mdp Feb 14, 2024
58bd343
Remove unneeded constructors in `quorum_certificate_extension` and `a…
greg7mdp Feb 14, 2024
226d01e
Add some extra checks for `last_qc_block_timestamp`
greg7mdp Feb 14, 2024
8b5ef60
Use `last_qc_block_timestamp` from `block_header_state_core` in `fina…
greg7mdp Feb 14, 2024
9b3b41e
Address PR comments
greg7mdp Feb 15, 2024
20534ff
Merge branch 'hotstuff_integration' of github.com:AntelopeIO/leap int…
greg7mdp Feb 15, 2024
86c48d7
Remove `block_timestamp` from `quorum_certificate`.
greg7mdp Feb 15, 2024
3921490
comment out weak-vote related tests from `unittests/finality_tests.cpp`
greg7mdp Feb 15, 2024
a512f17
Implement `extends` by using the same branch we already need to check…
greg7mdp Feb 15, 2024
0d94d71
Address PR review comments.
greg7mdp Feb 15, 2024
bc40291
Address PR comments.
greg7mdp Feb 15, 2024
9bc5937
Add descriptive comment for new file as suggested in review.
greg7mdp Feb 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions libraries/chain/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ set(CHAIN_WEBASSEMBLY_SOURCES
)

set(CHAIN_HOTSTUFF_SOURCES
hotstuff/finalizer.cpp
hotstuff/instant_finality_extension.cpp
hotstuff/hotstuff.cpp
)
Expand Down
12 changes: 7 additions & 5 deletions libraries/chain/block_header_state.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ block_header_state_core block_header_state_core::next(qc_claim_t incoming) const
return {*this};
}

EOS_ASSERT(incoming.last_qc_block_num > this->last_qc_block_num, block_validate_exception,
EOS_ASSERT(incoming.last_qc_block_num > this->last_qc_block_num &&
incoming.last_qc_block_timestamp > this->last_qc_block_timestamp, block_validate_exception,
"new last_qc_block_num ${new} must be greater than old last_qc_block_num ${old}",
("new", incoming.last_qc_block_num)("old", this->last_qc_block_num));

Expand Down Expand Up @@ -58,7 +59,8 @@ block_header_state_core block_header_state_core::next(qc_claim_t incoming) const
}

// new last_qc_block_num is always the input last_qc_block_num.
result.last_qc_block_num = incoming.last_qc_block_num;
result.last_qc_block_num = incoming.last_qc_block_num;
result.last_qc_block_timestamp = incoming.last_qc_block_timestamp;

return result;
}
Expand Down Expand Up @@ -165,7 +167,7 @@ block_header_state block_header_state::next(block_header_state_input& input) con

// Finally update block id from header
// -----------------------------------
result.id = result.header.calculate_id();
result.block_id = result.header.calculate_id();

return result;
}
Expand All @@ -182,7 +184,7 @@ block_header_state block_header_state::next(const signed_block_header& h, const
validator_t& validator) const {
auto producer = detail::get_scheduled_producer(active_proposer_policy->proposer_schedule.producers, h.timestamp).producer_name;

EOS_ASSERT( h.previous == id, unlinkable_block_exception, "previous mismatch" );
EOS_ASSERT( h.previous == block_id, unlinkable_block_exception, "previous mismatch" );
EOS_ASSERT( h.producer == producer, wrong_producer, "wrong producer specified" );
EOS_ASSERT( h.confirmed == 0, block_validate_exception, "invalid confirmed ${c}", ("c", h.confirmed) );

Expand All @@ -205,7 +207,7 @@ block_header_state block_header_state::next(const signed_block_header& h, const
auto& if_ext = std::get<instant_finality_extension>(if_entry->second);

building_block_input bb_input{
.parent_id = id,
.parent_id = block_id,
.timestamp = h.timestamp,
.producer = producer,
.new_protocol_feature_activations = std::move(new_protocol_feature_activations)
Expand Down
9 changes: 6 additions & 3 deletions libraries/chain/block_header_state_legacy.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -210,8 +210,11 @@ namespace eosio::chain {
if (new_finalizer_policy) {
new_finalizer_policy->generation = 1;
// set current block_num as qc_claim.last_qc_block_num in the IF extension
qc_claim_t initial_if_claim { .last_qc_block_num = block_num,
.last_qc_block_timestamp = timestamp,
.is_last_qc_strong = false };
emplace_extension(h.header_extensions, instant_finality_extension::extension_id(),
fc::raw::pack(instant_finality_extension{ { block_num, false }, std::move(new_finalizer_policy), {} }));
fc::raw::pack(instant_finality_extension{ initial_if_claim, std::move(new_finalizer_policy), {} }));
}

return h;
Expand Down Expand Up @@ -287,8 +290,8 @@ namespace eosio::chain {

block_header_state_legacy result( std::move( *static_cast<detail::block_header_state_legacy_common*>(this) ) );

result.id = h.calculate_id();
result.header = h;
result.id = h.calculate_id();
result.header = h;

result.header_exts = std::move(exts);

Expand Down
30 changes: 15 additions & 15 deletions libraries/chain/block_state.cpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#include <eosio/chain/block_state.hpp>
#include <eosio/chain/block_header_state_utils.hpp>
#include <eosio/chain/block_state_legacy.hpp>
#include <eosio/chain/hotstuff/finalizer.hpp>
#include <eosio/chain/exceptions.hpp>

#include <fc/crypto/bls_utils.hpp>
Expand All @@ -12,7 +13,7 @@ block_state::block_state(const block_header_state& prev, signed_block_ptr b, con
: block_header_state(prev.next(*b, pfs, validator))
, block(std::move(b))
, strong_digest(compute_finalizer_digest())
, weak_digest(compute_finalizer_digest())
, weak_digest(create_weak_digest(strong_digest))
, pending_qc(prev.active_finalizer_policy->finalizers.size(), prev.active_finalizer_policy->threshold, prev.active_finalizer_policy->max_weak_sum_before_weak_final())
{}

Expand All @@ -21,7 +22,7 @@ block_state::block_state(const block_header_state& bhs, deque<transaction_metada
: block_header_state(bhs)
, block(std::make_shared<signed_block>(signed_block_header{bhs.header})) // [greg todo] do we need signatures?
, strong_digest(compute_finalizer_digest())
, weak_digest(compute_finalizer_digest())
, weak_digest(create_weak_digest(strong_digest))
, pending_qc(bhs.active_finalizer_policy->finalizers.size(), bhs.active_finalizer_policy->threshold, bhs.active_finalizer_policy->max_weak_sum_before_weak_final())
, pub_keys_recovered(true) // probably not needed
, cached_trxs(std::move(trx_metas))
Expand All @@ -35,7 +36,7 @@ block_state::block_state(const block_header_state& bhs, deque<transaction_metada

// Used for transition from dpos to instant-finality
block_state::block_state(const block_state_legacy& bsp) {
block_header_state::id = bsp.id();
block_header_state::block_id = bsp.id();
header = bsp.header;
core.last_final_block_num = bsp.block_num(); // [if todo] instant transition is not acceptable
activated_protocol_features = bsp.activated_protocol_features;
Expand Down Expand Up @@ -80,10 +81,9 @@ std::pair<vote_status, std::optional<uint32_t>> block_state::aggregate_vote(cons

if (it != finalizers.end()) {
auto index = std::distance(finalizers.begin(), it);
const digest_type& digest = vote.strong ? strong_digest : weak_digest;
auto digest = vote.strong ? strong_digest.to_uint8_span() : std::span<const uint8_t>(weak_digest);
auto [status, strong] = pending_qc.add_vote(vote.strong,
#warning TODO change to use std::span if possible
std::vector<uint8_t>{digest.data(), digest.data() + digest.data_size()},
digest,
index,
vote.finalizer_key,
vote.sig,
Expand Down Expand Up @@ -158,22 +158,21 @@ void block_state::verify_qc(const valid_quorum_certificate& qc) const {

if( qc._weak_votes ) {
heifner marked this conversation as resolved.
Show resolved Hide resolved
pubkeys.emplace_back(aggregate_pubkeys(*qc._weak_votes));
digests.emplace_back(std::vector<uint8_t>{weak_digest.data(), weak_digest.data() + weak_digest.data_size()});
digests.emplace_back(std::vector<uint8_t>{weak_digest.begin(), weak_digest.end()});
}

// validate aggregated signature
EOS_ASSERT( fc::crypto::blslib::aggregate_verify( pubkeys, digests, qc._sig ), invalid_qc_claim, "signature validation failed" );
EOS_ASSERT( fc::crypto::blslib::aggregate_verify( pubkeys, digests, qc._sig ),
invalid_qc_claim, "signature validation failed" );
}

std::optional<quorum_certificate> block_state::get_best_qc() const {
auto block_number = block_num();
linh2931 marked this conversation as resolved.
Show resolved Hide resolved

// if pending_qc does not have a valid QC, consider valid_qc only
if( !pending_qc.is_quorum_met() ) {
if( valid_qc ) {
return quorum_certificate{ block_number, *valid_qc };
return quorum_certificate{ block_num(), *valid_qc };
} else {
return std::nullopt;;
return std::nullopt;
}
}

Expand All @@ -182,7 +181,7 @@ std::optional<quorum_certificate> block_state::get_best_qc() const {

// if valid_qc does not have value, consider valid_qc_from_pending only
if( !valid_qc ) {
return quorum_certificate{ block_number, valid_qc_from_pending };
return quorum_certificate{ block_num(), valid_qc_from_pending };
}

// Both valid_qc and valid_qc_from_pending have value. Compare them and select a better one.
Expand All @@ -191,6 +190,7 @@ std::optional<quorum_certificate> block_state::get_best_qc() const {
valid_qc->is_strong() == valid_qc_from_pending.is_strong() ?
*valid_qc : // tie broke by valid_qc
valid_qc->is_strong() ? *valid_qc : valid_qc_from_pending; // strong beats weak
return quorum_certificate{ block_number, best_qc };
}
return quorum_certificate{ block_num(), best_qc };
}

} /// eosio::chain
Loading
Loading