Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a mask for BLS aggregation and improve caching #604

Merged
merged 3 commits into from
Sep 23, 2024
Merged

Conversation

Stebalien
Copy link
Member

@Stebalien Stebalien commented Aug 29, 2024

Design:

  1. Include all public keys present in the power table in the BDN mask.
  2. Only select the relevant ones (via the bitmask) when aggregating signatures.
  3. Pre-compute the terms used in public key aggregation, making it much faster (it's a simple summation).
  4. Partially pre-compute the coefficients used in signature aggregation. It's slightly faster (can avoid some hashing) but not that much faster because we still need to compute the terms (multiply the coefficients by the signatures being aggregated).

There are three commits here and this PR should be merged by rebase, not squash.

  1. The first commit switches from drand's kyber fork back to dedis.
  2. The second imports the BDN and GNARK modules, along with the following dedis PRs:
    1. Optimize BDN Signature/Key Aggregation dedis/kyber#546
    2. Add gnark as bls-12-381 backend dedis/kyber#551
    3. Use the group-specific scalar type when hashing in BDN dedis/kyber#553
  3. The final commit actually makes use of the caching mask.

The code imported in commit 2 should be (nearly) identical to the code in the bdn and gnark packages in https://github.com/Stebalien/kyber/tree/f3.

fixes #592

Comment on lines +103 to 143
type Aggregate interface {
// Aggregates signatures from a participants.
//
// Implementations must be safe for concurrent use.
Aggregate(signerMask []int, sigs [][]byte) ([]byte, error)
// VerifyAggregate verifies an aggregate signature.
//
// Implementations must be safe for concurrent use.
VerifyAggregate(signerMask []int, payload, aggSig []byte) error
}

type Verifier interface {
// Verifies a signature for the given public key.
//
// Implementations must be safe for concurrent use.
Verify(pubKey PubKey, msg, sig []byte) error
// Aggregates signatures from a participants.
Aggregate(pubKeys []PubKey, sigs [][]byte) ([]byte, error)
// VerifyAggregate verifies an aggregate signature.
// Return an Aggregate that can aggregate and verify aggregate signatures made by the given
// public keys.
//
// Implementations must be safe for concurrent use.
VerifyAggregate(payload, aggSig []byte, signers []PubKey) error
Aggregate(pubKeys []PubKey) (Aggregate, error)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interface changes. Basically:

  1. You call Verifier.Aggregate(allPublicKeys).
  2. Then you call either Aggregate (name clash) or VerifyAggregate on the resulting Aggregate (naming?).

gpbft/participant.go Outdated Show resolved Hide resolved
Comment on lines 490 to 499
// TODO: this is slow and under a lock, but we only want to do it once per
// instance... ideally we'd have a per-instance lock/once, but that probably isn't
// worth it.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, punt till later (probably not much worse than fetching the committee under the lock).

@Stebalien Stebalien force-pushed the steb/bls-mask branch 3 times, most recently from fa0a974 to 0860f44 Compare August 29, 2024 17:00
@Stebalien Stebalien requested review from masih and Kubuxu August 29, 2024 17:05
@Stebalien Stebalien marked this pull request as ready for review August 29, 2024 17:06
@Stebalien
Copy link
Member Author

This is ready for review although still WIP until the upstream changes (drand/kyber#61) land. Reviews there would also be helpful (only look at the last two commits, everything else is from a previous PR).

Copy link

codecov bot commented Aug 29, 2024

Codecov Report

Attention: Patch coverage is 58.80452% with 255 lines in your changes missing coverage. Please review.

Project coverage is 76.90%. Comparing base (d392bbf) to head (f5af3ff).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
internal/bls/gnark/gt.go 7.69% 47 Missing and 1 partial ⚠️
internal/bls/gnark/scalar.go 21.05% 45 Missing ⚠️
internal/bls/gnark/g1.go 44.15% 42 Missing and 1 partial ⚠️
internal/bls/gnark/g2.go 51.94% 36 Missing and 1 partial ⚠️
internal/bls/bdn/bdn.go 75.55% 10 Missing and 12 partials ⚠️
internal/bls/gnark/suite.go 47.61% 21 Missing and 1 partial ⚠️
emulator/signing.go 47.82% 9 Missing and 3 partials ⚠️
blssig/aggregation.go 68.96% 6 Missing and 3 partials ⚠️
internal/bls/bdn/mask.go 96.00% 2 Missing and 2 partials ⚠️
internal/bls/gnark/adapter.go 66.66% 4 Missing ⚠️
... and 4 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #604      +/-   ##
==========================================
- Coverage   79.47%   76.90%   -2.58%     
==========================================
  Files          55       64       +9     
  Lines        4907     5451     +544     
==========================================
+ Hits         3900     4192     +292     
- Misses        631      857     +226     
- Partials      376      402      +26     
Files with missing lines Coverage Δ
blssig/signer.go 77.77% <100.00%> (ø)
blssig/verifier.go 68.00% <100.00%> (-4.55%) ⬇️
emulator/host.go 79.10% <100.00%> (+0.31%) ⬆️
emulator/instance.go 95.74% <100.00%> (+1.23%) ⬆️
gpbft/gpbft.go 88.05% <100.00%> (-0.04%) ⬇️
gpbft/powertable.go 85.07% <100.00%> (+0.57%) ⬆️
gpbft/participant.go 87.85% <75.00%> (+0.63%) ⬆️
certs/certs.go 93.82% <71.42%> (-1.11%) ⬇️
host.go 70.19% <75.00%> (-0.35%) ⬇️
internal/bls/bdn/mask.go 96.00% <96.00%> (ø)
... and 10 more

... and 3 files with indirect coverage changes

@masih masih changed the title [WIP] Use a mask for BLS aggregation and improve caching Use a mask for BLS aggregation and improve caching Aug 29, 2024
@masih masih changed the title Use a mask for BLS aggregation and improve caching [WIP] Use a mask for BLS aggregation and improve caching Aug 29, 2024
@masih
Copy link
Member

masih commented Aug 29, 2024

although still WIP until the upstream changes

Woops; sorry for PR title change noise.

Copy link
Contributor

@Kubuxu Kubuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach SGWM

@Stebalien
Copy link
Member Author

Ok, waiting on dedis/kyber#546 now.

@Stebalien Stebalien force-pushed the steb/bls-mask branch 2 times, most recently from 1c1c3ad to fdee71b Compare September 5, 2024 23:20
@Stebalien Stebalien changed the title [WIP] Use a mask for BLS aggregation and improve caching Use a mask for BLS aggregation and improve caching Sep 22, 2024
@Stebalien Stebalien force-pushed the steb/bls-mask branch 3 times, most recently from be0652f to 1c17db1 Compare September 22, 2024 14:32
@Stebalien Stebalien requested a review from Kubuxu September 22, 2024 14:33
byteIndex := i / 8
mask := byte(1) << uint(i&7)
if (m.mask[byteIndex] & mask) != 0 {
count++
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just popcount but doens't matter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make a PR upstream.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. Later. I have too many stacked.

It's probably going to take a while for upstream to merge the changes so
we're importing just the changed package (BDN) and the new
package (Gnark) into this repo. That way we avoid forking the entire
repo but can still import our changes.

Any changes to these pacakges should be submitted as PRs to upstream
_first_, then backported to this repo.

Includes:

- dedis/kyber#546
- dedis/kyber#551
- dedis/kyber#553
@Stebalien Stebalien merged commit f5af3ff into main Sep 23, 2024
11 checks passed
@Stebalien Stebalien deleted the steb/bls-mask branch September 23, 2024 11:57
@Stebalien
Copy link
Member Author

Merged by rebase to preserve the three separate commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change BLS signature/public key aggregation to enable better caching
3 participants