Merge branch 'main' into block-simulator-service

.github/ISSUE_TEMPLATE/release-checklist.md

@@ -7,41 +7,42 @@ assignees: ''
 
 ---
 
-- [ ] Confirm anything outstanding for release with other maintainers on #besu-release in Discord
+- [ ] Confirm at least 24 hours prior anything outstanding for release with other maintainers on #besu-release in Discord
 - [ ] Update changelog if necessary, and merge a PR for it to main
   - [ ] Notify maintainers about updating changelog for in-flight PRs 
 - [ ] Optional: for hotfixes, create a release branch and cherry-pick, e.g. `release-<version>-hotfix`
   - [ ] Optional: for hotfixes, create a PR into main from the hotfix branch to see the CI checks pass
 - [ ] On the appropriate branch/commit, create a calver tag for the release candidate, format example: `24.4.0-RC1`
-  - [ ] git tag 24.4.0-RC1
-  - [ ] git push upstream 24.4.0-RC1
+  - [ ] `git tag 24.4.0-RC1`
+  - [ ] `git push upstream 24.4.0-RC1`
 - [ ] Sign-off with team; announce the tag in #besu-release in Discord
   - [ ] Targeting this tag for the burn-in: https://github.com/hyperledger/besu/releases/tag/24.4.0-RC1
 - [ ] Consensys staff start burn-in using this tag
 - [ ] Seek sign off for burn-in
   - [ ] Pass? Go ahead and complete the release process
   - [ ] Fail? Put a message in #besu-release in Discord indicating the release will be aborted because it failed burn-in 
 - [ ] Optional: Perform a dry run with https://github.com/consensys/protocols-release-sandbox to test the workflows
-  - [ ] Sync fork
-  - [ ] git checkout <sha of 24.4.0-RC1>
-  - [ ] git tag 24.4.0
-  - [ ] git push origin 24.4.0
+  - [ ] Sync fork in github
+  - [ ] `git checkout <sha of 24.4.0-RC1>`
+  - [ ] `git tag 24.4.0`
+  - [ ] `git push <your remote name> 24.4.0`
   - [ ] Manually run https://github.com/Consensys/protocols-release-sandbox/actions/workflows/draft-release.yml using `main` branch and `24.4.0` tag
 - [ ] Back on besu, using the same git sha as 24.4.0-RC1, create a calver tag for the FULL RELEASE, example format `24.4.0`
-  - [ ] git checkout 24.4.0-RC1
-  - [ ] git tag 24.4.0
-  - [ ] git push upstream 24.4.0
+  - [ ] `git checkout 24.4.0-RC1`
+  - [ ] `git tag 24.4.0`
+  - [ ] `git push upstream 24.4.0`
 - [ ] Manually run https://github.com/hyperledger/besu/actions/workflows/draft-release.yml using `main` branch` and the FULL RELEASE tag name, i.e. `24.4.0`. Note, this workflow should always be run from `main` branch (hotfix tags will still be released even if they were created based on another branch)
     - publishes artefacts and version-specific docker tags but does not fully publish the GitHub release so subscribers are not yet notified
 - [ ] Check all draft-release workflow jobs went green
 - [ ] Check binary SHAs are correct on the release page
+- [ ] Check artifacts exist in https://hyperledger.jfrog.io/ui/repos/tree/General/besu-maven
 - [ ] Update release notes in the GitHub draft release, save draft and sign-off with team
 - [ ] Publish draft release ensuring it is marked as latest release (if appropriate)
     - this is now public and notifies subscribed users
     - makes the release "latest" in github
     - publishes the docker `latest` tag variants
-- [ ] Create homebrew release using [update-version workflow](https://github.com/hyperledger/homebrew-besu/actions/workflows/update-version.yml)
+- [ ] Create homebrew release PR using [update-version workflow](https://github.com/hyperledger/homebrew-besu/actions/workflows/update-version.yml)
   - If the PR has not been automatically created, create the PR manually using the created branch `update-<version>`
-  - Run commands `brew tap hyperledger/besu && brew install besu` on MacOSX and verify latest version has been installed
+- [ ] Verify homebrew release once the PR has merged using `brew tap hyperledger/besu && brew install besu` on MacOSX to verify latest version has been installed
 - [ ] Delete the burn-in nodes (unless required for further analysis eg performance)
 - [ ] Social announcements

CHANGELOG.md

@@ -3,23 +3,41 @@
 ## Unreleased
 
 ### Breaking Changes
+- `--host-whitelist` has been deprecated since 2020 and this option is removed. Use the equivalent `--host-allowlist` instead. 
 
 ### Upcoming Breaking Changes
-- Plugin API will be deprecating the BesuContext interface to be replaced with the ServiceManager interface.
 - `MetricSystem::createLabelledGauge` is deprecated and will be removed in a future release, replace it with `MetricSystem::createLabelledSuppliedGauge`
-- k8s (KUBERNETES) Nat method is now deprecated and will be removed in a future release
-- `--host-whitelist` has been deprecated in favor of `--host-allowlist` since 2020 and will be removed in a future release
+- k8s (KUBERNETES) Nat method is now deprecated and will be removed in a future release. Use docker or none instead.
 - Sunsetting features - for more context on the reasoning behind the deprecation of these features, including alternative options, read [this blog post](https://www.lfdecentralizedtrust.org/blog/sunsetting-tessera-and-simplifying-hyperledger-besu)
   - Tessera privacy
   - Smart-contract-based (onchain) permissioning
   - Proof of Work consensus
   - Fast Sync
 
 ### Additions and Improvements
-- Proper support for `pending` block tag when calling `eth_estimateGas` and `eth_createAccessList` [#7951](https://github.com/hyperledger/besu/pull/7951)
+- Add RPC HTTP options to specify custom truststore and its password [#7978](https://github.com/hyperledger/besu/pull/7978)
+- Retrieve all transaction receipts for a block in one request [#6646](https://github.com/hyperledger/besu/pull/6646)
+
 
 ### Bug fixes
-- Correct default parameters for frontier transactions in `eth_call` and `eth_estimateGas` [#7965](https://github.com/hyperledger/besu/pull/7965)
+- Fix serialization of state overrides when `movePrecompileToAddress` is present [#8204](https://github.com/hyperledger/besu/pull/8024)
+- Revise the approach for setting level_compaction_dynamic_level_bytes RocksDB configuration option [#8037](https://github.com/hyperledger/besu/pull/8037)
+
+## 24.12.2 Hotfix
+
+This is an optional hotfix to address serialization of state overrides parameter when `movePrecompileToAddress` is present. 
+
+There is no need to upgrade from 24.12.0 (or 24.12.1) to this release if you are not yet using this functionality.
+
+### Bug fixes
+- Fix serialization of state overrides when `movePrecompileToAddress` is present [#8204](https://github.com/hyperledger/besu/pull/8024)
+
+## 24.12.1 Hotfix
+
+This is a hotfix to address publishing besu maven artifacts.  There are no issues with 24.12.0 other than incomplete artifact publishing, and there is no functional difference between 24.12.0 and 24.12.1 release binaries.
+
+### Bug fixes
+- Fix BOM pom publication to Artifactory [#8201](https://github.com/hyperledger/besu/pull/8021)
 
 ## 24.12.0
 
@@ -68,11 +86,13 @@
 - Prometheus Java Metrics library upgraded to version 1.3.3 [#7880](https://github.com/hyperledger/besu/pull/7880)
 - Add histogram to Prometheus metrics system [#7944](https://github.com/hyperledger/besu/pull/7944)
 - Improve newPayload and FCU logs [#7961](https://github.com/hyperledger/besu/pull/7961)
+- Proper support for `pending` block tag when calling `eth_estimateGas` and `eth_createAccessList` [#7951](https://github.com/hyperledger/besu/pull/7951)
 
 ### Bug fixes
 - Fix registering new metric categories from plugins [#7825](https://github.com/hyperledger/besu/pull/7825)
 - Fix CVE-2024-47535 [7878](https://github.com/hyperledger/besu/pull/7878)
 - Fix QBFT prepared block based proposal validation [#7875](https://github.com/hyperledger/besu/pull/7875)
+- Correct default parameters for frontier transactions in `eth_call` and `eth_estimateGas` [#7965](https://github.com/hyperledger/besu/pull/7965)
 - Correctly parse nonce as hex in `eth_call` account overrides [#7999](https://github.com/hyperledger/besu/pull/7999)
 
 ## 24.10.0

acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/condition/clique/CliqueConditions.java

@@ -19,7 +19,7 @@
 import static org.hyperledger.besu.tests.acceptance.dsl.transaction.clique.CliqueTransactions.LATEST;
 
 import org.hyperledger.besu.config.CliqueConfigOptions;
-import org.hyperledger.besu.config.GenesisConfigFile;
+import org.hyperledger.besu.config.GenesisConfig;
 import org.hyperledger.besu.datatypes.Address;
 import org.hyperledger.besu.tests.acceptance.dsl.condition.Condition;
 import org.hyperledger.besu.tests.acceptance.dsl.condition.blockchain.ExpectBlockNotCreated;
@@ -89,9 +89,9 @@ public Condition awaitSignerSetChange(final Node node) {
 
   private int cliqueBlockPeriod(final BesuNode node) {
     final String config = node.getGenesisConfigProvider().create(emptyList()).get();
-    final GenesisConfigFile genesisConfigFile = GenesisConfigFile.fromConfig(config);
+    final GenesisConfig genesisConfig = GenesisConfig.fromConfig(config);
     final CliqueConfigOptions cliqueConfigOptions =
-        genesisConfigFile.getConfigOptions().getCliqueConfigOptions();
+        genesisConfig.getConfigOptions().getCliqueConfigOptions();
     return cliqueConfigOptions.getBlockPeriodSeconds();
   }
 

acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java

@@ -25,7 +25,7 @@
 import org.hyperledger.besu.cli.config.EthNetworkConfig;
 import org.hyperledger.besu.cli.config.NetworkName;
 import org.hyperledger.besu.components.BesuComponent;
-import org.hyperledger.besu.config.GenesisConfigFile;
+import org.hyperledger.besu.config.GenesisConfig;
 import org.hyperledger.besu.controller.BesuController;
 import org.hyperledger.besu.controller.BesuControllerBuilder;
 import org.hyperledger.besu.crypto.KeyPairUtil;
@@ -155,8 +155,8 @@ public void startNode(final BesuNode node) {
     networkConfigBuilder.setBootNodes(bootnodes);
     node.getConfiguration()
         .getGenesisConfig()
-        .map(GenesisConfigFile::fromConfig)
-        .ifPresent(networkConfigBuilder::setGenesisConfigFile);
+        .map(GenesisConfig::fromConfig)
+        .ifPresent(networkConfigBuilder::setGenesisConfig);
     final EthNetworkConfig ethNetworkConfig = networkConfigBuilder.build();
     final BesuControllerBuilder builder = component.besuControllerBuilder();
     builder.isRevertReasonEnabled(node.isRevertReasonEnabled());
@@ -166,9 +166,7 @@ public void startNode(final BesuNode node) {
     builder.nodeKey(new NodeKey(new KeyPairSecurityModule(KeyPairUtil.loadKeyPair(dataDir))));
     builder.privacyParameters(node.getPrivacyParameters());
 
-    node.getGenesisConfig()
-        .map(GenesisConfigFile::fromConfig)
-        .ifPresent(builder::genesisConfigFile);
+    node.getGenesisConfig().map(GenesisConfig::fromConfig).ifPresent(builder::genesisConfig);
 
     final BesuController besuController = component.besuController();
 

acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/clique/CliqueProposeRpcAcceptanceTest.java

@@ -21,6 +21,7 @@
 
 import java.io.IOException;
 
+import org.junit.jupiter.api.Disabled;
 import org.junit.jupiter.api.Test;
 
 public class CliqueProposeRpcAcceptanceTest extends AcceptanceTestBase {
@@ -59,6 +60,7 @@ public void shouldRemoveValidators() throws IOException {
     cluster.verify(clique.validatorsEqual(minerNode1, minerNode2));
   }
 
+  @Disabled
   @Test
   public void shouldNotAddValidatorWhenInsufficientVotes() throws IOException {
     final String[] initialValidators = {"miner1", "miner2"};
@@ -90,6 +92,7 @@ public void shouldNotRemoveValidatorWhenInsufficientVotes() throws IOException {
     cluster.verify(clique.validatorsEqual(minerNode1, minerNode2, minerNode3));
   }
 
+  @Disabled
   @Test
   public void shouldIncludeVoteInBlockHeader() throws IOException {
     final String[] initialValidators = {"miner1", "miner2"};

benchmark/docker-compose.yml

@@ -66,7 +66,7 @@ services:
     - --rpc-ws-enabled
     - --rpc-ws-apis=admin,eth,miner,web3,net,priv,eea
     - --rpc-ws-host=0.0.0.0
-    - --host-whitelist=*
+    - --host-allowlist=*
     - --graphql-http-enabled
     - --discovery-enabled=false
     - --privacy-enabled=true
@@ -84,7 +84,7 @@ services:
     - --rpc-ws-enabled
     - --rpc-ws-apis=admin,eth,miner,web3,net,priv,eea
     - --rpc-ws-host=0.0.0.0
-    - --host-whitelist=*
+    - --host-allowlist=*
     - --graphql-http-enabled
     - --discovery-enabled=false
     - --privacy-enabled=true

besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

@@ -85,7 +85,7 @@
 import org.hyperledger.besu.cli.util.VersionProvider;
 import org.hyperledger.besu.components.BesuComponent;
 import org.hyperledger.besu.config.CheckpointConfigOptions;
-import org.hyperledger.besu.config.GenesisConfigFile;
+import org.hyperledger.besu.config.GenesisConfig;
 import org.hyperledger.besu.config.GenesisConfigOptions;
 import org.hyperledger.besu.config.MergeConfiguration;
 import org.hyperledger.besu.controller.BesuController;
@@ -334,8 +334,8 @@ public class BesuCommand implements DefaultCommandValues, Runnable {
       new PreSynchronizationTaskRunner();
 
   private final Set<Integer> allocatedPorts = new HashSet<>();
-  private final Supplier<GenesisConfigFile> genesisConfigFileSupplier =
-      Suppliers.memoize(this::readGenesisConfigFile);
+  private final Supplier<GenesisConfig> genesisConfigSupplier =
+      Suppliers.memoize(this::readGenesisConfig);
   private final Supplier<GenesisConfigOptions> genesisConfigOptionsSupplier =
       Suppliers.memoize(this::readGenesisConfigOptions);
   private final Supplier<MiningConfiguration> miningParametersSupplier =
@@ -612,14 +612,6 @@ static class PrivacyOptionGroup {
       defaultValue = "localhost,127.0.0.1")
   private final JsonRPCAllowlistHostsProperty hostsAllowlist = new JsonRPCAllowlistHostsProperty();
 
-  @Option(
-      names = {"--host-whitelist"},
-      hidden = true,
-      paramLabel = "<hostname>[,<hostname>...]... or * or all",
-      description =
-          "Deprecated in favor of --host-allowlist. Comma separated list of hostnames to allow for RPC access, or * to accept any host (default: ${DEFAULT-VALUE})")
-  private final JsonRPCAllowlistHostsProperty hostsWhitelist = new JsonRPCAllowlistHostsProperty();
-
   @SuppressWarnings({"FieldCanBeFinal", "FieldMayBeFinal"})
   @Option(
       names = {"--color-enabled"},
@@ -1606,21 +1598,21 @@ private void validateChainDataPruningParams() {
     }
   }
 
-  private GenesisConfigFile readGenesisConfigFile() {
-    GenesisConfigFile effectiveGenesisFile;
+  private GenesisConfig readGenesisConfig() {
+    GenesisConfig effectiveGenesisFile;
     effectiveGenesisFile =
         network.equals(EPHEMERY)
             ? EphemeryGenesisUpdater.updateGenesis(genesisConfigOverrides)
             : genesisFile != null
-                ? GenesisConfigFile.fromSource(genesisConfigSource(genesisFile))
-                : GenesisConfigFile.fromResource(
+                ? GenesisConfig.fromSource(genesisConfigSource(genesisFile))
+                : GenesisConfig.fromResource(
                     Optional.ofNullable(network).orElse(MAINNET).getGenesisFile());
     return effectiveGenesisFile.withOverrides(genesisConfigOverrides);
   }
 
   private GenesisConfigOptions readGenesisConfigOptions() {
     try {
-      return genesisConfigFileSupplier.get().getConfigOptions();
+      return genesisConfigSupplier.get().getConfigOptions();
     } catch (final Exception e) {
       throw new ParameterException(
           this.commandLine, "Unable to load genesis file. " + e.getCause());
@@ -1712,15 +1704,6 @@ private void configure() throws Exception {
             unstableIpcOptions.getRpcIpcApis());
     inProcessRpcConfiguration = inProcessRpcOptions.toDomainObject();
     dataStorageConfiguration = getDataStorageConfiguration();
-    // hostsWhitelist is a hidden option. If it is specified, add the list to hostAllowlist
-    if (!hostsWhitelist.isEmpty()) {
-      // if allowlist == default values, remove the default values
-      if (hostsAllowlist.size() == 2
-          && hostsAllowlist.containsAll(List.of("localhost", "127.0.0.1"))) {
-        hostsAllowlist.removeAll(List.of("localhost", "127.0.0.1"));
-      }
-      hostsAllowlist.addAll(hostsWhitelist);
-    }
 
     permissioningConfiguration = permissioningConfiguration();
     staticNodes = loadStaticNodes();
@@ -2365,7 +2348,7 @@ private EthNetworkConfig updateNetworkConfig(final NetworkName network) {
       builder.setDnsDiscoveryUrl(null);
     }
 
-    builder.setGenesisConfigFile(genesisConfigFileSupplier.get());
+    builder.setGenesisConfig(genesisConfigSupplier.get());
 
     if (networkId != null) {
       builder.setNetworkId(networkId);