Steem 0.21.0

Steem 0.21.0 Release Notes

The hardfork is scheduled for Tue, 27 August 2019 15:00:00 UTC (11:00:00 EDT)

All nodes should upgrade to 0.21.0.

Reindexing

Steem 0.21.0 does require reindexing from all previous versions. If you have not upgraded to MIRA, please read the MIRA release notes on that process if you wish to run your Steem node backed by MIRA. Otherwise, a non-mira 0.21.0 is available under tag v0.21.0-no-mira.

Overview

This is a hard fork release. The primary changes in this release are the introduction of the Steem Proposal System (SPS) and the Steem Economic Improvement Proposal (EIP).

Steem Proposal System

The Steem Proposal System (SPS) is an on chain decentralized autonomous organization (DAO) that allows users to submit proposals for funding and vote on which proposals should be funded. The SPS is funded by means of inflation and donations. SBD (and only SBD) can be sent to the account steem.dao. Additionally anyone can make donations by setting the account as a comment reward beneficiary. Warning: There is no mechanism for refund from the steem.dao. Non SBD transfers will be rejected, but any SBD donated cannot be refunded.

The inflation budget has been changed to fund the SPS. The content rewards are being reduced from 75% of the budget to 65% to give 10% of the inflation budget to the SPS. The actual inflation schedule has not changed at all. The inflation schedule continues as planned. As of block 34 million, inflation was at 8.42% and will continue to narrow 0.01% every 250k blocks until it reaches a rate of 0.95%

One one-hundredth of the balance of the steem.dao account will be paid out every day in one hour intervals in order of approval. It is strongly recommended that the community creates and maintains refund proposal(s) to prevent proposals with little to no approval from receiving funding.

There are three new operations for interacting with the SPS.

struct create_proposal_operation : public base_operation
{
   account_name_type creator;
   account_name_type receiver;

   time_point_sec start_date;
   time_point_sec end_date;

   asset daily_pay;

   string subject;
   string permlink;

};

struct update_proposal_votes_operation : public base_operation
{
   account_name_type voter;

   flat_set_ex< int64_t > proposal_ids;

   bool approve = false;
};

struct remove_proposal_operation : public base_operation
{
   account_name_type proposal_owner;

   flat_set_ex< int64_t > proposal_ids;
};

When a proposal is created, a post must be specified that is linked to the proposal. It will be @creator/permlink. All proposal creators should use that post to keep their proposal up to date so that stake-holders can easily review and approve the proposal. There is a 10 SBD fee that must be paid in order to create a proposal. The fee is paid in to the SPS. A proposal can, at most, receive daily_pay per day, but if the proposal is not paid for a round either from losing approval votes or the budget not being able to cover the proposals costs, the lost pay will not be made up. Proposals are not owed anything by the blockchain until they are actually paid.

Proposal IDs are generated by the blockchain and are a global unique identifier of the proposal. The ID is returned with the proposal on all SPS API calls. Because the proposal ID is generated by the blockchain, it is strongly encouraged to use the transaction status API to wait for the transaction creating the proposal to become irreversible before approving it or else the proposal ID may change and the approval will be for the wrong proposal.

The following API calls have been added to support accessing proposal information. database_api.list_proposals, database_api.find_proposals, database_api.list_proposal_votes.

Economic Improvement Proposal

Convergent Linear Rewards Curve

We are changing the rewards curve to a convergent linear rewards curve. The new curve is of the form n^2/(n+1). It will start out super linear and then become linear as stake is concentrated on content. It will incentivize concentration of stake the same way the quadratic rewards did, but unlike quadratic rewards, this curve will not run away with the incentivization. As more stake is concentrated on content the incentivization to concentrate more stake will actually decrease to zero at infinity.

The curation reward curve has been updated alongside the content reward curve change. Steem has always derived the curation reward curve from the content reward curve to preserve certain properties. While the curve has changed, the expected behavior has not changed at all.

Curation Rewards

To encourage more thoughtful curation and user engagement, curation rewards will receive 50% of the rewards budget (up from 25%).

Downvote Pool

Each user now has a mana pool specifically for down votes. This pool is 25% of the upvote mana pool. Downvotes will consume this mana first before consuming regular mana. When voting, 1/50th of the remaining mana is used for a 100% vote. The same math is used for downvote mana but multiplied by 4 because the downvote mana pool is 25% the size of the normal mana pool. The maximum of these values is used. When both mana pools are at 100%, the values are the same. When downvote mana runs out normal mana is used in the same manner as the current behavior.

Other Changes

Reduce Curation Reverse Auction Window

In Hardfork 20 we reduced the curation reverse auction window from 30 minutes to 15 minutes. The change was received positively and the benefits of the system today are nebulous. We are reducing it further to 5 minutes.

New Account Update Operation

We are adding a new account update operation:

struct account_update2_operation : public base_operation
{
   account_name_type             account;
   optional< authority >         owner;
   optional< authority >         active;
   optional< authority >         posting;
   optional< public_key_type >   memo_key;
   string                        json_metadata;
   string                        posting_json_metadata;

   extensions_type               extensions;
};

This operation adds a new field, posting_json_metadata that can be updated using the posting key. json_metadata added to an account prior to the hardfork is copied to posting_json_metadata. posting_json_metadata is returned on the account_object in any API calls that return accounts. Apps that want to update metadata with the posting key can migrate to doing so. Apps that don't update will continue to work as they currently do.

Zero Mana Voting

Users that have 0 mana can still vote, but those votes will not receive curation rewards nor will they impact rewards.

SBD Haircut

There was a rounding error when enforcing the SBD "Haircut" rule that would allow SBD to continue to be printed at a slow rate despite the supply. This has been fixed and SBD printing can properly stop when the supply is too high. Note: SBD printing for the SPS does not follow this rule and will continue to print SBD regardless of the supply.

Comment Edit Lockout

Comments can only be created once per block but edits are just as impactful from a performance perspective and have no such lockout. We are adding a one comment edit per block lockout to accounts.

13 Week Powerdown

When calculating Steem Power power down rates, the rate was rounded down such that up to 0.000012 VESTS would withdraw on a 14th week. Those VESTS have been redistributed across the 13 week power down to eliminate the 14th week of a dust power down.

Forcibly Disconnect Peers

There was a bug in the p2p code that would not always disconnect from peers that the node wanted to disconnect from.

Steem 0.20.11

Steem Velocity 0.20.11 (MIRA) Release Notes

This is an optional update for the Steem database backend, called MIRA.

MIRA is a Multi Index Rocksdb Adapter that updates the database backend of Steem from memory mapped files to a disk log-structured merge-tree implemented via the RocksDB library.

Reindexing

0.20.11 requires reindexing if you want to use MIRA for the database backend of Steem.

Overview

MIRA is enabled via the cmake build options, ENABLE_MIRA=ON. This build option replaces the experimental build option ENABLE_STD_ALLOCATOR_SUPPORT. Due to the nature of C++ dynamically allocated data structures and the previous memory mapped database, MIRA is not compatible with the old database at runtime and must be enabled/disabled at compile time.

A reindex will be required to rebuild state in MIRA. The same command line argument, --replay-blockchain is used. Additionally, reindexing can be done entirely in memory with MIRA to improve reindex performance using the command line argument --memory-replay. This will keep all indices in memory until the end of the reindex. This can require a significant amount of memory. In the config file, memory-replay-indices can set which individual indices should be kept in memory to keep memory usage down while still benefitting from the increased performance.

RocksDB has many performance tuning options available. We have exposed a subset of those options in a new config file, database.cfg which lives in the data directory. We have provided a tuning guide to get node operators started on configuring their nodes for their hardware.

RocksDB does open a considerable number of files. You may need to configure your OS to allow for so many files to be open at once. These resources can help walk you through that process. ( OS X, Linux )

Impact

In our testing, memory requirements have decreased from 64G for consensus nodes to 16G when tuned correctly. Additionally, we were able to migrate from attached NVMe drives to network attached SSDs with similar, if not better, performance on a live node. Reindex performance with MIRA has much less variability. What this means is that it is much slower when compared to reindexing with an attached NVMe, but considerably faster than a network attached SSD. Your actual performance gain/loss may vary and are largely dependent on the hardware you were running on.

Steem 0.20.10

Steem Velocity 0.20.10 Release Notes

Reindexing

0.20.10 does not require reindexing from 0.20.9

Overview

This release is recommended for all nodes.

This release fixes a vulnerability in how the pending transaction queue is treated during block application. In the worst case, the previous behavior could result in block propagation delays and general instability/denial of service of the Steem network.

Steem 0.20.9

Steem Velocity 0.20.9 Release Notes

Reindexing

0.20.9 does not require reindexing from 0.20.8

Overview

This release fixes a security vulnerability in JSON parsing. This update is optional, but recommended for all nodes running public API endpoints or plugins that rely on custom JSON operations (i.e. follow plugin).

Steem 0.20.8

Steem Velocity 0.20.8 Release Notes

Reindexing

0.20.8 does not require reindexing from 0.20.7

Overview

This release fixes a fatal memory leak during reconstruction of the block log index. This update is optional so long as you do not need to reindex. However, if your block log index needs to be reconstructed, this update is required.

Steem 0.20.7

Steem Velocity 0.20.7 Release Notes

Reindexing

0.20.7 does not require reindexing from 0.20.6

Overview

This is a security release that only impacts serialization logic, not the serialization format or consensus logic. We recommend all node operators update to 0.20.7 as soon as it is convenient.

Steem 0.20.6

Steem Velocity 0.20.6 Release Notes

This is a bug fix release primarily targeting the RC Plugin.

Note: witness_api plugin no longer exists, and should be removed from config.ini if present.

Reindexing

• Nodes running the RC plugin need to manually force a reindex with --replay.

• Nodes that do not run the RC plugin do not need to upgrade or replay.

• At this time we recommend witnesses and API nodes upgrade.

• This is an optional release for exchanges. While the RC values will be different on 0.20.5 and 0.20.6, we have determined through our testing the difference will not be significant and should not cause problems for exchanges. If an exchange wants to update we recommend reindexing 0.20.6 up on a separate machine and switching only once the 0.20.6 node is functional.

Fixes

• RCs are no longer calculated prior to HF20 (#3007)

• Removed the old bandwidth system. This includes the removal of several API fields that were specific to bandwidth and the removal of the witness_api, which was used exclusively for returning information related to bandwidth. (#3029)

• Changes were made to the RC parameters to conserve current pricing while more aggressively protecting blockchain resources at high usage levels. (#3061)

• The state byte cost for votes has been reduced as votes are only required in consensus before a comment has been paid. (#3064)

• Negative RCs are limited to 2 hours worth of regeneration. (#3050)

• The execution time cost of custom json operations has been reduced by 20 times for all custom json operations except those used by the follow plugin. (#3027)

• RC Execution Time was not be charged against ops. This has been fixed and execution time is properly being checked. (#2972)

• Fixed command line interface compatibility issues with boolean flags. Flags work as intended on the command line but are settable to boolean values in the config file. For example, on the command line you can set --enable-stale-production or set it as enable-stale-production=true in the config. (#3025)

• The account creation fee in the command line wallet uses the correct HF 20 value. (#3019)

Steem 0.20.5

Release notes for 0.20.5

Steem Velocity 0.20.5 Release Notes

Reindexing

• Nodes that do not run the RC plugin do not need to upgrade or replay.
• Non-witness nodes that run the RC plugin should generally replay the blockchain.
• Exchanges and private full nodes used exclusively by accounts with large Steem Power balances do not urgently need to upgrade or replay [1] [2].

[1] If you choose not to upgrade, you should re-consider if you notice that transactions submitted to your node seem to frequently get "lost," i.e. not included in blocks. Upgrading to 0.20.5 should fix this issue.

[2] If you choose not to upgrade, you should re-consider if the accounts broadcasting transactions on your node have low Steem Power or RC. Upgrading to 0.20.5 should fix issues experienced by such accounts.

Witnesses

• To fix these known issues for most accounts, witnesses should upgrade immediately without replaying.
• Witnesses should replay the blockchain to retroactively fix RC of all affected accounts.

Fixes

• Fix a timing logic issue introduced when removing RC manabar regeneration cap. #2994
• Fix an issue which prevents newly created accounts from immediately receiving RC due to the account creation fee. #2997

Steem 0.20.4

Steem Velocity 0.20.4 Release Notes

This is a bug fix release regarding the rc plugin. All nodes running the rc plugin should to upgrade to 0.20.4.

Reindexing

Steem 0.20.4 requires reindexing from previous version if running the rc plugin. Please note: The witness plugin and network broadcast API specify the rc plugin as dependencies. All of the provided Docker configs run the rc plugin.

Fixes

• Virtual ops now properly appearing in get_ops_in_block api call. (#2936)
• Return the correct value for max_rcs via API calls. (#2944)
• Fixed condenser_api.get_accounts returning invalid values for voting power. (#2947) (#2962)
• Fixed new witness_properties not being reported correctly in condenser_api. (#2942)
• Properly credit burned account fees to account max RCs. (#2958)
• RC plugin enforcement now required for witnesses and broadcast nodes (#2965)
• Accounts with -100% of max RC can now regenerate RCs. (#2968)
• Increased RC parameters to better allow accounts with lower SP to transact at higher rates. (#2974)
• RCs are credited to the account on power up and incoming delegations. (#2961)

Steem 0.20.2

Steem Velocity 0.20.2 Release Notes

The hardfork is scheduled for Tue, 25 September 2018 15:00:00 UTC (11:00:00 EDT)

All nodes should to upgrade to 0.20.2.

Reindexing

Steem 0.20.2 does not require reindexing from 0.20.1. If upgrading from 0.20.0 or older, reindexing is required.

Overview

This is a hard fork release. The primary changes in this release are to the account creation and voting power systems and the addition of the resource credit system.

Steem 0.20.2 is built on top of the Appbase architectural changes released in Steem 0.19.10. If you are not familiar with the changes introduced in that release, we strongly recommend you read those release notes as well.

Steem 0.20.2 usurps 0.20.1 as the official release for Steem Velocity and contains some changes to how the RC system will behave at the time of the hardfork to create a better user experience.

Resource Credits

Steem Velocity moves beyond the bandwidth system to a an improved system based on Resource Credits (RCs). The RC Plugin defines blockchain resources and limits medium and long term use through stake based Resource Credits. Based on the usage of a particular resource, there will be a market price in RCs. When a transaction is included, the issuing account will be charged a number of RCs according to the resources consumed by the transaction. In this release those resources are history bytes, market bytes (these are analogous to the bandwidth plugin), state size, execution time, and subsidized accounts. The RC Plugin is designed to be extensible and more resources may be added if the need arises. This is the most comprehensive update to the resource management of the Steem Blockchain since the initial release in March of 2016.

There is also a new rc_api that will allow querying of various RC metrics. Those calls are get_resource_params, get_resource_pool, and find_rc_accounts.

The RC Plugin is listed as a dependency of the Witness Plugin. When HF 20 goes into effect, the Witness Plugin will automatically begin listening to the RC Plugin for bandwidth feedback instead of the existing bandwidth algorithm. For the time being, both algorithms will continue to operate but only one will be listened to for the purposes of rejecting transactions.

If, after HF 20 goes into effect, there is a problem with the RC Plugin, witnesses can go back to listening to the bandwidth algorithm by setting the following command line arguments. --witness-skip-enforce-bandwidth=false --rc-skip-reject-not-enough-rc=true. These arguments are exclusive and a steemd node will not start if they are both set to true or both set to false when running the Witness Plugin. In such an event, Steemit and the witnesses will coordinate to make the switchover.

Account Creation Changes

Account Claiming/Creating

There are a new pair of operations, claim_account_operation and create_claimed_account_operation that upgrade the account creation process. When combined, these operations work identically to the current account_create_operation. When used separately, this allows an account to pool claimed accounts and create them at will later on. When combined with subsidized account creation, this will lead to the smoothest user experience by allowing faucets to buffer subsidized accounts at a steady rate and consume them sporadically to meet user demand.

It is recommended that all faucets upgrade to use this creation flow as any possible future updates to account creation are more likely than not to be implemented using these operations.

struct claim_account_operation
{
   account_name_type creator;
   asset             fee;
   extensions_type   extensions;
};

struct create_claimed_account_operation
{
   account_name_type creator;
   account_name_type new_account_name;
   authority         owner;
   authority         active;
   authority         posting;
   public_key_type   memo_key;
   string            json_metadata;
   extensions_type   extensions;
};

Subsidized Account Creation

After HF 20 goes into effect, the blockchain will begin limited subsidization of account creations. A subsidized account can be created by paying a fee of 0 STEEM with the claim_account_operation. There are consensus and non-consensus limits on subsidized account creation. There are two primary consensus mechanisms. The first is a global pool of subsidized accounts that is enforced in consensus. This prevents unlimited creation when to pool is empty. The dynamics of the pool are controlled by the witnesses and discussed later. The second is a per witness consumption that prevents a compromised witness from draining the subsidized account pool. The non-consensus limit exists in the RC Plugin and charges account creators RCs to create subsidized accounts. The consensus supply is used as input to determine the cost in RCs of creating a subsidized account.

Account Creation with Delegation

The account_create_with_delegation_operation is deprecated and as of HF 20 will stop working. This was an initial attempt at cheaper account creation and is eclipsed by subsidized account creation.

Burned Account Creation Fee

The account creation fee is now burned upon account creation instead of being powered up to the new account. To compensate for the reduction in available RCs, each new account is granted a bump to their RCs proportional to the burned fee. This is retroactively applied to all accounts created prior to HF 20.

Minimum Power Down Restriction

We are removing the minimum power down restriction that prevents users from powering down their Steem Power until they have earned a certain amount. This was created to prevent new accounts from instantly powering down the STEEM used to fund the account. Because STEEM is now burned when an account is created, this restriction is not needed.

Witness Changes

Set Properties Operation

There are a number of new properties added in Steem 0.20.2 and the existing witness_update_operation does not support extension. witness_set_properties_operation is an extensible replacement that supports new properties added in 0.20.2 and can support properties that may be added in later releases.

Documentation on witness properties can be found here.

struct witness_set_properties_operation : public base_operation
{
   account_name_type                   owner;
   flat_map< string, vector< char > >  props;
   extensions_type                     extensions;

Account Creation Fee Change

When account_create_with_delegation_operation was introduced, we changed the semantics of the account creation fee. In Steem 0.20.2 we are reverting the semantics back to the original. After HF 20, the account creation fee is simply the required STEEM to create an account and the complications that were added will be removed. To ease this transition, at the time of the hardfork, all elected witnesses will have their account creation fees increased by 30x. The amount of STEEM required to make an account before and after the hardfork will not actually change. Witnesses are required to update their infrastructure to account for the change after the hardfork, but the blockchain will ease the transition.

Price Feed Format

It has already become customary to report price feeds as an SBD:STEEM pair. HF 20 will require the price feed to be reported this way for consistency and optimization.

Upper Limit on Votable Max Block Size

As a restriction of the p2p code, the highest votable max block size will be limited to 2MiB.

Voting Power Changes

Manabar

The way voting power is now tracked is via a "manabar". Rather than tracking percentage voting power remaining, it is now tracked as unspent rshares, also called mana. Your max mana is a function of your SP, delegations, and active power down. Mana regenerates proportional to your max mana and is updated whenever there is a change to your max mana or a vote takes place. This lazy updating is important for API consumers to be aware of. However, it is easy to extrapolate current mana. Mana is regenerated from 0 to 100% of max linearly over 5 days. The manabar records when it was last updated. The current mana is last_mana + (now - last_update) * max_mana / 5 days. This is, of course, capped by max mana.

Per Satoshi Steem Power

This change is largely motivated by a refinement of how we think about Steem Power in terms of voting power. Some voting exploits relied on this lack of a foundation to vote with more stake than an account should have. While it would be easy to assume that any maximally acting actor should use the exploit, we don't want users of Steemit.com, DTube, Busy, and other Steem DAPPS to be at a disadvantage because they don't use these exploits. What we came up with, or rather clarified internally, is a set of base rules defining how we treat Steem Power in this regard. Each satoshi of Steem has power associated with it regardless of whether it is powered up or not. That power is used for all stake based actions such as voting and RCs. The power regenerates linearly from 0 to 100% over five days. All liquid STEEM is treated as having 100% power even though it is not used for stake based calculations. The result of this decision is that when you power up STEEM, you instantly have access to it at 100% power, but the STEEM must be powered back up to 100% before it becomes liquid. To do this, the week before STEEM is powered down from Steem Power to liquid STEEM, it is not used for voting. This gives the STEEM seven days to regenerate its power.

This distinction is important and is a departure from the current rules. Currently, new Steem Power inherits the voting power of...