44 lovelace missing in calculated fee (12.0.0-beta3)

[kxbt-del]

Heya, I'm testing 12.0.0-beta3 transactions against a Preview 9.1.0 node.

After upgrading all the breaking changes, submitting a minting tx results in a 44 lovelace deficit in calculated fees. I'm using add_change_if_needed to calculate the fees.
Am I still missing something here or is there some kind of a bug present?

Error:

GenericError "\"Submitting TX failed due to error: ShelleyTxValidationError ShelleyBasedEraConway (ApplyTxError (ConwayUtxowFailure (UtxoFailure (FeeTooSmallUTxO (Coin 264136) (Coin 264092))) :| []))\"", TxSubmissionFailedError, 5a34fc0b3f0ded5a0e569c4d8c65db4cf967fc409ac78d44734df057a8b1e1c1, GenericError "\"Submitting TX failed due to error: ShelleyTxValidationError ShelleyBasedEraConway (ApplyTxError (ConwayUtxowFailure (UtxoFailure (FeeTooSmallUTxO (Coin 264136) (Coin 264092))) :| []))\"

Transaction:

84aa0082825820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9018258207b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e030184a3005839109e9dfb8ca4978774ef4b0b03705c73868cba2a2fda522836d845ac31af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e709701821a0233a6eca1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c5701028201d81858acd8799fa200a140a1401a004c4b4001a09fd8799f4040ffd8799f581cc82a4452eaebccb82aced501b3c94d3662cf6cd2915ad7148b459aec4341584fffffd8799f581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae45204953505f58724f414c57ffa347656e6444617465d905009f1b000003bbcc320998ff457072696365d87b9fd8799f011864ffff49737461727444617465d905009f1b0000009393056d98ffa0ff825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e7097821a0011b0dea1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c5701825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a30082f90825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a000ecc16021a0004079c075820b4f900a1f79ca3ff7859e15051d7170706fa26d64aebbf042a4390298a3b3e1e09a1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c57020b58206878390eea04bcbd76d564d8bc7fb480522b1d09069f693e3f850256123265290d818258207b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e070e81581c1b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba410825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a00463fd6111a00060b6aa300d9010281825820aef01e9f23cc52aa8716d6168dc29559be26578bbf84c81a5c556a659f41921358407a4c4e8246076b9b6d7062d17505c875e693ae753a76f345c7a1bacdfbd55fd13c79aab514bb02e1578d9d416ad3084286df752034737491f75a8d387683720306d901028159022659022301000032323232323232323232323232223232323232533300c3370e9000001099999199111980711299980a80089128008a99980919baf301630180010041300530180011300230170010012322230020033756602c0026ea4004dd7180900099199180511998011bab001232223002003374c002244a002464a66601e6ae8c0044894004488c00800ccc02894ccc03ccdd78009ba8480004894004488c00800c004004dd598091918091809180900098088021299980699807180780099111801001a4008264649319999806111299980a00089128008a9998089801180b800899111801001980b800899801801180b0009199119baf374e60300046e9cc06000530012bd8799fd8799f5820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9ff01ff00001001200116332300c22533301300114bd700998079801980b0009801180a8009180a180a8009bac30130051533300d3300e300f0013222300200332337029000000a40082930b0b0b180980118070009baa300f300e002300f300e001300e001223300422533300b00110051323330053011300f00223300933760601c602000600200420026004601a00200297adef6c602323002233002002001230022330020020015740ae6888cc0088cc0088cdc38010008a5013300124a0294494ccc00800448940044c94ccc00c0044c888c00800cdd69804180300109128009802000aab9f5573aae895d0918011baa0015573d0581840100d87980821a00018e901a01e31d8cf5a21902a2a1636d7367817141786f3a204e65772053747261746567791902d1a178383730373935323061666261636466363662366531643361376432613138326331643464663866343035306166366130336338616534353230a16953505f58724f414c57a964616c676fa46b6465736372697074696f6e6a4d61746368206174206c646c696e6b82784068747470733a2f2f6170702e61786f2d70696c6f742e74726164652f636f6d706f7365722f64343962636538662d363864632d343633662d626437302d61613569393037376563396661646e616d65654c696d6974647574786f82784037343535396238656164363235626339663035326664323135316238396238333064313831373832366238393961376434366336626664383533663162386237622330666173736574738260783e633832613434353265616562636362383261636564353031623363393464333636326366366364323931356164373134386234353961656334313538346665696d6167657835697066733a2f2f516d5a446d79553941324a6872734d3555744447717759764b7a5468656d3952636a5a745a5569345672635a724764696e6974a26762616c616e6365a2634144411a004c4b406341584f00657374617465a0696d656469615479706569696d6167652f706e67646e616d65704c696d6974203c4144412c2041584f3e66706172616d73a367656e64446174657818323130302d30312d33315432323a35393a35392e3030305a657072696365a26b64656e6f6d696e61746f721864696e756d657261746f7201697374617274446174657818313939302d30312d33315432323a35393a35392e3030305a657374726174a36b6465736372697074696f6e60646c696e6b783568747470733a2f2f6170702e61786f2d70696c6f742e74726164652f737472617465676965732f766965772f53505f58724f414c57646e616d656067776562736974657168747470733a2f2f61786f2e7472616465

Decoded:

{
  "body": {
    "inputs": [
      {
        "transaction_id": "259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9",
        "index": 1
      },
      {
        "transaction_id": "7b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e",
        "index": 3
      }
    ],
    "outputs": [
      {
        "address": "addr_test1zz0fm7uv5jtcwa80fv9sxuzuwwrgew329ld9y2pkmpz6cvd0z27pe2v3seud09qvgzfq5xarrtqyku684424yje7wztsu3dx7z",
        "amount": {
          "coin": "36939500",
          "multiasset": {
            "7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520": {
              "53505f58724f414c57": "1"
            }
          }
        },
        "plutus_data": {
          "Data": "{\"constructor\":0,\"fields\":[{\"map\":[{\"k\":{\"int\":0},\"v\":{\"map\":[{\"k\":{\"bytes\":\"\"},\"v\":{\"map\":[{\"k\":{\"bytes\":\"\"},\"v\":{\"int\":5000000}}]}}]}},{\"k\":{\"int\":1},\"v\":{\"map\":[]}}]},{\"list\":[{\"constructor\":0,\"fields\":[{\"bytes\":\"\"},{\"bytes\":\"\"}]},{\"constructor\":0,\"fields\":[{\"bytes\":\"c82a4452eaebccb82aced501b3c94d3662cf6cd2915ad7148b459aec\"},{\"bytes\":\"41584f\"}]}]},{\"constructor\":0,\"fields\":[{\"bytes\":\"7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520\"},{\"bytes\":\"53505f58724f414c57\"}]},{\"map\":[{\"k\":{\"bytes\":\"656e6444617465\"},\"v\":{\"constructor\":7,\"fields\":[{\"int\":4105119599000}]}},{\"k\":{\"bytes\":\"7072696365\"},\"v\":{\"constructor\":2,\"fields\":[{\"constructor\":0,\"fields\":[{\"int\":1},{\"int\":100}]}]}},{\"k\":{\"bytes\":\"737461727444617465\"},\"v\":{\"constructor\":7,\"fields\":[{\"int\":633826799000}]}}]},{\"map\":[]}]}"
        },
        "script_ref": null
      },
      {
        "address": "addr_test1qqd5p62dcuafcdejcagq3w8rkcy6x3x3yzk2nju49q63hf90z27pe2v3seud09qvgzfq5xarrtqyku684424yje7wztsmn77pu",
        "amount": {
          "coin": "1159390",
          "multiasset": {
            "7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520": {
              "53505f58724f414c57": "1"
            }
          }
        },
        "plutus_data": null,
        "script_ref": null
      },
      {
        "address": "addr_test1qqd5p62dcuafcdejcagq3w8rkcy6x3x3yzk2nju49q63hf90z27pe2v3seud09qvgzfq5xarrtqyku684424yje7wztsmn77pu",
        "amount": {
          "coin": "805842832",
          "multiasset": null
        },
        "plutus_data": null,
        "script_ref": null
      },
      {
        "address": "addr_test1qqd5p62dcuafcdejcagq3w8rkcy6x3x3yzk2nju49q63hf90z27pe2v3seud09qvgzfq5xarrtqyku684424yje7wztsmn77pu",
        "amount": {
          "coin": "969750",
          "multiasset": null
        },
        "plutus_data": null,
        "script_ref": null
      }
    ],
    "fee": "264092",
    "ttl": null,
    "certs": null,
    "withdrawals": null,
    "update": null,
    "auxiliary_data_hash": "b4f900a1f79ca3ff7859e15051d7170706fa26d64aebbf042a4390298a3b3e1e",
    "validity_start_interval": null,
    "mint": [
      [
        "7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520",
        {
          "53505f58724f414c57": "2"
        }
      ]
    ],
    "script_data_hash": "6878390eea04bcbd76d564d8bc7fb480522b1d09069f693e3f85025612326529",
    "collateral": [
      {
        "transaction_id": "7b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e",
        "index": 7
      }
    ],
    "required_signers": [
      "1b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4"
    ],
    "network_id": null,
    "collateral_return": {
      "address": "addr_test1qqd5p62dcuafcdejcagq3w8rkcy6x3x3yzk2nju49q63hf90z27pe2v3seud09qvgzfq5xarrtqyku684424yje7wztsmn77pu",
      "amount": {
        "coin": "4603862",
        "multiasset": null
      },
      "plutus_data": null,
      "script_ref": null
    },
    "total_collateral": "396138",
    "reference_inputs": null,
    "voting_procedures": null,
    "voting_proposals": null,
    "donation": null,
    "current_treasury_value": null
  },
  "witness_set": {
    "vkeys": [
      {
        "vkey": "ed25519_pk14mcpa8ere3f24pck6ctgms54txlzv4uth7zvsxju244xt86pjgfsw3w44l",
        "signature": "7a4c4e8246076b9b6d7062d17505c875e693ae753a76f345c7a1bacdfbd55fd13c79aab514bb02e1578d9d416ad3084286df752034737491f75a8d3876837203"
      }
    ],
    "native_scripts": null,
    "bootstraps": null,
    "plutus_scripts": [
   "59022301000032323232323232323232323232223232323232533300c3370e9000001099999199111980711299980a80089128008a99980919baf301630180010041300530180011300230170010012322230020033756602c0026ea4004dd7180900099199180511998011bab001232223002003374c002244a002464a66601e6ae8c0044894004488c00800ccc02894ccc03ccdd78009ba8480004894004488c00800c004004dd598091918091809180900098088021299980699807180780099111801001a4008264649319999806111299980a00089128008a9998089801180b800899111801001980b800899801801180b0009199119baf374e60300046e9cc06000530012bd8799fd8799f5820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9ff01ff00001001200116332300c22533301300114bd700998079801980b0009801180a8009180a180a8009bac30130051533300d3300e300f0013222300200332337029000000a40082930b0b0b180980118070009baa300f300e002300f300e001300e001223300422533300b00110051323330053011300f00223300933760601c602000600200420026004601a00200297adef6c602323002233002002001230022330020020015740ae6888cc0088cc0088cdc38010008a5013300124a0294494ccc00800448940044c94ccc00c0044c888c00800cdd69804180300109128009802000aab9f5573aae895d0918011baa0015573d"
    ],
    "plutus_data": null,
    "redeemers": [
      {
        "tag": "Mint",
        "index": "0",
        "data": "{\"constructor\":0,\"fields\":[]}",
        "ex_units": {
          "mem": "102032",
          "steps": "31661452"
        }
      }
    ]
  },
  "is_valid": true,
  "auxiliary_data": {
    "metadata": {
      "674": "{\"map\":[{\"k\":{\"string\":\"msg\"},\"v\":{\"list\":[{\"string\":\"Axo: New Strategy\"}]}}]}",
      "721": "{\"map\":[{\"k\":{\"string\":\"7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520\"},\"v\":{\"map\":[{\"k\":{\"string\":\"SP_XrOALW\"},\"v\":{\"map\":[{\"k\":{\"string\":\"algo\"},\"v\":{\"map\":[{\"k\":{\"string\":\"description\"},\"v\":{\"string\":\"Match at l\"}},{\"k\":{\"string\":\"link\"},\"v\":{\"list\":[{\"string\":\"https://app.axo-pilot.trade/composer/d49bce8f-68dc-463f-bd70-aa5\"},{\"string\":\"9077ec9fa\"}]}},{\"k\":{\"string\":\"name\"},\"v\":{\"string\":\"Limit\"}},{\"k\":{\"string\":\"utxo\"},\"v\":{\"list\":[{\"string\":\"74559b8ead625bc9f052fd2151b89b830d1817826b899a7d46c6bfd853f1b8b7\"},{\"string\":\"#0\"}]}}]}},{\"k\":{\"string\":\"assets\"},\"v\":{\"list\":[{\"string\":\"\"},{\"string\":\"c82a4452eaebccb82aced501b3c94d3662cf6cd2915ad7148b459aec41584f\"}]}},{\"k\":{\"string\":\"image\"},\"v\":{\"string\":\"ipfs://QmZDmyU9A2JhrsM5UtDGqwYvKzThem9RcjZtZUi4VrcZrG\"}},{\"k\":{\"string\":\"init\"},\"v\":{\"map\":[{\"k\":{\"string\":\"balance\"},\"v\":{\"map\":[{\"k\":{\"string\":\"ADA\"},\"v\":{\"int\":5000000}},{\"k\":{\"string\":\"AXO\"},\"v\":{\"int\":0}}]}},{\"k\":{\"string\":\"state\"},\"v\":{\"map\":[]}}]}},{\"k\":{\"string\":\"mediaType\"},\"v\":{\"string\":\"image/png\"}},{\"k\":{\"string\":\"name\"},\"v\":{\"string\":\"Limit <ADA, AXO>\"}},{\"k\":{\"string\":\"params\"},\"v\":{\"map\":[{\"k\":{\"string\":\"endDate\"},\"v\":{\"string\":\"2100-01-31T22:59:59.000Z\"}},{\"k\":{\"string\":\"price\"},\"v\":{\"map\":[{\"k\":{\"string\":\"denominator\"},\"v\":{\"int\":100}},{\"k\":{\"string\":\"numerator\"},\"v\":{\"int\":1}}]}},{\"k\":{\"string\":\"startDate\"},\"v\":{\"string\":\"1990-01-31T22:59:59.000Z\"}}]}},{\"k\":{\"string\":\"strat\"},\"v\":{\"map\":[{\"k\":{\"string\":\"description\"},\"v\":{\"string\":\"\"}},{\"k\":{\"string\":\"link\"},\"v\":{\"string\":\"https://app.axo-pilot.trade/strategies/view/SP_XrOALW\"}},{\"k\":{\"string\":\"name\"},\"v\":{\"string\":\"\"}}]}},{\"k\":{\"string\":\"website\"},\"v\":{\"string\":\"https://axo.trade\"}}]}}]}}]}"
    },
    "native_scripts": null,
    "plutus_scripts": null,
    "prefer_alonzo_format": false
  }
}

[lisicky]

Hi @kxbt-del , thanks for letting know, we will investigate and fix it asap

[lisicky]

@kxbt-del could you also provide code to reproduce this issue ?

[kxbt-del]

There's a bunch of stuff in there, iterating collateral, exunits etc. but the essence of building the tx is roughly this:

  const cslWalletAddress = CSL.Address.from_bech32(walletAddress)
  const cslStrategyOwnershipTokenName = CSL.AssetName.new(Buffer.from(ownershipTokenName, "hex"))

  const algoAddress = CSL.Address.from_bech32(algoPolicy)

  const cslStrategyPolicyId = CSL.ScriptHash.from_hex(strategyPolicy.policyId)
  const plutusData = CSL.PlutusData.new_empty_constr_plutus_data(CSL.BigNum.zero())
  const ownershipTokenRedeemer = CSL.Redeemer.new(
    CSL.RedeemerTag.new_mint(),
    CSL.BigNum.zero(),
    plutusData,
    redeemerExUnits
  )

  const ownershipTokenScript = CSL.PlutusScript.from_hex_with_version(
    strategyPolicy.policyCbor,
    CSL.Language.new_plutus_v2()
  )
  const ownershipTokenScriptWitness = CSL.MintWitness.new_plutus_script(
    CSL.PlutusScriptSource.new(ownershipTokenScript),
    ownershipTokenRedeemer
  )

  if (ownershipTokenScript.hash().to_hex() !== cslStrategyPolicyId.to_hex()) {
    throw new Error(
      `Ownership token script hash (${ownershipTokenScript
        .hash()
        .to_hex()}) and strategy policy ID (${cslStrategyPolicyId.to_hex()}) do not match!`
    )
  }

  const { txBuilder, protocolParams } = await createCSLTxBuilder(protocolInfo)

  addCip20Message(txBuilder, ["Axo: New Strategy"], CSL)

  const mintBuilder = CSL.MintBuilder.new()

  mintBuilder.add_asset(
    ownershipTokenScriptWitness,
    cslStrategyOwnershipTokenName,
    CSL.Int.new_i32(2) // Mint two NFTs, one for the wallet and one for the platform
  )

  txBuilder.set_mint_builder(mintBuilder)

  const gasLovelace = BigNumber(gas).multipliedBy(10 ** ADA_DECIMALS)

  const inputLovelace = nodeAllocation
    .map((assetDtos) =>
      assetDtos
        .filter((assetDto) => assetDto.asset.policyId === "")
        .reduce((acc, assetDto) => acc + assetDto.quantity, 0)
    )
    .reduce((acc, quantity) => acc + quantity, 0)

  const strategyLockLovelace = CSL.BigNum.from_str(gasLovelace.plus(inputLovelace).toString())

  const strategyLockTokens = CSL.MultiAsset.new()

  const ownershipToken = singleStrategyOwnershipTokenAsset(
    strategyPolicy.policyId,
    ownershipTokenName,
    CSL
  )
  strategyLockTokens.set_asset(...ownershipToken)

  const nonAdaInputs = groupBy(
    nodeAllocation
      .flat()
      .filter(({ asset: { policyId }, quantity }) => policyId !== "" && quantity > 0),
    ({ asset: { policyId, nameHex } }) => `${policyId}:${nameHex}`
  )

  Object.entries(nonAdaInputs).forEach(([key, assets]) => {
    const [policyId, nameHex] = key.split(":")
    const total = assets.reduce((acc, { quantity }) => acc + quantity, 0)

    strategyLockTokens.set_asset(
      CSL.ScriptHash.from_hex(policyId),
      CSL.AssetName.new(Buffer.from(nameHex, "hex")),
      CSL.BigNum.from_str(total.toString())
    )
  })

  const strategyValue: Value = CSL.Value.new_with_assets(strategyLockLovelace, strategyLockTokens)

  // To allow staking, we set strategy address as payment credential to algo script
  // and stake credential to wallet stake address
  const algoPaymentCredential = CSL.EnterpriseAddress.from_address(algoAddress)?.payment_cred()
  const walletStakeCredential = CSL.BaseAddress.from_address(cslWalletAddress)?.stake_cred()

  if (!algoPaymentCredential) {
    throw new Error("Cannot create BaseAddress from algoAddress")
  }

  if (!walletStakeCredential) {
    throw new Error("Cannot create BaseAddress from cslWalletAddress")
  }

  const strategyAddress = CSL.BaseAddress.new(
    algoAddress.network_id(),
    algoPaymentCredential,
    walletStakeCredential
  ).to_address()

  if (ownershipNftMetadata) {
    const metadata = {
      [strategyPolicy.policyId]: {
        [fromHex(ownershipTokenName).toString()]: ownershipNftMetadata,
      },
    }

    txBuilder.add_json_metadatum(
      CSL.BigNum.from_str("721"),
      JSON.stringify(metadata, (k, v) => {
        if (typeof v === "string" && Buffer.byteLength(v) > 64) {
          return splitStringToByteSizeLines(v)
        }

        return v
      })
    )
  }

  const strategyOutput = CSL.TransactionOutput.new(strategyAddress, strategyValue)
  const cslStrategyDatum = CSL.PlutusData.from_json(
    JSON.stringify(datum),
    CSL.PlutusDatumSchema.DetailedSchema
  )

  strategyOutput.set_plutus_data(cslStrategyDatum)

  const strategyOutputWithMinAda = adjustMinAdaValue(strategyOutput, protocolParams, CSL, {
    mode: "add",
  })

  const inputs = utxos.map((utxo) => {
    const key = utxo.input().transaction_id().to_hex() + utxo.input().index()

    const result = utxoRecord[key]
    if (!result) {
      throw new Error(`Could not find input ${key}`)
    }

    return result
  })
  const collateral = calculateCollateral(inFees, protocolParams).toString()

  const ownershipTokenSubject = `${strategyPolicy.policyId}${ownershipTokenName}`
  const walletLockTokens = CSL.MultiAsset.new()

  walletLockTokens.set_asset(
    ...singleStrategyOwnershipTokenAsset(strategyPolicy.policyId, ownershipTokenName, CSL)
  )

  const walletOutputValue = CSL.Value.new_from_assets(walletLockTokens)

  let walletOutputWithMinAda = adjustMinAdaValue(
    CSL.TransactionOutput.new(cslWalletAddress, walletOutputValue),
    protocolParams,
    CSL
  )

  const outputs = [strategyOutputWithMinAda, walletOutputWithMinAda]
  const requiredAssets = outputs.reduce<SimplifiedValue>((acc, output) => {
    const amount = output.amount()
    const simplified = toSimplifiedValue(amount)
    amount.free()

    delete simplified[ownershipTokenSubject]

    return Object.entries(simplified).reduce((acc, [subject, quantity]) => {
      acc[subject] = (acc[subject] || BigNumber("0")).plus(quantity)
      return acc
    }, acc)
  }, {})

  requiredAssets[""] = (requiredAssets[""] || BigNumber("0"))
    .plus(inFees.to_str())

  const { selectedInputs } = await selectInputsForRequiredAssets(
    requiredAssets,
    inputs,
    protocolParams,
    CSL  )

  const outUtxoPlaintext = utxoToPlaintext(outUtxo)
  const selectedPlaintexts = [...selectedInputs].map(utxoToPlaintext)

  // Need to make sure outref is included
  if (!selectedPlaintexts.includes(outUtxoPlaintext)) {
    selectedInputs.add(outUtxo)
  }

  const balancingOutputs = createRebalancingOutputs({
    selectedInputs,
    requiredAssets,
    protocolParams,
    buildCollateralCandidate: false,
    CSL,
    strategyOwnershipTokenNames,
    walletAddress,
  })

  const cslChangeAddress = cslWalletAddress

  const inputsBuilder = [...selectedInputs].reduce((builder, input) => {
    builder.add_input(input.output().address(), input.input(), input.output().amount())
    return builder
  }, CSL.TxInputsBuilder.new())

  outputs.forEach((output) => {
    txBuilder.add_output(output)
  })

  balancingOutputs.forEach((output) => {
    txBuilder.add_output(output)
  })

  txBuilder.set_inputs(inputsBuilder)

  // Get cost models from a newer protocol version if available
  const costModels = getCostModels(protocolInfo.protocolParams, CSL)

  txBuilder.calc_script_data_hash(costModels)

  const collateralInputSelection = selectInputsForRequiredAssets(
    { "": BigNumber(collateral) },
    inputs,
    protocolParams,
    CSL  )

  const collateralInputs = [...collateralInputSelection.selectedInputs]
  const collateralInputBuilder = collateralInputs.reduce((builder, { utxo }) => {
    builder.add_input(utxo!.output().address(), utxo!.input(), utxo!.output().amount())
    return builder
  }, CSL.TxInputsBuilder.new())

  txBuilder.set_collateral(collateralInputBuilder)
  txBuilder.set_total_collateral_and_return(CSL.BigNum.from_str(collateral), cslChangeAddress)

  const requiredSigners = new Set([...selectedInputs].map((x) => x.output().address().to_bech32()))

  for (const signer of requiredSigners) {
    txBuilder.add_required_signer(CSL.Ed25519KeyHash.from_hex(addrToPubkeyHash(signer, CSL)))
  }

  txBuilder.add_change_if_needed(cslChangeAddress)
  
  ...
  
      const rawTx = txBuilder.build_tx()

    const witnesses = CSL.TransactionWitnessSet.new()
    const witnessScripts = CSL.PlutusScripts.new()
    witnessScripts.add(ownershipTokenScript)
    witnesses.set_plutus_scripts(witnessScripts)

    const redeemers = CSL.Redeemers.new()
    redeemers.add(ownershipTokenRedeemer)
    witnesses.set_redeemers(redeemers)
    const txBody = rawTx.body()
    const tx = CSL.Transaction.new(txBody, witnesses, rawTx.auxiliary_data())

    const txCbor = tx.to_hex()

FWIW the only thing that differs between the HF and pre-HF branch is the CSL version change.

[lisicky]

Why do you modify witness set after build_tx() ?

[kxbt-del]

fair point, legacy leftover. Removing it does not seem to affect anything though

[lisicky]

It might affect redeemer index at least, could you try to not use these lines

    const witnesses = CSL.TransactionWitnessSet.new()
    const witnessScripts = CSL.PlutusScripts.new()
    witnessScripts.add(ownershipTokenScript)
    witnesses.set_plutus_scripts(witnessScripts)

    const redeemers = CSL.Redeemers.new()
    redeemers.add(ownershipTokenRedeemer)
    witnesses.set_redeemers(redeemers)
    const txBody = rawTx.body()
    const tx = CSL.Transaction.new(txBody, witnesses, rawTx.auxiliary_data())

[kxbt-del]

removed them, problem remains.
btw same code running on 11.5.0 builds as you'd expect, correct fees. Tried with and without the above lines.

[lisicky]

Okie, preparing test to find the issue

[lisicky]

@kxbt-del which other tool did you use to modify a tx ? CSL do not serialize sets under tag 258

cardano-serialization-lib/rust/src/serialization/witnesses/vkeywitnesses.rs

Lines 13 to 19 in b765c78

	//TODO: uncomment this line when we conway ero will come
	//serializer.write_tag(258)?;
	serializer.write_array(cbor_event::Len::Len(self.witnesses.len() as u64))?;
	for element in &self.witnesses {
	element.serialize(serializer)?;
	}
	Ok(serializer)

[kxbt-del]

There are 2 more things that may be relevant.

When signing TX, I'm using FixedTransaction

    const tx = CSL.FixedTransaction.from_hex(txCbor)
    const txWitnessSet = tx.witness_set()
    const witnessesCbor = await wallet.api.signTx(txCbor, true)
    const walletWitnessSet = CSL.TransactionWitnessSet.from_hex(witnessesCbor)
    const allWitnesses = await combineTxWitnessSets(txWitnessSet, walletWitnessSet)
    const auxData = tx.raw_auxiliary_data()

    const txFinal = auxData
      ? CSL.FixedTransaction.new_with_auxiliary(
          tx.raw_body(),
          allWitnesses.to_bytes(),
          auxData,
          true
        )
      : CSL.FixedTransaction.new(tx.raw_body(), allWitnesses.to_bytes(), true)
    return txFinal.to_hex()

and then the combineTxWitnessSets functionality:

const combineTxWitnessSets = async (
  ...wss: TransactionWitnessSet[]
): Promise<TransactionWitnessSet> => {
  const CSL = await getCsl()
  if (!CSL) {
    throw new Error(`Can't load CSL`)
  }

  const allVkeys = CSL.Vkeywitnesses.new()
  const allNativeScripts = CSL.NativeScripts.new()
  const allBootstraps = CSL.BootstrapWitnesses.new()
  const allPlutusScripts = CSL.PlutusScripts.new()
  const allPlutusData = CSL.PlutusList.new()
  const allRedeemers = CSL.Redeemers.new()

  const uniqueRedeemers = new Set<string>()

  wss.forEach((ws) => {
    const vkeys = ws.vkeys()
    if (vkeys) {
      for (let i = 0; i < vkeys.len(); i++) {
        allVkeys.add(vkeys.get(i))
      }
    }

    const natScripts = ws.native_scripts()
    if (natScripts) {
      for (let i = 0; i < natScripts.len(); i++) {
        allNativeScripts.add(natScripts.get(i))
      }
    }

    const bootstraps = ws.bootstraps()
    if (bootstraps) {
      for (let i = 0; i < bootstraps.len(); i++) {
        allBootstraps.add(bootstraps.get(i))
      }
    }

    const plutusScripts = ws.plutus_scripts()
    if (plutusScripts) {
      for (let i = 0; i < plutusScripts.len(); i++) {
        allPlutusScripts.add(plutusScripts.get(i))
      }
    }

    const plutusData = ws.plutus_data()
    if (plutusData) {
      for (let i = 0; i < plutusData.len(); i++) {
        allPlutusData.add(plutusData.get(i))
      }
    }

    const redeemers = ws.redeemers()
    if (redeemers) {
      for (let i = 0; i < redeemers.len(); i++) {
        const redeemerJson = redeemers.get(i).to_json()
        if (!uniqueRedeemers.has(redeemerJson)) {
          uniqueRedeemers.add(redeemerJson)
          allRedeemers.add(redeemers.get(i))
        }
      }
    }
  })

  const allWitnesses = CSL.TransactionWitnessSet.new()
  if (allVkeys.len() > 0) {
    allWitnesses.set_vkeys(allVkeys)
  }

  if (allNativeScripts.len() > 0) {
    allWitnesses.set_native_scripts(allNativeScripts)
  }

  if (allBootstraps.len() > 0) {
    allWitnesses.set_bootstraps(allBootstraps)
  }

  if (allPlutusScripts.len() > 0) {
    allWitnesses.set_plutus_scripts(allPlutusScripts)
  }

  if (allPlutusData.len() > 0) {
    allWitnesses.set_plutus_data(allPlutusData)
  }

  if (allRedeemers.len() > 0) {
    allWitnesses.set_redeemers(allRedeemers)
  }

  return allWitnesses
}

I believe the witness set combining can be adjusted given your new set logic in CSL, there's one case where the transaction is additionally signed by the platform after the wallet provides it's own sig.

[lisicky]

Could you check without signing by the platform ?

[kxbt-del]

Sorry for the confusion - the "sign by platform" was for context, to make it known that it does happen in one case. It is not the case for this transaction. So here what we see is what we get.

[lisicky]

Just reproduced whole your tx, and didn't get this "tag 258", CSL do not serialize it and it looks like:

1. You used modified CSL
2. You have non CSL tool in your workflow between tx creation and the final tx (the moment when you get tx hex that you provided in initial message)
3. CSL has very specific bug, but according to the current code it's impossible to have tag 258 in tx cbor if you build it by CSL

[lisicky]

Could you recheck that your code has no other tools that manipulates tx cbor

[kxbt-del]

ok confirmed.

what i send:

84aa0082825820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9018258207b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e030184a3005839109e9dfb8ca4978774ef4b0b03705c73868cba2a2fda522836d845ac31af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e709701821a027ff22ca1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c5701028201d81858acd8799fa200a140a1401a0098968001a09fd8799f4040ffd8799f581cc82a4452eaebccb82aced501b3c94d3662cf6cd2915ad7148b459aec4341584fffffd8799f581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae45204953505f58724f414c57ffa347656e6444617465d905009f1b000003bbcc320998ff457072696365d87b9fd8799f0118c8ffff49737461727444617465d905009f1b0000009393056d98ffa0ff825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e7097821a0011b0dea1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c5701825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a2fbbe450825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a000ecc16021a0004079c075820997a367cc8f8bbd034e2089166a796cc45333348e371880d42b693e0db4b02cb09a1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c57020b58206878390eea04bcbd76d564d8bc7fb480522b1d09069f693e3f850256123265290d818258207b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e070e81581c1b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba410825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a00463fd6111a00060b6aa30081825820aef01e9f23cc52aa8716d6168dc29559be26578bbf84c81a5c556a659f419213584016f03b333a81836c8342c89bb739443ff799dcd59d34300541c0d41c772121778d95436faa6fdaaaf5e21b5f111f9bbd3975f0dabb302e58ca33bab24e519800068159022659022301000032323232323232323232323232223232323232533300c3370e9000001099999199111980711299980a80089128008a99980919baf301630180010041300530180011300230170010012322230020033756602c0026ea4004dd7180900099199180511998011bab001232223002003374c002244a002464a66601e6ae8c0044894004488c00800ccc02894ccc03ccdd78009ba8480004894004488c00800c004004dd598091918091809180900098088021299980699807180780099111801001a4008264649319999806111299980a00089128008a9998089801180b800899111801001980b800899801801180b0009199119baf374e60300046e9cc06000530012bd8799fd8799f5820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9ff01ff00001001200116332300c22533301300114bd700998079801980b0009801180a8009180a180a8009bac30130051533300d3300e300f0013222300200332337029000000a40082930b0b0b180980118070009baa300f300e002300f300e001300e001223300422533300b00110051323330053011300f00223300933760601c602000600200420026004601a00200297adef6c602323002233002002001230022330020020015740ae6888cc0088cc0088cdc38010008a5013300124a0294494ccc00800448940044c94ccc00c0044c888c00800cdd69804180300109128009802000aab9f5573aae895d0918011baa0015573d0581840100d87980821a00018e901a01e31d8cf5a21902a2a1636d7367817141786f3a204e65772053747261746567791902d1a178383730373935323061666261636466363662366531643361376432613138326331643464663866343035306166366130336338616534353230a16953505f58724f414c57a964616c676fa46b6465736372697074696f6e6a4d61746368206174206c646c696e6b82784068747470733a2f2f6170702e61786f2d70696c6f742e74726164652f636f6d706f7365722f64343962636538662d363864632d343633662d626437302d61613569393037376563396661646e616d65654c696d6974647574786f82784037343535396238656164363235626339663035326664323135316238396238333064313831373832366238393961376434366336626664383533663162386237622330666173736574738260783e633832613434353265616562636362383261636564353031623363393464333636326366366364323931356164373134386234353961656334313538346665696d6167657835697066733a2f2f516d5a446d79553941324a6872734d3555744447717759764b7a5468656d3952636a5a745a5569345672635a724764696e6974a26762616c616e6365a2634144411a009896806341584f00657374617465a0696d656469615479706569696d6167652f706e67646e616d65704c696d6974203c4144412c2041584f3e66706172616d73a367656e64446174657818323130302d30312d33315432323a35393a35392e3030305a657072696365a26b64656e6f6d696e61746f7218c8696e756d657261746f7201697374617274446174657818313939302d30312d33315432323a35393a35392e3030305a657374726174a36b6465736372697074696f6e60646c696e6b783568747470733a2f2f6170702e61786f2d70696c6f742e74726164652f737472617465676965732f766965772f53505f58724f414c57646e616d656067776562736974657168747470733a2f2f61786f2e7472616465

(only see Unassigned(121) with empty arrays)

what gets submitted to the node:

84aa0082825820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9018258207b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e030184a3005839109e9dfb8ca4978774ef4b0b03705c73868cba2a2fda522836d845ac31af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e709701821a027ff22ca1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c5701028201d81858acd8799fa200a140a1401a0098968001a09fd8799f4040ffd8799f581cc82a4452eaebccb82aced501b3c94d3662cf6cd2915ad7148b459aec4341584fffffd8799f581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae45204953505f58724f414c57ffa347656e6444617465d905009f1b000003bbcc320998ff457072696365d87b9fd8799f0118c8ffff49737461727444617465d905009f1b0000009393056d98ffa0ff825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e7097821a0011b0dea1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c5701825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a2fbbe450825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a000ecc16021a0004079c075820997a367cc8f8bbd034e2089166a796cc45333348e371880d42b693e0db4b02cb09a1581c7079520afbacdf66b6e1d3a7d2a182c1d4df8f4050af6a03c8ae4520a14953505f58724f414c57020b58206878390eea04bcbd76d564d8bc7fb480522b1d09069f693e3f850256123265290d818258207b79cec3a3cc0e1035298a2f39bf22dc7e75c6c0c203145ddc0ee0e232cfa99e070e81581c1b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba410825839001b40e94dc73a9c3732c75008b8e3b609a344d120aca9cb9528351ba4af12bc1ca9918678d7940c40920a1ba31ac04b7347ad55524b3e70971a00463fd6111a00060b6aa300d9010281825820aef01e9f23cc52aa8716d6168dc29559be26578bbf84c81a5c556a659f419213584016f03b333a81836c8342c89bb739443ff799dcd59d34300541c0d41c772121778d95436faa6fdaaaf5e21b5f111f9bbd3975f0dabb302e58ca33bab24e51980006d901028159022659022301000032323232323232323232323232223232323232533300c3370e9000001099999199111980711299980a80089128008a99980919baf301630180010041300530180011300230170010012322230020033756602c0026ea4004dd7180900099199180511998011bab001232223002003374c002244a002464a66601e6ae8c0044894004488c00800ccc02894ccc03ccdd78009ba8480004894004488c00800c004004dd598091918091809180900098088021299980699807180780099111801001a4008264649319999806111299980a00089128008a9998089801180b800899111801001980b800899801801180b0009199119baf374e60300046e9cc06000530012bd8799fd8799f5820259ed76ff84d400c11eabbaed704ce77122d727a2b919f9c49f496c528601da9ff01ff00001001200116332300c22533301300114bd700998079801980b0009801180a8009180a180a8009bac30130051533300d3300e300f0013222300200332337029000000a40082930b0b0b180980118070009baa300f300e002300f300e001300e001223300422533300b00110051323330053011300f00223300933760601c602000600200420026004601a00200297adef6c602323002233002002001230022330020020015740ae6888cc0088cc0088cdc38010008a5013300124a0294494ccc00800448940044c94ccc00c0044c888c00800cdd69804180300109128009802000aab9f5573aae895d0918011baa0015573d0581840100d87980821a00018e901a01e31d8cf5a21902a2a1636d7367817141786f3a204e65772053747261746567791902d1a178383730373935323061666261636466363662366531643361376432613138326331643464663866343035306166366130336338616534353230a16953505f58724f414c57a964616c676fa46b6465736372697074696f6e6a4d61746368206174206c646c696e6b82784068747470733a2f2f6170702e61786f2d70696c6f742e74726164652f636f6d706f7365722f64343962636538662d363864632d343633662d626437302d61613569393037376563396661646e616d65654c696d6974647574786f82784037343535396238656164363235626339663035326664323135316238396238333064313831373832366238393961376434366336626664383533663162386237622330666173736574738260783e633832613434353265616562636362383261636564353031623363393464333636326366366364323931356164373134386234353961656334313538346665696d6167657835697066733a2f2f516d5a446d79553941324a6872734d3555744447717759764b7a5468656d3952636a5a745a5569345672635a724764696e6974a26762616c616e6365a2634144411a009896806341584f00657374617465a0696d656469615479706569696d6167652f706e67646e616d65704c696d6974203c4144412c2041584f3e66706172616d73a367656e64446174657818323130302d30312d33315432323a35393a35392e3030305a657072696365a26b64656e6f6d696e61746f7218c8696e756d657261746f7201697374617274446174657818313939302d30312d33315432323a35393a35392e3030305a657374726174a36b6465736372697074696f6e60646c696e6b783568747470733a2f2f6170702e61786f2d70696c6f742e74726164652f737472617465676965732f766965772f53505f58724f414c57646e616d656067776562736974657168747470733a2f2f61786f2e7472616465

there we see 2 additional entries with Unassigned(258).
Will investigate more and report back, thx

[kxbt-del]

The platform is using cardano-api ==9.0.0.0, to deserialize and reserialize the tx to perform additional checks before passing it forward. Need to update the dependencies and see the impact.

[kxbt-del]

hey, it's being looked into as we speak. Will be in touch when I have results.

[lisicky]

Thanks @kxbt-del for update

[kxbt-del]

So, the results are in - cardano-api is always adding this new optional set tag. You aren't and in cases where our platform needs to counter-sign a tx, it causes discrepancies in the calculated fees. Weird move by IOG, b/c it's obvious that this may happen if the tag is optional 😄
btw are you planning to start adding this tag? could do a 12.0.1 with this code uncommented - unless there are additional complications?

One thing is though - the tag is on all sets, not just vkwits, so i can see that being a "complication"

[lisicky]

@kxbt-del did you create an issue on cardano-api repo ?

[lisicky]

Since 13.0.0 all sets will be serialized with a tag 258