4.3.42: [4.3] fix send_file CTA and make file removal optional (#5941)

kazoo Changes

Changes to 4.3 after version 4.3.41

Unticketed Commits

• [4.3] fix send_file CTA and make file removal optional (#5941) * fix send_file CTA and make file removal optional * avoid deleting while still streaming the response body * the callers to this function all handle empty list conditions already * formatting * fix spec by bitbashing
• HELP-45578: if already unpublished dont change authority or re-unpublish (#5933) by karl anderson
• [4.3] HELP-44382: decrease 409s when saving accounts (#5930) * [4.3] HELP-44382: decrease 409s when saving accounts Due to changes in how account documents are saved (in the account db, then copied to the 'accounts' db), some systems were experiencing increased incidences of 409 API errors when modifying the account doc via /v2/accounts/{ACCOUNT_ID}. Currently the only "smoking gun" for why some systems experienced this elevated rate seems due to the Erlang scheduler's interleaving of processes and the way the kzs_cache works when storing a cache entry. Essentially the problem was distilled to the interplay of 3 processes, A, B, and C: A -> Cache: flush doc A -> Couch: save rev X B -> Couch: get doc Couch -> B: got rev X B -> Cache: cache rev X Couch -> A: saved rev Y C -> Cache: get rev X C -> Couch: save rev X Couch -> C: 409 See core/kazoo_data/doc/help_44382.png So the cache was being re-populated with the soon-to-be old revision while the save to Couch was occurring. To address this, after a successful save operation from Couch, kzs_doc will then cache the new document. One caveat is that if an existing doc is already cached, the revisions of the existing and new doc are compared and the higher revision is stored in the cache. Another issue that arose was if _rev is included in kzd_accounts:update/3 and is old, and ensure_saved is true, a loop will be started where the save will 409, updates re-applied to the actual doc (including the old _rev) and a save will again 409 - rinse and repeat. * update caching of saves * ignore return of set_default * no div by 0 by James Aimonetti
• PISTON-864 add new prompt for confirming a voicemail has been forwarded (#5935) Playing prompt on newcall appears to still play by Spencer Hutchinson

4.3.41

kazoo Changes

Changes to 4.3 after version 4.3.40

Unticketed Commits

• once the RECORD_START event is received, no need to maintain a timer (#5929) by James Aimonetti
• [4.3] HELP-45578: dont override the blacklist and improved logging (#5928) * HELP-45578: dont override the blacklist and improved logging * HELP-45578: dont hide port from resellers and ci issue * fixed a broken test by bitbashing
• [4.3] Timers for kzc_recording processes to make sure they terminate (#5926) * Timers for kzc_recording processes to make sure they terminate Start a timer to limit recording time; stop kzc_recording process if the timer expires Start a timer for RECORD_STOP when recv CHANNEL_DESTROY; stop kzc_recording if the timer expires * handle AMQP reconnect by checking call status * fix the test * remove unused vars * stop timers by James Aimonetti
• Handle disconnected channels during sync (#5923) - simplify channels as json - emit channel_disconnected for local channels during sync by lazedo
• HELP-45448: Use devices name for display name (#5925) by Harenson Henao
• Voicemail callback (#5910) * Added callcack feature to cf_voicemail. * Allow callback from offnet * Fixed stalled callflow * Fixed CI checks (cherry picked from commit f37d9f9b7ee0c8f69f92def1d94b0f91be9790ee) handle dialyzer complaints by sergey-safarov

4.3.40: HELP-44667: don't send authz resp on quick hangup (#5919)

kazoo Changes

Changes to 4.3 after version 4.3.39

Unticketed Commits

• HELP-44667: don't send authz resp on quick hangup (#5919) Processing the authz request can take longer than processing the subsequent CHANNEL_DESTROY. This causes the flushing in j5_channels to occur before the storing of the authorization payload. Since the channel is gone, the authorization payload isn't flushed and counts against the account/reseller limits. Instead, check the j5_channels ETS table for the authorization payload on CHANNEL_DESTROY; if none, store an entry with a destroyed:true flag. If a subsequent authz response is attempted to be stored, this check will keep it from happening. Introduce a test for sending CHANNEL_DESTROY followed by authz_req and ensuring totals remain 0 and no authz_resp is generated. by James Aimonetti
• help-44967: should use the faxDB not generate a new modb when saving new doc (#5913) by Sean Wysor
• KAZOO-6115: remove override defaults from assign and keep current when undefined (#5909) by bitbashing

4.3.39: Export function to handle auto compaction triggers (#5881) (#5900)

kazoo Changes

Changes to 4.3 after version 4.3.38

Unticketed Commits

• Export function to handle auto compaction triggers (#5881) (#5900) Use proper variable names by Harenson Henao
• multifactor view - 4.3 (#5904) * check if normalized result from view is a list (cherry picked from commit 9aff8442b352ce9d94911b58715876226fc955da) * cb_multi_factor: run views independently and merge results (cherry picked from commit be46fede3b8592c7373ad8e739cec98db30a6c75) by lazedo
• get doodle default connection from runtime (#5906) (cherry picked from commit 94bd87df0260663611059a0c0adfaded3d2ba8b4) by lazedo
• [4.3] KZ00-5: move call to cast (#5896) To avoid blocking the calling process when the gen_server is overly busy, move the channel updates to a cast instead of a call. In particular, we've observed the authz worker (during route req/resp for dialplan) getting blocked for 2s while waiting for this update to occur. Since the authz worker and dialplan response don't depend on the channel cache, no sense blocking for the ETS write. by James Aimonetti
• [4.3] Default true on whether to refresh ledger views across MODBs (#5894) by James Aimonetti
• [4.3] On init, check if modb creation should start (#5892) If the tasks app is started on or after the MODB creation date, the next month's MODBs won't be created (or may be partially created if tasks was restarted after MODB creation had begun). Check if the tasks app is starting up between configured creation date and the end of the month and start a MODB creation task if so. by James Aimonetti
• [4.3] Audit summary per day (#5821) by Hesaam Farhang
• Added voicemail playback order reversing (#5846) Schemas and APIs fixed for oldest_message_first voicemail A bunch of autogenerated fixes Added oldest_message_first/1,2 to kzd_vmboxes by Shvein Anton
• [4.3] KAZOO-6113: add exception for master account and refactor (#5902) * KAZOO-6113: add exception for master account and refactor * KAZOO-6113: typo * KAZOO-6113: change priority * KAZOO-6113: dont publish all, just those in the app_store doc by bitbashing

4.3.38

kazoo Changes

Changes to 4.3 after version 4.3.37

Unticketed Commits

• HELP-45073: Fix provisioning realm settings and include datetime (#5886) by Harenson Henao
• [[4.3 Doodle updates (#5848) stop after send add callee-id-number to payload add outbound handler don't loopback if number is not assigned remove dev logs 4.3 pool compat more 4.3 compat cherry-pick kz_maps & kz_json exec's less restrict app_store validation & add extends handle only numbers assigned to callflow make dialyzer happy ignore returns by lazedo
• dialyze hard (#5837) by James Aimonetti
• allow the sms api to be used with creds from superduper (#5837) add accessor for flows doodle updates * own lookup that failsback to callflow * add primary connection to connections * handle inbound for sip and not mdn devices * some renames add sms webhook * delivers from/to/body prevent devices without sip creds from crashing allow webhooks to deliver body in json * adds PUT to allowed verbs * adds format 'form-data' (default) , 'json' declare exchange once per connection that needs it by lazedo

4.3.37

kazoo Changes

Changes to 4.3 after version 4.3.36

Resolved Tickets

KAZOO-6108

• Summary
  • When the directory matches more than 1 user, it reads them back in reverse order Z - A
• Commits
  • KAZOO-6108: backport of #5237 for presorting directory users (#5874) by James Aimonetti

KAZOO-6107

• Summary
  • Porting notifications don't create alerts if service plans are assigned to an account
• Commits
  • KAZOO-6107: dont reset the resp data (#5861) by karl anderson

Unticketed Commits

• HELP-44996: Fix listing devices by owner for provisioning (#5877) by Harenson Henao
• [4.3] Added accessors for cluster zones with priority (#5868) * Added cluster zone accesors by yonkoma
• help-43894: just toggle retain-cid (#5871) by Sean Wysor
• [4.3] HELP-44761: fixed crashes in quantities view query (#5869) * HELP-44761: fixed crashes in quantities view query * HELP-44761: fmt fix by Sean Wysor
• HELP-44744: allow chaining children off temporal route actions (#5864) Previously, actions of menu, enable, disable, and reset would terminate the call after processing. Upon discussion, there wasn't a compelling reason to continue doing this (and restricting the ability to chain more actions off the action). The usecase is that when there is an "emergency" a callflow can be dialed to disable the main temporal route rules and enable the particular "emergency" temporal route rule. by James Aimonetti
• PISTON-811: Kazoo storage fixes for db operations on storage db (#5855) Backport of #5186 by Ben Bradford
• HELP-44667: for error replies wait for authz when enabled so the channel_destroy is not missed (#5850) use a macro for the timeout value by karl anderson
• [4.3] fix removing comments when rejected->submitted (#5856) * maybe this is removing comments? * format phonebook comments not context * remove time info from requested port date since user can't choose the time anyway * fix spec by icehess

4.3.36

kazoo Changes

Changes to 4.3 after version 4.3.35

Unticketed Commits

• process xml/binary for post, xml for get (#5844) by lazedo
• [4.3] Whitelabel SSO updates (#5560) * add basic SSO provider schema * build a better new whitelabel doc * use accessor module a bit * refactor functions and cleanup constants * handle when type is M:F() update schemas, docs, accessor modules * handle module name and add specs by James Aimonetti
• [4.3] fix improper devices.hotdesk schema (#5841) * fix improper devices.hotdesk schema by icehess
• remove andalso with non-boolean RHS (#5835) by James Aimonetti
• PISTON-769: do not rely on Hangup-Cause, will be set if transferred to vm (#5835) by Daniel Finke
• HELP-43867-4.3: export for integrations (#5836) by James Aimonetti

4.3.35

kazoo Changes

Changes to 4.3 after version 4.3.34

Unticketed Commits

• Moved config storage cluster_id and added cluster_id to webhooks (#5832) by yonkoma
• HELP-44059: flush registration when device is deleted using api_v1 (#5830) by icehess
• [4.3] HELP-43867: rework ledger/modb/services at month's end (#5829) * [4.3] HELP-43867: rework ledger/modb/services at month's end by James Aimonetti
• allow headers to be set as part of the notification (#5828) by lazedo
• add/fix missing event_name in samples (#5798) list things fix return and spec doc fmt asdf by Hesaam Farhang
• HELP-43911: create a consistent payload based on device/user/account (#5814) by James Aimonetti
• qf: fixed label for parking when it is integer (#5802) by swysor
• HELP-44281: Back port missing changes for fixing blind call transfer issue (#5813) by Harenson Henao

4.3.34

kazoo Changes

Changes to 4.3 after version 4.3.33

Resolved Tickets

KAZOO-6082

• Summary

  • As an admin I want to be able to rebuild the aggregate databases and the content so I can preform mainatenance

• Commits

  • [KAZOO-6082: [4.3] new maintenance commands (#5774)

• KAZOO-6082: new maintenance commands

• KAZOO-6082: create system db on refresh if not exists

• KAZOO-6082: return error on refresh if db not found

• wip

• disjoint wip

• multi branch

• fix log line

• add maintenance command to import account, fix reseller id

• fix log

• xref

• fix spec

• KAZOO-6082: basic pvt tree fixer

• KAZOO-6082: save fixed trees

• KAZOO-6082: clean up account to numbers db copy

• KAZOO-6082: copy from num_dbs to accounts, remove wrong assgined

• KAZOO-6082: get ride of fix_number

• KAZOO-6082: add back app_using

• KAZOO-6082: used_by fix numdb and accdb

• kzdoc kzjson

• something

• let. me. pass.

• xref

• typical

• make dialyzer happy

• stop complaining about my code

• wat now?

• STOP IT

• fmt

• fixes

• some fixes

• no retry conflict for now

• added faxbox to list of aggregates

• log stuff

• logging added to exported aggregates functions

• KAZOO-6082: fix some issues in ensure pvt tree

• fmt

• another round for dialyzer

• more efficient faxbox doc updates

• fixed a bug in the bulk save

• try conflict only for num -> account

• code cleanup](dc27004) by icehess

Unticketed Commits

• Backport (#5799) by Harenson Henao
• [4.3] HELP-39757: allow ledger amounts to include fractional values (#5787) * [4.3] HELP-39757: allow ledger amounts to include fractional values * update unit_amount spec + getter * use the accessor * remove unused clause * formatting * note usage of kzd_amounts * don't confuse edoc by James Aimonetti
• [4.3] limit inbound calls per did (#5793) by OnNet
• handle wrong_format schema errors by James Aimonetti
• fixed a crash in fax_jobs caused by start_job timeout condition (#5783) by Sean Wysor
• [4.3] help-43608: handle uppercase extensions (#5785) * help-43608: added upper case extensions for fax file types * another approach by Sean Wysor

4.3.33

kazoo Changes

Changes to 4.3 after version 4.3.32

Resolved Tickets

KAZOO-6099

• Summary

  • Kazoo object plan modifications do not trigger bookkeeper updates when using minimums on cumulative plans to force quantities

• Commits

  • [[4.3] KAZOO-6099: simplify annotate to fix last object removal quantity bug (#5777)

• KAZOO-6099: simplify annotate to fix last object removal quanity bug

• fix dialyzer

• fmt](2298603) by bitbashing

  • [[4.3] Kazoo 6099 4.3 (#5755)

• KAZOO-6099: check for object plan changes when generating invoices

• KAZOO-6099: fliped logic for debugging, didnt put it back](48891f2) by bitbashing

KAZOO-6098

• Summary
  • Unable to set new credit cards to default with additional braintree verifications enabled
• Commits
  • KAZOO-6098: verify only when adding a new card (#5751) by bitbashing

KAZOO-6088

• Summary

  • As a user I want the payment token alert to be suppressed if I dont' have service plans so that I am not confused

• Commits

  • [[4.3] KAZOO-6088: skip payment token alerts if there are no service plans (#5721)

• KAZOO-6088: skip payment token alerts if there are no service plans

• KAZOO-6088: update cb_alerts tests

• fix fix cb alerts tests](4d86f74) by bitbashing

Unticketed Commits

• prevent db listing to upgrade to node local connection in bigcouch (#5768) by John White
• fix deleting a single cc payment token (#5767) by icehess
• [4.3] - Ledger summary modb (#5771) * * Added new MODB view, ledgers/summary_by_account to return parent and subaccount ledger summaries. * Added new crossbar endpoint to cb_ledgers to return current modb ledger summary from ../ledgers/summary/{YYYYMM}. * api and doc updates * Revert "api and doc updates" This reverts commit 30cf6bb5e93a10de8ddf1916ad812a2a5b9c835f. * api and doc fixup * payload shape change and other requests * formatting * requested refactor and view changes * doc example by John White
• Fix function's return type (#5759) by James Aimonetti
• [4.3]: add api to get samples of webhook event payloads (#5759) add webhooks_all remove master specific event fix spec make apis? by Hesaam Farhang
• PORT-23: update return types (#5706) by James Aimonetti
• PORT-23: check old key for setting is_private first (#5706) migrate to is private key is_private_legacy/1 by Hesaam Farhang
• add code_checks.bash to code_checks target (4.3 only) (#5764) * add code_checks.bash to code_checks target * update circleci config files to properly set CHANGED by icehess
• [4.3] removing dialyzer race_conditions, it causing oom in cricleci (#5761) by Hesaam Farhang
• [4.3] tell processes with sticky channels about down brokers (#5760) * tell processes with sticky channels about down brokers * handle gen_listener casts by James Aimonetti
• [4.3] someone made dialyzer to not run and dialyzer is not happy about it (#5743) * someone made dialyzer to not run and dialyzer is not happy about it * this is all about items is not a list? by icehess