Add Versioning support for API Products

[HiranyaKavishani]

Problem

When an API Product developer wants to change an operational behaviour of an already published API product as per their new business requirement, for such as authentication mechanism, resources, business plans, etc, currently WSO2 API-M does not have a proper mechanism at API product support, unless the change is made directly in to the deployed API product. But, this may directly affect the API product consumers’ functionalities in their applications and can have disastrous effects on end user’s downstream products and services.

Solution

The solution is to add versioning support for API products and we plan to support the following sub-features to provide the proper versioning experience for API product developers.

1. Allow API developers to create a new api product version from an existing API product and allow to add version number during API Product creation.

2. Allow API developers to invoke the api product resources without specifying the version number by Introducing default version support

3. Allow API developers to select two options during life cycle management of an API Product Deprecate old versions after publishing the API Products and Require re-subscription by when publishing API Products.

4. Product developer to receive notification of API Product version creation
   Support context versioning for API Products in order to allow users to define the API Product version as a parameter of its context by adding the {version} into the context.

5. Allow API developers to manage their API product versions with different environments through WSO2 API CTL.

Affected Component

APIM

Version

4.3.0

Implementation

Followings are the sub tasks to achieve the whole versioning support for API Product version

• Initial discussion to finalize the scope of versioning support for API Products feature
• Prepare an initial design documentation for the versioning support to API Products

Backend Changes to support api product versioning

• Add a new REST API to copy API Product to create new API Product #2179 (Support new version create, update, delete flows)
• Support default version for API Products
• Support lifecycle checklist for API products (Show lifecycle checklist for new API Product versions, add the capability to deprecate old versions or copy subscription during APIProduct publish based on the checklist status )
• Support context versioning for API Products
• Display correct published default version of API product at devportal
• Enable email notifications for subscribers to receive notification of creation of a new version of APIProduct

UI modification for api product versioning suppport

Publisher UI modifications

• Add version field into API Product creation form
• Display API Product Version at product overview page
• Add option to "create new version from existing API product" at overview page
• Add default version option at basic info in design configuration page
• Enable two options Deprecate old versions after publishing the API Products and require re-subscription by when publishing API Products at life cycle management UI
• version support at tryout console for API Products

####Other

• Add versioning support to API Products in API CTL and testing #2205

Tests and Documentation_

• Manual Testing all the use cases related to API product versioning support
• Unit tests for the backend code changes
• UI tests for API Product versioning support
• Integration tests for the API Products versioning
• Documentation for versioning support to API Products #2291

Related Issues

Similar issue: wso2/product-apim#6971

Suggested Labels

No response

[HiranyaKavishani]

Progress Update

11th of Sep 2023 - 15th of Sep (5 days)

1. Finalized the scope
2. Prepared initial design documentation [1] and the proposed solution is emailed to Architecture group.
3. Started REST API implementation and testing

[1] https://docs.google.com/document/d/1cf-LazqdytaybVuHoiF8LUIiP8ENgwiaPCbWZGWF6Vo/edit?usp=sharing

[HiranyaKavishani]

Progress Update

18h of Sep 2023 - 22nd of Sep (5 days)

BackEnd Implemetation

1. Completed REST API to create New API Products Versioning and verify create/update/delete flow.
2. Completed the changes related to default version support for API Products
3. Completed the changes on API Product life cycle change REST API to introduce life cycle check list

FrontEnd Implementation

1. Made the UI changes to enable version field in the API Product creation form
2. Added UI changes to create new version from existing API product
3. Made changes in Product overview to show version
4. Enable default version option at API Product basic info

Ongoing

1. Testing the functionalities related to the above, completed task list (FrontEnd-BackEnd integration)

[HiranyaKavishani]

Progress Update

25th of Sep 2023 - 27th of Sep 2023 (3 days)

1. Tested the end to end API Product versioning related functionalities at publisher and fixed the issues.

Backend
I. Added changes to support context versioning for API Products
II. Modification on APIProduct listing Rest API to append context without version
III. Modification on API Product deployment flow (Bind version in to generated synapse API of API Products)

Front End
I. UI changes to display version at API tile during API Product listing and unified search
II. UI Changes on life cycle page to display life cycle check list for new API Product versions

Ongoing/ pending testing usecases.

1. Display multiple versions at devportal and verify their invocation.
2. Display default version at developer portal and invoke without version at url.

[HiranyaKavishani]

Progress Update

2nd of Oct 2023 - 6th of Oct 2023 (5 days)

1. Added backend changes to enable email notifications of new version creation of API Products

2. Added UI changes to API Products publisher tryout console to support versioning

3. Provided version support for API products in apictl

4. Identified testing scenarios and manually verified below
   I. Create/update/deleted API products with mutiple versions
   II Life Cycle support for API products based on the checklist to deprecate old versions or copy subscriptions or both
   III Deploy revisions of new API product versions and invoke at publisher and devportal
   IV Display multiple versions at devportal and verify their invocation.
   V Display default version at developer portal and invoke without version at url.
   VI. UI changes in publisher portal (Overview, design config, lifecycle, tryout UI)
   VII. Context versioning support for API products

Ongoing/ pending testing usecases and remain work

1. Testing API product use cases on apictl tool and fixing issues.
2. Unit tests/integration tests/Documentation

[HiranyaKavishani]

Progress Update

9th of Oct 2023 - 11th of Oct 2023 (3 days)

1. Tested the API product use cases on apictl tool and fixed affected areas issues
2. Finalize the PRs of the feature and those can be found at [1] [2] [3]
3. Conducted the code review

[1] carbon-apimgt: wso2/carbon-apimgt#12168
[2] apim-apps: wso2/apim-apps#506
[3] product-apim-tooling: wso2/product-apim-tooling#1022

[HiranyaKavishani]

Code Review of the feature is completed on last week, The review comments which were suggested during code review are fixed and latest PRs are now available for the further review.

How do migrated API products work?

1. In order to work the migrated API products without any issue, we have set the default value of isDefaultVersion= null for APIProducts. So, that, If there is no any entry at AM_API_DEFAULT_VERSION table, it will be null as per the default value. We can identify migrated api products from that point.

2. In publisher design configuration UI, DefaultVersion of all migrated API Products will be shown as true if the isDefaultAPIVersion is null. When an APIProduct update action is happened, new entry will be saved at AM_API_DEFAULT_VERSION table for those API Products.

3. For a creation of a new API Product (not a version) at API-M 4.3.0, We internally set isDefaultVersion to true in order to make consistent with old behaviour. So that, new entry will be saved at AM_API_DEFAULT_VERSION table during new API product creation.

We can port the available fixes to API-M 4.2.0 without any issue. Only tests and documentation are pending to complete the feature.

@nandika please note.

[HiranyaKavishani]

As @SavinduDimal worked on API Product Versioning support for API-M 4.2.0, we noticed some blockers during testing phase and introduced some changes to work migrated API Product as it is. Following are some of key highlights.

1. By default, isDefaultVersion=true for all API products (Migrated and new API Products )
2. Migrated API products are identified by the value of the ContextTemplate which is stored at AM_API table in AM_DB
   (It is null for old API Products since versioning didn't support for older APIM versions)
3. Migrated API Products should be able to identify with it's value of publishedDefaultVersion (If revision has been already deployed and ContextTemplate=null => IsPublishedDefaultVersion = true and publishedDefaultVersion=1.0.0)
4. Since, version can be there in the context it self for migrated API Products, extra validation of context is added to identify duplicateContext.

[HiranyaKavishani]

Closing the issue since all tasks are completed