Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMR-10257: [Fix] Updating API Docs to simplify language #2195

Merged
merged 5 commits into from
Nov 26, 2024
Merged

Conversation

jmaeng72
Copy link
Contributor

Overview

What is the feature/fix?

After review of API docs for new subscription api changes, there needed to be more simplification and clarification to them

What is the Solution?

Organized information for easier understanding

What areas of the application does this impact?

Ingest and Search API Docs

Checklist

  • I have updated/added unit and int tests that prove my fix is effective or that my feature works
  • New and existing unit and int tests pass locally and remotely
  • clj-kondo has been run locally and all errors corrected
  • I have removed unnecessary/dead code and imports in files I have changed
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have cleaned up integration tests by doing one or more of the following:
    • migrated any are2 tests to are3 in files I have changed
    • de-duped, consolidated, removed dead int tests
    • transformed applicable int tests into unit tests
    • refactored to reduce number of system state resets by updating fixtures (use-fixtures :each (ingest/reset-fixture {})) to be :once instead of :each

@jmaeng72 jmaeng72 self-assigned this Nov 20, 2024
@jmaeng72 jmaeng72 changed the title CMR-10257: [FIx] Updating API Docs to simplify language CMR-10257: [Fix] Updating API Docs to simplify language Nov 20, 2024
@codecov-commenter
Copy link

codecov-commenter commented Nov 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.23%. Comparing base (01b6628) to head (1e9727a).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2195   +/-   ##
=======================================
  Coverage   58.23%   58.23%           
=======================================
  Files        1056     1056           
  Lines       71009    71009           
  Branches     2025     2023    -2     
=======================================
  Hits        41351    41351           
- Misses      27768    27769    +1     
+ Partials     1890     1889    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:


Subscription metadata is in JSON format and conforms to [UMM-Sub Schema](https://git.earthdata.nasa.gov/projects/EMFD/repos/unified-metadata-model/browse/subscription). There is a background job that processes the search subscriptions periodically (configurable), to see if there are any collections/granules that are created/updated since the last time the subscription has been processed and notify the subscription user with any matches.
There are two kinds of subscriptions: Search and Ingest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than describe the subscription types as "search" and "ingest" we should give a more user-facing description. Something like: "batch notification" and "near-real-time notification".


Subscription concepts can be created by sending an HTTP POST or PUT with the metadata sent as data to the URL `%CMR-ENDPOINT%/subscriptions/<native-id>`. The response will include the [concept id](#concept-id) ,the [revision id](#revision-id), and a [native-id](#native-id).
- Ingest subscriptions are processed on ingest and are only for granules. When a user subscribes, notifications are sent out to a provided AWS SQS endpoint.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sent out via the provided notification endpoint, such as an AWS SQS messaging queue.


`Type` is a required field in subscription request body. The valid values of `Type` are: `"collection"` or `"granule"`. It indicates if the subscription is a collection subscription or granule subscription. Subscriptions of type granule must supply a requisite CollectionConceptId, and subscriptions of type collection cannot have a CollectionConceptId field. When the CollectionConceptId field is used, the subscription subscriber must have read access to the collection, otherwise the ingest of the subscription will fail.
- Search subscriptions run periodically via CMR search queries and notify users by the email address that is registered in Earthdata Login. There are two types of subscriptions (identified by the `Type` field of the subscription):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Batch subscription processing is run periodically and execute CMR search queries, as defined by the Subscription definition, and notify users ...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...two types of batch process subscriptions...


If a SubscriberId is not provided, then the user ID associated with the token used to ingest the subscription will be used as the SubscriberId.
There is a configurable background job that processes the search subscriptions periodically, to see if there are any collections/granules that are created/updated since the last time the subscription has been processed and notify the subscription user with any matches.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Batch subscription notification processing is executed periodically...
Notification of updates is via the email address associated with the SubscriberId's EarthData Login (URS).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do see that the parameter description section describes this but if feels like a good thing to mention as an overview also, since that is the notification mechanism.


If a SubscriberId is not provided, then the user ID associated with the token used to ingest the subscription will be used as the SubscriberId.
There is a configurable background job that processes the search subscriptions periodically, to see if there are any collections/granules that are created/updated since the last time the subscription has been processed and notify the subscription user with any matches.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this is not configurable by the client, so no need to include it in the API docs

- `EmailAddress`: [deprecated] was previously a required field, but is now deprecated. Instead, the email address associated with the SubscriberId's EarthData Login (URS) account will be used as the EmailAddress. If an EmailAddress is specified at subscription creation it will be ignored.
- `MetadataSpecification`: [required] Specifies which metadata version schema you are using for this subscription. Currently, that version is 1.1.1.

##### Additional Data Fields for Ingest Subscriptions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...for Near Real-time Subscriptions

Basically, rename "Ingest subscription" everywhere to NRT or whatever we land on.

ingest-app/docs/api.md Show resolved Hide resolved
ingest-app/docs/api.md Show resolved Hide resolved

Subscription metadata is in JSON format and conforms to [UMM-Sub Schema](https://git.earthdata.nasa.gov/projects/EMFD/repos/unified-metadata-model/browse/subscription). Subscriptions of type `granule` must supply a requisite CollectionConceptId, and subscriptions of type `collection` cannot have a CollectionConceptId field. There is a background job that processes the search subscriptions periodically (configurable), to see if there are any collections/granules that are created/updated since the last time the subscription has been processed and notify the subscription user with any matches.
- Ingest subscriptions are processed on ingest and are only for granules. When a user subscribes, notifications are sent out to a provided AWS SQS endpoint.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar comments for the search doc, replace "ingest subscriptions" and "search subscriptions"

@jmaeng72 jmaeng72 merged commit e911b0e into master Nov 26, 2024
6 checks passed
@jmaeng72 jmaeng72 deleted the CMR-10257-Fix branch November 26, 2024 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants