TRADE-187: Add tests for SAMBAH

[danielfromearth]

Description

Adds a regression test notebook for the SAMBAH service chain.

Supersedes #93, in order to track references files using git lfs.

Jira Issue ID

• TRADE-187 (closed)
• TRADE-410

Also related to:

• TRADE-500

Local Test Steps

Ran notebook tests for UAT environment successfully.

PR Acceptance Checklist

• Acceptance criteria met
• Tests added/updated (if needed) and passing
• [N/A] Documentation updated (if needed)

[danielfromearth]

@owenlittlejohns, @ank1m,

I've revised the Harmony regression test notebook, and I thiiiink it's good. The tests for UAT are passing.

The tests in PROD don't yet produce the same output as the tests in UAT, however, despite using matching granules. I believe that is because of the difference in versions currently deployed in the environments:

PROD: l2ss-2.11.0, batchee-1.1.0, stitchee-1.2.1, concise-0.9.0
UAT: l2ss-2.12.0rc14, batchee-1.2.0, stitchee-1.5.0, concise-0.10.0rc4

The latest versions in UAT have important changes/features—such as stitchee applying a dataset sorting, which enables consistent results—that the PROD versions don't yet have. Once the latest versions are deployed to PROD, I'd expect the tests to pass for PROD too.

[danielfromearth]

Versions in Production have been updated — so now stitchee and batchee are the same in both places!

PROD: l2ss-2.11.0, batchee-1.3.0, stitchee-1.6.1, concise-0.9.0
UAT: l2ss-2.12.0rc14, batchee-1.3.0, stitchee-1.6.1, concise-0.10.0rc5

[chris-durbin]

@ygliuvt - Is there anything that needs to be updated in this PR to ensure the SAMBAH tests run anytime STITCHEE or Batchee are deployed?

[ygliuvt]

@ygliuvt - Is there anything that needs to be updated in this PR to ensure the SAMBAH tests run anytime STITCHEE or Batchee are deployed?

The test needs to be added in config/services_tests_config_uat.json and config/services_tests_config_prod.json for batchee and stitchee services.

[flamingbear]

This mostly looks great with a couple of things to add. Like we mentioned in slack, I do have a couple of commits on my fork that I can push here or you can add them how you like.

As for the tests, I can't see any granules for that time period and you may need to add permissions for harmony_dev_user so that they can run automatically.

[danielfromearth]

@flamingbear , thanks for all of your efforts in reviewing this PR!

This mostly looks great with a couple of things to add. Like we mentioned in slack, I do have a couple of commits on my fork that I can push here or you can add them how you like.

As for the tests, I can't see any granules for that time period and you may need to add permissions for harmony_dev_user so that they can run automatically.

Since you've already made the changes, I'd say go ahead and push them from your fork. We can then assess if there's anything still remaining. I've reached out to some folks about the UAT visibility question, will update here when that is resolved.

[danielfromearth]

oh, I also want to note here that we discovered there is currently a difference between TEMPO's collections in UAT versus PROD — which happened because of recent reprocessing that only updated PROD granules. That needs to be addressed in order for the sambah regression tests to all pass, because there is currently one set of reference files — i.e., both UAT and PROD need to produce the same output. (This will be addressed in ticket TRADE-500)

[flamingbear]

@danielfromearth I pushed the changes to here. It should be ready for you to test. I still don't see any granules during the time period specified in the tests.

[danielfromearth]

I've been told that the permissions got out of sorts during some MMT updates, but will be worked on... standing by.