Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update @cumulus/sftp not autoClose stream to fix sync-granule unit test #3889

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jennyhliu
Copy link
Contributor

@jennyhliu jennyhliu commented Dec 20, 2024

Summary: Summary of changes

Addresses CUMULUS-XX: Develop amazing new feature

This thread suggested a workaround:
mscdex/ssh2#1175

sync-granule unit test error:

@cumulus/sync-granule:   Uncaught exception in tests/sync_granule_test.js
@cumulus/sync-granule:   Error: No response from server
@cumulus/sync-granule:   › cleanupRequests (/cumulus/packages/sftp-client/node_modules/ssh2/lib/protocol/SFTP.js:2730:15)
@cumulus/sync-granule:   › SFTP.push (/cumulus/packages/sftp-client/node_modules/ssh2/lib/protocol/SFTP.js:191:7)
@cumulus/sync-granule:   › onCHANNEL_CLOSE (/cumulus/packages/sftp-client/node_modules/ssh2/lib/utils.js:50:13)
@cumulus/sync-granule:   › ChannelManager.cleanup (/cumulus/packages/sftp-client/node_modules/ssh2/lib/utils.js:200:7)
@cumulus/sync-granule:   › Socket.<anonymous> (/cumulus/packages/sftp-client/node_modules/ssh2/lib/client.js:829:********)
@cumulus/sync-granule: {"level":"debug","message":"returning {\"size\":1098034,\"bucket\":\"abae********73c473c04b4d4fc3537f5d770cb307fbc1\",\"key\":\"file-staging/lpdaac-cumulus-phaseIII/MOD09GQ___6/036a63acbd3a8bd3da8faa2700166fb2/MOD09GQ.A2017224.h27v08.006.2017227165029.hdf\",\"source\":\"/granules/MOD09GQ.A2017224.h27v08.006.2017227165029.hdf\",\"fileName\":\"MOD09GQ.A2017224.h27v08.006.2017227165029.hdf\",\"type\":\"data\",\"checksumType\":\"CKSUM\",\"checksum\":\"143571********44\"}","sender":"unknown","timestamp":"2024-12-07T05:24:30.301Z"}
@cumulus/sync-granule: {"level":"debug","message":"finished ingest()","sender":"unknown","timestamp":"2024-12-07T05:24:30.301Z"}
@cumulus/sync-granule: {"level":"debug","message":"SyncGranule Complete. Returning output: {\"granules\":[{\"granuleId\":\"MOD09GQ.A2017224.h27v08.006.2017227165029\",\"dataType\":\"MOD09GQ\",\"version\":\"6\",\"provider\":\"MODAPS\",\"files\":[{\"size\":1098034,\"bucket\":\"abae********73c473c04b4d4fc3537f5d770cb307fbc1\",\"key\":\"file-staging/lpdaac-cumulus-phaseIII/MOD09GQ___6/036a63acbd3a8bd3da8faa2700166fb2/MOD09GQ.A2017224.h27v08.006.2017227165029.hdf\",\"source\":\"/granules/MOD09GQ.A2017224.h27v08.006.2017227165029.hdf\",\"fileName\":\"MOD09GQ.A2017224.h27v08.006.2017227165029.hdf\",\"type\":\"data\",\"checksumType\":\"CKSUM\",\"checksum\":\"143571********44\"}],\"sync_granule_duration\":7981,\"createdAt\":1733549062320}],\"granuleDuplicates\":{},\"process\":\"modis\"}","sender":"unknown","timestamp":"2024-12-07T05:24:30.305Z"}
@cumulus/sync-granule:   ✘ tests/sync_granule_test.js exited with a non-zero exit code: 1
@cumulus/sync-granule:   ─
@cumulus/sync-granule:   `--fail-fast` is on. At least 25 tests were skipped, as well as 1 test file.
@cumulus/sync-granule:   32 tests passed
@cumulus/sync-granule:   1 uncaught exception

Changes

  • Unfortunately, unit test still fails some times with the change.

PR Checklist

  • Update CHANGELOG
  • Unit tests
  • Ad-hoc testing - Deploy changes and test manually
  • Integration tests

@jennyhliu jennyhliu changed the title Update @cumulus/sftp not autoClose stream Fix sync-granule unit test Update @cumulus/sftp not autoClose stream to fix sync-granule unit test Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant