Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not possible to use @hypermod/cli to run transforms located in a private NPM package, right? #210

Open
alistairwilliamtaylor opened this issue Oct 20, 2023 · 9 comments
Assignees
Labels
📚 docs Improvements or additions to documentation 🥸 private packages

Comments

@alistairwilliamtaylor
Copy link

alistairwilliamtaylor commented Oct 20, 2023

Hello, and thank you for this wonderful project!

I'm just a junior dev trying to figure out how to set up codemods for our company's design system. I'd been playing around with setting up a script in our package which would use the jsCodeshift executable to run the transforms I'd written, but then I found hypermod and thought that could potentially save me from myself and I could instruct consumers to run something like:

npx @hypermod/cli --packages @myob/[email protected] ./src

However, our package @myob/myob-widgets is a private package, so I'm imagining that's why I'm getting the following error:

Error: Unable to locate package from Hypermod Community or NPM.
Make sure the package name "@myob/myob-widgets" is correct and try again.

Am I right in thinking that it won't work because @myob/myob-widgets is a private package? And are my options therefore to either (a) create a separate public NPM package with transforms which hypermod would be able to access, or (b) try to set up a script myself to run the transforms with jsCodeshift in our private package.

Thanks for your help, and my apologies if this question is in the wrong place / wildly off the mark

@danieldelcore
Copy link
Contributor

Hey @alistairwilliamtaylor 👋 Thanks for raising an issue! I should really create a page for this on the site.

It should be possible to use hypermod for private packages, you might need to use either the --registry and/or --registryToken flags to point the CLI to your private npm registry.

https://www.codeshiftcommunity.com/docs/cli#--registry
https://www.codeshiftcommunity.com/docs/cli#--registrytoken

Assuming developers within your company are using the correct .npmrc configuration, the --registry URL should be sufficient 🤔. Could you give it a go and let me know if that solves your problem?

@danieldelcore danieldelcore added 📚 docs Improvements or additions to documentation 🥸 private packages labels Oct 22, 2023
@alistairwilliamtaylor
Copy link
Author

Oh thank you so much for getting back to me about this - I'll give it a go in the next few days and let you know

@danieldelcore
Copy link
Contributor

Wonderful, feel free to reach out whenever! I'd love to make sure you guys can use hypermod successfully! 🙏

@alistairwilliamtaylor
Copy link
Author

Thanks Daniel!

I've been having a go over the past few days, but sadly no success yet.

I might potentially have a bit of a lead, though? I initially tried just with --registry but haven't had any luck with that, so tried providing --registryToken as well. This produced a somewhat different result, because the CLI would get to:

⚠️  Unable to locate Hypermod package: @hypermod/mod-myob__myob-widgets
ℹ Attempting to download npm package: @myob/myob-widgets

And then the cursor just hangs and hangs on that - it never produces an error message, but it also doesn't work. I tried providing a bogus registry and/or bogus registry token and it fails immediately if either of them is wrong, which leads me to believe that when it hangs it is a somewhat encouraging sign?

I'm happy to keep playing around with this, please feel free to let me know if you have any suggestions about things I could try

@danieldelcore
Copy link
Contributor

Ah really interesting, I wonder why it's hanging like that 🤔 Does the npm package have a large bundlesize? It could be hanging while it downloads and maybe fail silently. Could you share some logs if there are any?

@alistairwilliamtaylor
Copy link
Author

Initially I was cancelling the command because it was hanging for so long, but if I leave it for 5-6 minutes I eventually get:

<--- Last few GCs --->

[53233:0x130008000]   509554 ms: Scavenge 3931.4 (4128.2) -> 3919.4 (4128.2) MB, 6.8 / 0.0 ms  (average mu = 0.227, current mu = 0.136) task
[53233:0x130008000]   509623 ms: Scavenge 3931.7 (4128.2) -> 3920.6 (4129.2) MB, 8.9 / 0.0 ms  (average mu = 0.227, current mu = 0.136) task
[53233:0x130008000]   512496 ms: Mark-sweep 3933.4 (4129.2) -> 3919.7 (4130.0) MB, 2807.4 / 0.0 ms  (average mu = 0.200, current mu = 0.172) task scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x102c82984 node::Abort() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 2: 0x102c82b74 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 3: 0x102dc56e8 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 4: 0x102dc56ac v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 5: 0x102f474d4 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 6: 0x102f4ad30 v8::internal::Heap::CollectSharedGarbage(v8::internal::GarbageCollectionReason) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 7: 0x102f47c98 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 8: 0x102f455bc v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 9: 0x102fc5bcc v8::internal::ScavengeJob::Task::RunInternal() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
10: 0x102cde5d8 node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task>>) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
11: 0x102cdd2f4 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
12: 0x1034f5244 uv__async_io [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
13: 0x103506de4 uv__io_poll [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
14: 0x1034f56d4 uv_run [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
15: 0x102bc96c0 node::SpinEventLoop(node::Environment*) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
16: 0x102cbc800 node::NodeMainInstance::Run(int*, node::Environment*) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
17: 0x102cbc4ec node::NodeMainInstance::Run() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
18: 0x102c54370 node::Start(int, char**) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
19: 0x19bfeff28 start [/usr/lib/dyld]
[1]    53232 abort      npx @hypermod/cli --registry https://{PRIVATE_REGISTRY_HERE}/npm/ --registryToken

@danieldelcore
Copy link
Contributor

Ah I see, the package loader we use has some drawbacks. I suspect the bundlesize is too large which is causing it to run out of memory. I have implemented a more robust loader behind the [--experimentalLoader](https://github.com/hypermod-io/hypermod-community/blob/2b9179a34efcd9d1edcdbead5b1fd6c839f72d75/packages/cli/src/index.ts#L70C5-L70C5) flag. It should be able to handle larger packages but doesn't yet have support for --registryToken` 😭 I'll have to update it and see if that fixes your issue.

Is the package you're deploying you're codemods with quite large?

@alistairwilliamtaylor
Copy link
Author

Yeah it is quite large - it has all of our components for the design system in it. Sorry about that! I'm not opposed to creating a separate package for the hypermod codemods for now, so that I can start playing around more with hypermod. I'll try to have a go this week.

Thanks for all of your support with this! 😄

@danieldelcore
Copy link
Contributor

danieldelcore commented Oct 31, 2023

No worries at all! Feel free to reach out if you run into any more issues, I'd be more than happy to help!

Yeah, a standalone package in this case would be the best way to go!
I recommend having a look at this guide: https://www.codeshiftcommunity.com/docs/external-packages 😄

@danieldelcore danieldelcore self-assigned this Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📚 docs Improvements or additions to documentation 🥸 private packages
Projects
None yet
Development

No branches or pull requests

2 participants