-
Notifications
You must be signed in to change notification settings - Fork 42
Added more detail for a working SAM template example for a serverless lambda #88
Conversation
wattry
commented
Jun 15, 2020
- Added a makefile and deno directory
- Tested SAM template deployment
Does the SAM example deployment no longer work? I don't understand the need for this mv and empty makefile. Is that required/why does that help? Could you explain? Thanks! Edit: I tested the existing code with sam (version 0.44) and it worked... so I am confused at the issue! (I would definitely like to include it in the test suite somehow.) Edit2: I tested this and |
With regards to Edit1, I am relatively new to AWS and Deno so I have no problem figuring out how to add this to the test suit when I have some free time. With regards to Edit2: So that is my bad, I removed something in the makefile which apparently is required. I will push up a fix for this. Edit > I don't understand the need for this mv and empty makefile. Is that required/why does that help? Could you explain? Thanks! Does the move make sense in the context of the issue? |
Can you move these back up a directory? (I think it should work without the /deno/ change.) If not, I can fix it later. Thanks again for the investigation. 👍
build-HelloWorldFunction:
DENO_DIR=.deno_dir deno cache hello.ts
cp -R .deno_dir/gen/file$(ARTIFACTS_DIR)/ .deno_dir/LAMBDA_TASK_ROOT
cp -R .deno_dir/gen/file$(ARTIFACTS_DIR)/ .deno_dir/LAMBDA_TASK_ROOT But, at least for me (on OSX), this fails due to make using /private/var directory and confusing deno's compilation, which is worrisome. This means that compilation will not be cached. |
I've got it: build-HelloWorldFunction:
DENO_DIR=.deno_dir deno cache hello.ts
cp -R .deno_dir/gen/file$(CURDIR)/ .deno_dir/LAMBDA_TASK_ROOT |
Please correct me if my understanding of SAM is incorrect, does using Is there a specific reason the Lambda needs to have the .npmignore or README.md files stored? I suppose that was my rationale behind the |
sam respects .npmignore, so those wouldn't be included in the deployment. |
That's good to know! Thanks. |
Is this indentation purposeful, since I see you've used 2 spaces in the rest of the codebase? |
At least my Makefile required tabs 😞 |
Weird, I'm using VSCodium and it did not complain. Even weirder my build succeeded:
Edit: I am also using OSX Edit1: It looks like the ~/deno-lambda/example-sam/.aws-sam/build/HelloWorldFunction directory is empty. |
and still
strange. Are we going down the wrong path here? Very confusing. Note: I don't really understand! 😢 |
I gave up on SAM local functionality a while ago. There are too many moving parts and it's not stable enough in complex builds. I just had a laugh because I spent a good chunk of yesterday trying to compile deno for AWS and it lead me to your issue and eventually this repo. I didn't connect your account to the issue until right now. |
😄 . Yeah compiling is for sure a pain, we publish a binary in releases. I guess the question here is what is the point of Actually, I take back that
which sounds like the same issue (.aws-sam/build/HelloWorldFunction directory is empty.) I'm optimistic that if we can work this out then local invoke will work... |
@hayd yeah they are handy! I was a little hesitant to have someone else's lambda layer executing possibly sensitive API requests. However, this is so far in dev that it's unlikely we will be moving from Node to Deno anytime soon. So I am relatively new to the SAM world. We tried running local templates but it just created headaches because some of the intrinsic functions relied on resources that were created in other accounts or that did not work for the resources that we needed yet. I used a tool called dive to check the image and it looks like the lambda layer isn't getting pulled down/ created. That's what is throwing the error you posted, we had issues referencing layers locally, last I checked you needed an ARN version. |
I think it actually may be this issue aws/aws-sam-cli#1489 Since the lambda layer is referenced by an SAR application. That would be annoying. I am loath to publish a lambda layer explicitly as I have no idea how much that could end up costing. |
Agreed, I see now I have conflated two issues here. Most likely due to my limited work with SAM and Deno. The issue I originally was trying to solve is that build requires a runtime at build time, however we are not creating the runtime for the lambda we are accessing the runtime from a lambda layer. The makefile inclusion is a workaround I was using to trick SAM into thinking it was building the runtime locally but I was getting it from the reference to the SAR. However, that is not actually pulling down the layer and on top of that, I see now in your docs that you need to compile the deno function and dependencies. This means that the deploy and the function work but it must be compiling the code itself. What we could do is provide the lambda layer as an input parameter? Then if we can get the makefile to actually do the work you provided we would likely be able to run it locally. |
The first problem was solved by just referencing the AWS lambda layer ARN as an input parameter. Until I can figure out how better to do this I'm going to use this method, I'm very open to suggestions. Now the layer gets pulled down and it is a top of the base image. After much head bashing (excuse the pun) I have come to this makefile:
The reason why the original makefile did not work is that cp cannot recursively make the nested directory .deno_dir/LAMBDA_TASK_ROOT so in a long-winded way I solved that issue by making the directory before the copy. Wonderfully now you can see the docker image now has the bootstrap file, the deno executable, and the compiled code. This is all great, however, when I run the deploy the ARTIFACTS_DIR is not uploaded to the lambda and it is empty. I followed the following example and copied the example-sam project and was able to get this to work. However, they are not using a custom lambda layer as the runtime they are instead building the custom bootstrap as part of the lambda build. From my observations, it seems that if I manually compile the code for the deno application it will still pick up the uncompiled file and compile it. This leads me to believe that since the lambda is hot the then compiled code is executed on subsequent requests. This may be a design limitation on SAM's part or something that needs to be requested. Edit1: I deployed the rust application to AWS and I noticed there was a function name property on the SAM template, which was not on the HelloWorld template once I added that the compiled code was pushed to AWS. Edit 2: I have isolated the issue to the bootstrap file, I'm not sure yet how to debug it but because I'm passing transpiled code to the lambda there is no handler file that may be what is causing the error I am seeing in the lambda. |
Thanks. I'm going to sit on this for a while. It's a shame that SAR layer doesn't "just work" since IME it's kinda a pain to manage layer versions (especially when they can be reuploaded frequently, it's annoying they don't share more efficiently). It's nice that SAR references by version string rather than meaningless incremental number. I'm not happy you would have to upload the deno binary everytime (?). /rant I feel like there has to be a better way. Just to step back: what is the benefit of building vs only deploying? |
I apologise for the length here. TLDR;The build step is not required for Deno because all dependencies are listed in the src code so at runtime deno will simply download and transpile. Use the makefile at the bottom of this post and update the bootstrap file with suggestions. ReasoningThe build step is crucial and language-specific. Let me illustrate this with a Node project since this is my background. Ignore the runtime for the moment, assume we have the need for a specific version of Node because of some dependency.
Deno has a different execution cycle due to the way package management works, or lack thereof.
We can use the bundle subcommand to wrap all dependencies and transpile them in the makefile then store it as JS: build-HelloWorldFunction:
DENO_DIR=.deno_dir deno bundle hello.ts hello.js
mv hello.js hello.ts $(ARTIFACTS_DIR) We also need to have a logical test in the bootstrap script that tests to see if there is a .js file first then fall back on the .ts file else throw an error. Secondly, the cp logic can be removed, and lastly, the hello.ts can be removed from the bundled layer. Or alternatively, you could provide an alternative output bundle. To test this I ran a sam build then manually updated the hello.ts file with a I know that this is a long post so I'll give you a while to digest it, if I have any incorrect assumptions please correct me. |
There is HANDLER_EXT env variable which you can set to js https://github.com/hayd/deno-lambda#configuration I am erring (for the moment) two template files
In Example 2 it looks like you can define the makefile inside the template, that way the version is in the template which is IMO desirable. (Oh, it looks like you already use that feature above 👍 ) I still think this is a bug in SAM (that they don't support SAR layers). |
This repo is now archived because we are discontinuing the AWS Lambda layer for Deno. Instead we recommend you use a Docker image on AWS Lambda to run Deno. You can find a full guide in the docs. |