Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: improve image cache error handling #214

Merged
merged 26 commits into from
May 13, 2024
Merged

Conversation

rafaelcr
Copy link
Collaborator

@rafaelcr rafaelcr commented May 8, 2024

Add the option to handle different exit codes from the image cache script depending on error circumstances, to determine if the API should retry the cache later or not.

Copy link

github-actions bot commented May 8, 2024

Vercel deployment URL: https://token-metadata-rfnpjwjsn-blockstack.vercel.app 🚀

Copy link

codecov bot commented May 8, 2024

Codecov Report

Attention: Patch coverage is 86.90476% with 11 lines in your changes are missing coverage. Please review.

❗ No coverage uploaded for pull request base (beta@a3aad1d). Click here to learn what that means.

❗ Current head f305cb8 differs from pull request most recent head dd286c7. Consider uploading reports for the commit dd286c7 to get more accurate results

Files Patch % Lines
src/token-processor/util/image-cache.ts 86.66% 10 Missing ⚠️
src/token-processor/util/metadata-helpers.ts 85.71% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             beta     #214   +/-   ##
=======================================
  Coverage        ?   86.29%           
=======================================
  Files           ?       44           
  Lines           ?     5179           
  Branches        ?      509           
=======================================
  Hits            ?     4469           
  Misses          ?      699           
  Partials        ?       11           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rafaelcr rafaelcr marked this pull request as ready for review May 9, 2024 15:20
@rafaelcr rafaelcr changed the title fix: handle cache script errors fix: improve image cache error handling May 9, 2024
@rafaelcr rafaelcr requested a review from zone117x May 9, 2024 15:40
Comment on lines +54 to +58
// Cache the token so we can reuse it for other images.
process.env['IMAGE_CACHE_GCS_AUTH_TOKEN'] = json.access_token;
return json.access_token;
} catch (error) {
throw new Error(`Error fetching GCS access token: ${error.message}`);
throw new Error(`GCS access token error: ${error}`);
Copy link
Member

@zone117x zone117x May 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The @google-cloud/storage library can be used to handle the auth and upload a lot easier, it would be like 3-4 lines, I'm using it over here https://github.com/hirosystems/stacks-node-crawler/blob/720d4f9f79155a84a19c3d1028f0df1a54b3fbfb/src/index.ts#L5

E.g.

import { Storage } from '@google-cloud/storage';
const bucket = new Storage().bucket(myBucket); // auth handled automatically
const uploadStream = bucket.file(myfile).createWriteStream({ contentType: 'image/png' });
stream.pipe(uploadStream);

Copy link
Member

@zone117x zone117x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a comment about maybe simplifying the GCS code. Up to you though, if the current code works then seems fine to me.

@rafaelcr
Copy link
Collaborator Author

Thanks @zone117x I opened a PR for this because k8s has some unique rules on auth tokens so AFAIK using the library is not as simple, we'd need to create a service account and some other things

@rafaelcr rafaelcr merged commit 115a745 into beta May 13, 2024
5 checks passed
@rafaelcr rafaelcr deleted the fix/image-cache-tweaks branch May 13, 2024 16:33
blockstack-devops pushed a commit that referenced this pull request May 13, 2024
## [0.7.0-beta.5](v0.7.0-beta.4...v0.7.0-beta.5) (2024-05-13)

### Bug Fixes

* improve image cache error handling ([#214](#214)) ([115a745](115a745))
@blockstack-devops
Copy link

🎉 This PR is included in version 0.7.0-beta.5 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

blockstack-devops pushed a commit that referenced this pull request May 13, 2024
## [0.7.0](v0.6.3...v0.7.0) (2024-05-13)

### Features

* add admin rpc to reprocess token image cache ([#205](#205)) ([2fdcb33](2fdcb33))
* update ts client with image thumbnails ([#206](#206)) ([c24cb56](c24cb56))
* upload token images to gcs ([#204](#204)) ([1cec219](1cec219))

### Bug Fixes

* get access token properly ([a6b98c5](a6b98c5))
* get gcs auth token dynamically for image cache ([#210](#210)) ([8434b22](8434b22))
* image cache agent arg types ([5826628](5826628))
* improve image cache error handling ([#214](#214)) ([115a745](115a745))
* reuse gcs token and validate image cache script errors ([#213](#213)) ([5e1af5c](5e1af5c))
@blockstack-devops
Copy link

🎉 This PR is included in version 0.7.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants