Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 transient tcp errors on large uploads #175

Open
Joe-Improbable opened this issue Oct 28, 2020 · 0 comments
Open

S3 transient tcp errors on large uploads #175

Joe-Improbable opened this issue Oct 28, 2020 · 0 comments

Comments

@Joe-Improbable
Copy link
Contributor

Joe-Improbable commented Oct 28, 2020

I've encountered a transient issue when performing large uploads to an S3 store.

I occasionally run into this error (perhaps once in 100 uploads):

Put "<s3 bucket name>/<chunk id>/<chunk id>.cacnk": write tcp <ip>-><ip>: use of closed network connection

A simple solution to this problem would just be to retry any failed PUT (or GET) requests, similar to how the RemoteHTTPStore does with error-retry option.

Minio-go implements retries but only for errors it deems retry-able (listed here), unfortunately any TCP errors won't be retried even though a number of them can be solved by retrying the request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant