From 8de557e136d985af3ee210d1477c44e7469de077 Mon Sep 17 00:00:00 2001 From: Marius Kleidl Date: Fri, 1 Mar 2024 10:44:35 +0100 Subject: [PATCH] docs: Improve language --- docs/_advanced-topics/hooks.md | 20 +++++++------ docs/_getting-started/configuration.md | 40 +++++++++++++------------- docs/_getting-started/installation.md | 8 +++--- docs/_getting-started/usage.md | 18 ++++++------ docs/_storage-backends/aws-s3.md | 9 +++--- docs/_storage-backends/local-disk.md | 4 +-- docs/_storage-backends/overview.md | 6 ++-- docs/faq.md | 8 +++--- docs/index.md | 10 +++---- 9 files changed, 62 insertions(+), 61 deletions(-) diff --git a/docs/_advanced-topics/hooks.md b/docs/_advanced-topics/hooks.md index 84db84b1..dee431f9 100644 --- a/docs/_advanced-topics/hooks.md +++ b/docs/_advanced-topics/hooks.md @@ -17,10 +17,10 @@ When integrating tusd into an application, it is important to establish a commun When a specific event happens during an upload, a corresponding hook is triggered. This hook is then delivered to you application using one of many ways: -1. File hooks: tusd executes a provided executable file or script (similar to Git hooks). -2. HTTP hooks: tusd sends HTTP POST request to a custom endpoint. -3. gRPC hooks: tusd invokes a method on a remote gRPC endpoint. -3. Plugin hooks: tusd loads a plugin from disk and invokes its methods. +1. [File hooks](#file-hooks): tusd executes a provided executable file or script (similar to Git hooks). +2. [HTTP hooks](#https-hooks): tusd sends HTTP POST request to a custom endpoint. +3. [gRPC hooks](#grpc-hooks): tusd invokes a method on a remote gRPC endpoint. +3. [Plugin hooks](#plugin-hooks): tusd loads a plugin from disk and invokes its methods. ## List of Available Hooks @@ -38,7 +38,7 @@ The table below provides an overview of all available hooks. Users should be aware of following things: - If a hook is _blocking_, tusd will wait with further processing until the hook is completed. This is useful for validation and authentication, where further processing should be stopped if the hook determines to do so. However, long execution time may impact the user experience because the upload processing is blocked while the hook executes. - If a hook is _non-blocking_, tusd will continue processing the request while the hook is being executed. The hook is able to influence the upload in some way, but the hook must be aware that an HTTP response might already be sent. This is useful for logging upload progress or starting the post-processing of uploaded data. -- During the lifecycle of an upload, multiple hooks may be triggered. Their execution can happen concurrently and in general the order of hooks is not guaranteed. This is especially true if hooks are delivered over the network. For example, the post-finish hook might be before post-create. Your hooks should be designed to not rely on an order. The only guarantees are that pre-create will always be the first hook for any upload and that post-finish is started after pre-finish has been completed. +- During the lifecycle of an upload, multiple hooks may be triggered. Their execution can happen concurrently and in general the order of hooks is not guaranteed. This is especially true if hooks are delivered over the network. For example, the post-finish hook might be delivered before post-create. Your hooks should be designed to not rely on an order. The only guarantees are that pre-create will always be the first hook for any upload and that post-finish is started after pre-finish has been completed. - Not all hooks are enabled by default for performance reasons. You can enable/disable each hook individually using the `-hooks-enabled-events` flag. ## Hook Requests and Responses @@ -222,7 +222,7 @@ When the process exits with the zero exit code, tusd reads the process' stdout a The process' stderr is redirected to tusd's stderr and can be used for logging from inside the hook. -An example is available at [/examples/hooks/file]({{ site.baseurl }}/examples/hooks/file). +An example is available at [github.com/tus/tusd/examples/hooks/file](https://github.com/tus/tusd/tree/main/examples/hooks/file). ### HTTP(S) Hooks @@ -250,7 +250,7 @@ When the endpoint responds with a non-2XX status code, tusd interprets this as a When the endpoint responds with a 2XX status code, tusd reads the response body and parses it as a JSON-encoded hook response. This allows the hook to customize the HTTP response, reject and abort uploads. -An example is available at [/examples/hooks/http]({{ site.baseurl }}/examples/hooks/http). +A Python-based example is available at [github.com/tus/tusd/examples/hooks/http](https://github.com/tus/tusd/tree/main/examples/hooks/http). #### Retries @@ -273,7 +273,9 @@ $ tusd -hooks-grpc localhost:8081 ... ``` -The endpoint must implement the hook handler service as specified in [/pkg/hooks/grpc/proto/hook.proto]({{ site.baseurl }}/pkg/hooks/grpc/proto/hook.proto). Its `InvokeHook` method will be invoked for each triggered events and will be passed the hook request. +The endpoint must implement the hook handler service as specified in [github.com/tus/tusd/pkg/hooks/grpc/proto/hook.proto](https://github.com/tus/tusd/blob/main/pkg/hooks/grpc/proto/hook.proto). Its `InvokeHook` method will be invoked for each triggered events and will be passed the hook request. + +A Python-based example is available at [github.com/tus/tusd/examples/hooks/grpc](https://github.com/tus/tusd/tree/main/examples/hooks/grpc). #### Retries @@ -290,7 +292,7 @@ File hooks are an easy way to receive events from tusd, but can induce overhead Plugin hooks provide a sweet spot between these two worlds. You can create a plugin with any programming language. tusd then loads this plugin by starting it as a standalone process, restarting it if necessary, and communicating with it over local sockets. This system is powered by [go-plugin](https://github.com/hashicorp/go-plugin), which is designed for Go, but provides cross-language support. The approach provides a low-overhead hook handler, which is still able to track state between hooks. -To learn more, have a look at the example at [/examples/hooks/plugin]({{ site.baseurl }}/examples/hooks/plugin). +To learn more, have a look at the example at [github.com/tus/tusd/examples/hooks/plugin](https://github.com/tus/tusd/tree/main/examples/hooks/plugin). ## Common Uses diff --git a/docs/_getting-started/configuration.md b/docs/_getting-started/configuration.md index dcb8ff83..192f50e0 100644 --- a/docs/_getting-started/configuration.md +++ b/docs/_getting-started/configuration.md @@ -12,11 +12,11 @@ nav_order: 3 ## Configuration options -tusd can be configured and customized by passing flags when starting the process. Please consult the output of `tusd -help` for all available flags. +Tusd can be configured and customized by passing flags when starting the process. Please consult the output of `tusd -help` for all available flags. ## Network configuration -By default, tusd listens on port 8080 and all available interface. This can be changed by the `-host` and `-port` flags: +By default, tusd listens on port 8080 and all available interfaces. This can be changed using the `-host` and `-port` flags: ```bash $ tusd -host 127.0.0.1 -port 1337 @@ -34,7 +34,7 @@ $ tusd -unix-sock /var/my-tusd.sock ### Base path -Uploads can be created by sending a `POST` request to the upload creation endpoint. This endpoint is, by default, available under the `/files/` path, e.g. `http://localhost:8080/files/`. Other paths cannot be used to create uploads. This path can be customized using the `-base-path` flag: +Uploads can be created by sending a [`POST` request](https://tus.io/protocols/resumable-upload#creation) to the upload creation endpoint. This endpoint is, by default, available under the `/files/` path, e.g. `http://localhost:8080/files/`. Paths other than the base path cannot be used to create uploads. The base path can be customized using the `-base-path` flag: ```bash # Upload creation at http://localhost:8080/api/uploads/ @@ -45,29 +45,29 @@ $ tusd -base-path / ### Proxies -When tusd is utilized behind a reverse proxy (Nginx, HAProxy etc), tusd and the proxy must be configured appropriate to works together. +In some cases, it is necessary to run tusd behind a reverse proxy (Nginx, HAProxy etc), for example for TLS termination or serving multiple services on the same hostname. To properly do this, tusd and the proxy must be configured appropriately. -Firstly, you must set `-behind-proxy` flag indicating tusd that it a reverse proxy is in use and it should respect the `X-Forwarded-*`/`Forwarded` headers: +Firstly, you must set the `-behind-proxy` flag indicating tusd that a reverse proxy is in use and that it should respect the `X-Forwarded-*`/`Forwarded` headers: ```bash $ tusd -behind-proxy ``` -Secondly, some of the reverse proxy's settings should be adjusted, depending on the used software: +Secondly, some of the reverse proxy's settings should be adjusted. The exact steps depend on the used proxy, but the following points should be checked: -- *Disable request buffering.* Nginx, for example, reads the entire incoming HTTP request, including its body, before sending it to the backend, by default. This behavior defeats the purpose of resumability where an upload is processed while it's being transferred. Therefore, such as feature should be disabled. +- *Disable request buffering.* Nginx, for example, reads the entire incoming HTTP request, including its body, before sending it to the backend, by default. This behavior defeats the purpose of resumability where an upload is processed and saved while it's being transferred, allowing it be resumed. Therefore, such a feature must be disabled. - *Adjust maximum request size.* Some proxies have default values for how big a request may be in order to protect your services. Be sure to check these settings to match the requirements of your application. -- *Forward hostname and scheme.* If the proxy rewrites the request URL, the tusd server does not know the original URL which was used to reach the proxy. This behavior can lead to situations, where tusd returns a redirect to a URL which can not be reached by the client. To avoid this confusion, you can explicitly tell tusd which hostname and scheme to use by supplying the `X-Forwarded-Host` and `X-Forwarded-Proto` headers. +- *Forward hostname and scheme.* If the proxy rewrites the request URL, the tusd server does not know the original URL which was used to reach the proxy. This behavior can lead to situations, where tusd returns a redirect to a URL which can not be reached by the client. To avoid this issue, you can explicitly tell tusd which hostname and scheme to use by supplying the `X-Forwarded-Host` and `X-Forwarded-Proto` headers. Configure the proxy to set these headers to the original hostname and protocol when forwarding requests to tusd. Explicit examples for the above points can be found in the [Nginx configuration](https://github.com/tus/tusd/blob/main/examples/nginx.conf) which is used to power the [tusd.tusdemo.net](https://tusd.tusdemo.net) instance. -## Protocol extensions +## Protocol settings ### Maximum upload size -By default, tusd does not restrict the maximum size of a single upload. If you want to apply such a limit, use the `-max-size` flag: +By default, tusd does not restrict the maximum size of a single upload and allows infinitely large files. If you want to apply such a limit, use the `-max-size` flag: ```bash # Allow uploads up to 1000000000 bytes (= 1GB) @@ -76,7 +76,7 @@ $ tusd -max-size 1000000000 ### Disable downloads -tusd allows any user to retrieve a previously uploaded file by issuing an HTTP GET request to the corresponding upload URL. This is possible as long as the uploaded files have not been deleted or moved to another location in the storage backend. While it is a handy feature for debugging and testing your setup, there are situations where you don't want to allow downloads. To completely disable downloads, use the `-disable-download` flag: +Tusd allows any user to retrieve a previously uploaded file by issuing an HTTP GET request to the corresponding upload URL. This is possible as long as the uploaded files have not been deleted or moved to another location in the storage backend. While it is a handy feature for debugging and testing your setup, there are situations where you don't want to allow downloads. To completely disable downloads, use the `-disable-download` flag: ```bash $ tusd -disable-download @@ -84,7 +84,7 @@ $ tusd -disable-download ### Disable upload termination -The [tus termination extensions](https://tus.io/protocols/resumable-upload#termination) allows clients to terminate uploads (complete or incomplete) they are no longer interested in. In this case, the associated files in the storage backend will be removed. If you don't want to allow users to delete uploads, use the `-disable-termination` flag to disable this extension: +The [tus termination extension](https://tus.io/protocols/resumable-upload#termination) allows clients to terminate uploads (complete or incomplete) in which they are no longer interested. In this case, the associated files in the storage backend will be removed and the upload cannot be used anymore. If you don't want to allow users to delete uploads, use the `-disable-termination` flag to disable this extension: ```bash $ tusd -disable-termination @@ -92,19 +92,19 @@ $ tusd -disable-termination ## Storage backend -Tusd has been designed with flexible storage backends in mind and can store the received uploads on local disk or various cloud provides (AWS S3, Azure Cloud Storage, and Google Cloud Storage). By default, tusd will store uploads in the directory specified by the `-upload-dir` flag (which defaults to `./data`). Please consult the dedicated [Storage Backends section]({{ site.baseurl }}/storage-backends/overview/) for details on how to use different storage backend and configure them. +Tusd has been designed with flexible storage backends in mind and can store the received uploads on local disk or various cloud provides (AWS S3, Azure Cloud Storage, and Google Cloud Storage). By default, tusd will store uploads in the directory specified by the `-upload-dir` flag (which defaults to `./data`). Please consult the dedicated [Storage Backends section]({{ site.baseurl }}/storage-backends/overview/) for details on how to use a different storage backend and configure them. ## Integrations into applications with hooks -When integrating tusd into an application, it is important to establish a communication channel between tusd and your main application. For this purpose, tusd provides a hook system which triggers user-defined actions when certain events happen, for example when an upload is created or finished. This simple-but-powerful system enables many uses, such as logging, validation, authorization, and post-processing of the uploaded files. Please consult the dedicated [Storage Backends section]({{ site.baseurl }}/advanced-topics/hooks/) for details on how to use the hook system. +When integrating tusd into an application, it is important to establish a communication channel between tusd and your main application. For this purpose, tusd provides a hook system which triggers user-defined actions when certain events happen, for example when an upload is created or finished. This simple-but-powerful system enables many uses, such as logging, validation, authorization, and post-processing of the uploaded files. Please consult the dedicated [hooks section]({{ site.baseurl }}/advanced-topics/hooks/) for details on how to use the hook system. ## Cross-Origin Resource Sharing (CORS) -When tusd is used in a web application and the tusd server is reachable under a different origin (domain, scheme, or port) than the frontend itself, browsers put restrictions on cross-origin requests for security reasons. For example, your main application is running on `https://example.org` but your tusd server is hosted at `https://uploads.example.org`. In this case, the server needs to use the [Cross-Origin Resource Sharing (CORS) mechanism](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) to signal the browser that it will accept requests from `https://example.org`. +When tusd is used in a web application and the tusd server is reachable under a different origin (domain, scheme, or port) than the frontend itself, browsers put restrictions on the cross-origin requests from the frontend to tusd for security reasons. For example, your main application is running on `https://example.org` but your tusd server is hosted at `https://uploads.example.org`. In this case, the server needs to use the [Cross-Origin Resource Sharing (CORS) mechanism](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) to signal the browser that it will accept requests from `https://example.org`. -To make your setup easier, tusd already includes the necessary CORS configuration to allow communication with tus clients. By default, tusd will allow cross-origin requests from any origin. +To make your setup easier, tusd already includes the necessary CORS configuration to allow communication with tus clients. By default, tusd will allow cross-origin requests from any origin. If these defaults work for your application, you don't have to change the CORS configuration. -If you want to restrict the origins or add additional header fields to the CORS configuration, utilize the `-cors-*` flags: +If you do want to restrict the origins or add additional header fields to the CORS configuration, utilize the `-cors-*` flags: ```bash $ tusd \ @@ -130,7 +130,7 @@ $ tusd -disable-cors If you want tusd to be accessible via HTTPS, there are two options: -1. Use a TLS-terminating reverse proxy, such as Nginx. The proxy is configured to accept HTTPS requests from the clients and forwards unencrypted HTTP requests to tusd. This approach is the most flexible and recommended approach as such proxies provide detailed configuration options for HTTPS and are well tested. Please see the [section on proxies](#proxies) for additional details when using tusd with proxies. +1. Use a TLS-terminating reverse proxy, such as Nginx. The proxy must be configured to accept HTTPS requests from clients and forward unencrypted HTTP requests to tusd. This approach is the most flexible and recommended method as such proxies provide detailed configuration options for HTTPS and are well tested. Please see the [section on proxies](#proxies) for additional considerations when using tusd with proxies. 2. Tusd itself provides basic TLS support for HTTPS connections. In contrast to dedicated TLS-terminating proxies, tusd supports less configuration options for tuning the TLS setup. However, the built-in HTTPS support is useful for development, testing and encrypting internal traffic. It can be enabled by supplying a certificate and private key. Note that the certificate file must include the entire chain of certificates up to the CA certificate and that the key file must not be encrypted/require a passphrase. The available modes are: @@ -167,6 +167,6 @@ If tusd receives a SIGINT or SIGTERM signal, it will initiate a graceful shutdow Once the graceful shutdown is started, tusd will stop listening on its port and won't accept new connections anymore. Idle connections are closed down. Already running requests will be given a grace period to complete before their connections are closed as well. PATCH and POST requests with a request body are interrupted, so that data stores can gracefully finish saving all the received data until that point. If all requests have been completed, tusd will exit. -If not all requests have been completed in the period defined by the `-shutdown-timeout` flag, tusd will exit regardless. By default, tusd will give all requests 10 seconds to complete their processing. If you do not want to wait for requests, use `-shutdown-timeout=0`. +If not all requests have been completed in the period defined by the `-shutdown-timeout` flag, tusd will exit regardless. By default, tusd will give all requests 10 seconds to complete their processing. If you do not want to wait for requests, use `-shutdown-timeout=0` to shut down immediately. -tusd will also immediately exit if it receives a second SIGINT or SIGTERM signal. It will also always exit immediately if a SIGKILL is received. +tusd will also immediately exit if it receives a second SIGINT or SIGTERM signal. It will also always exit immediately if a SIGKILL signal is received. diff --git a/docs/_getting-started/installation.md b/docs/_getting-started/installation.md index c0d0bf1c..cd3f6d94 100644 --- a/docs/_getting-started/installation.md +++ b/docs/_getting-started/installation.md @@ -13,7 +13,7 @@ There are multiple methods for installing tusd. Please choose one that fits your {:toc} -## Download pre-builts binaries (recommended) +## Download pre-built binaries (recommended) You can download ready-to-use packages including binaries for OS X, Linux and Windows in various formats of the @@ -23,18 +23,18 @@ Once the archive is extracted, the file `tusd` (or `tusd.exe`) is ready to be ex ## Compile from source -The only requirement for building tusd is [Go](http://golang.org/doc/install). +If you don't want to download a pre-built package or none is available for your system, you can also compile tusd on your own. The only requirement is [Go](http://golang.org/doc/install). We only test and support the [two latest major releases](https://go.dev/dl/) of Go, although tusd might also run with older versions. -Once a recent Go version is installed, you can clone the git repository, install -the remaining dependencies and build the binary: +Once a recent Go version is installed, you can clone the git repository and build the binary: ```bash git clone https://github.com/tus/tusd.git cd tusd go build -o tusd cmd/tusd/main.go +# The binary is saved in ./tusd ``` ## Docker container diff --git a/docs/_getting-started/usage.md b/docs/_getting-started/usage.md index d31432d7..cbe76ae8 100644 --- a/docs/_getting-started/usage.md +++ b/docs/_getting-started/usage.md @@ -16,13 +16,13 @@ Starting the tusd upload server is as simple as invoking a single command. For e ``` $ tusd -upload-dir=./data -2024/02/19 12:10:48.069284 Using '/Users/marius/workspace/tus/tusd/data' as directory storage. -2024/02/19 12:10:48.069576 Using 0.00MB as maximum size. -2024/02/19 12:10:48.069634 Supported tus extensions: creation,creation-with-upload,termination,concatenation,creation-defer-length -2024/02/19 12:10:48.069638 Using 0.0.0.0:8080 as address to listen. -2024/02/19 12:10:48.069639 Using /files/ as the base path. -2024/02/19 12:10:48.069658 Using /metrics as the metrics path. -2024/02/19 12:10:48.069982 You can now upload files to: http://[::]:8080/files/ +Using '/Users/marius/workspace/tus/tusd/data' as directory storage. +Using 0.00MB as maximum size. +Supported tus extensions: creation,creation-with-upload,termination,concatenation,creation-defer-length +Using 0.0.0.0:8080 as address to listen. +Using /files/ as the base path. +Using /metrics as the metrics path. +You can now upload files to: http://[::]:8080/files/ ``` The last line from tusd's output indicates the *upload creation URL*: @@ -40,7 +40,7 @@ Uploaded files will be stored by default in the directory specified with the `-u # Connecting clients -Once tusd is running, any tus-compatible client can connect to tusd and upload files. Usually, the only required client configuration is point the client's endpoint setting to tusd's upload creation URL. +Once tusd is running, any tus-compatible client can connect to tusd and upload files. To achieve this, point the client's endpoint setting to tusd's upload creation URL. Below you can find a few examples for common tus client, assuming that tusd is accepting uploads at `http://localhost:8080/files/`, which is the default upload creation URL. @@ -78,7 +78,7 @@ new Uppy() ## tus-java-client -For [tus-java-client](https://github.com/tus/tus-java-client), pass the upload creation URL in the [`TusClient#setUploadCreationURL` method](https://javadoc.io/doc/io.tus.java.client/tus-java-client/latest/io/tus/java/client/TusClient.html): +For [tus-java-client](https://github.com/tus/tus-java-client), pass the upload creation URL to the [`TusClient#setUploadCreationURL` method](https://javadoc.io/doc/io.tus.java.client/tus-java-client/latest/io/tus/java/client/TusClient.html): ```java TusClient client = new TusClient(); diff --git a/docs/_storage-backends/aws-s3.md b/docs/_storage-backends/aws-s3.md index dd088448..5494308e 100644 --- a/docs/_storage-backends/aws-s3.md +++ b/docs/_storage-backends/aws-s3.md @@ -21,7 +21,7 @@ $ tusd -s3-bucket=my-test-bucket.com ... ``` -Credentials can also be loaded from a shared credentials file (`~/.aws/credentials`) as described in the [AWS SDK for Go](https://github.com/aws/aws-sdk-go#configuring-credentials). You still need to declare the `AWS_REGION` value which isn't conventionally associated with credentials. +Credentials can also be loaded from a shared credentials file (`~/.aws/credentials`) as described in the [AWS SDK for Go](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/#specifying-credentials). You still need to declare the `AWS_REGION` value which isn't conventionally associated with credentials. ### Alternative endpoints @@ -43,7 +43,7 @@ $ tusd -s3-bucket=my-test-bucket.com -s3-object-prefix=uploads/ ### AWS S3 Transfer Acceleration -If your S3 bucket has been configured for [AWS S3 Transfer Acceleration](https://aws.amazon.com/s3/transfer-acceleration/) and you want to make use of that advanced service, you can direct tusd to automatically use the designated AWS acceleration endpoint for your bucket by including the optional +If your S3 bucket has been configured for [AWS S3 Transfer Acceleration](https://aws.amazon.com/s3/transfer-acceleration/) and you want to make use of that service, you can direct tusd to automatically use the designated AWS acceleration endpoint for your bucket by including the optional command line flag `-s3-transfer-acceleration` as follows: ```bash @@ -54,7 +54,7 @@ $ tusd -s3-bucket=my-test-bucket.com -s3-transfer-acceleration ## Permissions -For full functionality of the S3 backend, the user accessing the bucket must have at least following AWS IAM policy permissions for the bucket and all of its subresources: +For full functionality of the S3 backend, the user accessing the bucket must have at least the following AWS IAM policy permissions for the bucket and all of its subresources: ``` s3:AbortMultipartUpload @@ -85,8 +85,7 @@ The file object is not visible in the S3 bucket before the upload is finished be ### Metadata -If [metadata](https://tus.io/protocols/resumable-upload#upload-metadata) is associated with the upload during creation, it will be added to the file object once the upload is finished. Because the metadata on S3 objects must only contain ASCII characters, tusd will replace every non-ASCII character -with a question mark (for example, "Menü" will be "Men?"). +If [metadata](https://tus.io/protocols/resumable-upload#upload-metadata) is associated with the upload during creation, it will be added to the file object once the upload is finished. Because the metadata on S3 objects must only contain ASCII characters, tusd will replace every non-ASCII character with a question mark. For example, "Menü" will become "Men?". In addition, the metadata is also stored in the informational object, which can be used to retrieve the original metadata without any characters being replaced. diff --git a/docs/_storage-backends/local-disk.md b/docs/_storage-backends/local-disk.md index aa65b0ff..fb3fdf68 100644 --- a/docs/_storage-backends/local-disk.md +++ b/docs/_storage-backends/local-disk.md @@ -19,10 +19,10 @@ When a new upload is created, for example with the upload ID `abcdef123`, tusd c ## Issues with NFS and shared folders -Tusd maintains [upload locks]({{ site.baseurl }}/advanced-topics/locks/) on disk to ensure exclusive access to uploads and prevent data corruption. These disk-based locks utilize hard links, which might not supported by older NFS versions or when a folder is shared in a VM using VirtualBox or Vagrant. In these cases, you might get errors like this: +Tusd maintains [upload locks]({{ site.baseurl }}/advanced-topics/locks/) on disk to ensure exclusive access to uploads and prevent data corruption. These disk-based locks utilize hard links, which might not be supported by older NFS versions or when a folder is shared in a VM using VirtualBox or Vagrant. In these cases, you might get errors like this: ``` TemporaryErrors (Lockfile created, but doesn't exist) ``` -We recommend you to ensure that your file system supports hard links, use a different file system, or use one of tusd's cloud storage abilities. If the problem still persists, please open a bug report. +We recommend you to ensure that your file system supports hard links, use a different file system, or use one of tusd's cloud storage backends. If the problem still persists, please open a bug report. diff --git a/docs/_storage-backends/overview.md b/docs/_storage-backends/overview.md index ea2f4dba..a95b51f7 100644 --- a/docs/_storage-backends/overview.md +++ b/docs/_storage-backends/overview.md @@ -6,7 +6,7 @@ nav_order: 1 # Storage backends -Tusd can store the uploaded files either on local disk or various cloud storage services: +Tusd can store the uploaded files either locally or on various cloud storage services: - [Local disk]({{ site.baseurl }}/storage-backends/local-disk/) - [AWS S3 (and S3-compatible services)]({{ site.baseurl }}/storage-backends/aws-s3/) @@ -17,8 +17,8 @@ Tusd can store the uploaded files either on local disk or various cloud storage While the exact details of how uploaded files are stored depend on the chosen backend, usually two files/objects are created and modified while the upload is progressing: -- An informational file/object with the `.info` extension holds meta information about the upload, such as its size, its metadata, and whether it is used in conjunction with the [concatenation extension](https://tus.io/protocols/resumable-upload#concatenation). The data encoded using [JSON](https://www.json.org/json-en.html). More details can be found in the [documentation of the underlying data structure](https://pkg.go.dev/github.com/tus/tusd@v1.13.0/pkg/handler#FileInfo). -- Another file/object is created to store the actual file data. It does not have a characteristic extension and its name is either set by the [pre-create hook]({{ site.baseurl }}/advanced-topics/hooks/) or chosen by the storage backend. Once the upload is finished, it will contain the entire uploaded data. Please note depending on the storage backend (e.g. S3), this file/object might not be present until all data has been transferred. +- An informational file/object with the `.info` extension holds meta information about the upload, such as its size, its metadata, and whether it is used in conjunction with the [concatenation extension](https://tus.io/protocols/resumable-upload#concatenation). The data is encoded using [JSON](https://www.json.org/json-en.html). A list of the stored properties and their purpose can be found in the [documentation of the underlying data structure](https://pkg.go.dev/github.com/tus/tusd/pkg/handler#FileInfo). +- Another file/object is created to store the actual file data. It does not have a characteristic extension and its name is either set by the [pre-create hook]({{ site.baseurl }}/advanced-topics/hooks/) or chosen by the storage backend. Once the upload is finished, it will contain the entire uploaded data. Please note that depending on the storage backend (e.g. S3), this file/object might not be present until all data has been transferred. Once an upload is finished, both files/objects are preserved for further processing depending on your application's needs. The informational file/object is useful to retrieve upload metadata and thus not automatically removed by tusd. diff --git a/docs/faq.md b/docs/faq.md index 5764fd71..ebbf87fd 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -15,7 +15,7 @@ nav_order: 2 ### How can I access tusd using HTTPS? -Yes, you can achieve this by either running tusd behind a TLS-terminating proxy (such as Nginx or Apache) or configuring tusd to serve HTTPS directly. Details on both approaches are given in the [configuration guide]({{ site.baseurl }}/getting-started/configuration/#httpstls). +You can achieve this by either running tusd behind a TLS-terminating proxy (such as Nginx or Apache) or configuring tusd to serve HTTPS directly. Details on both approaches are given in the [configuration guide]({{ site.baseurl }}/getting-started/configuration/#httpstls). ### Can I run tusd behind a reverse proxy? @@ -23,7 +23,7 @@ Yes, it is absolutely possible to do so. Please consult the [section about confi ### Can I run custom verification/authentication checks before an upload begins? -Yes, this is made possible by the [hook system]({{ site.baseurl }}/docs/hooks.md) inside the tusd binary. It enables custom routines to be executed when certain events occurs, such as a new upload being created which can be handled by the `pre-create` hook. Inside the corresponding hook file, you can run your own validations against the provided upload metadata to determine whether the action is actually allowed or should be rejected by tusd. Please have a look at the [corresponding documentation]({{ site.baseurl }}/docs/hooks.md#pre-create) for a more detailed explanation. +Yes, this is made possible by the [hook system]({{ site.baseurl }}/advanced-topics/hooks/) inside the tusd binary. It enables custom routines to be executed when certain events occurs, such as a new upload being created which can be handled by the `pre-create` hook. Inside the corresponding hook logic, you can run your own validations against the provided upload metadata to determine whether the action is actually allowed or should be rejected by tusd. Please have a look at the [corresponding example]({{ site.baseurl }}/advanced-topics/hooks/#receiving-and-validating-user-data) for a more detailed explanation. ### I am getting TemporaryErrors (Lockfile created, but doesn't exist)! What can I do? @@ -31,11 +31,11 @@ This error can occur when you are running tusd's disk storage on a file system w ### How can I prevent users from downloading the uploaded files? -Tusd allows any user to retrieve a previously uploaded file by issuing a HTTP GET request to the corresponding upload URL. This is possible as long as the uploaded files on the datastore have not been deleted or moved to another location. This can be completely disabled using the [`-disable-download` flag]({{ site.baseurl }}/getting-started/configuration/#disable-downloads). +Tusd allows any user to retrieve a previously uploaded file by issuing an HTTP GET request to the corresponding upload URL. This is possible as long as the uploaded files on the datastore have not been deleted or moved to another location. This can be completely disabled using the [`-disable-download` flag]({{ site.baseurl }}/getting-started/configuration/#disable-downloads). ### How can I keep the original filename for the uploads? -tusd will generate a unique ID for every upload, e.g. `1881febb4343e9b806cad2e676989c0d`, which is also used as the filename for storing the upload. If you want to keep the original filename, e.g. `my_image.png`, you will have to rename the uploaded file manually after the upload is completed. One can use the [`post-finish` hook](https://github.com/tus/tusd/blob/main/docs/hooks.md#post-finish) to be notified once the upload is completed. The client must also be configured to add the filename to the upload's metadata, which can be [accessed inside the hooks](https://github.com/tus/tusd/blob/main/docs/hooks.md#the-hooks-environment) and used for the renaming operation. +Tusd will generate a unique ID for every upload, e.g. `1881febb4343e9b806cad2e676989c0d`, which is also used as the filename for storing the upload. If you want to keep the original filename, e.g. `my_image.png`, you will have to rename the uploaded file manually after the upload is completed. One can use the [`post-finish` hook]({{ site.baseurl }}/advanced-topics/hooks/) to be notified once the upload is completed. The client must also be configured to add the filename to the upload's metadata, which can be [accessed inside the hooks]({{ site.baseurl }}/advanced-topics/hooks/#hook-requests-and-responses) and used for the renaming operation. Please have a look at the [corresponding post-processing use case]({{ site.baseurl }}/advanced-topics/hooks/#post-processing-files) for a more detailed explanation. ### Does tusd support Cross-Origin Resource Sharing (CORS)? diff --git a/docs/index.md b/docs/index.md index a2ed5184..d0a7da59 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,27 +17,27 @@ protocol](http://www.tus.io/protocols/resumable-upload.html): ## Highlights -- Multiple storage options: +- Multiple storage options for uploaded files: - [Local disk]({{ site.baseurl }}/storage-backends/local-disk/) - [AWS S3]({{ site.baseurl }}/storage-backends/aws-s3/) - [Azure Blob Storage]({{ site.baseurl }}/storage-backends/azure-blob-storage/) - [Google Cloud Storage]({{ site.baseurl }}/storage-backends/google-cloud-storage/) -- Fully customize using [hook system]({{ site.baseurl }}/advanced-topics/hooks/) via [scripts]({{ site.baseurl }}/advanced-topics/hooks/#file-hooks), [HTTP requests]({{ site.baseurl }}/advanced-topics/hooks/#https-hooks), or [gRPC]({{ site.baseurl }}/advanced-topics/hooks/#grpc-hooks): +- Fully customizable using [hooks]({{ site.baseurl }}/advanced-topics/hooks/) executed via [scripts]({{ site.baseurl }}/advanced-topics/hooks/#file-hooks), [HTTP]({{ site.baseurl }}/advanced-topics/hooks/#https-hooks), or [gRPC]({{ site.baseurl }}/advanced-topics/hooks/#grpc-hooks), such as: - [Upload validation]({{ site.baseurl }}/advanced-topics/hooks/#receiving-and-validating-user-data) - [User authentication and authorization]({{ site.baseurl }}/advanced-topics/hooks/#authenticating-users) - [File post-processing]({{ site.baseurl }}/advanced-topics/hooks/#post-processing-files) - Supports arbitrarily large files - Can receive uploads from any [tus-compatible client](https://tus.io/implementations) - Distributed as a [single binary without needing a runtime]({{ site.baseurl }}/getting-started/installation/#download-pre-builts-binaries-recommended) -- Easily [embedded in Go applications]({{ site.baseurl }}/advanced-topics/usage-package/) +- Easily [embeddable in Go applications]({{ site.baseurl }}/advanced-topics/usage-package/) ## Getting Started -To get started, have a look at the available [installation methods]({{ site.baseurl }}/getting-started/installation/) for tusd. Once that's done, you can check out the [usage guide]({{ site.baseurl }}/getting-started/usage/) to get tusd running and connect clients to it. Further details are available in the [configuration section]({{ site.baseurl }}/getting-started/configuration/) and the [storage overview]({{ site.baseurl }}/storage-backends/overview/). +To get started, have a look at the available [installation methods]({{ site.baseurl }}/getting-started/installation/) for tusd. Once you have tusd installed, you can check out the [usage guide]({{ site.baseurl }}/getting-started/usage/) to get tusd running and connect clients to it. As next steps, we recommend reading through the [configuration section]({{ site.baseurl }}/getting-started/configuration/) and the [storage overview]({{ site.baseurl }}/storage-backends/overview/). ## Help -If you have questions or problem, please read the [frequently asked questions]({{ site.baseurl }}/faq.html). If these didn't cover your issue, feel free to ask us in the [GitHub repository](https://github.com/tus/tusd) or in our [community forum](https://community.transloadit.com/c/tus/6). If you wish for private consulting, Transloadit offers [commercial support for tus](https://transloadit.com/open-source/support/). +If you have questions or run into problems, please read the [frequently asked questions]({{ site.baseurl }}/faq.html). If these didn't cover your issue, feel free to ask us in the [GitHub repository](https://github.com/tus/tusd) or in our [community forum](https://community.transloadit.com/c/tus/6). If you need private consulting, Transloadit offers [commercial support for tus](https://transloadit.com/open-source/support/). ## License