diff --git a/rubygems-org-compact-index-api.md b/rubygems-org-compact-index-api.md index bba9964..d2f67b7 100644 --- a/rubygems-org-compact-index-api.md +++ b/rubygems-org-compact-index-api.md @@ -10,6 +10,45 @@ next: /rubygems-org-rate-limits Compact Index API ----------------- +The Compact Index API is considered a stable public API. + +The primary index file is the `versions` file, which provides the name, versions and the MD5 checksum of the per-gem `info` files. + +### Fetching and Caching + +Example Response (with some headers ignored): + + $ curl -I https://rubygems.org/info/bundler + + HTTP/2 200 + last-modified: Fri, 22 Mar 2024 13:09:59 GMT + etag: "40148273a7c7cd16b49d00a9935fd445" + cache-control: max-age=60, public + content-type: text/plain; charset=utf-8 + repr-digest: sha-256="pf7Ts7NTRac//JxO9ke3IYbcWx8kcVn9N0mbm5EJpP0=" + accept-ranges: bytes + +All compact index endpoints support ETags with the `If-None-Match` header. + + If-None-Match: "40148273a7c7cd16b49d00a9935fd445" + +The compact index is designed to be fetched using the HTTP `Range` header. +When a previously fetched copy is present, a ranged request to take advantage of the appended line pattern. + + Range: bytes=#{range_start}- + +Responses from the `/versions` and `/info/[GEM]` endpoints will include the `Repr-Digest` header. +The digests will have at least the SHA256 checksum of the entire file whether the response includes the full file or a partial response. +*Please note that a `Digest` header is present, but it is deprecated and may be removed without notice.* + +When the Range header is satisfied, append the contents at exactly the starting byte, then computer the SHA256 checksum of the resulting file. +If the result matches the `Repr-Digest` header's SHA256 checksum, the file is considered complete and up-to-date. + +Each line in the `/versions` file includes the latest MD5 calculation of the matching `/info` file for that gem at the time the line was written. +The latest MD5, closest to the end of the file, will match the MD5 of the up-to-date `/info` file. + +Exact implementation details are available in the [Bundler CompactIndexClient::Updater](https://github.com/rubygems/rubygems/blob/master/bundler/lib/bundler/compact_index_client/updater.rb). + ### GET - `/versions` Returns a custom text based format of all versions of all gems. @@ -81,7 +120,9 @@ Returns a custom text based format for a single gem name with a line for each ve #### `info` File Format -The format of the `info` file uses one line per version. It is recalculated when a version is yanked., with additional lines appended to the end that may include new or yanked versions of a gem already present earlier in the file. +The format of the `info` file uses one line per version. +Additional lines are appended to the end that for new versions of a gem. +The `info` file is recalculated when a version is yanked so the file will never list a version as yanked, but rather exclude the version entirely. The file starts with three minus characters on a new line.