Alan.donham/hackathon compression #67
Conversation
Adding hooks for zstd compression in _oci_image_layer_impl.
…auses invalid memory access when the writer closes. We already have calls to Close(), so we really don't need to defer this. Updating oci_image_layer rule to define outputs inside the impl, so that it can choose the correct extension for the archive.
go/cmd/ocitool/createlayer_cmd.go
Outdated
| var tw *tar.Writer | ||
| var zstdWriter *zstd.Writer | ||
| var gzipWriter *gzip.Writer | ||
| var mediaType string | ||
|
|
||
| if config.UseZstd { | ||
| zstdWriter = zstd.NewWriter(wc) | ||
| mediaType = ocispec.MediaTypeImageLayerZstd | ||
| tw = tar.NewWriter(zstdWriter) | ||
| } else { | ||
| gzipWriter = gzip.NewWriter(wc) | ||
| gzipWriter.Name = path.Base(out.Name()) | ||
| mediaType = ocispec.MediaTypeImageLayerGzip | ||
| tw = tar.NewWriter(gzipWriter) | ||
| } |
There was a problem hiding this comment.
Suggestion: other than tar.Writer being its own thing, you shouldn't need to declare compression-specific Writers. Consider instead having the compression writer being a generic interface. io.WriteCloser might be appropriate.
There was a problem hiding this comment.
I'll look into this. I attempted this approach initially, which resulted in some compilation errors due to the interface not being fully implemented, so I put this together to keep making progress.
There was a problem hiding this comment.
The gzip and zstd Writer interfaces are not 1 to 1. They both implement a similar interface, but diverge enough that we cannot use a common variable to store the Writer.
There was a problem hiding this comment.
Yes, but they don't need to be.
If all you need is Write and Close, then you can just have the type be an io.WriteCloser, and pass that around everywhere. That way, your function implementations can be generic with respect to what the types can do, not what they are, if that makes sense.
…pression library to use.
This changes adds an option to compress using the zstd library. The library used is in pure Go, which allows cross compilation to work as expected.
Going forward, we likely will want to look at tuning the compression level, though for now it remains at the default value.