Argon testuite integration is broken

[ndufresne]

As documented in "Argon Streams AV1 Bit-stream Generator Specification", the MD5 provided are based on all layers, which is not what we expect in fluster. We strictly don't want "all layers" since a proper decoder should only output the frames from higher spatial layers. It is also not relevant to keep both in the md5, since the higher sized frame are builts from the lower size one, so any corruption in lower layer will result in a bad md5 anyway.

There is many issue in the Argon suite integration, but once we have resolved them, we should regenerate all MD5 based on how we call aomdec reference deocder rather how Argon wants it to be called. This will make the MD5 compatible with other decoders. Argon reports that all these streams should work with aom.

In reference to that, with current integration, AOM current give 37/3181 core. With a quick hack, we can get 1854/3181.

diff --git a/fluster/decoders/av1_aom.py b/fluster/decoders/av1_aom.py
index cec5767..db41617 100644
--- a/fluster/decoders/av1_aom.py
+++ b/fluster/decoders/av1_aom.py
@@ -41,11 +41,11 @@ class AV1AOMDecoder(Decoder):
         """Decodes input_filepath in output_filepath"""
         fmt = None
         if output_format in [OutputFormat.YUV420P, OutputFormat.YUV420P10LE]:
-            fmt = "--i420"
+            fmt = "--rawvideo"
         else:
             fmt = "--rawvideo"
         run_command(
-            [self.binary, fmt, input_filepath, "-o", output_filepath],
+            [self.binary, "--annexb", "--all-layers", fmt, input_filepath, "-o", output_filepath],
             timeout=timeout,
             verbose=verbose,
         )

There is ~200 test that get fixed by the --rawvideo, some investigation needed. On top if this, Argon suite covers Anchors, which no-one really support. I don't know if anchors are the ALL the remaining, but we don't provide the other files needed to decode these (since obu with anchors are not self contained). If we keep these, we should make sure to provide information about that and skip them in decoders that can't support it, notably aomdec can't handle you need to use the lightfield_tile_list_decoder from the same project and requires and which is lost in our suite. As the spec says:

Methods to pack large scale tile bit-streams are not included in the AV1 Specification, which only covers the decoding of the individual tiles. This section describes the method used by the reference decoder, which is also the method adopted by Argon Streams AV1.

Which makes me think we should just not include these are "conformance" tests.

[mdimopoulos]

@mcesariniflu fyi