Introduce strict and legacy validation modes

[mtojek]

As we have started deprecating some properties introduced earlier (for example: release tag), it might be necessary to introduce two validation modes:

• legacy
• strict

Legacy mode

The mode validates all features ever introduced to package spec, so that we can include also deprecated and old packages. We may need it for publishing logic to resign and revalidate all package revisions.

Strict mode

The strict mode focuses only on current features and guards that developers don't use deprecated features accidentally.

Checklist:

• Make a dry run on the entire package registry to see how many packages will be marked as invalid
• Enforce the use of formatVersion
• Define a mechanism to freeze package spec version
• Discuss about: How to implement different levels of validation?

[jsoriano]

I think we need more fine-grained validation modes, or stricter versioning.

Some use cases:

• Additional contents are added to a directory or object with additionalContents: false. This can be a problem if consumers of packages like Fleet reuse the package-spec for validation. A package supporting a broad range of versions will fail if it starts providing the new field or files, even if otherwise this field wouldn't be problematic.
  • An example of this can be the setting we will have to add to enable TSDB mode. Packages can work the same with this mode enabled or not (only with different performance), but they would fail validation with older spec if we add this setting in an object with additionalContents: false. Package developers would need to increase kibana.version to use a version of the package-spec that supports their package.
  • A solution for this can be to disable the additionalContents check.
  • Another solution can be to start using format_version, and make the validator to support multiple versions.
  • Another solution can be to make the package-spec upgradeable on runtime on consumers of packages, but this can open a can of worms.
• We add a new check, that produces validation failures on existing packages. We can find a case where a package needs to introduce breaking changes to solve this validation failure. It could be nice to provide a way to disable checks, as many linters have. Another option would be to use format_version too.

[jsoriano]

Another thing we may need to add is different levels (for things like #316).
Now the validation is all or nothing, we may need to report different levels of issues found.

[jsoriano]

Related elastic/package-storage#1013

[jsoriano]

I think that the changes for Package Spec v2 will cover most of this.

The thing that would be pending would be to have the different levels of validation, for this maybe #342 would help.

[jsoriano]

Closing this, leaving #342 for the different validation levels.