Use the canonical copy of the CWL conformance tests

[mr-c]

https://github.com/common-workflow-language/common-workflow-language is the canonical location for the CWL v1.0 conformance tests

[dleehr]

Thanks @mr-c! I updated to using the v1.0.1 release on 82-canonical-conformance-tests and have fewer failures now.

calrissian/conformance/results.txt

Lines 314 to 316 in 16f78bf

	105 tests passed, 3 failures, 0 unsupported features
	1 tool tests failed

However I only find 2 failures in the output:

I know for example that many of the DockerRequirement features are not supported (#12):

calrissian/calrissian/job.py

Lines 378 to 390 in d60a31d

	# Dictionary of supported features.
	# Not yet complete, only checks features of DockerRequirement
	supported_features = {
	'DockerRequirement': ['class', 'dockerPull']
	}
	def check_requirements(self):
	for feature in self.supported_features:
	requirement, is_required = self.get_requirement(feature)
	if requirement and is_required:
	for field in requirement:
	if not field in self.supported_features[feature]:
	raise UnsupportedRequirement('Error: feature {}.{} is not supported'.format(feature, field))

so I would expect this one to fail:

calrissian/conformance/results.txt

Lines 282 to 284 in 16f78bf

	Test [94/108] Test dockerOutputDirectory
	Test [95/108] Test hints with $import

but it doesn't appear to. Any idea how to root out this or other failures?

[mr-c]

When a test fails, I re run it manually using the provided command line without --quiet and adding engine specific debugging options like --debug or similar

[dleehr]

I would expect this one to fail ... but it doesn't appear to. Any idea how to root out this or other failures?

This was caused by using a 2-year old release of https://github.com/common-workflow-language/common-workflow-language. Updating to today's release (v1.0.2) runs a total of 197 tests for 1.0 with 191 passing. And the failures are all accounted for.