v3.1.0

What's Changed

Framework Enhancements

• Auto discovery
  • Configurable namespace list for all label based test target discovery
  • Deployment auto discovery
  • Node auto discovery
  • CRD auto discovery
  • Improved pod discovery with running status check
• Robustness
  • Greatly improved external shell session management in terms of creating/closing, monitoring and error handling for unexpected and induced events (i.e. intrusive tests),
  • External command return code check added to the core test engine for better error handling/reporting
  • Reliable node access using the pods from the debug pod daemon set provided by the cnf-certification-test-partner repo
• Usability
  • Disconnected environment support with partner pods image source URL override
  • Logging
    • Timestamp added
    • Streamlined log levels
    • Added mirroring of the stderr/stdout output of shell sessions
  • Refactoring of the claim.json file:
    • New format
    • Better failure root cause identification
    • Possibility to add logs
    • OCP client/server versions now captured in the claim file
  • Consolidated cmd tool for managing claims, generating test handler stubs and test catalogs, test report grading
  • References to the CNF best practice document added to each test case in the test catalog
• General
  • Yamlint and "go vet" added to github CI test
  • Replaced the deprecated "go get" with "go install" in build scripts
  • Go update to 1.17
  • Ginkgo update to 2.0rc1

Improved Test Cases

• Stateful set added to the pod-owner test
• Tainted kernel test now decodes the result into human readable reason text
• NUMA aware huge page config support added to the huge page test
• SCC capability checks for IPC_LOCK and NET_RAW added to the host resource test
• Network connectivity test now skips containers/pods that don't have required binary or interfaces

New Test Cases

• CRD spec status check

Full Changelog: v3.0.0...v3.1.0