enable testing without autokit on physical DUTs #1060
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
How to use:
set your
workspace/config.js
to:Test suite specific caveats:
OS suite: Must put device in local mode first (via dashboard). Also manually enable persistent logging in the dashboard too. If your device has wifi, provision it to use your wifi (same network as laptop).
Cloud suite: Must not be in local mode, "SSH authentication test" must be set as unstable
Hup suite: Annoying in general - make sure you provision your DUT into the OLD OS version you want to hup from. Then in the
image
property put the path to the<hostapp>.docker
hostapp image, instead offalse
You must run this branch of the tests from meta balena: https://github.com/balena-os/meta-balena/tree/ryan/manual_tests
You must provision your DUT manually to a balena cloud fleet, with no release, and your
organization
must have access to it. The DUT must be on the same network as your laptop. Ensure the DUT is in development mode first. Then run:If there are ssh authentication errors, then the
organization
you put inconfig.js
doesn't have access to the DUT you provisioned.Other considerations:
should be added in the above config.js, right after
manual:true,
only if testing in staging.The device should have both local mode enabled in the dashboard, and developmentMode:true in it's config.json, for the OS suite.
If the OS test suite need to be re-run, /mnt/boot/system-proxy should be deleted, the device should then be re-booted and local-mode should be toggled to off and back on prior to restarting the test suite. Depending on the leftovers from a failed or interrupted test suite, re-flashing may be necessary.
On particular devices that also have a p2p-dev-wlP1p1s0 interface for example, the connectivity test should be modified to
For the cloud suite, the following line should be changed to use "that" instead of "this": https://github.com/balena-os/meta-balena/blob/ryan/manual_tests/tests/suites/cloud/suite.js#L248