WIP: PoC First smoke test #19
Conversation
skt basic functionality: - fetch/merge git branches - build - publish: cp/scp - run bkr job based on a template
Remember commit ids for each branch used, so that we can publish it along with the build to be able to later recreate the same tree.
Add timestamp to tgz before uploading and upload buildinfo as well.
Use timestamp as UID in beaker jobs to make whiteboard unique.
Change from millisecond representation to human readable timestamp format.
Instead of relying on assumptions of where interesting data should be in the output stream - actually parse it and find the parts we are interested in.
Allow the __main__ function to properly trap on various errors, like user exit. Signed-off-by: Don Zickus <[email protected]>
Just code cleanup. Signed-off-by: Don Zickus <[email protected]>
Makes it easier to parse configs later. The returned value is a dict cfg of available options.
Code cleanup.
GitPython brinfs a bunch of compatability issues between versions so dropping it in favor of direct subprocess calls.
Add subcommands merge/build/publish/run/cleanup and 'all' which would allow to execute separate steps of the process.
Add INSTALL_MOD_STRIP=1 to make execution so that we get smaller tarballs
Add an option for skt to not exit until runner is done.
Do not require 'uid' argument for runner.run method, use the last part of url as uid instead.
Add an option to save/read states on each step to provide a communication method between steps.
Switch from using heads only to any refs allowing user to specify full ref in 'refs/xxx/yyy' format or just use 'yyy' and rely on git to guess full ref. THIS IS NOT JUST A RENAME. This commit actually changes the way refs are fetched.
Don't use userprovided refs for in local ref names. It might result in ambigious and/or weird names and only clutters ref list. Use "master" for base and "merge" for merge branches instead.
Allow switching jow owner for submitted job.
Doing so makes it easier to navigate local refs especially in case when multiple branches from the same remote are used.
Make --state option actually mater, before this commit state would be used whether this option was provided or not.
buildinfo and tarpkg might be deleted/moved by publishers, so silently ignore unlink errors
change retcode based on test results
Beaker results are not binary, apart from pass/fail there is also 'cancelled' and 'aborted'. Logging those might be useful.
Instead of relying on bkr's "--wait" option implement own wait, that would monitor separate recipes. This is needed for future failure handling features.
Respect max-line-length=79 and left 1 blank line on nested function
It runs only over the test folder for now, because the rest of the source code is not compliant yet. Fixes cki-project#12
It follows the README instructions using merge, patch and build actions of skt
|
You can see the output pipeline here https://travis-ci.org/agustinhenze/skt/builds/355851651. It uses secret variables therefore it won't pass here. |
|
I forgot to ping you, please take a look at it ☝️ @spbnick @stefwalter |
|
Wow, impressive work, Agustin! Here's my perspective. First thing, I would like to keep CI fast, at least until development slows down. For that reason, I wonder if we can perhaps create a small repo in this "organization", which would look like a kernel to skt, but would "build" very fast. Or rather generate that repo as part of the integration tests, so they don't depend on GitHub being available, and so that they don't break independently of this repo. I would also like to have any non-trivial tests, added to Travis CI runs, available for easy execution without it. I.e., first of all, they should be available to developers for running on their machines easily, and only then used by Travis CI. I'm on the fence regarding Docker. On one hand, sure, it's easy to create a test environment with it, but on the other hand it's hard to debug. It's not helped by my minimal experience with it, so at the very least I would like more explanations of what's going on there :) Lastly, I am very reluctant to bring such an explosion of complexity to our CI. Do you think you can remake this as a shell or Python script running skt without Docker in a temporary or, perhaps, specified directory, and using a dummy git repo generated on the fly, and then add that script to .travis.yml? @stefwalter, what's your opinion? |
|
By "generating a git repo" I meant something like this (from an old project of mine): https://github.com/spbnick/carton/blob/master/tests/test_git.sh |
Great idea! Create a bare git repository with the minimal things needed, so skt
Yes, I totally agree with you about both the testing and development
Actually it's the opposite, it's easy to debug because if something fails on CI
Ups, my idea is to write an integration test using docker-compose. In this way
Ok, I could do that, but I'm not so convinced it's a good idea. IMO, having a |
|
Alright, let's try to go with Docker. It seems to be unavoidable anyway :) And I like the idea of not having to track the dependencies and install them all on the host. One of the things I worry about, though, is how it would fare without an internet connection. Would it keep its images reliably cached and not decide to suddenly update them, without a request, at the most unsuitable moment, when a developer has no connection? I think it would be awesome to have a Patchwork instance automatically set up for testing, both locally and in CI. It's been something I was doing my own research on. We can then feed it with test data as part of the setup and run our tests against it. However, I understand it can be quite complicated, and I don't want you to apply a huge effort to this. I absolutely agree that it should be easy for people to start hacking on the project. It's with this intention @stefwalter has been hacking on a Dockerfile for skt in #28. Our own, automatically-setup Beaker would be simply amazing, but I wonder how you're going to provision it with systems from inside a Docker container. I also expect it would be a big effort to get it to run as we need. @major, what do you think about Agustin's ideas? |
Here is a PoC of the smoke test following the steps described on #17. The minimal config is not so minimal yet as it could be and it does the ci pipeline too slow. I guess more than 25 minutes (at the moment that I'm writing this, it didn't finish yet...). I have rewritten the .travis.yml using stages + docker, so you can see if the lint check pass before wait the whole pipeline ends. I'm sure we can improve the speed a lot from here. The most important for me now, is to discuss if the approach is good or not.
"Easy" to speed up
I would like to know your thoughts about it before following in that direction