Add multihost testing

[engelmi]

Fixes #164

This PR adds support for specifying pipeline settings - including the pipeline type (currently only multihost). The API specification of testing farm for requesting new test run is used as basis:
https://api.testing-farm.io/redoc#operation/request_a_new_test_v0_1_requests_post
(Some inline comments to the old API spec have been updated as well)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.50%. Comparing base (979f50e) to head (2b2789a).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #169      +/-   ##
==========================================
+ Coverage   99.32%   99.50%   +0.18%     
==========================================
  Files           8        8              
  Lines         591      608      +17     
  Branches       68       68              
==========================================
+ Hits          587      605      +18     
+ Misses          4        3       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[engelmi]

The code coverage is failing due to the formatting. For example:

const envSettings = envSettingsParsed.success ? envSettingsParsed.data : {};

Since it is in one line, the else part of the ternary operator is considered covered even though its not executed.

The formatting of the newly added pipeline settings creates a separate line, so its considered not covered:

const pipelineSettings = pipelineSettingsParsed.success
    ? pipelineSettingsParsed.data
    : {};

Should I implement a new test for this or is it ok that the coverage drops a bit?

[jamacku]

You can ignore codecov failure. It is also missing for other instances of input parsing.

[jamacku]

@engelmi Thank you for the PR. It looks great. Could you please also add the test into tests/action.test.ts?

You can take a Hardware test as an example:

testing-farm-as-github-action/tests/action.test.ts

Lines 255 to 326 in 080971e

	test('Hardware test', async () => {
	setDefaultInputs();
	// Mock required Action inputs
	// api_key - A testing farm server api key
	vi.stubEnv('INPUT_API_KEY', 'abcdef-123456');
	// git_url - An url to the GIT repository
	vi.stubEnv(
	'INPUT_GIT_URL',
	'https://github.com/sclorg/testing-farm-as-github-action'
	);
	// tmt_plan_regex - A tmt plan regex which will be used for selecting plans. By default all plans are selected
	vi.stubEnv('INPUT_TMT_PLAN_REGEX', 'fedora');
	vi.stubEnv('INPUT_TMT_HARDWARE', '{"memory": ">= 8 GB"}');
	// Mock Testing Farm API
	vi.mocked(mocks.unsafeNewRequest).mockImplementation(
	async (_request: unknown) => {
	return Promise.resolve({
	id: '1',
	});
	}
	);
	vi.mocked(mocks.requestDetails)
	.mockResolvedValueOnce({ state: 'new', result: null })
	.mockResolvedValueOnce({ state: 'queued', result: null })
	.mockResolvedValueOnce({ state: 'pending', result: null })
	.mockResolvedValueOnce({ state: 'running', result: null })
	.mockResolvedValueOnce({
	state: 'complete',
	result: { overall: 'passed', summary: '\\o/' },
	});
	// Run action
	const octokit = new Octokit({ auth: 'mock-token' });
	const pr = await PullRequest.initialize(1, octokit);
	await action(pr);
	expect(mocks.newRequest).not.toHaveBeenCalled();
	expect(mocks.unsafeNewRequest).toHaveBeenCalledOnce();
	// Check if we have waited for Testing Farm to finish
	expect(mocks.requestDetails).toHaveBeenCalledTimes(5);
	// Test outputs
	expect(process.env['OUTPUT_REQUEST_ID']).toMatchInlineSnapshot('"1"');
	expect(process.env['OUTPUT_REQUEST_URL']).toMatchInlineSnapshot(
	'"https://api.dev.testing-farm.io/requests/1"'
	);
	expect(process.env['OUTPUT_TEST_LOG_URL']).toMatchInlineSnapshot(
	'"https://artifacts.dev.testing-farm.io/1"'
	);
	// First call to request PR details, next two calls for setting the status
	expect(mocks.request).toHaveBeenCalledTimes(1);
	expect(mocks.request).toHaveBeenLastCalledWith(
	'GET /repos/{owner}/{repo}/pulls/{pull_number}',
	{
	owner: 'sclorg',
	pull_number: 1,
	repo: 'testing-farm-as-github-action',
	}
	);
	// Test summary
	await assertSummary(`<h1>Testing Farm as a GitHub Action summary</h1>
	<table><tr><th>Compose</th><th>Arch</th><th>Infrastructure State</th><th>Test result</th><th>Link to logs</th></tr><tr><td>${process.env['INPUT_COMPOSE']}</td><td>${process.env['INPUT_ARCH']}</td><td>OK</td><td>success</td><td>[pipeline.log](https://artifacts.dev.testing-farm.io/1/pipeline.log)</td></tr></table>
	`);
	expect(mocks.TFError).not.toHaveBeenCalled();
	});

[engelmi]

@engelmi Thank you for the PR. It looks great. Could you please also add the test into tests/action.test.ts?

Hi @jamacku,
sure thing. I added two tests - valid and invalid pipeline settings. I also switched from safeParse to parse so that an error in the schema doesn't get "covered up" and shows directly before submitting the TF request.

[jamacku]

Thank you for adding tests. Please remove unused import in tests and please rebase to main and run yarn && yarn run all

[jamacku]

Thank you @engelmi, LGTM 👍