Skip to content

Commit

Permalink
feat: refactor hooks,screenshots,attachments, address #28, #26, #20, #17
Browse files Browse the repository at this point in the history
  • Loading branch information
Shelex authored Dec 29, 2020
1 parent 1c259df commit 722ee6e
Show file tree
Hide file tree
Showing 12 changed files with 352 additions and 140 deletions.
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,12 @@ Cypress.Allure.reporter.runtime.writer;
See [cypress-allure-plugin-example](https://github.com/Shelex/cypress-allure-plugin-example) project, which is already configured to use this plugin, hosting report as github page and run by github action. It has configuration for basic allure history saving (just having numbers and statuses in trends and history).
For complete history (allure can display 20 build results ) with links to older reports and links to CI builds check [cypress-allure-historical-example](https://github.com/Shelex/cypress-allure-historical-example) with basic and straightforward idea how to achieve it.

There are also existing solutions that may help you prepare your report infrastructure:

- [Allure Server](https://github.com/kochetkov-ma/allure-server) - self-hosted portal with your reports
- [Github Action](https://github.com/simple-elf/allure-report-action) - report generation + better implementation for historic reports described above
- [Allure TestOps](https://docs.qameta.io/allure-testops/) - Allure portal for those who want more than report

## How to open report

Assuming allure is already installed:
Expand Down Expand Up @@ -207,11 +213,14 @@ Allure API available:
- suite(name: string)
- label(name: LabelName, value: string)
- parameter(name: string, value: string)
- testParameter(name: string, value: string)
- link(url: string, name?: string, type?: LinkType)
- issue(name: string, url: string)
- tms(name: string, url: string)
- description(markdown: string)
- testDescription(markdown: string)
- descriptionHtml(html: string)
- testDescriptionHtml(html: string)
- owner(owner: string)
- severity(severity: Severity)
- tag(tag: string)
Expand All @@ -221,8 +230,6 @@ Allure API available:
- endStep()
- step(name: string, isParent: boolean)
It may be assumed that Allure API method used in hooks (before/after all/each) would be applied to all/each test, but actually it is dealing with current allure executable (current test or hook) and obviously will not work. In case you need such behaviour it is better to try `test:before:run` or `test:after:run` cypress events to do so, when Allure interface commands will be applied to current test.
## VS Code for cypress + cucumber
In case you are using VS Code and [Cypress Helper](https://marketplace.visualstudio.com/items?itemName=Shelex.vscode-cy-helper) extension, it has configuration for allure cucumber tags autocompletion available:
Expand All @@ -237,7 +244,7 @@ In case you are using VS Code and [Cypress Helper](https://marketplace.visualstu
## Screenshots and Videos
Screenshots are attached automatically, for other type of content use `testAttachment` (for current test) or `attachment` (for current executable).
Screenshots are attached automatically, for other type of content feel free to use `testAttachment` (for current test), `attachment` (for current executable), `attachFile` (for existing file).
Videos are attached for failed tests only from path specified in cypress config `videosFolder` and in case you have not passed video=false to Cypress configuration.
Please take into account, that in case spec files have same name, cypress is trying to create subfolders in videos folder, and it is not handled from plugin unfortunately, so video may not have correct path in such edge case.
Expand Down
10 changes: 9 additions & 1 deletion cypress/integration/basic/allure.spec.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
context('Allure API Context', () => {
before(() => {
cy.log('This will run before scenarios');
});

after(() => {
cy.log('This will run after scenarios');
});

describe('Allure API Context', () => {
beforeEach(() => {
cy.log('This will run before every scenario');
});
Expand Down
8 changes: 8 additions & 0 deletions cypress/integration/cucumber/allure/hooks.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
before(() => {
cy.log('This will run before scenarios');
});

after(() => {
cy.log('This will run after scenarios');
});

beforeEach(() => {
cy.log('This will run before every scenario');
});
Expand Down
100 changes: 69 additions & 31 deletions cypress/integration/results/main.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,32 +10,58 @@ describe('Allure results', () => {
['basic', 'cucumber'].forEach((mode) => {
it(`should contain suite results for ${mode}`, () => {
const { suites, tests } = result[mode];
expect(suites).to.have.length(mode === 'basic' ? 4 : 1);
expect(suites.every((s) => s.children.length === 1)).to.be.eq(
true,
'every suite has single test'
);
expect(suites).to.have.length(1);
expect(suites[0].children).to.have.length(mode === 'basic' ? 4 : 1);
expect(
tests.every((t) =>
suites.find((s) => s.children.includes(t.uuid))
)
).to.be.eq(true, 'every test is linked to suite by uuid');
});

it(`should contain before all and after all hooks for ${mode}`, () => {
const [suite] = result[mode].suites;
expect(suite.befores).to.have.length(mode === 'basic' ? 1 : 2);
expect(
suite.befores.every(
(hook) =>
hook.status === 'passed' && hook.stage === 'finished'
)
).to.be.eq(true, 'before all hooks attached');
expect(suite.afters).to.have.length(mode === 'basic' ? 1 : 2);
expect(
suite.afters.every(
(hook) =>
hook.status === 'passed' && hook.stage === 'finished'
)
).to.be.eq(true, 'after all hooks attached');
});

it(`should contain before each and after each hooks for ${mode}`, () => {
const verifyEachHook = (name, suite) => {
const data = suite[`${name}s`];
const hook = data.filter((h) => h.steps.length)[0];
expect(hook.status).to.be.eq('passed');
expect(hook.steps).to.have.length(1);
expect(hook.steps[0].name).to.be.eq(
`log ("This will run ${name} every scenario")`
);
};
result[mode].suites.forEach((suite) => {
verifyEachHook('before', suite);
verifyEachHook('after', suite);
});
const hooks = result[mode].tests
.map((test) =>
test.steps.filter((step) => step.name.includes('each'))
)
.flat();

expect(
hooks.every(
(hook) =>
hook.stage === 'finished' && hook.status === 'passed'
)
).to.be.eq(true, 'every hook passed');

mode === 'cucumber' && hooks.splice(1, 1);
expect(
hooks.every(
(hook) =>
hook.steps.length === 1 &&
hook.steps[0].name ===
`log ("This will run ${hook.name
.replace(' each" hook', '')
.substring(1)} every scenario")`
)
).to.be.eq(true, 'step attached');
});

it(`should contain test results for ${mode}`, () => {
Expand Down Expand Up @@ -133,10 +159,11 @@ describe('Cucumber specific', () => {
}
];

expect(test.steps).to.have.length(expectedSteps.length);
const steps = test.steps.filter((s) => !s.name.includes('each'));
expect(steps).to.have.length(expectedSteps.length);

expectedSteps.forEach(({ name, child }, index) => {
const parentStep = test.steps[index];
const parentStep = steps[index];
expect(parentStep.status).to.be.eq('passed');
expect(parentStep).to.have.property('start');
expect(parentStep).to.have.property('stop');
Expand All @@ -158,8 +185,9 @@ describe('Basic specific', () => {
it('should have every attachment be linked to test', () => {
const { attachments, tests } = result.basic;
attachments.forEach(({ fileName }) => {
console.warn(`checking ${fileName}`);
const test = tests.find((t) =>
t.attachments.find((a) => a.source === fileName)
t.attachments.some((a) => a && a.source.endsWith(fileName))
);
expect(test.attachments).to.have.length(1);
const [testAttachment] = test.attachments;
Expand Down Expand Up @@ -246,32 +274,42 @@ describe('Basic specific', () => {

it('should contain test steps', () => {
const { tests } = result.basic;

const test = tests.find(
(t) => t.name === 'should attach cypress command log as steps'
);

const verifyStep = (step) => {
expect(step.status).to.be.eq('passed');
expect(step.stage).to.be.eq('finished');
expect(step).to.have.property('start');
expect(step).to.have.property('stop');
};

expect(test.steps).to.have.length(2);
expect(test.steps).to.have.length(4);
console.log(test.steps);
const steps = [
'"before each" hook',
`first parent step`,
`second parent step`,
'"after each" hook'
];
test.steps.forEach((step, i) => {
const order = i === 0 ? 'first' : 'second';
const stepName = (index) => steps[index];

expect(step.name).to.be.eq(`${order} parent step`);
expect(step.name).to.be.eq(stepName(i));

verifyStep(step);

//child steps
expect(step.steps).to.have.length(1);
const [childStep] = step.steps;
verifyStep(childStep);
expect(childStep.name).to.be.eq(
`log ("child command for ${order} step")`
);
if (i > 0 && i < steps.length - 1) {
const order = i === 1 ? 'first' : 'second';
expect(step.steps).to.have.length(1);
const [childStep] = step.steps;
verifyStep(childStep);
expect(childStep.name).to.be.eq(
`log ("child command for ${order} step")`
);
}
});
});
});
7 changes: 0 additions & 7 deletions reporter/afterHook.js

This file was deleted.

34 changes: 34 additions & 0 deletions reporter/commands.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@ Cypress.Commands.add(
}
);

Cypress.Commands.add(
'testParameter',
{ prevSubject: true },
(allure, name, value) => {
allure.testParameter(name, value);
cy.wrap(allure, { log: false });
}
);

Cypress.Commands.add('severity', { prevSubject: true }, (allure, level) => {
allure.severity(level);
cy.wrap(allure, { log: false });
Expand Down Expand Up @@ -41,6 +50,15 @@ Cypress.Commands.add(
}
);

Cypress.Commands.add(
'attachFile',
{ prevSubject: true },
(allure, name, path, type) => {
allure.attachFile(name, path, type);
cy.wrap(allure, { log: false });
}
);

Cypress.Commands.add(
'step',
{ prevSubject: true },
Expand Down Expand Up @@ -116,6 +134,22 @@ Cypress.Commands.add(
cy.wrap(allure, { log: false });
}
);
Cypress.Commands.add(
'testDescription',
{ prevSubject: true },
(allure, markdown) => {
allure.testDescription(markdown);
cy.wrap(allure, { log: false });
}
);
Cypress.Commands.add(
'testDescriptionHtml',
{ prevSubject: true },
(allure, html) => {
allure.testDescriptionHtml(html);
cy.wrap(allure, { log: false });
}
);
Cypress.Commands.add('tag', { prevSubject: true }, (allure, tag) => {
allure.tag(tag);
cy.wrap(allure, { log: false });
Expand Down
36 changes: 32 additions & 4 deletions reporter/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ type ContentType =
| 'application/json'
| 'video/webm'
| 'video/mp4'
| 'image/jpeg';
| 'image/jpeg'
| 'application/pdf';
type Status = 'failed' | 'broken' | 'passed' | 'skipped';
type Severity = 'blocker' | 'critical' | 'normal' | 'minor' | 'trivial';

Expand Down Expand Up @@ -84,11 +85,17 @@ declare global {
*/
label(name: LabelName, value: string): Allure;
/**
* Add Parameter
* Add Parameter for current executable (step/test)
* @param name
* @param value
*/
parameter(name: string, value: string): Allure;
/**
* Add Parameter for current test
* @param name
* @param value
*/
testParameter(name: string, value: string): Allure;
/**
* Add customized link
* @param {string} url
Expand All @@ -109,15 +116,25 @@ declare global {
*/
tms(name: string, url: string): Allure;
/**
* Add test description in markdown format
* Add test description in markdown format for step/test
* @param markdown
*/
description(markdown: string): Allure;
/**
* Add test description in html format
* Add test description in html format for step/test
* @param html
*/
descriptionHtml(html: string): Allure;
/**
* Add test description in markdown format for test
* @param markdown
*/
testDescription(markdown: string): Allure;
/**
* Add test description in html format for test
* @param html
*/
testDescriptionHtml(html: string): Allure;
/**
* Add test owner
* @param owner
Expand Down Expand Up @@ -171,6 +188,17 @@ declare global {
content: Buffer | string,
type: ContentType
): Allure;
/**
* Attach existing file to current test
* @param name
* @param path
* @param type
*/
attachFile(
name: string,
path: string,
type: ContentType
): Allure;
/**
* Log step into Test Execution Body
* @param name - step name
Expand Down
Loading

0 comments on commit 722ee6e

Please sign in to comment.