System tests for transforms #2242
Conversation
💔 Build Failed
Failed CI StepsHistorycc @jsoriano |
| running := false | ||
|
|
||
| // Reset transform to clean any previous state. | ||
| /* XXX: It fails to create the index after reset :? |
There was a problem hiding this comment.
This is failing now because it tries to create indexes with a name that is matched by a data stream template, and transforms need to use indexes, not data streams.
|
|
||
| func (r *tester) checkTransformAuditMessages(ctx context.Context, transformId string) error { | ||
| // XXX: This is an internal API, are these audit messages available somewhere else? | ||
| const internalTransformsPath = "/internal/transform/transforms" |
There was a problem hiding this comment.
I have found that some failures only appear in the audit messages, but these messages seem to be available only on internal APIs, that I guess don't have compatibility guarantees, nor will be available in all deployments.
We have to look for some other API.
@kpollich do you know how safe these internal APIs are to use? Or do you know if there are alternatives?
Though the errors found here also make the transform to fail with real-time data, so maybe this is an issue with test data that is discarded by being old. Then we'd have to review the test data too.
|
test integrations |
|
Created or updated PR in integrations repository to test this version. Check elastic/integrations#11831 |
Check that transforms can work properly once installed, after documents are ingested. It ensures that the transform is executed at least once while monitoring its behaviour and looks for failures in transform stats and audit messages.
It doesn't check the validity of documents by now, we do this with the preview API, maybe in a future change we can do this validation at this point too, with real documents.
These tests would be intended to earlier prevent issues like the ones mentioned in #2218.
Fixes #859