Integration tests in CI

[maxjakob]

Run integration tests in CI.

• Remove check for OPENAI_API_KEY because this key is actually not required
• Create ES service in GitHub Action environment
• Check if a model is deployed in ES to determine if a test needs to be skipped
• Some refactoring for the previous point
• Remove obsolete compile test

[efriis]

I think this should work!

• the reason we test for "compiling integration tests" but don't actually run them in check_diffs.yml (which runs on every PR) is because integration tests typically rely on secrets, which are not available when PRs come from forks (e.g. community contributions). It seems your integration tests don't rely on any secrets, so this could work.
• my light suggestion would be to reformat some of these "integration tests" as "unit tests" (if you want to rename those two folders to "pr tests" and "release tests" that works for me). That way you can keep the integration testing folder if you ever wanted to run release integration tests to make sure cloud connection is working, or integration with other components that need api keys works.

But will defer to you, and lmk if you have questions!

[maxjakob]

Appreciate the feedback @efriis!

It seems your integration tests don't rely on any secrets, so this could work.

Exactly they don't and I would argue that for this Elasticsearch package we can keep secrets out of the tests going forward since we can stub the other parts (like embeddings).

my light suggestion would be to reformat some of these "integration tests" as "unit tests" (if you want to rename those two folders to "pr tests" and "release tests" that works for me)

I had a look through the integration tests but didn't find an instance that could be ported to a unit test (that can run without ES). Regarding the distinction between PR pushes and merges to main: the integration tests currently take less than a minute in CI including ES setup. From my perspective this is acceptable to run on each push to a PR.

That way you can keep the integration testing folder if you ever wanted to run release integration tests to make sure cloud connection is working [...]

It's still possible to run the integration tests locally pointing at a cloud instance so I think we don't loose this option with this PR.