Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support matching JSON body with CEL expressions #1255

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

juho9000
Copy link

@juho9000 juho9000 commented Jun 5, 2024

This PR implements parsing a JSON response from the body and checking the result using a CEL query. Looking for feedback on this.

Sample usage:

modules:
  http_2xx:
    prober: http
    http:
      fail_if_body_json_not_matches_cel: "body.foo.bar == 'baz'"

If response is {"foo": { "bar": "qux" } } the probe fails. fail_if_body_json_matches_cel is also implemented and should work as expected.

@juho9000 juho9000 force-pushed the feature/cel-json-body branch from 45e8b9c to 2f2d0fb Compare June 5, 2024 12:25
@juho9000
Copy link
Author

juho9000 commented Jun 17, 2024

@roidelapluie @mem Ping :)

Copy link
Contributor

@mem mem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of minor comments.

CONFIGURATION.md Outdated Show resolved Hide resolved
CONFIGURATION.md Outdated Show resolved Hide resolved
@@ -297,6 +351,11 @@ func ProbeHTTP(ctx context.Context, target string, module config.Module, registr
Help: "Indicates if probe failed due to regex",
})

probeFailedDueToCel = prometheus.NewGauge(prometheus.GaugeOpts{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking this should be added conditionally, if the configuration asks for validation using CEL expressions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was following the same logic that is applied to regex checks, following your logic that should also be conditionally added, no?

@mem
Copy link
Contributor

mem commented Jul 23, 2024

Thank you for this, this seems useful!

Question, I have no experience with CEL... is it possible to use it to match HTML, too? I see in the code that Eval takes a map[string]any, but I couldn't spot how that any is handled.

The reason I'm asking is because I'm trying to figure out if it's possible to extend this to more inputs, or if it needs to be restricted to JSON.

prober/http.go Outdated
}

if httpConfig.FailIfBodyJSONMatchesCel != nil {
result, details, err := httpConfig.FailIfBodyJSONMatchesCel.Eval(evalPayload)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: shouldn't this be EvalContext instead? I'm not clear about the guarantees that the evaluation will terminate, and the fact that EvalContext exists suggests that there's a benefit from imposing a deadline.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're probably right, I've changed it to use ContextEval

@juho9000 juho9000 force-pushed the feature/cel-json-body branch 2 times, most recently from 0c9b9a0 to f9a0036 Compare August 1, 2024 12:56
@juho9000
Copy link
Author

juho9000 commented Aug 1, 2024

Thank you for this, this seems useful!

Question, I have no experience with CEL... is it possible to use it to match HTML, too? I see in the code that Eval takes a map[string]any, but I couldn't spot how that any is handled.

The reason I'm asking is because I'm trying to figure out if it's possible to extend this to more inputs, or if it needs to be restricted to JSON.

I atleast do not know how CEL could be used to match HTML in any sane way. You might want to check this out https://github.com/google/cel-spec/blob/master/doc/langdef.md#json-data-conversion

Thanks for your comments and suggestions so far. Really hoping we can get this merged. I think that having the ability to validate JSON responses would be an awesome addition to blackbox exporter. :)

@juho9000 juho9000 requested a review from mem August 1, 2024 13:06
@zetaab
Copy link

zetaab commented Aug 7, 2024

@mem kindly ping :) we are interested about this feature and would like to see it as well

@juho9000 juho9000 force-pushed the feature/cel-json-body branch from f9a0036 to b9b903b Compare August 8, 2024 10:13
@juho9000 juho9000 force-pushed the feature/cel-json-body branch 3 times, most recently from fbb6791 to 9131539 Compare August 29, 2024 08:36
@juho9000 juho9000 force-pushed the feature/cel-json-body branch 2 times, most recently from 3101080 to 2b1939d Compare September 3, 2024 07:40
@zetaab
Copy link

zetaab commented Sep 3, 2024

@mem @roidelapluie any comments to this PR?

@juho9000
Copy link
Author

juho9000 commented Oct 1, 2024

@electron0zero @mem I haven't heard from any of the maintainers in a while. I fixed or clarified everything that was mentioned in the review comments. The initial message I got was that this seems useful, but it seems that it's forgotten now. Any update? I'd love to get this merged in order to avoid maintaining my own fork. :)

@zetaab
Copy link

zetaab commented Dec 9, 2024

ping @electron0zero @mem could you give some opinion about this?

Signed-off-by: Juho Majasaari <[email protected]>

� Conflicts:
�	go.mod
�	go.sum
�	prober/http.go
Signed-off-by: Juho Majasaari <[email protected]>

# Conflicts:
#	prober/http.go
@juho9000 juho9000 force-pushed the feature/cel-json-body branch 2 times, most recently from 787242f to a07544b Compare December 17, 2024 11:31
Signed-off-by: Juho Majasaari <[email protected]>
@juho9000 juho9000 force-pushed the feature/cel-json-body branch from a07544b to 704e04a Compare December 17, 2024 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants