Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

option to not include _entry in loki.pack #2200

Open
yosefy opened this issue Dec 1, 2024 · 2 comments
Open

option to not include _entry in loki.pack #2200

yosefy opened this issue Dec 1, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@yosefy
Copy link

yosefy commented Dec 1, 2024

Request

sometimes _entry value is not needed and it is too long
so may be we can add parameter to not include it in output json in stage.pack

thanks

Use case

i am parsing mysql slow query log which is pretty long and not beautiful lines
so i add the mysql query itself to query field with pack

but i really don't need it twice because it becomes almost unreadable this way

@yosefy yosefy added the enhancement New feature or request label Dec 1, 2024
@ngc4579
Copy link

ngc4579 commented Dec 13, 2024

I second that. We're processing complex log lines from Java applications in JSON format; after applying a series of processing stages extracting structured metadata, I'd like to re-pack the remaining actual log content (message and potentially exceptions and call stack) into a more lightweight JSON object without repeating the already extracted structured metadata into an _entry element.

@ngc4579
Copy link

ngc4579 commented Dec 16, 2024

I've forked the repo and tried to prepare a PR... but actually I'm not quite sure if this is a simple thing to achieve. It all depends on this: can the logproto.Entry.Line contain null as the value of the logqlmodel.PackedEntryKey (_entry), such as shown here:

Entry: logproto.Entry{
    Timestamp: time.Unix(1, 0),
    Line: "{\"foo\":\"bar\",\"" + logqlmodel.PackedEntryKey + "\":null}",
},

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants