feat: Add the otelcol syslog receiver #2263

dehaansa · 2024-12-11T18:37:47Z

PR Description

While adding the otelcol.exporter.syslog component, I discovered that there was not a clean way to proxy syslog using the loki.source.syslog component as a source. For the purposes of syslog proxying, the otelcol.receiver.syslog component emits logs in the exact syntax expected by the exporter.

This does not add support for enhancing parsing with the numerous stanza operators available in upstream opentelemetry. Most or all of the use cases that would be supported by those can also be implemented through processors like the otelcol.processor.transform component, and the effort to add them in seemed significant for no known customer requirements at this time.

Which issue(s) this PR fixes

Related to #312

Notes to the Reviewer

PR Checklist

CHANGELOG.md updated
Documentation added
Tests updated
Config converters updated

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

Co-authored-by: Clayton Cornell <[email protected]>

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

…log.md Co-authored-by: Clayton Cornell <[email protected]>

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

…log.md Co-authored-by: Clayton Cornell <[email protected]>

…loy into feat/otelcol-syslog-receiver

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

mattdurham · 2024-12-12T14:44:15Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

+The `location` argument specifies a Time Zone identifier. The available locations depend on the local IANA Time Zone database.
+See [this wikipedia entry][tz-wiki] for a non-comprehensive list.
+
+The `non_transparent_framing_trailer` argument must be one of `LF`, `NUL`.


Can we add some context around this? Like how can the default be nil? Even if its a string it should be "" but that seems to not be LF or NUL.

Yeah, that's fair. It's a *string, so it's nil if not set. If it is set, then the character chosen is expected to terminate the message. If not set, it's expected that you're using octet counting or that the syslog frame contains only one message.

@clayton-cornell I'd appreciate any feedback you have on this docs change as well as Matt's.

mattdurham · 2024-12-12T14:46:46Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

+| Name                            | Type     | Description                                                                                                  | Default | Required |
+|---------------------------------|----------|--------------------------------------------------------------------------------------------------------------|---------|----------|
+| `listen_address`                | `string` | The `<host:port>` address to listen to for syslog messages.                                                  |         | yes      |
+| `max_log_size`                  | `int`    | The maximum size of a log entry to read before failing.                                                      | `1MiB`  | no       |


This should be a string.

mattdurham · 2024-12-12T14:48:34Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

+| Name               | Type       | Description                                                                                               | Default      | Required |
+|--------------------|------------|-----------------------------------------------------------------------------------------------------------|--------------|----------|
+| `enabled`          | `bool`     | If true, the receiver will pause reading a file and attempt to resend the current batch of logs on error. | `false`      | no       |
+| `initial_interval` | `duration` | The time to wait after first failure to retry.                                                            | `1 second`   | no       |


Durations should be in formatted to 1s or 5m

mattdurham · 2024-12-12T14:50:11Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

@@ -0,0 +1,271 @@
+---
+canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.receiver.syslog/


@clayton-cornell do we need to add a tag for public preview?

Yes. I missed that :-( I saw the shared lookup... forgot to make sure the badge was added.

Parallel to this, we are looking at better ways of adding badges to the docs... so we can avoid HTML spans.

mattdurham · 2024-12-12T14:53:13Z

internal/component/otelcol/receiver/syslog/syslog.go

+}
+
+// Values taken from tcp input Build function
+const defaultMaxLogSize = helper.ByteSize(tcp.DefaultMaxLogSize)


nit: Rename to tcpDefaultMaxLogSize

mattdurham

Looks good aside from a few doc changes.

…log.md Co-authored-by: Clayton Cornell <[email protected]>

clayton-cornell

The updated description makes sense when I read it.

In reviewing again, I found a couple other minor points I missed in the first pass.

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

clayton-cornell · 2024-12-12T17:47:09Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

@@ -0,0 +1,271 @@
+---
+canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.receiver.syslog/


Yes. I missed that :-( I saw the shared lookup... forgot to make sure the badge was added.

Parallel to this, we are looking at better ways of adding badges to the docs... so we can avoid HTML spans.

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

clayton-cornell · 2024-12-12T18:01:18Z

Ooops double posted my review. I was logged out Git right when I submitted the review.

…log.md Co-authored-by: Clayton Cornell <[email protected]>

Co-authored-by: Clayton Cornell <[email protected]>

clayton-cornell

Approving for docs

dehaansa added 5 commits December 11, 2024 10:12

Implementation of otelcol syslog receiver wrapper

1aec317

update unmarshal code

69a93f3

Merge branch 'main' into feat/otelcol-syslog-receiver

306a43f

Add changelog entry for syslogreceiver

ad69096

Change field key, add converter

cefb4c4

dehaansa requested review from clayton-cornell and a team as code owners December 11, 2024 18:37

clayton-cornell reviewed Dec 11, 2024

View reviewed changes

dehaansa and others added 5 commits December 11, 2024 14:33

Apply suggestions from code review

587d4a1

Co-authored-by: Clayton Cornell <[email protected]>

Update whitespace attribute descriptions

85b2cd0

Clean up commented out code

b22ebfb

Clean up comments

520eeea

Update autogenerated docs

ad716de

clayton-cornell reviewed Dec 11, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Dec 11, 2024

dehaansa and others added 4 commits December 11, 2024 14:59

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

d00f726

…log.md Co-authored-by: Clayton Cornell <[email protected]>

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

47deaad

…log.md Co-authored-by: Clayton Cornell <[email protected]>

Put a little more maximum delay in syslog test

f79a9ac

clean up validation, try not to use localhost for ci tests

3a438a7

clayton-cornell reviewed Dec 11, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

dehaansa and others added 4 commits December 11, 2024 21:47

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

1072d10

…log.md Co-authored-by: Clayton Cornell <[email protected]>

Fix syslogreceiver converter

27dcf40

Merge branch 'feat/otelcol-syslog-receiver' of github.com:dehaansa/al…

d0e59ac

…loy into feat/otelcol-syslog-receiver

Add test sleep

1d0b32a

clayton-cornell reviewed Dec 12, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

mattdurham reviewed Dec 12, 2024

View reviewed changes

mattdurham requested changes Dec 12, 2024

View reviewed changes

dehaansa and others added 3 commits December 12, 2024 10:20

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

f2942e4

…log.md Co-authored-by: Clayton Cornell <[email protected]>

Address some PR feedback

ad373b7

Update docs for rfc6587 arguments

70997f7

dehaansa requested a review from mattdurham December 12, 2024 15:59

Add todo to test comment

f278703

clayton-cornell reviewed Dec 12, 2024

View reviewed changes

dehaansa and others added 2 commits December 12, 2024 13:47

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

548b173

…log.md Co-authored-by: Clayton Cornell <[email protected]>

Apply suggestions from code review

a9fc448

Co-authored-by: Clayton Cornell <[email protected]>

clayton-cornell approved these changes Dec 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add the otelcol syslog receiver #2263

feat: Add the otelcol syslog receiver #2263

dehaansa commented Dec 11, 2024

mattdurham Dec 12, 2024

dehaansa Dec 12, 2024

dehaansa Dec 12, 2024

mattdurham Dec 12, 2024

mattdurham Dec 12, 2024

mattdurham Dec 12, 2024

clayton-cornell Dec 12, 2024

mattdurham Dec 12, 2024

mattdurham left a comment

clayton-cornell left a comment

clayton-cornell Dec 12, 2024

clayton-cornell commented Dec 12, 2024

clayton-cornell left a comment

		@@ -0,0 +1,271 @@
		---
		canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.receiver.syslog/

feat: Add the otelcol syslog receiver #2263

Are you sure you want to change the base?

feat: Add the otelcol syslog receiver #2263

Conversation

dehaansa commented Dec 11, 2024

PR Description

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattdurham left a comment

Choose a reason for hiding this comment

clayton-cornell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clayton-cornell commented Dec 12, 2024

clayton-cornell left a comment

Choose a reason for hiding this comment