Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
6b7cdbb
Work on adding auth so far
aidanleuck Nov 27, 2024
d3812c2
Cleanup
aidanleuck Nov 27, 2024
c7155c7
Made a ton of progress
aidanleuck Nov 28, 2024
37e5880
Fix test fails?
aidanleuck Nov 28, 2024
3732a54
Refactor
aidanleuck Nov 29, 2024
8e82871
Add auth blocks to implementing extensions
aidanleuck Nov 29, 2024
93377d0
Refactor to use feature flag
aidanleuck Nov 30, 2024
497276a
Comments
aidanleuck Nov 30, 2024
cdb1c86
Cleanup
aidanleuck Nov 30, 2024
a7d45c7
Spacing
aidanleuck Nov 30, 2024
3f21422
Update docs
aidanleuck Nov 30, 2024
6f477ff
Update CHANGELOG
aidanleuck Nov 30, 2024
6510731
Last auth extension missing
aidanleuck Nov 30, 2024
9998e92
We also need grpc auth
aidanleuck Nov 30, 2024
dade331
Fix opencensus docs
aidanleuck Dec 1, 2024
055bb1b
Fix extra comment
aidanleuck Dec 1, 2024
fd537ff
Update comment with findings
aidanleuck Dec 1, 2024
e8c8e36
Merge with main to fix conflicts
aidanleuck Dec 2, 2024
ddbca7b
Properly fix merge conflict
aidanleuck Dec 2, 2024
592e535
Fix changelog
aidanleuck Dec 10, 2024
883ccee
Save file
aidanleuck Dec 10, 2024
b6ba127
Spelling error
aidanleuck Dec 10, 2024
cf178fa
That has been released now
aidanleuck Dec 10, 2024
23ab3be
Add auth support to influxdb receiver
aidanleuck Dec 12, 2024
f5c42c2
Merge remote-tracking branch 'origin' into feature/add_auth_to_otlp_r…
aidanleuck Dec 12, 2024
c0d3ed0
Fix failing auth test/MAIL
aidanleuck Dec 15, 2024
1139424
Merge remote-tracking branch 'origin' into feature/add_auth_to_otlp_r…
aidanleuck Dec 16, 2024
4665aaf
Comment cleanup
aidanleuck Dec 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ Main (unreleased)

- Add relevant golang environment variables to the support bundle (@dehaansa)

- Add support for server authentication to otelcol components. (@aidaleuc)

- Update mysqld_exporter from v0.15.0 to v0.16.0 (including 2ef168bf6), most notable changes: (@cristiangreco)
- Support MySQL 8.4 replicas syntax
- Fetch lock time and cpu time from performance schema
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ title: otelcol.auth.headers
`otelcol.auth.headers` exposes a `handler` that can be used by other `otelcol`
components to authenticate requests using custom headers.

This extension only supports client authentication.

{{< admonition type="note" >}}
`otelcol.auth.headers` is a wrapper over the upstream OpenTelemetry Collector `headerssetter` extension.
Bug reports or feature requests will be redirected to the upstream repository, if necessary.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ title: otelcol.auth.oauth2

`otelcol.auth.oauth2` exposes a `handler` that can be used by other `otelcol` components to authenticate requests using OAuth 2.0.

This extension only supports client authentication.

The authorization tokens can be used by HTTP and gRPC based OpenTelemetry exporters.
This component can fetch and refresh expired tokens automatically.
Refer to the [OAuth 2.0 Authorization Framework](https://datatracker.ietf.org/doc/html/rfc6749#section-4.4) for more information about the Auth 2.0 Client Credentials flow.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ title: otelcol.auth.sigv4
components to authenticate requests to AWS services using the AWS Signature Version 4 (SigV4) protocol.
For more information about SigV4 see the AWS documentation about [Signing AWS API requests][].

This extension only supports client authentication.

[Signing AWS API requests]: https://docs.aws.amazon.com/general/latest/gr/signing-aws-api-requests.html

> **NOTE**: `otelcol.auth.sigv4` is a wrapper over the upstream OpenTelemetry
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ Name | Type | Description
`max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

### tls block

Expand Down Expand Up @@ -125,6 +126,7 @@ Name | Type | Description
`read_buffer_size` | `string` | Size of the read buffer the gRPC server will use for reading from clients. | `"512KiB"` | no
`write_buffer_size` | `string` | Size of the write buffer the gRPC server will use for writing to clients. | | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

### keepalive block

Expand Down Expand Up @@ -293,3 +295,23 @@ otelcol.extension.jaeger_remote_sampling "example" {
}
}
```

## Enabling Authentication

You can create a `jaeger_remote_sampling` extensions that requires authentication for requests. This is useful for limiting access to the sampling document. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.

```alloy
otelcol.extension.jaeger_remote_sampling "default" {
http {
auth = otelcol.auth.basic.creds.handler
}
grpc {
auth = otelcol.auth.basic.creds.handler
}
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Name | Type | Description
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | `false` | no
`read_timeout` | `duration` | Read timeout for requests of the HTTP server. | `"60s"` | no
`compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

By default, `otelcol.receiver.datadog` listens for HTTP connections on `localhost`.
To expose the HTTP server to other machines on your network, configure `endpoint` with the IP address to listen on, or `0.0.0.0:8126` to listen on all network interfaces.
Expand Down Expand Up @@ -134,6 +135,25 @@ otelcol.exporter.otlp "default" {
}
}
```

## Enabling Authentication

You can create a `datadog` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.

```alloy
otelcol.receiver.datadog "default" {
output {
metrics = [otelcol.processor.batch.default.input]
traces = [otelcol.processor.batch.default.input]
}
auth = otelcol.auth.basic.creds.handler
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```
<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ otelcol.receiver.influxdb "influxdb_metrics" {
| `max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no |
| `include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no |
| `compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no |
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

By default, `otelcol.receiver.influxdb` listens for HTTP connections on `localhost`.
To expose the HTTP server to other machines on your network, configure `endpoint` with the IP address to listen on, or `0.0.0.0:8086` to listen on all network interfaces.
Expand Down Expand Up @@ -150,6 +151,24 @@ prometheus.remote_write "mimir" {
}
```

## Enabling Authentication

You can create a `influxdb` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.

```alloy
otelcol.receiver.influxdb "influxdb_metrics" {
output {
metrics = [...]
}
auth = otelcol.auth.basic.creds.handler
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```

<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ Name | Type | Description
`read_buffer_size` | `string` | Size of the read buffer the gRPC server will use for reading from clients. | `"512KiB"` | no
`write_buffer_size` | `string` | Size of the write buffer the gRPC server will use for writing to clients. | | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

### tls block

Expand Down Expand Up @@ -154,6 +155,7 @@ Name | Type | Description
`max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

### cors block

Expand Down Expand Up @@ -262,6 +264,28 @@ otelcol.exporter.otlp "default" {

`otelcol.receiver.jaeger` supports [Gzip](https://en.wikipedia.org/wiki/Gzip) for compression.

## Enabling Authentication

You can create a `jaeger` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility. This functionality is currently limited to the GRPC/HTTP blocks.

```alloy
otelcol.receiver.jaeger "default" {
protocols {
grpc {
auth = otelcol.auth.basic.creds.handler
}
thrift_http {
auth = otelcol.auth.basic.creds.handler
}
}
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```

<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,13 @@ Name | Type | Description | Default | Required
`read_buffer_size` | `string` | Size of the read buffer the gRPC server will use for reading from clients. | `"512KiB"` | no
`write_buffer_size` | `string` | Size of the write buffer the gRPC server will use for writing to clients. | | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

`cors_allowed_origins` are the allowed [CORS](https://github.com/rs/cors) origins for HTTP/JSON requests.
An empty list means that CORS is not enabled at all. A wildcard (*) can be
used to match any origin or one or more characters of an origin.


The "endpoint" parameter is the same for both gRPC and HTTP/JSON, as the protocol is recognized and processed accordingly.

To write traces with HTTP/JSON, `POST` to `[address]/v1/trace`. The JSON message format parallels the gRPC protobuf format. For details, refer to its [OpenApi specification](https://github.com/census-instrumentation/opencensus-proto/blob/master/gen-openapi/opencensus/proto/agent/trace/v1/trace_service.swagger.json).
Expand Down Expand Up @@ -207,6 +209,21 @@ otelcol.exporter.otlp "default" {
}
}
```

## Enabling Authentication

You can create a `opencensus` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.

```alloy
otelcol.receiver.opencensus "default" {
auth = otelcol.auth.basic.creds.handler
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```
<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ Name | Type | Description | Default | Required
`read_buffer_size` | `string` | Size of the read buffer the gRPC server will use for reading from clients. | `"512KiB"` | no
`write_buffer_size` | `string` | Size of the write buffer the gRPC server will use for writing to clients. | | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

### tls block

Expand Down Expand Up @@ -145,6 +146,7 @@ Name | Type | Description | Default | Required
`metrics_url_path` | `string` | The URL path to receive metrics on. | `"/v1/metrics"` | no
`logs_url_path` | `string` | The URL path to receive logs on. | `"/v1/logs"` | no
`compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

To send telemetry signals to `otelcol.receiver.otlp` with HTTP/JSON, POST to:
* `[endpoint][traces_url_path]` for traces.
Expand Down Expand Up @@ -240,6 +242,30 @@ otelcol.exporter.otlp "default" {
## Technical details

`otelcol.receiver.otlp` supports [gzip](https://en.wikipedia.org/wiki/Gzip) for compression.

## Enabling Authentication

You can create a `otlp` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.

```alloy
otelcol.receiver.otlp "default" {
http {
auth = otelcol.auth.basic.creds.handler
}
grpc {
auth = otelcol.auth.basic.creds.handler
}

output {
...
}
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```
<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Name | Type | Description | Default | Required
`max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no
`include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no
`compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no

If `parse_string_tags` is `true`, string tags and binary annotations are
converted to `int`, `bool`, and `float` if possible. String tags and binary
Expand Down Expand Up @@ -141,6 +142,21 @@ otelcol.exporter.otlp "default" {
}
}
```

## Enabling Authentication

You can create a `zipkin` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.

```alloy
otelcol.receiver.zipkin "default" {
auth = otelcol.auth.basic.creds.handler
}

otelcol.auth.basic "creds" {
username = sys.env("USERNAME")
password = sys.env("PASSWORD")
}
```
<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ func TestWithOtelcolConsumer(t *testing.T) {
err := otelcolExporter.Run(ctx, otlphttp.Arguments{
Client: otlphttp.HTTPClientArguments(otelcol.HTTPClientArguments{
Endpoint: finalOtelServer.URL,
Auth: &otelcolAuthHeaderExport.Handler,
Auth: otelcolAuthHeaderExport.Handler,
TLS: otelcol.TLSClientArguments{
Insecure: true,
InsecureSkipVerify: true,
Expand Down
Loading
Loading