Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable optional trust domain label for all metrics #5673

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions doc/telemetry/telemetry_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,21 @@ You may use all, some, or none of the collectors. The following collectors suppo

## Telemetry configuration syntax

| Configuration | Type | Description | Default |
|-----------------------|---------------|---------------------------------------------------------------|--------------------------|
| `InMem` | `InMem` | In-memory configuration | running |
| `Prometheus` | `Prometheus` | Prometheus configuration | |
| `DogStatsd` | `[]DogStatsd` | List of DogStatsd configurations | |
| `Statsd` | `[]Statsd` | List of Statsd configurations | |
| `M3` | `[]M3` | List of M3 configurations | |
| `MetricPrefix` | `string` | Prefix to add to all emitted metrics | spire_server/spire_agent |
| `EnableHostnameLabel` | `bool` | Enable adding hostname to labels | true |
| `AllowedPrefixes` | `[]string` | A list of metric prefixes to allow, with '.' as the separator | |
| `AllowedPrefixes` | `[]string` | A list of metric prefixes to allow, with '.' as the separator | |
| `BlockedPrefixes` | `[]string` | A list of metric prefixes to block, with '.' as the separator | |
| `AllowedLabels` | `[]string` | A list of metric labels to allow, with '.' as the separator | |
| `BlockedLabels` | `[]string` | A list of metric labels to block, with '.' as the separator | |
| Configuration | Type | Description | Default |
|--------------------------|---------------|---------------------------------------------------------------|--------------------------|
| `InMem` | `InMem` | In-memory configuration | running |
| `Prometheus` | `Prometheus` | Prometheus configuration | |
| `DogStatsd` | `[]DogStatsd` | List of DogStatsd configurations | |
| `Statsd` | `[]Statsd` | List of Statsd configurations | |
| `M3` | `[]M3` | List of M3 configurations | |
| `MetricPrefix` | `string` | Prefix to add to all emitted metrics | spire_server/spire_agent |
| `EnableTrustDomainLabel` | `bool` | Enable optional trust domain label for all metrics | false |
| `EnableHostnameLabel` | `bool` | Enable adding hostname to labels | true |
| `AllowedPrefixes` | `[]string` | A list of metric prefixes to allow, with '.' as the separator | |
| `AllowedPrefixes` | `[]string` | A list of metric prefixes to allow, with '.' as the separator | |
| `BlockedPrefixes` | `[]string` | A list of metric prefixes to block, with '.' as the separator | |
| `AllowedLabels` | `[]string` | A list of metric labels to allow, with '.' as the separator | |
| `BlockedLabels` | `[]string` | A list of metric labels to block, with '.' as the separator | |

### `Prometheus`

Expand Down Expand Up @@ -79,7 +80,6 @@ telemetry {
]

InMem {}

AllowedLabels = []
BlockedLabels = []
AllowedPrefixes = []
Expand Down
1 change: 1 addition & 0 deletions pkg/agent/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ func (a *Agent) Run(ctx context.Context) error {
FileConfig: a.c.Telemetry,
Logger: a.c.Log.WithField(telemetry.SubsystemName, telemetry.Telemetry),
ServiceName: telemetry.SpireAgent,
TrustDomain: a.c.TrustDomain.Name(),
})
if err != nil {
return err
Expand Down
14 changes: 8 additions & 6 deletions pkg/common/telemetry/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ type MetricsConfig struct {
Logger logrus.FieldLogger
ServiceName string
Sinks []Sink
TrustDomain string
}

type FileConfig struct {
Expand All @@ -19,12 +20,13 @@ type FileConfig struct {
M3 []M3Config `hcl:"M3"`
InMem *InMem `hcl:"InMem"`

MetricPrefix string `hcl:"MetricPrefix"`
EnableHostnameLabel *bool `hcl:"EnableHostnameLabel"`
AllowedPrefixes []string `hcl:"AllowedPrefixes"` // A list of metric prefixes to allow, with '.' as the separator
BlockedPrefixes []string `hcl:"BlockedPrefixes"` // A list of metric prefixes to block, with '.' as the separator
AllowedLabels []string `hcl:"AllowedLabels"` // A list of metric labels to allow, with '.' as the separator
BlockedLabels []string `hcl:"BlockedLabels"` // A list of metric labels to block, with '.' as the separator
MetricPrefix string `hcl:"MetricPrefix"`
EnableTrustDomainLabel *bool `hcl:"EnableTrustDomainLabel"`
EnableHostnameLabel *bool `hcl:"EnableHostnameLabel"`
AllowedPrefixes []string `hcl:"AllowedPrefixes"` // A list of metric prefixes to allow, with '.' as the separator
BlockedPrefixes []string `hcl:"BlockedPrefixes"` // A list of metric prefixes to block, with '.' as the separator
AllowedLabels []string `hcl:"AllowedLabels"` // A list of metric labels to allow, with '.' as the separator
BlockedLabels []string `hcl:"BlockedLabels"` // A list of metric labels to block, with '.' as the separator

UnusedKeyPositions map[string][]token.Pos `hcl:",unusedKeyPositions"`
}
Expand Down
1 change: 1 addition & 0 deletions pkg/common/telemetry/dogstatsd_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ func testDogStatsdConfig() *MetricsConfig {
return &MetricsConfig{
Logger: l,
ServiceName: "foo",
TrustDomain: "test.org",
FileConfig: FileConfig{
DogStatsd: []DogStatsdConfig{
{
Expand Down
1 change: 1 addition & 0 deletions pkg/common/telemetry/inmem_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ func testInmemConfig() *MetricsConfig {
return &MetricsConfig{
Logger: logger,
ServiceName: "foo",
TrustDomain: "test.org",
FileConfig: FileConfig{InMem: &InMem{}},
}
}
Expand Down
1 change: 1 addition & 0 deletions pkg/common/telemetry/m3_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ func testM3Config() *MetricsConfig {
return &MetricsConfig{
Logger: l,
ServiceName: "foo",
TrustDomain: "test.org",
FileConfig: FileConfig{
M3: []M3Config{
{
Expand Down
41 changes: 28 additions & 13 deletions pkg/common/telemetry/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ type MetricsImpl struct {
c *MetricsConfig
runners []sinkRunner
// Each instance of metrics.Metrics in the slice corresponds to one metrics sink type
metricsSinks []*metrics.Metrics
metricsSinks []*metrics.Metrics
enableTrustDomainLabel bool
}

var _ Metrics = (*MetricsImpl)(nil)
Expand Down Expand Up @@ -83,12 +84,18 @@ func NewMetrics(c *MetricsConfig) (*MetricsImpl, error) {
} else {
conf.EnableHostnameLabel = true
}

conf.EnableTypePrefix = runner.requiresTypePrefix()
conf.AllowedLabels = c.FileConfig.AllowedLabels
conf.BlockedLabels = c.FileConfig.BlockedLabels
conf.AllowedPrefixes = c.FileConfig.AllowedPrefixes
conf.BlockedPrefixes = c.FileConfig.BlockedPrefixes

impl.enableTrustDomainLabel = false
if c.FileConfig.EnableTrustDomainLabel != nil {
impl.enableTrustDomainLabel = *c.FileConfig.EnableTrustDomainLabel
}

metricsSink, err := metrics.New(conf, fanout)
if err != nil {
return nil, err
Expand All @@ -112,13 +119,15 @@ func (m *MetricsImpl) ListenAndServe(ctx context.Context) error {
}

func (m *MetricsImpl) SetGauge(key []string, val float32) {
for _, s := range m.metricsSinks {
s.SetGauge(key, val)
}
m.SetGaugeWithLabels(key, val, nil)
}

// SetGaugeWithLabels delegates to embedded metrics, sanitizing labels
func (m *MetricsImpl) SetGaugeWithLabels(key []string, val float32, labels []Label) {
if m.enableTrustDomainLabel {
labels = append(labels, Label{Name: TrustDomain, Value: m.c.TrustDomain})
}

sanitizedLabels := SanitizeLabels(labels)
for _, s := range m.metricsSinks {
s.SetGaugeWithLabels(key, val, sanitizedLabels)
Expand All @@ -132,41 +141,47 @@ func (m *MetricsImpl) EmitKey(key []string, val float32) {
}

func (m *MetricsImpl) IncrCounter(key []string, val float32) {
for _, s := range m.metricsSinks {
s.IncrCounter(key, val)
}
m.IncrCounterWithLabels(key, val, nil)
}

// IncrCounterWithLabels delegates to embedded metrics, sanitizing labels
func (m *MetricsImpl) IncrCounterWithLabels(key []string, val float32, labels []Label) {
if m.enableTrustDomainLabel {
labels = append(labels, Label{Name: TrustDomain, Value: m.c.TrustDomain})
}

sanitizedLabels := SanitizeLabels(labels)
for _, s := range m.metricsSinks {
s.IncrCounterWithLabels(key, val, sanitizedLabels)
}
}

func (m *MetricsImpl) AddSample(key []string, val float32) {
for _, s := range m.metricsSinks {
s.AddSample(key, val)
}
m.AddSampleWithLabels(key, val, nil)
}

// AddSampleWithLabels delegates to embedded metrics, sanitizing labels
func (m *MetricsImpl) AddSampleWithLabels(key []string, val float32, labels []Label) {
if m.enableTrustDomainLabel {
labels = append(labels, Label{Name: TrustDomain, Value: m.c.TrustDomain})
}

sanitizedLabels := SanitizeLabels(labels)
for _, s := range m.metricsSinks {
s.AddSampleWithLabels(key, val, sanitizedLabels)
}
}

func (m *MetricsImpl) MeasureSince(key []string, start time.Time) {
for _, s := range m.metricsSinks {
s.MeasureSince(key, start)
}
m.MeasureSinceWithLabels(key, start, nil)
}

// MeasureSinceWithLabels delegates to embedded metrics, sanitizing labels
func (m *MetricsImpl) MeasureSinceWithLabels(key []string, start time.Time, labels []Label) {
if m.enableTrustDomainLabel {
labels = append(labels, Label{Name: TrustDomain, Value: m.c.TrustDomain})
}

sanitizedLabels := SanitizeLabels(labels)
for _, s := range m.metricsSinks {
s.MeasureSinceWithLabels(key, start, sanitizedLabels)
Expand Down
1 change: 1 addition & 0 deletions pkg/common/telemetry/prometheus_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ func testPrometheusConfig() *MetricsConfig {
return &MetricsConfig{
Logger: l,
ServiceName: "foo",
TrustDomain: "test.org",
FileConfig: FileConfig{
// Let prometheus listen on a random port
Prometheus: &PrometheusConfig{},
Expand Down
1 change: 1 addition & 0 deletions pkg/common/telemetry/statsd_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ func testStatsdConfigWithPort(port int) *MetricsConfig {
return &MetricsConfig{
Logger: l,
ServiceName: "foo",
TrustDomain: "test.org",
FileConfig: FileConfig{
Statsd: []StatsdConfig{
{
Expand Down
1 change: 1 addition & 0 deletions pkg/server/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ func (s *Server) run(ctx context.Context) (err error) {
FileConfig: s.config.Telemetry,
Logger: s.config.Log.WithField(telemetry.SubsystemName, telemetry.Telemetry),
ServiceName: telemetry.SpireServer,
TrustDomain: s.config.TrustDomain.Name(),
})
if err != nil {
return err
Expand Down