v0.6.1

This is release v0.6.1 of the Grafana Cloud Agent. This is a minor patch release recommended for all users.

Notable changes:

• [BUGFIX] Fix issue where updating a config in the scraping service may fail to pick up new targets. (@rfratto)
• [BUGFIX] Fix deadlock that slowly prevents the Agent from scraping targets at a high scrape volume. (@rfratto)
• [BUGFIX] Fix issue where v0.6.0 build information was empty when running the Agent with --version. (@rfratto)

Thanks to @amckinley for discovering the deadlock and reporting the missing build info!

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

The following scripts will download and install two Kubernetes manifests for the
Agent. The first manifest will collect metrics and the second will collect logs.
You will be prompted for input for each manifest. The script requires curl and
envsubst (GNU gettext).

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.6.1/production/kubernetes/install.sh)" | kubectl -ndefault apply -f -
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.6.1/production/kubernetes/install-loki.sh)" | kubectl -ndefault apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.6.1"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.6.1/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

agentctl

agentctl, a tool for helping you interact with the Agent,
is available as a Docker image:

docker pull "grafana/agentctl:v0.6.1"

Or as a binary. Like before, choose the assets below that matches your
operating system. For example, with linux on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.6.1/agentctl-linux-amd64.zip"

# extract the binary
unzip "agentctl-linux-amd64.zip"

# make sure it is executable
chmod a+x "agentctl-linux-amd64"

v0.6.0

NOTE: Unfortunately, FreeBSD builds are not available for this release. There is a bug in an upstream library that prevents builds from working. FreeBSD builds will return in a future release.

The Agent is back with its biggest release ever! A lot of work was done in just three weeks and I want to extend a thanks to everyone who assisted in reviewing code: @mem, @electron0zero, @hoenn, @jeschkies, and a special thanks to @gotjosh who went above and beyond expectations in helping to move things along.

Notable changes:

• Support for Loki by embedding the official Loki client, Promtail.
• Support for a new process_exporter integration that uses the lovely process-exporter.
• The embedded Prometheus version has been updated to 2.20.1.

For once, the number of changes is too long to list here so check out the full CHANGELOG for a detailed list of what has changed.

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

The following scripts will download and install two Kubernetes manifests for the
Agent. The first manifest will collect metrics and the second will collect logs.
You will be prompted for input for each manifest. The script requires curl and
envsubst (GNU gettext).

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.6.0/production/kubernetes/install.sh)" | kubectl -ndefault apply -f -
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.6.0/production/kubernetes/install-loki.sh)" | kubectl -ndefault apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.6.0"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.6.0/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

agentctl

agentctl, a tool for helping you interact with the Agent,
is available as a Docker image:

docker pull "grafana/agentctl:v0.6.0"

Or as a binary. Like before, choose the assets below that matches your
operating system. For example, with linux on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.6.0/agentctl-linux-amd64.zip"

# extract the binary
unzip "agentctl-linux-amd64.zip"

# make sure it is executable
chmod a+x "agentctl-linux-amd64"

v0.5.0

It's been a while since the last release, but the Agent is back with a new performance-focused minor version! The v0.5.0 release is notable primarily for its new scrape targets API and shared instance mode.

Shared instances mode combines the scrape configs of "compatible" instance configs together, lowering the number of Prometheus components that are spawned. This affects users of integrations and the scraping service, and will reduce the overall CPU and memory utilization for the Agent process. Shared instances mode is now the default mode.

The previous default behavior, now called "distinct instances mode" is deprecated and may be removed in a future release. Please report any problems you may see with the shared instances mode and your use cases for distinct instances if you do not want it to be removed.

Switching to shared instances mode should be transparent with the following exceptions:

• Metrics for Prometheus components (service discovery, scraping, remote_write, WAL) will be aggregated by the shared instance rather than individual instance config names. When shared instances mode is enabled, the instance_name label changes to instance_group_name.
• The scrape targets API will report the instance_group_name rather than the individual instance name. This behavior may change in a future release.

Notable changes:

• [FEATURE] A scrape targets API has been added to show every target the Agent is currently scraping, when it was last scraped, how long it took to scrape, and errors from the last scrape, if any. (@rfratto)
• [FEATURE] "Shared Instance Mode" is the new default mode for spawning Prometheus instances, and will improve CPU and memory usage for users of integrations and the scraping service. (@rfratto)
• [ENHANCEMENT] Memory stability and utilization of the WAL has been improved, and the reported number of active series in the WAL will stop double-counting recently churned series. (@rfratto)
• [ENHANCEMENT] Changing scrape_configs and remote_write configs for an instance will now be dynamically applied without restarting the instance. This will result in less missing metrics for users of the scraping service that change a config. (@rfratto)
• [ENHANCEMENT] The Tanka configuration now uses k8s-alpha. (@Duologic)
• [BUGFIX] The Tanka configuration will now also deploy a single-replica deployment specifically for scraping the Kubernetes API. This deployment acts together with the Daemonset to scrape the full cluster and the control plane. (@gotjosh)
• [BUGFIX] The node_exporter filesystem collector will now work on Linux systems without needing to manually set the blocklist and allowlist of filesystems. (@rfratto)

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

The following script will download a Kubernetes manifest for the Agent and
prompt for remote_write credentials. It requires curl and envsubst (GNU gettext).

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.5.0/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.5.0"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.5.0/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

agentctl

agentctl, a tool for helping you interact with the Agent,
is available as a Docker image:

docker pull "grafana/agentctl:v0.5.0"

Or as a binary. Like before, choose the assets below that matches your
operating system. For example, with linux on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.5.0/agentctl-linux-amd64.zip"

# extract the binary
unzip "agentctl-linux-amd64.zip"

# make sure it is executable
chmod a+x "agentctl-linux-amd64"

v0.4.0

This is release v0.4.0 of the Grafana Cloud Agent. Thanks to @robx for helping out with bug fixes in this one!

Notable changes:

This is a big feature release! Most notably, the Grafana Cloud Agent now includes support for integrations, which are embedded exporters that can scrape metrics on your behalf and remote_write them without having to configure Prometheus or install an exporter yourself.

The two integrations in v0.4.0 are the agent integration (allowing the Agent to scrape metrics from itself) and the node_exporter integration (allowing the Agent to scrape metrics from a UNIX host). The node_exporter integration built by vendoring the upstream node_exporter from Prometheus. More integrations will be added in future releases.

v0.4.0 includes some important bug fixes from the previous release and is recommended for all users of v0.3.2, namely:

• Prevent the Agent from crashing when a global Prometheus config stanza is not provided. (@robx)
• Enable agent host_filter in the Tanka configs, which was disabled by default by mistake. (@rfratto)

As always, the full changelog can be found within the repo.

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

The following script will download a Kubernetes manifest for the Agent and
prompt for remote_write credentials. It requires curl and envsubst (GNU gettext).

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.4.0/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.4.0"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.4.0/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

agentctl

agentctl, a tool for helping you interact with the Agent,
is available as a Docker image:

docker pull "grafana/agentctl:v0.4.0"

Or as a binary. Like before, choose the assets below that matches your
operating system. For example, with linux on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.4.0/agentctl-linux-amd64.zip"

# extract the binary
unzip "agentctl-linux-amd64.zip"

# make sure it is executable
chmod a+x "agentctl-linux-amd64"

v0.3.2

A little over a week from v0.3.1 and it's time for a new patch release with v0.3.2!

Notable changes:

This release primarily focuses on fixing bugs from v0.3.0 and v0.3.1 and is recommended for all v0.3.x users:

• [BUGFIX] A deadlock in scraping service mode when updating a config that shards to the same node has been fixed (@rfratto)

• [BUGFIX] remote_write config stanzas will no longer ignore password_file (@rfratto)

• [BUGFIX] scrape_config client secrets (e.g., basic auth, bearer token, password_file) will now be properly retained in scraping service mode (@rfratto)

• [BUGFIX] Labels for CPU, RX, and TX graphs in the Agent Operational dashboard now correctly show the pod name of the Agent instead of the exporter name. (@rfratto)

There is one other major change that requires caution: the Tanka configs and Kubernetes manifest now use different labels for metrics scraped from pods discovered using Kubernetes SD. The previous instance label has been renamed to pod, and a new instance label with fully unique values has taken its place. A container label has also been added.

This change was made to sync with the Prometheus Ksonnet found in grafana/jsonnet-libs, and is now matches up with labeling standards from the Kubernetes ecosystem.

The agent dashboard mixins have been updated to accomodate the new labels, so if you update your grafana/agent to v0.3.2 and use our Tanka configs or our Kubernetes manifest, you should also update your mixins (if deployed) for the dashboards to work properly. No other changes are required when updating to v0.3.2.

As always, the full list of changes can be found in the CHANGELOG.

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.3.2/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.3.2"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.3.2/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

agentctl

agentctl, a tool for helping you interact with the Agent,
is available as a Docker image:

docker pull "grafana/agentctl:v0.3.2"

Or as a binary. Like before, choose the assets below that matches your
operating system. For example, with linux on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.3.2/agentctl-linux-amd64.zip"

# extract the binary
unzip "agentctl-linux-amd64.zip"

# make sure it is executable
chmod a+x "agentctl-linux-amd64"

v0.3.1

This is release v0.3.1 of the Grafana Cloud Agent.

Notable changes:

• [BUGFIX] A typo in the Tanka configs and Kubernetes manifests that prevents the Agent launching with v0.3.0 has been fixed (@captncraig)

• [BUGFIX] Fixed a bug where Tanka mixins could not be used due to an issue with the folder placement enhancement (@rfratto)

• [ENHANCEMENT] agentctl and the config API will now validate that the YAML they receive are valid instance configs. (@rfratto)

• [FEATURE] The Agent has upgraded its vendored Prometheus to v2.18.1 (@gotjosh, @rfratto)

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.3.1/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.3.1"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.3.1/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

agentctl

agentctl, a tool for helping you interact with the Agent,
is available as a Docker image:

docker pull "grafana/agentctl:v0.3.1"

Or as a binary. Like before, choose the assets below that matches your
operating system. For example, with linux on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.3.1/agentctl-linux-amd64.zip"

# extract the binary
unzip "agentctl-linux-amd64.zip"

# make sure it is executable
chmod a+x "agentctl-linux-amd64"

v0.3.0

This is release v0.3.0 of the Grafana Cloud Agent! 🎉

Before we dive into the notable changes, a huge shoutout is deserved for @hoenn and @gotjosh who have been spending a lot of time helping out with the Agent and pushing it to grow ❤️. Every contribution, whether
it's review feedback or a PR, is greatly appreciated.

Notable changes:

We've been hard at work implementing a new operational mode for the Agent, which we're calling the "scraping service mode." The scraping service mode allows you to deploy a clustered set of Agents that automatically balance scrape load between them based on the number of Agents within the cluster. The scraping service mode is experimental and should be used with caution.

Released alongside the scraping service mode is agentctl, which will aim to help users interact with the Agent. The v0.3.0 release gives agentctl the ability to sync config files to the KV store used for the scraping service mode. For more info on the scraping service mode and agentctl, check out its dedicated docs.

Full release notes are located within the repo's CHANGELOG.md

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.3.0/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.3.0"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.3.0/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

v0.2.0

The v0.2.0 release of the Grafana Cloud Agent approaches! v0.2.0 is a small release, but important as it fixes a critical bug around scraping cAdvisor and Kubelet metrics when using Kubernetes service discovery with role: node configured.

Notable changes:

• [FEATURE] The Prometheus remote write protocol will now send scraped metadata (metric name, help, type and unit). This results in almost negligent bytes sent increase as metadata is only sent every minute. It is on by default. (@gotjosh)

  These metrics are available to monitor metadata being sent:

  • prometheus_remote_storage_succeeded_metadata_total
  • prometheus_remote_storage_failed_metadata_total
  • prometheus_remote_storage_retried_metadata_total
  • prometheus_remote_storage_sent_batch_duration_seconds and
    prometheus_remote_storage_sent_bytes_total have a new label “type” with
    the values of metadata or samples.

• [FEATURE] The Agent has upgraded its vendored Prometheus to v2.17.1 (@rfratto)

• [BUGFIX] Invalid configs passed to the agent will now stop the process after they are logged as invalid; previously the Agent process would continue. (@rfratto)

• [BUGFIX] Enabling host_filter will now allow metrics from node role Kubernetes service discovery to be scraped properly (e.g., cAdvisor, Kubelet). (@rfratto)

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.2.0/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.2.0"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.2.0/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

v0.1.1

Roughly five hours from the last one, this is release v0.1.1 of the Grafana Cloud Agent!

Notable changes:

Only some minor bug fixes have been made since the last release. Namely:

• Nits in documentation (@sh0rez)
• Fix various dashboard mixin problems from v0.1.0 (@rfratto)
• Pass through release tag to docker build (@rfratto)

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.1.1/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.1.1"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.1.1/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"

v0.1.0

This is the first release of the Grafana Cloud Agent! 🎉

Installation:

Grafana Cloud Agent is currently distributed in plain binary form, Docker
container images, and a Kubernetes install script. Choose whichever fits your
use-case best.

Kubernetes Install Script

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.1.0/production/kubernetes/install.sh)" | kubectl apply -f -

Docker container:

• https://hub.docker.com/r/grafana/agent

docker pull "grafana/agent:v0.1.0"

Binary

We provide precompiled binary executables for the most common operating systems.
Choose from the assets below for your matching operating system. Example for the
linux operating system on amd64:

# download the binary
curl -O -L "https://github.com/grafana/agent/releases/download/v0.1.0/agent-linux-amd64.zip"

# extract the binary
unzip "agent-linux-amd64.zip"

# make sure it is executable
chmod a+x "agent-linux-amd64"