telegraf

Telegraf module

This module allows enabling of telegraf service for metrics reporting. Meant to be used in instances containing services consul, nomad_client, nomad_server and vault.

Telegraf Configuration

By default, the Packer templates in the Core module will install Telegraf and the relevant configuration script. A default configuration template is copied to the AMI. The default configuration will discard all outputs.

You will need to provide your own configuration file to output to your desired sinks. In your Packer template, you should copy your configuration files ending with .conf to the directory /etc/telegraf/telegraf.d/.

By default, all the default Packer templates will install consul-template to the AMIs and is configured to start consul-template by default. You might want to use this to render out the configuration files for Telegraf using values from Consul and/or Vault.

If you are using the provided Elasticsearch or Prometheus modules, you can set the appropriate variables in this module for outputs to these two modules to be automatically configured.

Example usage

module "telegraf" {
  source = "github.com/GovTechSG/terraform-modules.git//terraform-modules/modules/telegraf"

  # Optional, default is true
  core_integration = true

  # Optional, default is terraform/
  consul_key_prefix = "terraform/"

  # Optional, default is true
  consul_enabled = true

  # Optional, default is true
  nomad_server_enabled = true

  # Optional, default is true
  nomad_client_enabled = true

  # Optional, default is true
  vault_enabled = true
}

Defining Additional Server Types

If you have a new "server type" or a different category of servers to report, you can make use of the automated bootstrap and configuration that this repository. You can always configure Telegraf manually if you elect not to do so.

For example, you might want to add a separate cluster of Nomad clients and have their metrics reported separately.

The following pre-requisites must be met when you want to make use of the automation:

• You should have installed Telegraf and the bootstrap using the Ansible role that is included by default using the default Packer images for the Core AMIs.
• Your AMI must have Consul installed and configured to run Consul agent. Installation of Consul agent can be done using this module and Consul Agent can be started and run using this module.
• You will also need to provide the appropriate Telegraf configuration file while using the Ansible role.
• Define the key under the path ${prefix}telegraf/server_type/${server_type}/enabled in Consul KV store with value yes. The default prefix is terraform/.
• If you want to enable automatic configuration of Elasticsearch, you will have to set the key ${prefix}telegraf/server_type/${server_type}/output/elasticsearch/enabled to yes and set the key ${prefix}telegraf/server_type/${server_type}/output/elasticsearch/service_name to the service name of Elasticsearch in Consul.
• Run the bootstrap script to initialise Telegraf after Consul agent has been started. By default, the Ansible role installs the script to /opt/run-telegraf. For example, you can run /opt/run-telegraf --type "${server_type}". Use the --help flag for more options.

For the required Consul keys, you can use the helper module to configure.

For more information and examples, refer to the Packer templates and user_data scripts for the various types of servers in the core module.

Inputs

Name	Description	Type	Default	Required
consul_enabled	Enable Telegraf for Consul servers	string	true	no
consul_key_prefix	Path prefix to the key in Consul to set for the core module to know that this module has been applied. If you change this, you have to update the integration_consul_prefix variable in the core module as well.	string	terraform/	no
consul_output_elasticsearch_service_name	Service name in Consul to lookup Elasticsearch URLs	string	elasticsearch	no
consul_output_elastisearch	Enable metrics output to Elasticsearch	string	false	no
consul_output_prometheus	Create a Prometheus Client to serve the metrics for a Prometheus server to scrape	string	false	no
consul_output_prometheus_service_cidrs	List of CIDRs that the Prometheus client will permit scraping. Remember to allow 127.0.0.1/32 for Consul health checks.	string	<list>	no
consul_output_prometheus_service_name	Name of the service to advertise in Consul	string	prometheus-client	no
consul_output_prometheus_service_port	Port of the Prometheus Client	string	9273	no
core_integration	Enable integration with the core module by setting some values in Consul so that the user_data scripts in core know that this module has been applied	string	true	no
nomad_client_enabled	Enable Telegraf for Nomad clients	string	true	no
nomad_client_output_elasticsearch_service_name	Service name in Consul to lookup Elasticsearch URLs	string	elasticsearch	no
nomad_client_output_elastisearch	Enable metrics output to Elasticsearch	string	false	no
nomad_client_output_prometheus	Create a Prometheus Client to serve the metrics for a Prometheus server to scrape	string	false	no
nomad_client_output_prometheus_service_cidrs	List of CIDRs that the Prometheus client will permit scraping. Remember to allow 127.0.0.1/32 for Consul health checks.	string	<list>	no
nomad_client_output_prometheus_service_name	Name of the service to advertise in Consul	string	prometheus-client	no
nomad_client_output_prometheus_service_port	Port of the Prometheus Client	string	9273	no
nomad_server_enabled	Enable Telegraf for Nomad servers	string	true	no
nomad_server_output_elasticsearch_service_name	Service name in Consul to lookup Elasticsearch URLs	string	elasticsearch	no
nomad_server_output_elastisearch	Enable metrics output to Elasticsearch	string	false	no
nomad_server_output_prometheus	Create a Prometheus Client to serve the metrics for a Prometheus server to scrape	string	false	no
nomad_server_output_prometheus_service_cidrs	List of CIDRs that the Prometheus client will permit scraping. Remember to allow 127.0.0.1/32 for Consul health checks.	string	<list>	no
nomad_server_output_prometheus_service_name	Name of the service to advertise in Consul	string	prometheus-client	no
nomad_server_output_prometheus_service_port	Port of the Prometheus Client	string	9273	no
vault_enabled	Enable Telegraf for Vault servers	string	true	no
vault_output_elasticsearch_service_name	Service name in Consul to lookup Elasticsearch URLs	string	elasticsearch	no
vault_output_elastisearch	Enable metrics output to Elasticsearch	string	false	no
vault_output_prometheus	Create a Prometheus Client to serve the metrics for a Prometheus server to scrape	string	false	no
vault_output_prometheus_service_cidrs	List of CIDRs that the Prometheus client will permit scraping. Remember to allow 127.0.0.1/32 for Consul health checks.	string	<list>	no
vault_output_prometheus_service_name	Name of the service to advertise in Consul	string	prometheus-client	no
vault_output_prometheus_service_port	Port of the Prometheus Client	string	9273	no