Description

This cookbook provides providers and resources to configure and manage Diamond Diamond is a python daemon that collects system metrics and publishes them to Graphite. It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source.

Requirements

Chef Version

Requires Chef 11 or higher.

Cookbooks

Opscode Cookbooks:

• build-essential
• python
• git

Default Collectors

Diamond has the following Collectors enabled by default:
Cpu, DiskSpace, DiskUsage, LoadAvg, Memory, SockStat, Vmstat.

Resources and Providers

This cookbook provides three resources and corresponding providers.

install.rb

Install Diamond with this resource.

Actions:

• git - Clones the source from Github and installs using python setup.py install
• tarball - Fetches a tarball containing using HTTP and installs using python setup.py install

Attribute Parameters:

diamond::install

Attribute	Type	Description	Default
prefix	String	Prefix of where to install diamond to	/opt/diamond
git_repository_uri	String	Location of where the cookbook fetches Diamond from to install	https://github.com/BrightcoveOS/Diamond.git
git_reference	String	Which git reference to install Diamond from	master
required_python_packages	Hash	List of python packages to install for Diamond	{ "configobj" => "4.7.2","psutil" => "1.2.1"}>
cookbook	String	Name of the cookbook that holds the erb templates	diamond
tarball_path	String	Url of the Tarball to install Diamond with	https://github.com/BrightcoveOS/Diamond/archive/master.tar.gz
tarball_extract_fldr	String	Name of the directory that holds Diamond in the tarball	Diamond-master

configure.rb

Configure Diamond with this resource.

Actions:

• config - Creates #{new_resource.prefix}/etc/diamond/diamond.conf

Attribute Parameters:

diamond::configure

Attribute	Type	Description	Default
cookbook	String	Name of the cookbook that holds the erb templates	diamond
runit_name	String	Name of the runit service to restart when modifying this configuration	diamond
diamond_configuration_source>	String	Name of the ERB Template used to generate diamond.conf	diamond.conf.erb
diamond_handlers	Array	Array of the Handlers to configure Diamond with	["diamond.handler.graphitepickle.GraphitePickleHandler"]
diamond_user	String	Name of the user for diamond to run as
diamond_group	String	Name of the group the diamond user will run as
collectors_config_path	String	Path where Diamond will store and retrieve configuration files for the Collectors	/opt/diamond/etc/diamond/collectors
diamond_collectors_path	String	Directory where the collectors are installed	/opt/diamond/share/diamond/collectors
archive_handler	Hash	Configuration parameters for the archive handler	{ "log_file" => "/var/log/diamond/diamond.log", "days" => 7 }
graphite_handler	Hash	Configuration parameters for the graphite handler	{ "host" => "localhost", "port" => 2003, "batch" => 256, "timeout" => 15 }
graphite_picklehandler	Hash	Configuration parameters for the graphite pickle handler	{ "host" => "localhost", "port" => 2004, "batch" => 256, "timeout" => 15 }
statsdhandler	Hash	Configuration parameters for the statsd handler	{ "host" => "127.0.0.1", "port" => 8125 }
tsdbhandler	Hash	Configuration parameters for the tsdb handler	{ "host" => "127.0.0.1", "port" => 4242, "timeout" => 15 }
mysqlhandler	Hash	Configuration parameters for the mysql handler	{ "host" => "127.0.0.1", "port" => 3306, "username" => String.new, "password" => String.new, "database" => "diamond", "table" => "metrics", "col_time" => "timestamp", "col_metric" => "metric", "col_value" => "value" }
collectors	Hash	Configuration for the collectors allowing you to configure suffix, prefix, how to determine hostname and/or set the hostname manually	{ "hostname_method" => "fqdn_short", "hostname" => String.new, "path_prefix" => String.new, "path_suffix" => String.new, "interval" => 300 }

plugin.rb

Enable / Disable plugins with this resource.

Actions:

• enable - Creates a configuration file for the collector specified and restarts Diamond
• disable - Deletes the configuration file for the collector specified and restarts Diamond

diamond::plugin

Attribute	Type	Description	Default
source	String	Name of the ERB Template to be used generating the Collector configuration files	generic_collector_config.conf.erb
cookbook	String	Name of the cookbook holding the ERB template for the source attribute	/opt/diamond/etc/diamond/collectors
runit_name	String	Name of the runit service to restart when modifying this plugin	diamond
options	Hash	Allows you to specify options outside of enabling a collector

Generic Usage

The following example will just configure Diamond with the GraphiteHandler and with the default collectors enabled only.

diamond_install "#{node['hostname']}" do  
  action :git  
end  
diamond_configure "#{node['hostname']}" do   
  action :config  
  diamond_handlers [ "diamond.handler.graphite.GraphiteHandler" ]    
  graphite_handler({"host" => "127.0.0.1","port" => 2003, "timeout" => 15})  
end

Basic Usage with Plugins

The following example will configure Diamond with the archive.ArchiveHandler and graphite.GraphiteHandler with the PingCollector and CPUCollector enabled.

diamond_install "#{node['hostname']}" do  
  action :git  
end  
diamond_configure "#{node['hostname']}" do  
  action :config  
  diamond_handlers [ "diamond.handler.archive.ArchiveHandler", "diamond.handler.graphite.GraphiteHandler" ]  
  graphite_handler({"host" => "127.0.0.1","port" => 2003, "timeout" => 15})  
end  
  diamond_plugin "CPUCollector" do  
  action :enable  
  options({})  
end  
  diamond_plugin "PingCollector" do  
  action :enable  
  options({  
    "enabled" => "True",  
    "bin" => "/bin/ping",  
    "use_sudo" => "False",  
    "sudo_cmd" => "/usr/bin/sudo",  
    "target_1" => "google.com",  
    "target_2" => "aol.com"  
    })  
  end  

Advanced Usage with Custom Collectors Installed

diamond_install "#{node['hostname']}" do  
  action :git  
end  
diamond_configure "#{node['hostname']}" do  
  action :config  
  diamond_handlers [ "diamond.handler.archive.ArchiveHandler", "diamond.handler.graphite.GraphiteHandler" ]  
  graphite_handler({"host" => "127.0.0.1","port" => 2003, "timeout" => 15})  
end  
directory "/usr/local/share/diamond/collectors/latency_http" do  
  action :create  
end  
cookbook_file "/usr/local/share/diamond/collectors/latency_http/latency_http.py" do  
  source "latency_http.py"
  cookbook "our_custom"
end
diamond_plugin "LatencyHTTTPCollector" do  
  action :enable  
  options({  
    "enabled" => "True",  
    "target_1" => "host1.com",  
    "target_2" => "host2.com"  
  })  
end  

Author:: Scott M. Likens ([email protected])

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.