Working with Namshi has opened my mind to a wide of tools for effectiveness. Like always a quote “Graph everything”. Initially, we were using icinga-web for graphing but seems like graphite introduces something missing and better than other tools.

We started using collectd for graphing system metrics to graphite. Its amazingly easy to setup and configure. You can find the salt state for collectd here. Collectd sends metrics to graphite straight of the box which is beauitful. Our systems has multiple cores and really not intuitive to have cores being analyzed separately, its good for individual core analysis but with hyperthreading. Then I got to know of aggregation plugin and Postcache chain which does exatly what I want.

Postcache chain sends the cpu metric to aggregate and truncates the rest.

1
2
3
4
5
6
7
8
9
10
11

<Chain "PostCache">
  <Rule> 
    <Match regex>
      Plugin "^cpu$"
    </Match>
    <Target write>
      Plugin "aggregate"
    </Target>
    Target stop
  </Rule>
</Chain>

Configure the aggregation plugin for cpu to merge all of them and only output average and sum of the metric.

1
2
3
4
5
6
7
8
9
10
11
12

<Plugin "aggregation">
  <Aggregation>
    Plugin "cpu"
    Type "cpu"

    GroupBy "Host"
    GroupBy "TypeInstance"

    CalculateSum true
    CalculateAverage true
  </Aggregation>
</Plugin>

This is much more better to analysis and do prefer it to others.