We have updated our Terms of Service, Code of Conduct, and Addendum.

How do I aggregate multiple logs into metrics?

Kyle McCririe
Kyle McCririe Posts: 29 ✭✭
in General Discussions

I am trying to send the count or average of certain fields instead of the whole events for a specific time period.

Best Answer

  • Chris
    Chris Posts: 13 mod
    Answer ✓

    Here is how I would do it. In Cribl, there is an Aggregations function that does just this. For example, if you had a metric like node_cpu_percent_active_all that you wanted to gather and send an average over 10s, you would build the Aggregations function similar to this:

    What this is doing is averaging these values over 10seconds and sending just the aggregate event to my destination. I also have Metrics mode turned on (at the bottom) because I send this to a Metrics index in Splunk, but you can send it as a log event as well by disabling that.

    With the Aggregations function, you can do these aggregate functions such as sum, count, average and then only send those aggregate events to your destination. You also are able to group by fields in the event (in the example above Im grouping by host) or you can do an overall aggregate.

    More in-depth documentation can be found here: Aggregations | Cribl Docs

Answers

  • Chris
    Chris Posts: 13 mod

    Here is how I would do it. In Cribl, there is an Aggregations function that does just this. For example, if you had a metric like node_cpu_percent_active_all that you wanted to gather and send an average over 10s, you would build the Aggregations function similar to this:

    What this is doing is averaging these values over 10seconds and sending just the aggregate event to my destination. I also have Metrics mode turned on (at the bottom) because I send this to a Metrics index in Splunk, but you can send it as a log event as well by disabling that.

    With the Aggregations function, you can do these aggregate functions such as sum, count, average and then only send those aggregate events to your destination. You also are able to group by fields in the event (in the example above Im grouping by host) or you can do an overall aggregate.

    More in-depth documentation can be found here: Aggregations | Cribl Docs

  • Chris
    Chris Posts: 13 mod
    Answer ✓

    Here is how I would do it. In Cribl, there is an Aggregations function that does just this. For example, if you had a metric like node_cpu_percent_active_all that you wanted to gather and send an average over 10s, you would build the Aggregations function similar to this:

    What this is doing is averaging these values over 10seconds and sending just the aggregate event to my destination. I also have Metrics mode turned on (at the bottom) because I send this to a Metrics index in Splunk, but you can send it as a log event as well by disabling that.

    With the Aggregations function, you can do these aggregate functions such as sum, count, average and then only send those aggregate events to your destination. You also are able to group by fields in the event (in the example above Im grouping by host) or you can do an overall aggregate.

    More in-depth documentation can be found here: Aggregations | Cribl Docs

Categories