How do you get the workers to send metrics (like CPU usage, etc.) to Splunk?
Noob question here... How do you get the workers to send metrics (like CPU usage, etc.) to Splunk? We have the Leader sending cribl metrics, but not the workers.
Answers
-
Enable the internal Cribl Metrics input to send them through the pipelines.
0 -
Since I didn't set up the Cribl env, I will have to look to see how that leader was set up, I guess to see how to do that for the workers.
0 -
Not sure what you mean by the leader sending metrics. The leader doesn't send out any metrics.
0 -
0
0 -
These are going to the cribl_logs index. The cpuPerc is what I'm most interested in.
0 -
I think....
0 -
Those are only logs. Not actual metrics in a metrics format. Metrics need to go to a metrics index. If what you have there is sufficient then you won't need the cribl metrics source enabled but you'll want the cribl logs source enabled instead.
0 -
So, this is what we have that i would consider the metrics going to a metrics index, but there is nothing in the index (which is a metrics index):
0 -
I didn't set that up, BTW, so please don't think I know why it was set up this way.
0 -
Is the cribl metrics source enabled ?
0 -
How do I find that out?
0 -
Go into the inputs for each worker group to see if that input is on.
0 -
From the Sources -> Cribl Internal -> CriblMetrics, they are enabled. Is that what you mean?
0 -
yes
0 -
So if they are already enabled, why would I not see the metrics in that index?
0 -
Thank you for your help, BTW. I truly appreciate it.
0 -
hard to say, i've only seen a small section of your configuration. Maybe a route for them isn't setup? Maybe they aren't even going thru that pipeline.
0 -
Route is there.
0 -
Pipeline:
0 -
I recommend filing a support case so we can review the entire config in depth and possibly conduct a zoom call if necessary. Please include a diag bundle from a worker node (not the leader node) for our review.
0 -
Gov't won't let out diags that aren't completely cleaned and then they have to go through a clearance process. But I can file a support case, and do a screen share.
0
