We have updated our Terms of Service, Code of Conduct, and Addendum.

Can only see splunk metrics.log events and not other events from monitored files via the Splunk UF

mikeylee
mikeylee Posts: 6

Hi guys,
I’m testing cribl stream for the 1st time by getting a splunk UF to forward to a Worker on the same centos7 host (ip: 10.0.2.9).

When checking Live Data, I only see events from /opt/splunkforwarder/var/log/splunk/metrics.log
I was expecting to see events from /var/log/audit/audit.log and /var/log/secure which are listed as being monitored files:

/opt/splunkforwarder/bin/splunk list monitor

Your session is invalid. Please login.
Splunk username: admin
Password:
Monitored Directories:
$SPLUNK_HOME/var/log/splunk
/opt/splunkforwarder/var/log/splunk/audit.log
/opt/splunkforwarder/var/log/splunk/btool.log
/opt/splunkforwarder/var/log/splunk/conf.log
/opt/splunkforwarder/var/log/splunk/configuration_change.log
/opt/splunkforwarder/var/log/splunk/dfm_stderr.log
/opt/splunkforwarder/var/log/splunk/dfm_stdout.log
/opt/splunkforwarder/var/log/splunk/first_install.log
/opt/splunkforwarder/var/log/splunk/health.log
/opt/splunkforwarder/var/log/splunk/license_usage.log
/opt/splunkforwarder/var/log/splunk/mongod.log
/opt/splunkforwarder/var/log/splunk/remote_searches.log
/opt/splunkforwarder/var/log/splunk/scheduler.log
/opt/splunkforwarder/var/log/splunk/search_messages.log
/opt/splunkforwarder/var/log/splunk/searchhistory.log
/opt/splunkforwarder/var/log/splunk/splunkd-utility.log
/opt/splunkforwarder/var/log/splunk/splunkd_access.log
/opt/splunkforwarder/var/log/splunk/splunkd_stderr.log
/opt/splunkforwarder/var/log/splunk/splunkd_stdout.log
/opt/splunkforwarder/var/log/splunk/splunkd_ui_access.log
/opt/splunkforwarder/var/log/splunk/wlm_monitor.log
$SPLUNK_HOME/var/log/splunk/license_usage_summary.log
/opt/splunkforwarder/var/log/splunk/license_usage_summary.log
$SPLUNK_HOME/var/log/splunk/metrics.log
/opt/splunkforwarder/var/log/splunk/metrics.log
$SPLUNK_HOME/var/log/splunk/splunk_instrumentation_cloud.log*
/opt/splunkforwarder/var/log/splunk/splunk_instrumentation_cloud.log
$SPLUNK_HOME/var/log/splunk/splunkd.log
/opt/splunkforwarder/var/log/splunk/splunkd.log
$SPLUNK_HOME/var/log/watchdog/watchdog.log*
/opt/splunkforwarder/var/log/watchdog/watchdog.log
$SPLUNK_HOME/var/run/splunk/search_telemetry/search_telemetry.json
$SPLUNK_HOME/var/spool/splunk/tracker.log

Monitored Files:
$SPLUNK_HOME/etc/splunk.version
/var/log/audit/audit.log
/var/log/secure

When the splunk UF was forwarding directly top my standlaone instance of splunk(ip: 10.0.2.9) , I could see these events in splunk.

Below is the configuration on the splunk-UF/Worker-node:

/opt/splunkforwarder/bin/splunk list forward-server

Active forwards:
10.0.2.9:9997
Configured but inactive forwards:
None

#cat /opt/splunkforwarder/etc/system/local/outputs.conf
[tcpout:cribl]
server = 10.0.2.9:9997
sendCookedData = true
negotiateProtocolLevel = 0

[tcpout]
defaultGroup = cribl

Any ideas what I might be doing wrong?

Best Answer

  • mikeylee
    mikeylee Posts: 6
    Answer ✓

    My bad. Ive found the events - just had to really increase the "Capture Up to N Events" and duration. We good now. Cheers

Answers