Handling the missing date_ fields for Splunk Enterprise Security

asc_me

Splunk Enterprise Security… there's a lot of SPL out there that leverages the date_ fields that the TAs on splunk HFs create when parsing time. How is everyone dealing with lack of these fields, and aliasing to CIM etc by Splunk TAs when leveraging cribl stream?

xpac xpac

They're actually using the date_ fields? Oo

iirc not all data in Splunk has them, and they're also not timezone aware, so in the Splunk Community we usually discourage using them...

asc_me

let me check with our professional services team on this, but I believe we have some customers using HEC from cribl workers to splunk indexers and can remove the time value from the output events. This gets the indexers to recreate this during ingestion, and also creates the `date` fields as expected.If you are using the standard splunk tcp/load balanced destination, then this data is "cooked" already, hence why the date_ fields are not created by the indexing tier.

asc_me

Definitely keen on any advice/guidance that can be provided… particular if there's ways to not have cribl send the data as "cooked"…

asc_me

sorry for the late response (i was on vacation).Data from Cribl >> Splunk Indexers will always be "cooked" as of today. This may be something that we change in the future, but today, this is the only way we deliver to Splunk.If you want to recreate the date_ fields, that Splunk was automatically doing, its a very simple EVAL function you can add to you pipeline or packs to create those fields. You can do for all Splunk data, or just filter for certain indexes/source types where you have users or TA's that rely on those fields.A few examples below...

xpac xpac

Remember there may be quirks around this regarding timezones...

asc_me

100%, just showing that you can, if desired, reproduce those missing fields in your output from Cribl Stream. `_time` still needs to be correct though,,,

Clint Sharp

Don't use `date_` fields. This has been guidance from Splunk for probably 10 years. If you need them, it's easy to recreate them.

asc_me

Thanks for the replies… I have created those BUT have no idea how to create the logic behind the date_zone field‚ which is set to either 'local' or an offset in minutes from utc

For context‚ I'm of the understanding it's the splunk security team that are creating spl that is referencing date_ fields

xpac xpac

Can you show a sample SPL where it's used? Just because some Splunk search uses those fields doesn't make that usage smart...

asc_me

This link is an example of one - https://research.splunk.com/cloud/d4c4d4eb-3994-41ca-a25e-a82d64e125bb/

asc_me

I've been doing some testing with this… Are customers using the Splunk HEC destination when doing this? I'm finding I end up with the _raw field wrapped in an extra level of json with a fields array which exists in the json even if I've removed all those extra fields. Curious how to make it so that the output is only the _raw field… or mirror what a HF would expect from a UF so that the HF parses using the TA… Cheers