What would the Source field be for a json parsed event?
Good morning, I'm trying to remove some key/values with null values in a tcpjson event w/ the parser function. I noticed that having Source field equal to _raw doesn't access the JSON parsed log data. What would the Source field be for a json parsed event?
Answers
-
I have found the following Parser() function options effective at removing keys whose values are null. You should set _raw to `JSON.stringify(_raw)` afterwards though.
0 -
Yup, I have something similar. My problem is using a tcpjson as your source automatically parses the JSON, and it doesn't create an _raw field. So, I'm not sure how to go about accessing the keys in the event
0 -
So when your events hit the pipeline, the JSON object has already been parsed and the contents are extracted as top-level fields (including keys w/null values).
0 -
Gotchya, is there a way to access the parsed JSON without re-writing it to _raw & back to the event?
0 -
Sorry, my comment was not meant to be a statement. It was meant to be a question. I have a TCP JSON data source on my Stream v3.5.4 instance and events come in as JSON in _raw. So, in my case the TCP JSON source is not automatically parsing the JSON. Which version of Stream are you running?
0 -
Looks like I'm on 4.1
0 -
and the answer to your question above is yes. There's no _raw key, everything is a top-level field
0 -
If there is a specific key you are looking at, you can use that key in parser instead of _raw. However, I assume that you are trying to apply the filter to all fields. I am not sure how to do this without a code function
0 -
Do you serialize some of the top-level fields back into _raw as a JSON object at any point in your pipeline? That would be an opportunity to filter out the keys w/null values.
0
