How to parse a JSON object?
I have a JSON object that I'm trying to pull apart using the parser function but it's as if the parser doesn't see it as a json object. I put it into other JSON extraction tools and it works but not in Cribl. It should just be a standard "Extract JSON Object from _raw"
Answers
-
Can you share the JSON object? Also, try an Eval with `newfield` = `JSON.parse(_raw)`
0 -
Beat me to it Jon
0 -
Sensitive info in the event and might be difficult to scrub. The eval to a new object didn't seem to work. For more context, they are zScaler events but I have all sources sending to the same port load balanced across our workers. In order to separate out the sourcetype because they are sending to the same port they are appending the sourcetype to the beginning of the event. The first function in each sourcetype pipeline I do a Mask and `"^<sourcetype>" -->""` and just drop the sourcetype and write it to metadata. Then all that is left is the JSON object.
0 -
Not sure why it would work in other parsers and not Cribl after that is done though
0 -
Sample data would help, but there is a known issue with array-based JSON objects. Eg: ```[ { "field1": "value" }, { "field2": "value" }, { "field3": "value" } ]``` You'll notice this a) does not show as parseable in the preview pane; and b) will not be parsed by the Parser function. You can use the JSON.parse() method I mentioned above though.
0 -
```{"LogTimestamp": "Mon Feb 27 13:59:51 2023","Customer": "-----","SessionID": "4asdasdasdasdg","ConnectionID": "4asasdsdasdqbFiS9,bJasd/asddsfdsOv","InternalReason": "BRK_MT_TERMINATED","ConnectionStatus": "close","IPProtocol": 6,"DoubleEncryption": 0,"Username": "bdfsdfsdfsdfsdf","ServicePort": 443,"ClientPublicIP": "123.1.2.3","ClientPrivateIP": "123.5.6.7","ClientLatitude": 40.000000,"ClientLongitude": -75.000000,"ClientCountryCode": "US","ClientZEN": "US","Policy": "Policy","Connector": "sdasdasd","ConnectorZEN": "US","ConnectorIP": "10.0.0.1","ConnectorPort": 56246,"Host": "host","Application": "application","AppGroup": "Apps","Server": "0","ServerIP": "10.123.33.4","ServerPort": 445,"PolicyProcessingTime": 21,"ServerSetupTime": 1085,"TimestampConnectionStart": "2023-02-27T13:59:51.001Z","TimestampConnectionEnd": "2023-02-27T13:59:51.981Z","TimestampCATx": "","TimestampCARx": "2023-02-27T13:59:51.001Z","TimestampAppLearnStart": "","TimestampZENFirstRxClient": "2023-02-27T13:59:51.101Z","TimestampZENFirstTxClient": "2023-02-27T13:59:51.036Z","TimestampZENLastRxClient": "2023-02-27T13:59:51.352Z","TimestampZENLastTxClient": "2023-02-27T13:59:51.187Z","TimestampConnectorZENSetupComplete": "2023-02-27T13:59:51.017Z","TimestampZENFirstRxConnector": "2023-02-27T13:59:51.036Z","TimestampZENFirstTxConnector": "2023-02-27T13:59:51.017Z","TimestampZENLastRxConnector": "2023-02-27T13:59:51.187Z","TimestampZENLastTxConnector": "2023-02-27T13:59:51.352Z","ZENTotalBytesRxClient": 2115,"ZENBytesRxClient": 1598,"ZENTotalBytesTxClient": 5331,"ZENBytesTxClient": 5331,"ZENTotalBytesRxConnector": 5331,"ZENBytesRxConnector": 5331,"ZENTotalBytesTxConnector": 2115,"ZENBytesTxConnector": 2115,"Idp": "IDP","ClientToClient": "0"}```
0 -
Here's a scrubbed event. Not array based
0 -
The event starts as: ```zscalerlss-zpa-app{"LogTimestamp": "Mon Feb 27 13:59:51 2023","Customer": "-----","SessionID": "4asdasdasdasdg","ConnectionID": "4asasdsdasdqbFiS9,bJasd/asddsfdsOv","InternalReason": "BRK_MT_TERMINATED","ConnectionStatus": "close","IPProtocol": 6,"DoubleEncryption": 0,"Username": "bdfsdfsdfsdfsdf","ServicePort": 443,"ClientPublicIP": "123.1.2.3","ClientPrivateIP": "123.5.6.7","ClientLatitude": 40.000000,"ClientLongitude": -75.000000,"ClientCountryCode": "US","ClientZEN": "US","Policy": "Policy","Connector": "sdasdasd","ConnectorZEN": "US","ConnectorIP": "10.0.0.1","ConnectorPort": 56246,"Host": "host","Application": "application","AppGroup": "Apps","Server": "0","ServerIP": "10.123.33.4","ServerPort": 445,"PolicyProcessingTime": 21,"ServerSetupTime": 1085,"TimestampConnectionStart": "2023-02-27T13:59:51.001Z","TimestampConnectionEnd": "2023-02-27T13:59:51.981Z","TimestampCATx": "","TimestampCARx": "2023-02-27T13:59:51.001Z","TimestampAppLearnStart": "","TimestampZENFirstRxClient": "2023-02-27T13:59:51.101Z","TimestampZENFirstTxClient": "2023-02-27T13:59:51.036Z","TimestampZENLastRxClient": "2023-02-27T13:59:51.352Z","TimestampZENLastTxClient": "2023-02-27T13:59:51.187Z","TimestampConnectorZENSetupComplete": "2023-02-27T13:59:51.017Z","TimestampZENFirstRxConnector": "2023-02-27T13:59:51.036Z","TimestampZENFirstTxConnector": "2023-02-27T13:59:51.017Z","TimestampZENLastRxConnector": "2023-02-27T13:59:51.187Z","TimestampZENLastTxConnector": "2023-02-27T13:59:51.352Z","ZENTotalBytesRxClient": 2115,"ZENBytesRxClient": 1598,"ZENTotalBytesTxClient": 5331,"ZENBytesTxClient": 5331,"ZENTotalBytesRxConnector": 5331,"ZENBytesRxConnector": 5331,"ZENTotalBytesTxConnector": 2115,"ZENBytesTxConnector": 2115,"Idp": "IDP","ClientToClient": "0"}``` And i just run a mask `^zscalerlss-zpa-app` --> `""`
0 -
I use that header to assign sourcetype
0 -
Hmm. The sample you gave me above works with Parser.
0 -
This is really strange. I run the original event with the sourcetype header through and can see _raw transformed to just a json object and it won't pull it apart with parser. I copy the event to a new sample file and run it through the pipeline and it will parse it as a JSON object no problem
0 -
possibly a pack would help to troubleshoot? create a new pack, add your scrubbed sample data to it, and your pipeline. export and paste here or DM me. or open a ticket with <mailto:support@cribl.io|support@cribl.io>
0 -
or we can try a screenshare
0 -
i can join a screen share after 3p PST
0
