Is there a way to order/sort the JSON keys prior to hitting the destination?
Is there a way to order/sort the JSON keys prior to hitting the destination? I thought prepending the key names with an underscore would bring them on the top but they show up at the bottom of the JSON structure. The ordering looks the way we want it in the pipeline view but come out differently in the actually output.
Answers
-
you can probably do it with regex or other hacks but note that JSON, unlike CSV or some others, is not positional.
0 -
Python has sort_keys param. So nothing like this in JS/cribl?https://www.w3schools.com/python/gloss_python_json_sort.asp
0 -
We have a regex on the destination side to parse the source identifier and that coming towards the end makes our regex expensive.
0 -
if you're using regex, do you need JSON? Maybe switch you K=V or CSV, where you can specify order.
0 -
Yes we need JSON. Our SIEM parser is only compatible with JSON for this specific source (Azure).
0 -
but requires regex?
0 -
correct
0 -
SIEM does have a JSON parser but that's available only post processing for property extraction.
0 -
The Stringify() function in typescript doesn't have a sorting option. Your alternative would be to use the Code function to build your own stringifier where you specify the order
0 -
How do we do that at the event level? Iterate through __e?
0 -
(Code or Eval)
0 -
iterating is one option. But since you have a requirement around order, I'd think you'd want to just layout the fields
0 -
built the textual json by hand, 1 field at a time
0 -
or, dump all the keys, sort them, then loop through that list to build the string
0 -
Will try different options but would be great if we could have this option as a feature enhancement.
0 -
```try { var ans = "{" var a = Object.keys(__e.f).sort() for (var k in a) { ans = `${ans}"${a[k]}":"${__e.f[a[k]]}",` } __e.ans = ans.replace(/,$/,'}') } catch (e) { __e.debug = e.message }```
0 -
where f is your object you want to sort into a stringified JSON
0 -
of course this may be a problem if your object isn't flat. just a starting point :slightly_smiling_face:
0 -
I'll need to do some testing since the webhook destination dumps the entire event except for the hidden fields. Not sure if it is even possible to sort at the top level.
0 -
i guess i'm still not clear then on what the goal is
0 -
If the destination supported syslog, I'll just have to modify the host.
0 -
With webhooks, it gets a bit tricky
0 -
Force order (Code function): (put your desired fields in the `first` array, everything else is included with the `...rest`) ```let first = ["ProviderId","ProviderName","MachineName"]; let [...rest] = Object.keys(e).filter(key => !first.includes(key) && !key.startsWith("")); const ordered = [...first, ...rest]; __e['__httpOut'] = JSON.stringify(ordered.reduce((current, val) => Object.assign(current, {[val]: __e[val]}), {}));```
0 -
Then change your webhook settings to use the "Custom" format with `__httpOut` as the expression.
0 -
-
Input
0 -
Output (note the first 3 fields match what I wanted first)
0 -
baller
0 -
Thanks <@U01LSBF5953>. Will give it a try.
0 -
For those curious on what the dots do: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment
0
