We have updated our Terms of Service, Code of Conduct, and Addendum.

Cribl function for removing keys based off value

As a part of a cribl log processing pipeline, I want to add a function to remove any JSON keys that have empty strings or empty objects.

I have a serialization function, such as this:

  - id: serde
    filter: "true"
    disabled: null
    conf:
      mode: reserialize
      type: json
      srcField: text
      fieldFilterExpr: value !==""
      fields: []
      remove:
        - .*

but I don't want to reserialize, as it converts a JSON object during my pipeline into a JSON string. Is there another built-in function I can use to achieve this?

Example input:

{"my-log": {}, "my-other-log": "", "my-third-log": false} 

Expected output:

{"my-third-log": false} 

Answers

  • David Maislin
    David Maislin Posts: 229 mod
    edited June 2023

    Just use Parser with Extract instead of Reserialize and set the Destination to _raw and it will be an Object.

    Set the filter expression to value!=null && value

  • dritan
    dritan Posts: 51 ✭✭

    here’s a quick and dirty Code function. There is probably better ways but should get them going

    Object.keys(__e).forEach((k) => (__e[k] === '' || JSON.stringify(__e[k])==='{}') && delete __e[k])