We have updated our Terms of Service, Code of Conduct, and Addendum.

Anyone know the proper method to extract something prior to pipelines to use for filtering?

Anyone know the proper method to extract something prior to pipelines to use for filtering? Case: I'm looking to use kubernetes.namespace_name as a field to compare against to send through different routes. It's nested inside _raw and I can't seem to get it to be referenced without doing some nasty .search() or .match() that could match with other data streams coming through that source

Answers

  • Jon Rust
    Jon Rust Posts: 445 mod

    Solid thinking

  • Try `JSON.parse(_raw)`

  • Jon Rust
    Jon Rust Posts: 445 mod

    Use a pre-proc pipeline, or a pipeline chain

  • assuming _raw is a JSON object. That allows you to avoid a generic search/match of the data by being able to reference specific fields using JSON syntax. Does that help?

  • mferris
    mferris Posts: 27 admin

    _raw is a JSON object indeed, so this worked like a charm! I had totally assumed it wouldn't want functions like that being run in the filter, but i'm glad it lets it.

  • mferris
    mferris Posts: 27 admin

    To Jon: That wasn't necessary for this go-round but I'll keep that in the back pocket for all the other data sources

  • That is a pretty awesome feature when needed. TIL

  • <@U034GKZLVB8&gt; any field that has a blue vertical bar on the left side allows JS expressions to be used.

  • Jon Rust
    Jon Rust Posts: 445 mod

    if i'm going to reference things inside the object more than once, i'll usually set-up a pre-proc to set `__parsed` => `JSON.parse(_raw)` to save cycles

  • mferris
    mferris Posts: 27 admin

    I did know that, just expected some limitations on some of those fields that weren't in the pipelines. Love the freedom. And another helpful tip from Jon as well. Always appreciate the expedient help!