We have updated our Terms of Service, Code of Conduct, and Addendum.

Anyone know the proper method to extract something prior to pipelines to use for filtering?

mferris
mferris Posts: 27 admin
in General Discussions

Anyone know the proper method to extract something prior to pipelines to use for filtering? Case: I'm looking to use kubernetes.namespace_name as a field to compare against to send through different routes. It's nested inside _raw and I can't seem to get it to be referenced without doing some nasty .search() or .match() that could match with other data streams coming through that source

Answers

  • Jon Rust
    Jon Rust Posts: 443 mod

    Solid thinking

  • Brandon McCombs
    Brandon McCombs Posts: 150 mod

    Try `JSON.parse(_raw)`

  • Jon Rust
    Jon Rust Posts: 443 mod

    Use a pre-proc pipeline, or a pipeline chain

  • Brandon McCombs
    Brandon McCombs Posts: 150 mod

    assuming _raw is a JSON object. That allows you to avoid a generic search/match of the data by being able to reference specific fields using JSON syntax. Does that help?

  • mferris
    mferris Posts: 27 admin

    _raw is a JSON object indeed, so this worked like a charm! I had totally assumed it wouldn't want functions like that being run in the filter, but i'm glad it lets it.

  • mferris
    mferris Posts: 27 admin

    To Jon: That wasn't necessary for this go-round but I'll keep that in the back pocket for all the other data sources

  • Sheldon Carmichael
    Sheldon Carmichael Posts: 20 ✭

    That is a pretty awesome feature when needed. TIL

  • Brandon McCombs
    Brandon McCombs Posts: 150 mod

    <@U034GKZLVB8&gt; any field that has a blue vertical bar on the left side allows JS expressions to be used.

  • Jon Rust
    Jon Rust Posts: 443 mod

    if i'm going to reference things inside the object more than once, i'll usually set-up a pre-proc to set `__parsed` => `JSON.parse(_raw)` to save cycles

  • mferris
    mferris Posts: 27 admin

    I did know that, just expected some limitations on some of those fields that weren't in the pipelines. Love the freedom. And another helpful tip from Jon as well. Always appreciate the expedient help!

Categories