We have updated our Terms of Service, Code of Conduct, and Addendum.

Anyone know the proper method to extract something prior to pipelines to use for filtering?

Options

Anyone know the proper method to extract something prior to pipelines to use for filtering? Case: I'm looking to use kubernetes.namespace_name as a field to compare against to send through different routes. It's nested inside _raw and I can't seem to get it to be referenced without doing some nasty .search() or .match() that could match with other data streams coming through that source

Answers

  • Jon Rust
    Jon Rust Posts: 440 mod
    Options

    Solid thinking

  • Brandon McCombs
    Options

    Try `JSON.parse(_raw)`

  • Jon Rust
    Jon Rust Posts: 440 mod
    Options

    Use a pre-proc pipeline, or a pipeline chain

  • Brandon McCombs
    Options

    assuming _raw is a JSON object. That allows you to avoid a generic search/match of the data by being able to reference specific fields using JSON syntax. Does that help?

  • mferris
    mferris Posts: 27 admin
    Options

    _raw is a JSON object indeed, so this worked like a charm! I had totally assumed it wouldn't want functions like that being run in the filter, but i'm glad it lets it.

  • mferris
    mferris Posts: 27 admin
    Options

    To Jon: That wasn't necessary for this go-round but I'll keep that in the back pocket for all the other data sources

  • Sheldon Carmichael
    Options

    That is a pretty awesome feature when needed. TIL

  • Brandon McCombs
    Options

    <@U034GKZLVB8&gt; any field that has a blue vertical bar on the left side allows JS expressions to be used.

  • Jon Rust
    Jon Rust Posts: 440 mod
    Options

    if i'm going to reference things inside the object more than once, i'll usually set-up a pre-proc to set `__parsed` => `JSON.parse(_raw)` to save cycles

  • mferris
    mferris Posts: 27 admin
    Options

    I did know that, just expected some limitations on some of those fields that weren't in the pipelines. Love the freedom. And another helpful tip from Jon as well. Always appreciate the expedient help!