We have updated our Terms of Service, Code of Conduct, and Addendum.

What cribl internal fields are available to be used in route filters in pre processing packs?

Govardhanen Gopal
Govardhanen Gopal Posts: 19 ✭
in Stream

Hello,

I am re-factoring a lot of our code + workflow and trying to simplify stuff. I ran into this issue and I wanted to check with the community + cribl folks if they could give some guidance.

We have a few groups, some groups have an rsyslogd input configured. This is configured as "networkrsyslogd". As you can imagine we have networking gear sending out its events over.

I stood up a new "Pre-Processing-Pack" and I tried to use create a route filtering via the cribl internal field __inputId but alas this did not work, I further configured the above input to include a new field inputName="networkingsyslog" and further when I used this as as a filter in routes, things work.

That said, other than fields that are emitted by the source, what cribl internal fields can I use for filtering purposes? I thought logically this should have been stamped on the events as soon as it hits stream or edge, but doesn't look like it does.

Thank you!
g

Answers

  • Jon Rust
    Jon Rust Posts: 487 mod

    If you capture sample data, in the capture window you can hit the 3 dots on the right side of the window and "show internal fields." Every internal field is available in expressions, filters, etc.

    image.png

  • Govardhanen Gopal
    Govardhanen Gopal Posts: 19 ✭

    Hey Jon,

    Thank you for responding. Unfortunately the __internalID is not respected in route filters.

    gov

  • Govardhanen Gopal
    Govardhanen Gopal Posts: 19 ✭
    edited April 2024

    Hey Jon,

    I am unable to edit the prior comment which had a typo in it. I confirmed that the cribl internal fields(__inputId & __srcIpPort) are not stamped on an event when it its the "Pre routing pipeline". The funny thing is if I introduce my own internal field or field that does get stamped before hitting the "Pre routing pipeline".

    I also tried doing creating this new field and the field myInternalId just gets set to 'NOTHING_FOUND'. Please give it a shot in one of your instances and you will see the same behavior. It would be great if the cribl internal fields inputId & srcIpPort AND others are stamped on events as soon as it hits the app.

    myInternalId : __e['__inputId'] ? __e['_inputId'] : 'NOTHING_FOUND'

    gov

  • Jon Rust
    Jon Rust Posts: 487 mod

    __inputId and others should be available at any stage of the event life. However, be aware that in the case of using CriblTCP or CriblHTTP to send data in a Cribl→Cribl situation, as with Edge for example, the original values will be put into an object named __forwardedAttrs

    See the docs for more info: https://docs.cribl.io/stream/sources-cribl-tcp/#internal-fields

Categories