We have updated our Terms of Service, Code of Conduct, and Addendum.

Data Enrichment on the fly using cribl stream with the enrichments are from a CSV lookup

Options

Hello everyone, I am having an interesting use case that I am curious if Cribl offer any solution for or not, Now we need to perform Data Enrichment on the fly using cribl stream, The enrichments are based on a CSV lookup, this CSV file will be updated on a daily basis based on new findings and new enrichments rules are added everyday. I am hosting this CSV file at Github, The Question now Is there any functionality in Cribl that can capture such CSV file once committed and pushed to github to reflect directly in Cribl Stream Knowledge > Lookups Library. I know I can workaround such by having a CI/CD pipelines updating the backend `$CRIBL_HOME/groups/<groupname>/data/lookups/` But I am just checking if anyone had similar problem before and what was the solution you thought of ?

Answers

  • Paul Hoffman
    Options

    you can work with a database, whether using Concanon's database connector or other options...

  • Paul Hoffman
    Options

    and you can use CICD to do an API call to update the CSV IIRC.