We have updated our Terms of Service, Code of Conduct, and Addendum.

How do you guys get data into Cribl to transform before hitting Splunk indexers?

Splunk does not recommend a load balancer between a forwarder and receiver. How do you guys get data into Cribl to transform before hitting Splunk indexers?

Answers

  • We support S2S and the SplunkLB output does its own load balancing. Does that answer your question ?

  • I will be configuring a Splunk Universal Forwarder to output to the Load Balancer that sits in front of Cribl, which will then feed into Splunk Indexers.

  • So you are referring to cribl cloud ?

  • No, we have a separate load balancer in front of Cribl hosted in instances.

  • Ok

  • xpac xpac
    xpac xpac Posts: 148 ✭✭✭

    You should still not put a load balancer between your UF and Cribl. It still speaks S2S and that can still result in broken data

  • xpac xpac
    xpac xpac Posts: 148 ✭✭✭

    Give your UF a DNS target that resolves to all your Cribl workers, and the UF will auto load balance

  • xpac xpac
    xpac xpac Posts: 148 ✭✭✭

    That is, unless you use HTTPOUT on the UFs

  • A simple TCPOUT-Server stanza wont work

  • xpac xpac
    xpac xpac Posts: 148 ✭✭✭

    Why?

  • I think it would, how would I verify the data is hitting the LB and coming back into Cribl?

  • we have this working, we just give cribl the Cluster manager and splunk handled the rest. You can check this in the status page the Destination. It would have listed out all your splunk indexers

  • Can you see it in Splunk?

  • Clint Sharp
    Clint Sharp Posts: 27 mod

    It's actually less of a problem with Cribl in the middle because we do event breaking and spread the events properly at the index tier.

  • xpac xpac
    xpac xpac Posts: 148 ✭✭✭

    Yeah, Cribl distributes very well. Putting a LB between UF and Cribl can cause partial events, or stuck UFs. Splunk has loadbalancing built-in, use it :wink:

  • Clint Sharp
    Clint Sharp Posts: 27 mod

    Stuck UFs, no big deal, doesn't really matter what worker it hits other than in theory you could end up with a very busy worker process. Partial events, also shouldn't really happen, what have you seen and why?

  • All I did was specify outputs.conf [<tcpout-server://host>:port]