We have updated our Terms of Service, Code of Conduct, and Addendum.

I'm having a hard time get Redis running on my Cribl Workers

Hi all, I'm having a hard time get Redis running on my Cribl Workers and I don't know what the issue is. What I've done so far: 1. Setup a Redis Server in Docker (Standalone) 2. Set Up a pipeline as demonstarted in https://docs.cribl.io/stream/redis-function/#scenario-a|Docs 3. Setup Redis URL `redis://<password>@&lt;host>:<port>` 4. Authentication: None 5. Tested the connection from my cribl worker via `redis-cli` from which I was able to write and retrive Keys from the Redis-Server Basically, my question is: Why doesn't it work using Cribl? Any ideas on that? Cheers :zebra_face:

«1

Answers

  • David Maislin
    David Maislin Posts: 229 mod

    if your Redis endpoint is encrypted you may need to use rediss (double S)

  • Rob Franz
    Rob Franz Posts: 33 mod

    <@U01AWTNNHQB&gt; you mean: Setup Redis URL `rediss://<password>@&lt;host>:<port>` If so, it is still not working.

  • David Maislin
    David Maislin Posts: 229 mod

    Can you share a screenshot of your redis function in Cribl?

  • Rob Franz
    Rob Franz Posts: 33 mod

    Yep, rediss:// is necessary for encrypted endpoints, you can think of it as http(s). We use redis<s://fqdn:port|s://fqdn:port> and add the credentials under Authentication Method as manual.

  • David Maislin
    David Maislin Posts: 229 mod
    1. Is the Redis URL quoted`'redis://<password>@&lt;host>:<port>'`
  • David Maislin
    David Maislin Posts: 229 mod
    1. Is the Key quoted?
  • David Maislin
    David Maislin Posts: 229 mod

    Just an example

  • Rob Franz
    Rob Franz Posts: 33 mod

    First Screenshot shows settings, the second one that there is data on the server. Tried both, with quoted and unquoted URL.

  • David Maislin
    David Maislin Posts: 229 mod

    I just set and get

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Do you have any special characters in the password? Colons, percents, etc?

  • Rob Franz
    Rob Franz Posts: 33 mod

    Still not working... Getting an Error, if I check the logs nothing of interesst appears in there, just the two rows you can see in the screenshot

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Basically would this regex fail? `[a-zA-Z0-9_-]+`

  • Rob Franz
    Rob Franz Posts: 33 mod

    <@U01LSBF5953&gt; no special characters. You're regex would catch

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Can you share more about this Docker setup? How are you handling exposing the port?

  • Rob Franz
    Rob Franz Posts: 33 mod

    Think that is not the issue, as I retrieve data from the Redis-Server when using the `redis-cli -h <redis-server-ip> -p 6379` command and `get <key>` from cribl worker. Or do you think that still could be an issue?

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    No, that's a good sign.

  • Rob Franz
    Rob Franz Posts: 33 mod

    Although I couldn't find any information about this, but are there maybe other requirements, such as packages, libraries I need to install on the OS?

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Hold on... you can't put your password in the URL like that.

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    You need to use "Manual" authentication and enter it into the password box.

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Like so:

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Additionally, do you require a password using the `redis-cli` by specifying the `-a <password>` flag?

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    If not, I don't think you require any authentication at all.

  • Rob Franz
    Rob Franz Posts: 33 mod

    In regards to the docs, I can set the password in the URL (see screenshot). For `redis-cli` before quering any data I use `AUTH <password>` but I dont need a password in order to execute this one successfully `redis-cli -h <redis-server-ip> -p 6379`

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Cool. Try taking the password out of the URL and entering it under Manual auth

  • Rob Franz
    Rob Franz Posts: 33 mod

    Doesn't work either. I tried without an entry in Username and with `default` as the Username

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod

    Would you be willing to share your docker setup? What version of Cribl are you running? Standalone or distributed?

  • Rob Franz
    Rob Franz Posts: 33 mod

    » Cribl 4.0.2 » Distributed Setup » Docker: ‚ó¶ Image: redis:latest ‚ó¶ PORTS: 0.0.0.:6379 -> 6379/tcp Docker isn't really my business and someone else set up the Container. Do you need some more information? If so, what do you like to see?

  • Raanan Dagan
    Raanan Dagan Posts: 101 mod

    Are you using the first or second option? `'redis..'` ``redis..``

  • Rob Franz
    Rob Franz Posts: 33 mod

    <@U01J549PR6Y&gt; the first with single-quotes, if you're asking for the URL.

  • Raanan Dagan
    Raanan Dagan Posts: 101 mod

    try the second option ( ``redis..``).. at least that seems to work for me ..