Cribl Edge on Windows - How to handle proxy?

Andrew Farmer

I deployed Cribl on some Windows Servers. They are routing events to an on-premise Cribl Stream instance. Everything was working fine, until collection just stopped. It appears that the Edge agents aren't connecting to "https://cdn.cribl.io/telemetry". These servers are behind a web proxy. The proxy settings are applied properly in Windows.

If I pull up a browser on the server, and surf to https://cdn.cribl.io/telemetry, I get the expected output --- “cribl /// living the stream!”

I have tried:

• Manually configuring the proxy settings in Windows itself rather than auto-config.
• Defining actual environment variables for “HTTP_PROXY” and “HTTPS_PROXY” within Windows Server.
• Tried flipping the Cribl Edge “default_fleet” toggle for “Proxy” -> “use environment variables” on and off.

I have manually restarted the services and the Edge agent itself multiple times. Nothing has worked. What is the proper combination of settings in Windows and Cribl Edge to force the agent to use the proxy to phone home?

Brendan Dalpe

Try this PowerShell please:

$environment = [string[]]@("HTTP_PROXY=http://proxy:3128,HTTPS_PROXY=http://proxy:3128")

Set-ItemProperty HKLM:SYSTEM\CurrentControlSet\Services\Cribl -Name Environment -Value $environment

Please change your proxy URLs to the correct values. This will set the environment variables on the Cribl Edge service.

Brendan Dalpe

Try this PowerShell please:

$environment = [string[]]@("HTTP_PROXY=http://proxy:3128,HTTPS_PROXY=http://proxy:3128")

Set-ItemProperty HKLM:SYSTEM\CurrentControlSet\Services\Cribl -Name Environment -Value $environment

Please change your proxy URLs to the correct values. This will set the environment variables on the Cribl Edge service.

Andrew Farmer

This worked properly, and it DOES persist across reboots. Problem solved. Thank you Brendan!

Brendan Dalpe

Excellent! I don't know about persisting through upgrades, so let us know if/when you do that what happens.

Andrew Farmer

Whoopsie…follow-up…. Two things:

(1) The Powershell changes do NOT PERSIST across upgrades.

After the bump from 4.4.0 to 4.4.3, I had to run them again. The output of "Get-ItemProperty HKLM:SYSTEM\CurrentControlSet\Services\Cribl -Name Environment" was blank. Re-running the Powershell restored them.

(2) Separate Issue: BEFORE the upgrade, the Powershell environment variables worked…until they didn't. Everything was still set properly, as configured by the previous commands. However, the Edge agents started failing Telemetry checks again at some point, despite the fact that the Environment variables were set properly at the time. Everything flat-lined except the regular system stats/processes/etc. POST-upgrade, even after applying the environment variables and restarting the Cribl agents (both from the Leader node and the local Windows service), there is still no event flow.

Do subsequent telemetry attempts fail to use the proxy? Is there a way to FORCE the Edge clients to try and reconnect again? A good old-fashioned kick? I'm not sure what's going on in the background.