How does the primary server schedule out script collectors?
how does the primary server schedule out script collectors? We were looking at the stats for one of the collectors (looking at the data in Splunk) and it appears that one worker node pulled 10,000 events and another pulled in maybe 257 events (number varies on second worker). Does the primary keep track of the state of a collector to switch it to another worker to pull the remaining events or something? Just trying to understand
Answers
-
Just in case, read up https://docs.cribl.io/stream/collectors#how-do-collectors-work|here. in particular: > A collection job is typically made up of one or more tasks that: discover the data to be fetched; fetch data that match the run filter; and finally, pass the results either through the Routes or (optionally) into a specific Pipeline and Destination. The discover phase is first (if it is set-up). The leader picks a worker process to run it. The results from the discover phase will be distributed amongst other worker processes. For heavier collection jobs, i'd recommend having a dedicated WG just for collections to isolate the work from incoming stream workloads.
0 -
-
> the Leader Node oversees the task distribution, and tries to maintain a fair balance across jobs. > When Workers ask for tasks, the Leader will normally try to assign the next task from a job that has the least tasks in progress. This is known as "Least-In-Flight Scheduling," and it provides the fairest task distribution for most cases. If desired, you can change this default behavior by opening Settings > Global Settings > General Settings > Job Limits, and then setting Job Dispatching to Round Robin
0 -
i dont see a Job Limits setting in global settings - general settings
0 -
is that a licensed feature maybe?
0 -
I think that's an artifact that hasn't been updated for 4.0... This has been moved to Group Settings under the Worker Group.
0 -
<!subteam^S030575HHKL|@cribldocs> FYI this needs an update ^^
0 -
also we dont use discover stage to do much, we do it all in the collect script. discover script we had to put something in so it is just an echo line. we dont reference it in the collector script
0 -
<@U027NM6JPEC> - Could you take a look at https://docs.cribl.io/stream/collectors/#collectors-in-distributed-deployments and make sure it's up-to-date? Looks like the Job Limits setting moved and we didn't catch it.
0 -
sure, <@U03CJ90F91A>, will do
0
