Are there default limits for JSON processing and are they tunable?
Answers
-
Did you create an event breaker and adjust the max event length?
0 -
i did not. what event breaker might it be using since I didn't define one?
0 -
You can go under Knowledge/Event Breakers and create a new RuleSet
0 -
from that it appears "JSON Newline Delimited" is the default, and would apply in my case
0 -
i'm sending via TCPJSON, in which each event is newline delimited (not pretty-printed). that appears to be working
0 -
Have you considered using the TCP source instead which supports EventBreakers?
0 -
"max event bytes " appears to be 1024000 by default in the "JSON Newline Delimited" breaker.. so that must be what i'm up against
so the TCPJSON source does not support event breakers? meaning the default "cribl" event breaker (with ndjson rule) is not even being applied?
0 -
we just using fallback breaker on new lines on TCP JSON right?
I haven't tried, but have you tried upping the ndson ruleset to: `134217728`?
If not, feel free to test with the TCP source and do an event breaker there.
0 -
i haven't touched it since I'm just now learning about these limits. what is that number? seems very specific
0 -
It is the max size of an event we support in bytes.
128 MB
0 -
oohhh. ok well then that's a hard limit across all sources / destinations?
that's very large but I'll have to see if any single event we have exceeds that
0 -
That is the biggest event size we currently support IIRC.
0 -
to answer your question, probably (I'm assuming it's). It doesn't look like TCP JSON source can have an event breaker configured for it so he'll probably have to switch over to the raw TCP to use an event breaker with a larger max event size even though the event delimiter () would be the same.
0 -
Thanks
0 -
That's interesting. Any particular reason there's not configurable event breaker for TCP JSON sources?
0 -
not that i'm aware
0 -
alright cool, thanks guys. I'll test but assuming nothing else pops up, sounds like it should work
i've implemented this but now it's deleted my _raw field in every event
0 -
That's very odd. Maybe need to tweak some things. I can Zoom later if you like.
0 -
nevermind… i was putting __raw into _raw via TCPJSON, but __raw doesn't exist in TCP sources
0 -
Yep indeed. Only _raw exists.
0
