Searching for keywords across all fields in live data.
it might be a weird question… But as i look at live Data (cribl Stream) (to look at whats coming in the system) i see filter expression to see and capture but is there a quick way to search the data real fast like what kind of keywords across all fields as i dont know what fields there are. like a mini search like | %keyword% or on the fly matching (i have 1000s of new data to hunt for
Best Answer
-
There's pretty much two options: send data to Splunk or any comparable destination, use that to search. Or create a pipeline that serializes all fields to one field in e.g. KV. format, then use the filter to search on that field. For the latter, make sure the point you capture at is after the pipeline is applied, e.g. "Before destination"
0
Answers
-
Search can be done in a captured set of data like this using a javascript expression
0 -
ahhh i tried to do that with the filter but it didnt let me free search
i had to go to Sources / then Live Data
0 -
it's touchy, click in there type search and hit enter
0 -
is this not like splunk search
but kind of like %contain%
0 -
lol, that's what Cribl Search (product) is for... This is a simple way to just display records you are looking for in a capture.
Any valid javascript expression can be used to match the records... For example if u have an array of strings (named arr): `arr.includes('someValue')`
0 -
so cribl stream doesnt really have that search baked in
so searching ALL would it be _raw
0 -
You can access the search product. Stream is designed to move data from point A to point B.
0 -
cause im getting 10000s of promethus but im not sure what/wehre it is but i know
oh is that only cloud view
0 -
yes, Search is only available in Cloud for now. Hopefully the filtering technique listed above will help you sift through the records you are looking for
0 -
ya trying to find how can you filter through all fields
as each event i dont know what fields im seeing (this is the data learning porttion on how / what to look for)
*.includes(‘something’)
0 -
if looking in raw, `_raw.includes('some text')` is a quick way to do it
0 -
if _raw has all in 1 but the prometheus write i dont know
does _raw come in all
0 -
yes
well, most cases have _raw
0 -
ya this one i dont
but 100k records im not sure how to look for what i need but i do know what keyword globally i just dont know what fields are there to search. in splunk world (super lazy but quick was search the text and it would search through everything matching
kind of real world example on how to idenitfy inbound data when i dont know what it is to look for hehe
0 -
the m indicates its a metric. Those two rows are metrics
0 -
ya so i dont know if alll rows are metrics theres other scenarios so its like guessing what is where
0 -
if your trying to explore some captured data or sample data from pipeline you can also use following options to further refine which fields in the dataset to display, which columns to show.
0 -
ya it just doesnt work if i dont know what field that data is going to show in
i found it out instance (but i had to reverse engineer what the tcpdump shows) can that filter search ALL text through ALL fields even if you dont know what field is what im after
0 -
what type of data are you trying to analyze?
0 -
There's pretty much two options: send data to Splunk or any comparable destination, use that to search. Or create a pipeline that serializes all fields to one field in e.g. KV. format, then use the filter to search on that field. For the latter, make sure the point you capture at is after the pipeline is applied, e.g. "Before destination"
0 -
oh wow ( im in the information discovery phase so sometimees i dont know what to look for but maybe could regex have a filter on scanning 80 fields but not sure where the text i need is in one of those (kind of interesting as cribl main thing is part of data inbound) but doing simple IF anything contains i thought would be out of the box
0
