Passing data to another host with no extra meta data
Hello all. I feel this should be simple, but not working for me. I receive lots of different data types on one port. For one particular type, I want to just forward it on, completely untouched, to a different host on a different port as JSON. I am using the TCP JSON output. The issue is, Cribl is setting the metadata and parsing fields, such as _raw. I don't want any parsing, no fields, no metadata- just the raw data. I must be missing something? Thanks in advance for any suggestions.
Answers
-
Did you try sending it through the passthru pipeline?
0 -
What destination are you sending to? If it's not Splunk, you can drop fields you don't want going out the door before you transmit.
0 -
post processing pipeline/Pack
0 -
If I use passthru pipeline, Crible sets metadata fields host and _time
0 -
so can't use that
0 -
you can remove host and _time fields
0 -
use a post processing pipelne to do that
0 -
and the eval function
0 -
I guess I will have to mask the _raw at the beginning also?
0 -
that will work I guess, but I thought there must be a simpler way
0 -
_raw is the raw event I would think you need that
0 -
I need the data untouched, so no field name
0 -
it comes in as raw json, like {data1:"blah"}
0 -
in the output I have "_raw":"{bl;ah.."
0 -
ok guess i thought _raw was the raw data. Iwill wait for someone from Cribl to chime in.
0 -
You can also create a pipeline that doesn't do anything other than remove the fields you don't want passed on. I do that in Packs, becaus they're more portable, but to each his own.
0 -
use that instead of a pass-thru. it acts like a pass-thru
0 -
but gives you more control.
0 -
Thanks for the suggestion of using post-processing. That fixed the metadata stuff.
0 -
However, I still have a _raw field
0 -
so my incoming message looks like {json_field1:blah...
0 -
outgoing looks like raw: {json....
0 -
I need to get rid of the field name _raw
0 -
I tried masking it
0 -
but the quotes are still there, so now my outgoing starts with "":"
0 -
Hey Steve, What source is receiving the data? I'd need to doublecheck - there is probably a better way to do this - but you could just create a 2-function pipeline to parse out the fields from _raw and then remove _raw, _time, and host:
0 -
let me give that a try, thanks
0 -
It's raw TCP in
0 -
from Vectra
0 -
So I am using raw tcp source, JSON output
0
