We have updated our Terms of Service, Code of Conduct, and Addendum.

what does "Cannot advance offset 2023514992 past end of event 74" mean?

Johan Woger
Johan Woger Posts: 16 ✭
in Stream

Would it be possible to confirm:

  1. What does Cribl do with the events when this issue occurs
  2. What is the maximum character limit CRIBL can parse

Tagged:

Answers

  • Brian Yearwood
    Brian Yearwood Posts: 14 ✭✭
    edited January 8

    • 1/ When this issues occurs Cribl drops the data, we see this in the logs:
    • First we see the bad payload:
      {"time":"2023-12-19T05:08:12.331Z","cid":"w0","channel":"input:Splunk","level":"error","message":"Failed to parse s2s payload","src":"10.10.10.10:60184","error":{"message":"Cannot advance offset 2023514992 past end of event 74","stack":"Error: Cannot advance offset 2023514992 past end of event 74\n    at l.advance (/opt/cribl/bin/cribl.js:14:16486037)\n    at f._readEvents (/opt/cribl/bin/cribl.js:14:16488397)\n    at f._transform (/opt/cribl/bin/cribl.js:14:16489865)\n    at f.Transform._read (_stream_transform.js:205:10)\n    at f.Transform._write (_stream_transform.js:193:12)\n    at writeOrBuffer (_stream_writable.js:352:12)\n    at f.Writable.write (_stream_writable.js:303:10)\n    at Socket.ondata (_stream_readable.js:719:22)\n    at Socket.emit (events.js:315:20)\n    at Socket.EventEmitter.emit (domain.js:486:12)"}}
    • Then we see the worker process closing the socket where the forwarder is connected to:
      {"time":"2023-12-19T05:08:12.331Z","cid":"w0","channel":"input:Splunk","level":"error","message":"closed connection","src":"10.10.10.10:60184","error":{"message":"Cannot advance offset 2023514992 past end of event 74","stack":"Error: Cannot advance offset 2023514992 past end of event 74\n    at l.advance (/opt/cribl/bin/cribl.js:14:16486037)\n    at f._readEvents (/opt/cribl/bin/cribl.js:14:16488397)\n    at f._transform (/opt/cribl/bin/cribl.js:14:16489865)\n    at f.Transform._read (_stream_transform.js:205:10)\n    at f.Transform._write (_stream_transform.js:193:12)\n    at writeOrBuffer (_stream_writable.js:352:12)\n    at f.Writable.write (_stream_writable.js:303:10)\n    at Socket.ondata (_stream_readable.js:719:22)\n    at Socket.emit (events.js:315:20)\n    at Socket.EventEmitter.emit (domain.js:486:12)"},"r":137366,"b":765138}

    2/ What is the maximum character limit CRIBL can parse

    • The maximum support size of an event is 64MB which is the same limit that is imposed by Splunk receivers.

Categories