We have updated our Terms of Service, Code of Conduct, and Addendum.

How do I extract AWS EC2 metadata from an Edge File Collector's metadata fields?

Dean Solomon
Dean Solomon Posts: 2 ✭
edited November 2023 in Edge

I am running Edge Nodes on our EC2 instances.

When reviewing internal fields from our File Collector sources, I noticed that there is alot of really useful data that our SREs use to investigate which regions & servers are experiencing issues during incidents.

These include:

- AMI image

- Region

- Availability Zone

- Architecture

- Public IP

- Private IP

I'd like to extract these internal fields as event fields that get sent to our destinations. What is the simplest way to do that?

Tagged:

Answers

  • Jeremiah Ross
    Jeremiah Ross Posts: 8 ✭
    edited November 2023

    Hi Dean,

    It sounds like an pre-processing pipeline that includes an eval function will do the trick. I took a few screenshots from the lab to help you along.

    1. Review Your metadata. For an example, see the below fields nested in __metadata.aws
    image.png

    2. Create a new pipeline that can be assigned as the source's pre-processing pipeline

    3. Create an eval function in your new pipeline

    4. Configure your eval function like this:

    Filter: true (to catch all events) - or -

    Filter: source.includes('file') ( to filter by file)

    Eval:

    • Name: <desired field name>
    • Value Expression: __metadata.aws.<desired field to extract>
    image.png

    5. Add this new pipeline to your source (after testing)

    image.png

    6. Commit & Deploy to save changes

    I hope that this helps

Categories